Transcription: OpenAgents Episode 072 - LLM Inference Plugin

Source: https://twitter.com/OpenAgentsInc/status/1755001183784075769 Wiki source: https://raw.githubusercontent.com/wiki/OpenAgentsInc/openagents/Video-Series.md Media title: OpenAgents - Episode 072: LLM Inference Plugin We write our new agent's third and ... Upload date: 20240206 Transcription model: mlx-whisper/mlx-community/whisper-tiny Generated at: 2026-06-01T19:05:56Z

Machine-generated transcript. Review speaker labels and wording before using this as quote-grade source material.

[00:00] Christopher David: Alright, let's write our third and for this project final plugin using our new PHP host function support from the last episode.

[00:11] Christopher David: We're now going to write this to urban.

[00:14] Christopher David: So to be clear, we wrote our first plugin as a basic Rustxism plugin.

[00:23] Christopher David: We wrote this one number two as a Go plugin, also using host functions, just for the HTTP request.

[00:33] Christopher David: And we deployed that to a separate Go service using L4 or two in video 69.

[00:47] Christopher David: Nope, 70.

[00:49] Christopher David: While we built it in 69 and deployed it in 70.

[00:52] Christopher David: And now we're going to use the new host function support that we tested in the last video to pull in a plugin to our code base with host functions.

[01:06] Christopher David: Passing it in a PHP version of HTTP host function.

[01:12] Christopher David: It has even necessary.

[01:14] Christopher David: This one we were able to use the Go HTTP function built into the SDK.

[01:24] Christopher David: Pity-key.

[01:28] Christopher David: Lots of fun architectural things.

[01:30] Christopher David: Okay, let's do this in one and rust.

[01:40] Christopher David: Let's fork this from the other one that we made.

[01:46] Christopher David: Plug in your extractor.

[01:50] Christopher David: So let's do.

[01:58] Christopher David: Let's do this.

[02:03] Christopher David: Get clone.

[02:05] Christopher David: As plugin, LLM influencer.

[02:12] Christopher David: Okay.

[02:17] Christopher David: Go into there.

[02:20] Christopher David: Oops.

[02:30] Christopher David: So what we got.

[02:37] Christopher David: So one thing is let's change this to LLM influencer.

[02:51] Christopher David: Let's do it with a relevant thing that we'll need.

[03:01] Christopher David: Let's go back to the, what was it?

[03:04] Christopher David: The X-Sysm rust quick start.

[03:09] Christopher David: Let's keep this in one place.

[03:13] Christopher David: This is to build.

[03:25] Christopher David: And this is to call it.

[03:29] Christopher David: Although I don't think we're going to be able to call it from the command line.

[03:32] Christopher David: Maybe because we're doing host functions.

[03:35] Christopher David: But let's do.

[03:39] Christopher David: Oops.

[03:41] Christopher David: Cargo build for.

[03:46] Christopher David: Release.

[03:50] Christopher David: Okay, let's see.

[03:52] Christopher David: Let's just make sure that this works.

[03:55] Christopher David: I forget what the command is.

[03:57] Christopher David: Let's do.

[04:10] Christopher David: Okay, let's just take a look at that.

[04:23] Christopher David: Let's go to.

[04:27] Christopher David: Throw that into the.

[04:31] Christopher David: This thing here.

[04:37] Christopher David: Target release.

[04:45] Christopher David: Oops target was 32 release.

[04:55] Christopher David: Okay.

[04:59] Christopher David: Okay, extract URLs.

[05:01] Christopher David: That's there.

[05:03] Christopher David: And it was it called the right thing.

[05:13] Christopher David: Plug in LLM influencer.

[05:18] Christopher David: Great.

[05:19] Christopher David: Oops.

[05:24] Christopher David: Okay, so we have a function called extract URLs.

[05:27] Christopher David: Now we're going to change this.

[05:32] Christopher David: First, let's commit this.

[05:42] Christopher David: Okay.

[05:44] Christopher David: And we need to create a repo for this.

[05:54] Christopher David: Okay.

[06:04] Christopher David: Get remote.

[06:05] Christopher David: Remove origin.

[06:06] Christopher David: Paste the origin.

[06:08] Christopher David: Push to origin name.

[06:14] Christopher David: Okay.

[06:17] Christopher David: Okay.

[06:27] Christopher David: Now, one thing I'm not clear on is the idea of a.

[06:40] Christopher David: Like host function.

[06:42] Christopher David: We're using the like default Rust version.

[06:46] Christopher David: So.

[06:52] Christopher David: It's going to the Rust PDK.

[06:56] Christopher David: GitHub.

[06:58] Christopher David: Let's see what their example is for host functions here.

[07:04] Christopher David: Sometimes it is useful to let a plugin make HTTP calls.

[07:12] Christopher David: So maybe let's just try getting the HTTP call working first.

[07:30] Christopher David: Keep it simple.

[07:34] Christopher David: Out of curiosity.

[07:40] Christopher David: What if I just do that?

[07:42] Christopher David: Is there anything else that I need?

[07:44] Christopher David: Can I build it like that?

[07:50] Christopher David: The trade to bytes is not implemented for Extras and PDK HTTP.

[07:56] Christopher David: We should.

[08:05] Christopher David: I'm going to update the PDK.

[08:15] Christopher David: I'm going to update the PDK.

[08:25] Christopher David: I'm going to update the PDK.

[08:32] Christopher David: I'm on 101.

[08:41] Christopher David: All right.

[08:59] Christopher David: Let's start getting.

[09:04] Christopher David: Let's start with the HTTP.

[09:14] Christopher David: We are writing a Rust.

[09:18] Christopher David: Extrism plugin to be consumed by our layer of our application.

[09:22] Christopher David: We want to use a.

[09:27] Christopher David: Calls.

[09:28] Christopher David: Here are the docs.

[09:30] Christopher David: I don't say anything on to I ask you something.

[09:46] Christopher David: My code here.

[09:52] Christopher David: Generates this error.

[09:58] Christopher David: I'm going to update the code.

[10:08] Christopher David: I'm going to update the code.

[10:18] Christopher David: I'm going to update the code.

[10:32] Christopher David: It seems like there might be a mismatch or an oversight in the XSNPDK's handling of HTTP response objects as return types for plugin functions.

[10:44] Christopher David: I'm going to update the code.

[10:54] Christopher David: I'm going to update the code.

[11:04] Christopher David: I'm going to update the code.

[11:28] Christopher David: How do curiosity?

[11:38] Christopher David: Does this build to Wazi?

[11:42] Christopher David: No.

[12:12] Christopher David: I don't want to update the code.

[12:34] Christopher David: I don't want to update the code.

[12:44] Christopher David: I'm going to update the code.

[12:54] Christopher David: I'm going to update the code.

[12:58] Christopher David: I'm going to update the code.

[13:06] Christopher David: I'll string of the response code.

[13:22] Christopher David: I'll try that.

[13:52] Christopher David: I'll try that.

[14:06] Christopher David: I'll try that.

[14:34] Christopher David: There we go.

[14:52] Christopher David: What?

[15:06] Christopher David: I'll try that.

[15:28] Christopher David: I'll try to try.

[15:38] Christopher David: I'll try that.

[16:08] Christopher David: I'll try that.

[16:22] Christopher David: I'll try that.

[16:32] Christopher David: I'll try that.

[17:02] Christopher David: I'll try that.

[17:32] Christopher David: I'll try that.

[17:54] Christopher David: I'll try that.

[18:18] Christopher David: I'll try that.

[18:42] Christopher David: I'll try that.

[19:06] Christopher David: I'll try that.

[19:36] Christopher David: I'll try that.

[19:59] Christopher David: the

[20:20] Christopher David: day, dark-faced viewer quest you change it.

[28:45] Christopher David: Rustnube

[30:07] Christopher David: you don't have to tell me all this, you just need to do it and then give me code that passes.

[30:24] Christopher David: Assuming you have some mechanism, no, you do the magnific looks.

[30:36] Christopher David: You don't have to tell me all this, you just need to do it and then give me code.

[30:46] Christopher David: Thank you.

[31:08] Christopher David: Thank you.

[31:38] Christopher David: Thank you.

[32:08] Christopher David: Thank you.

[32:38] Christopher David: Thank you.

[33:08] Christopher David: Thank you.

[33:23] Christopher David: Thank you.

[33:50] Christopher David: Now, I have curiosity, can I, well, I have curiosity first, let's go to ModSurf and dump this

[33:58] Christopher David: and make sure it looks like we'd expected to look.

[34:08] Christopher David: We have a function called inference.

[34:15] Christopher David: Now, yeah, that compiled, thank you.

[34:32] Christopher David: Now give me an example, see I come in with the right formatting for request JSON to test this out.

[34:44] Christopher David: Let's see.

[35:13] Christopher David: Accept that build it for release.

[35:35] Christopher David: Let's see if it works.

[35:59] Christopher David: Let's call release and it's called plugin LM infernser maybe.

[36:17] Christopher David: HTTP request to chat completions is not allowed.

[36:25] Christopher David: Is that because I'm on a local environment didn't we have this before?

[36:35] Christopher David: Let's see if it's allowed.

[36:50] Christopher David: Let's see if it's allowed.

[37:20] Christopher David: I know where to find the answers.

[37:50] Christopher David: Ever accept two arms received six.

[38:03] Christopher David: Thank you.

[38:24] Christopher David: Boo-yah, Boo-yah, allow host, fuck yes.

[38:45] Christopher David: That works.

[38:55] Christopher David: Let's do it again.

[39:01] Christopher David: GPT4.

[39:16] Christopher David: What do you know about ex-dism?

[39:24] Christopher David: I'm sorry, but as an AI, I cannot find an incredible information on ex-dism.

[39:59] Christopher David: the

[40:01] Christopher David: web assembly is a binary. Okay. So that's fine. That works. Let's go to our LLM

[40:06] Christopher David: inferencer thingy. Let's make a release.

[40:13] Christopher David: It's also paste on the

[40:29] Christopher David: you may not use my api key.

[40:43] Christopher David: Okay.

[40:55] Christopher David: Okay. So we now have a plugin. Let's try to

[40:58] Christopher David: zoom it from page p. First let me

[41:02] Christopher David: make a release for this.

[41:28] Christopher David: Okay.

[41:58] Christopher David: Okay. Okay. So what? Maybe.

[42:06] Christopher David: So give me the persistent URL.

[42:12] Christopher David: And then let's pop over to our code base.

[42:16] Christopher David: Earlier I was planning out the JavaScript.

[42:18] Christopher David: I'm kind of having a pan of the ass. So let's try.

[42:22] Christopher David: I don't know that we'll need a close function.

[42:26] Christopher David: Let's try.

[42:32] Christopher David: Can execute LM inferencer plugin.

[42:42] Christopher David: I was a plugin. He equals plugin.

[42:44] Christopher David: Factory.

[42:46] Christopher David: Create.

[42:48] Christopher David: Wasm URL.

[42:50] Christopher David: Okay.

[42:54] Christopher David: Okay.

[42:58] Christopher David: Okay.

[43:00] Christopher David: Okay.

[43:02] Christopher David: Okay.

[43:04] Christopher David: Okay.

[43:06] Christopher David: Okay.

[43:08] Christopher David: Okay.

[43:10] Christopher David: Okay.

[43:12] Christopher David: Okay.

[43:14] Christopher David: Okay.

[43:16] Christopher David: Okay.

[43:18] Christopher David: Okay.

[43:22] Christopher David: So data equals.

[43:24] Christopher David: Model name is a GPT2.

[43:28] Christopher David: No.

[43:30] Christopher David: I believe we have it open AI.

[43:32] Christopher David: API key here. Let's try this.

[43:34] Christopher David: Whoa.

[43:36] Christopher David: Whoa.

[43:38] Christopher David: Oh.

[43:40] Christopher David: Oh.

[43:42] Christopher David: Oh.

[43:46] Christopher David: Yeah.

[43:48] Christopher David: Now.

[43:50] Christopher David: Plug in.

[43:52] Christopher David: Call.

[43:54] Christopher David: I want it.

[43:56] Christopher David: Inference.

[43:58] Christopher David: Ditty.

[44:00] Christopher David: My output.

[44:02] Christopher David: Be really cool.

[44:04] Christopher David: This works.

[44:06] Christopher David: Be really cool if this works.

[44:08] Christopher David: Inferware rinse.

[44:10] Christopher David: I'm sorry.

[44:12] Christopher David: I did not find the function called Inferware rinse.

[44:14] Christopher David: You're supposed to inferer when I'm asking for.

[44:18] Christopher David: Okay.

[44:20] Christopher David: Infer.

[44:22] Christopher David: Inferencer.

[44:23] Christopher David: Inferencer.

[44:24] Christopher David: All right.

[44:26] Christopher David: Err.

[44:28] Christopher David: I love errors.

[44:30] Christopher David: Oh.

[44:32] Christopher David: You know what? We are not doing.

[44:34] Christopher David: I don't think.

[44:36] Christopher David: We're not doing our equivalent of allow hosts.

[44:40] Christopher David: Which we need to do in the

[44:44] Christopher David: Man Fest.

[44:46] Christopher David: Yeah.

[44:48] Christopher David: C.C. I think this like wasm time stuff.

[44:50] Christopher David: We're getting the same like wasm.

[44:52] Christopher David: Wasm time complaining probably because of that same thing.

[44:54] Christopher David: Now let's remember where did we.

[45:04] Christopher David: Did that hear?

[45:06] Christopher David: None parse.

[45:08] Christopher David: Err.

[45:10] Christopher David: Wasm.

[45:12] Christopher David: How do we get some allowed hosts?

[45:16] Christopher David: A wasm is in array of blah blah blah blah.

[45:22] Christopher David: Okay. The manifest.

[45:24] Christopher David: A loud host equals array of star maybe.

[45:36] Christopher David: And down here we say call.

[45:44] Christopher David: If there's no plug in the search.

[45:46] Christopher David: See if it will call in the search.

[45:48] Christopher David: So this is going to be called in a matter of what.

[45:50] Christopher David: So let's do.

[45:52] Christopher David: Man Fest.

[45:54] Christopher David: Allowed hosts.

[45:56] Christopher David: No.

[45:58] Christopher David: Allowed underscore.

[46:04] Christopher David: Allowed underscore hosts.

[46:20] Christopher David: Who I'll be able to set that it to.

[46:24] Christopher David: Open agents.com or something.

[46:26] Christopher David: What you see if that does.

[46:28] Christopher David: What I only wanted to do.

[46:30] Christopher David: Okay.

[46:40] Christopher David: Incorrect API key provided.

[46:42] Christopher David: Okay.

[46:44] Christopher David: Okay.

[46:46] Christopher David: Okay.

[46:48] Christopher David: Made of use a deprecated key.

[46:50] Christopher David: Okay.

[47:00] Christopher David: Okay.

[47:10] Christopher David: Okay.

[47:16] Christopher David: Okay.

[47:18] Christopher David: Okay.

[47:20] Christopher David: Okay.

[47:22] Christopher David: Okay.

[47:24] Christopher David: Okay.

[47:26] Christopher David: Okay.

[47:28] Christopher David: Okay.

[47:30] Christopher David: Okay.

[47:32] Christopher David: Okay.

[47:34] Christopher David: Okay.

[47:36] Christopher David: Okay.

[47:38] Christopher David: Okay.

[47:40] Christopher David: Okay.

[47:42] Christopher David: Okay.

[47:44] Christopher David: Okay.

[47:46] Christopher David: Okay.

[47:48] Christopher David: Okay.

[47:50] Christopher David: Okay.

[47:52] Christopher David: Okay.

[47:54] Christopher David: Okay.

[47:56] Christopher David: Okay.

[47:58] Christopher David: Okay.

[48:00] Christopher David: Okay.

[48:02] Christopher David: Okay.

[48:04] Christopher David: Okay.

[48:06] Christopher David: Okay.

[48:08] Christopher David: Okay.

[48:10] Christopher David: Okay.

[48:12] Christopher David: Okay.

[48:14] Christopher David: Okay.

[48:16] Christopher David: Okay.

[48:18] Christopher David: Okay.

[48:20] Christopher David: Okay.

[48:22] Christopher David: Okay.

[48:24] Christopher David: Okay.

[48:26] Christopher David: Okay.

[48:28] Christopher David: Okay.

[48:30] Christopher David: Okay.

[48:32] Christopher David: Okay.

[48:34] Christopher David: Okay.

[48:36] Christopher David: Okay.

[48:38] Christopher David: Okay.

[48:40] Christopher David: Okay.

[48:42] Christopher David: Okay.

[48:44] Christopher David: Okay.

[48:46] Christopher David: Okay.

[48:48] Christopher David: Okay.

[48:50] Christopher David: Okay.