Transcription: OpenAgents Episode 059 - Agent Node Graphs, Litegraph Edition

Source: https://twitter.com/OpenAgentsInc/status/1749850948296380668 Wiki source: https://raw.githubusercontent.com/wiki/OpenAgentsInc/openagents/Video-Series.md Media title: OpenAgents - Episode 059: Agent Node Graphs, Litegraph Edition We build a UI to sh... Upload date: 20240123 Transcription model: mlx-whisper/mlx-community/whisper-tiny Generated at: 2026-06-01T20:06:16Z

Machine-generated transcript. Review speaker labels and wording before using this as quote-grade source material.

[00:00] Christopher David: Alright, we are building a UI for the back end functionality we finished in the last video,

[00:08] Christopher David: giving our agency ability to use Wazom plugins in their task and step workflows.

[00:14] Christopher David: We had previously done some work on this in episode 38.

[00:18] Christopher David: We built a little initial, like start of a library, we were going to call agent graphs,

[00:26] Christopher David: something that might be separate.

[00:29] Christopher David: And that was kind of inspired by Leva, this gorgeous, like GUI panel library from Pymondras,

[00:38] Christopher David: great React developer, collective thing, and then Light Graph, JS.

[00:43] Christopher David: And I found Light Graph originally seeing who is doing what for node graphs and comfy UI, which is this,

[00:53] Christopher David: stable diffusion GUIs, like what are they used?

[00:57] Christopher David: They use all of their, this stuff is powered by this library called Light Graph.js.

[01:05] Christopher David: And my previous thinking was, oh, this is just standalone, you know, vanilla JavaScript.

[01:11] Christopher David: I want to make it reactified, and so I was like, I'm actually getting together with the, like,

[01:16] Christopher David: React to GUI component from level, but now I don't want it reactified.

[01:21] Christopher David: And a standalone vanilla JS library may actually just be great.

[01:26] Christopher David: So in this video, I want to take a spin through the agent, or so through the light graph code base, you know,

[01:34] Christopher David: pull it in, see if this is something that will work for our purposes.

[01:41] Christopher David: So the like demo page linked from the repo there.

[01:47] Christopher David: The got some demos, different node types.

[01:53] Christopher David: This is all great. I wouldn't want to have to like rebuild this kind of stuff.

[01:58] Christopher David: And even though it feels like a little bit janky sometimes.

[02:04] Christopher David: We like a little bit janky.

[02:07] Christopher David: So yeah, we have some big ass agent workflows in here.

[02:13] Christopher David: And then you can like double click and add a node just searching.

[02:17] Christopher David: So imagine I want to like search for embedding or inference or I know I want to use mixed role and such.

[02:26] Christopher David: So I like that that people who know enough to know that they want to use mixed role here or embedding here,

[02:33] Christopher David: different APIs here, I want them to have that kind of out there fingertips and be able to kind of compose things.

[02:40] Christopher David: And it'll be cool that like the discoverability of other people's workflows will be awesome.

[02:45] Christopher David: So you don't have to like build your own everything from scratch.

[02:49] Christopher David: You can pull in someone else's study and like kind of reason about where you want to extend it.

[02:54] Christopher David: And then to be able to have these little like you tells up here for playing, you know, like running the workflow,

[03:00] Christopher David: maybe being able to, I'm guessing you can like inspect what the results are of these things.

[03:07] Christopher David: And step through live blah blah blah.

[03:12] Christopher David: So I think this it looks fine.

[03:16] Christopher David: It looks fine.

[03:19] Christopher David: So let's see if we can get.

[03:23] Christopher David: I guess maybe an objective for this video would be to see if we can get kind of like we had about into these.

[03:28] Christopher David: So this is an example.

[03:32] Christopher David: We kind of stopped on the previous one kind of showing we were passing the given like the tasks and steps that we did.

[03:39] Christopher David: In the last video for the concierge like here are those task and steps represented.

[03:44] Christopher David: As.

[03:48] Christopher David: Steps in these little widgets so about it input embedded put similar research call.

[03:53] Christopher David: If we could get a light graph version of that.

[03:58] Christopher David: Uh, working.

[04:00] Christopher David: That would be a good stopping point for this video.

[04:03] Christopher David: See if we can do that easily.

[04:05] Christopher David: Okay, so like graph.

[04:08] Christopher David: A library in JavaScript to create graphs in the browser similar to Unreal Blueprints.

[04:12] Christopher David: Those can be programmed easily and it includes an editor to construct and test the graphs.

[04:17] Christopher David: It can be integrated easily in existing web applications and graphs can be run without the need of the editor.

[04:22] Christopher David: Try it in the demo site.

[04:26] Christopher David: Windows on canvas 2D editor.

[04:33] Christopher David: Under the notes per graph customizable theme callbacks to personalize every option drawing event of nodes sub graphs nodes that contain graphs themselves.

[04:42] Christopher David: That's cool.

[04:44] Christopher David: Live mode system highs the graph but calls nodes surrender whatever they want useful to create you eyes.

[04:49] Christopher David: Graphs can be executed in the node.

[04:52] Christopher David: We'll do our own execution.

[04:54] Christopher David: Highly customizable nodes color shape slots vertical or horizontal widgets custom hiring.

[04:59] Christopher David: Easy to integrate in a JS applications one single found out dependency test.

[05:03] Christopher David: Okay.

[05:11] Christopher David: You do NPM or download the like graph.js and CSS like graph.js.

[05:18] Christopher David: So what we can do in our app is just throw that into only the page where we're going to be actually rendering it.

[05:30] Christopher David: First project.

[05:31] Christopher David: Yeah, let's try some copypasta here.

[05:35] Christopher David: So let's set up a.

[05:40] Christopher David: Page to test this on.

[05:47] Christopher David: See.

[05:51] Christopher David: Make any branch code like graph.

[05:56] Christopher David: And maybe for the test.

[06:05] Christopher David: Let's just make a view called like graph.

[06:34] Christopher David: And let's just copypaste this all in there and see what happens.

[06:38] Christopher David: Okay, I need to get the JavaScript.

[06:50] Christopher David: I'll get the main.js so we can inspect it.

[07:00] Christopher David: And I can mess bundling for now.

[07:03] Christopher David: We'll just put this in vendor like graph.

[07:10] Christopher David: Like graph.js.

[07:16] Christopher David: And we'll grab the CSS.

[07:30] Christopher David: And we'll just add the code.

[08:11] Christopher David: And I'll make this vendor.

[08:17] Christopher David: That might be right.

[08:41] Christopher David: Are we loading the JavaScript now?

[08:50] Christopher David: Oops vendor like graph.

[08:57] Christopher David: Okay.

[09:02] Christopher David: So unmanify that's 200 kb.

[09:05] Christopher David: Yeah.

[09:36] Christopher David: All right, we'll call this like graph demo.

[09:41] Christopher David: And now let's see if we can connect this to our agents.

[09:48] Christopher David: And no, it's like our agent of the step in tasks.

[09:56] Christopher David: And for our previous version of this, we had data being passed to what we called the agent view

[10:23] Christopher David: with all the data that we would need.

[10:28] Christopher David: I think.

[10:31] Christopher David: So remember where we put that agent controller.

[10:36] Christopher David: Yeah.

[10:40] Christopher David: So on the show agent page, we'll do that same thing.

[10:51] Christopher David: We have agent controller.

[10:56] Christopher David: Oh, we have us already.

[10:58] Christopher David: So now just instead of rendering the agent view.

[11:06] Christopher David: Let's render a new components.

[11:16] Christopher David: All right, let's first get the route set up for that.

[11:43] Christopher David: All right, let's go into our routes web and just see how we did it there.

[11:58] Christopher David: There we go.

[12:06] Christopher David: So we have the agent controller.

[12:13] Christopher David: And let's make sure what they're seeder doing these days.

[12:19] Christopher David: It's seeding the plug and seeder.

[12:29] Christopher David: Let's actually change that to see the, what do we call it, the concierge with plug-in seeder.

[12:37] Christopher David: So we're seeding a plug-in and the agent, the task and all the steps for an actual workflow.

[12:45] Christopher David: That's what we did in the last video.

[12:49] Christopher David: Concierge with plug-in seeder.

[12:57] Christopher David: Okay, let's reset our database and seed.

[13:07] Christopher David: Concierge with spellets.

[13:18] Christopher David: And then motions see.

[13:30] Christopher David: Okay.

[13:34] Christopher David: So what happens if I go to agent slash one?

[13:37] Christopher David: Okay, that comes up.

[13:41] Christopher David: And I think I should hear have the agent name properly.

[13:46] Christopher David: The concierge.

[13:47] Christopher David: Okay.

[13:48] Christopher David: So let's give our friend chat, jippity something to do here.

[14:11] Christopher David: We want to show a show agent page in our larval blade hgmxdill end using this controller.

[14:26] Christopher David: And we want it to render the relevant step data connected to each other, sorted by the order of the steps.

[14:40] Christopher David: Using like graph, adapting this demo code.

[15:01] Christopher David: You can infer the shape of the data from these tests.

[15:18] Christopher David: Agent step.

[15:23] Christopher David: Task.

[15:33] Christopher David: Please write the agent view blade up.

[15:44] Christopher David: Certainly.

[16:10] Christopher David: You're taking so long.

[16:20] Christopher David: This is pretty cool.

[16:40] Christopher David: That for each work in a job script like that.

[16:59] Christopher David: Oh, yeah.

[17:16] Christopher David: Agent details.

[17:21] Christopher David: A chatbot that response user message after consulting on knowledge base.

[17:29] Christopher David: Agent balance zero task and steps.

[17:37] Christopher David: That's not bad for 10 minutes of work.

[17:42] Christopher David: Let's save this as initial agent view nodes.

[17:52] Christopher David: Yeah, so yeah, just write on the code.

[17:56] Christopher David: So thanks that is not bad except you drew a line from what I presume is the task node.

[18:05] Christopher David: What's that initial node?

[18:13] Christopher David: And I don't see.

[18:21] Christopher David: Why not do we not have task name?

[18:32] Christopher David: Oh, first of all, that's not an agent task is task.

[18:47] Christopher David: Oh, task factory.

[18:57] Christopher David: So we added a name to our task.

[19:00] Christopher David: We do it's nullable.

[19:02] Christopher David: So task.

[19:06] Christopher David: Hang on, hang on.

[19:07] Christopher David: We're using a factory.

[19:08] Christopher David: We're using the concierge with plug and cedar.

[19:14] Christopher David: Oh.

[19:26] Christopher David: Respond with vowels.

[19:31] Christopher David: All right, I want to see that on the screen.

[19:33] Christopher David: Why this?

[19:37] Christopher David: Let's see that refresh.

[19:50] Christopher David: Okay.

[19:51] Christopher David: Respond with vowels.

[19:59] Christopher David: Fucking cool.

[20:08] Christopher David: That does not, that does not matter except you drew a connector line from each step to the task.

[20:19] Christopher David: And I want each step connected to each other.

[20:27] Christopher David: One after the other.

[20:29] Christopher David: And we can maybe do notes for like input output.

[20:35] Christopher David: And then we can see the difference between the two.

[20:42] Christopher David: And then we can see the difference between the two.

[20:55] Christopher David: All the time I spend sitting here.

[20:57] Christopher David: I mean, it's, I'm of course spoiled because a year ago this didn't really exist.

[21:01] Christopher David: But all the time that I'm sitting here like I want agents doing this stuff for me.

[21:06] Christopher David: I want agents building out this code base.

[21:13] Christopher David: And then we can see the difference between the two.

[21:21] Christopher David: Come on.

[21:22] Christopher David: Copy paste.

[21:43] Christopher David: I can't really believe it ever.

[21:52] Christopher David: Okay, so this is, this is not working, this is not getting me any.

[22:14] Christopher David: First was null and then it seems to think that it's connected although it's not.

[22:19] Christopher David: Why would it work there and not there.

[22:49] Christopher David: Oh, make sure they have compatible output and input ports for connections.

[23:04] Christopher David: Let's see if we can get it on trying to.

[23:07] Christopher David: And then we can try feeding it to like graph docs.

[23:19] Christopher David: And then we can see the difference between the two.

[23:26] Christopher David: And then we can see the difference between the two.

[23:33] Christopher David: And then we can see the difference between the two.

[23:39] Christopher David: And then we can see the difference between the two.

[23:47] Christopher David: Task index for each.

[23:56] Christopher David: I don't know that you can do that.

[24:06] Christopher David: Oops.

[24:12] Christopher David: That gives this.

[24:16] Christopher David: I don't think that syntax is supported.

[24:20] Christopher David: Rewrite it.

[24:35] Christopher David: That looks better.

[24:41] Christopher David: I don't think that's going to work.

[24:51] Christopher David: I don't think that's going to work.

[24:56] Christopher David: I don't think that's going to work.

[25:01] Christopher David: I don't think that's going to work.

[25:04] Christopher David: I don't think that's going to work.

[25:08] Christopher David: I'm missing the main task.

[25:14] Christopher David: Okay, so here's what we're going to do.

[25:15] Christopher David: We're going to go to the read me.

[25:37] Christopher David: I like the more logical positions of the nodes,

[25:45] Christopher David: but there are still no connections between them.

[25:49] Christopher David: We may need to use a different base node type.

[25:57] Christopher David: Let's define our own node type.

[26:07] Christopher David: That's exactly right.

[26:12] Christopher David: Thank you.

[26:22] Christopher David: Oh, yeah.

[26:27] Christopher David: I like it.

[26:36] Christopher David: I like it.

[26:58] Christopher David: There we go.

[27:07] Christopher David: What a text.

[27:10] Christopher David: Oh, my goodness.

[27:15] Christopher David: Oh, my goodness.

[27:27] Christopher David: Wow.

[27:37] Christopher David: Okay.

[27:39] Christopher David: Connect nodes together.

[27:45] Christopher David: And that comes here.

[27:48] Christopher David: That works great.

[28:06] Christopher David: Now let's please add relevant fields to that step node.

[28:17] Christopher David: Editable where you think it makes sense for the other step.

[28:26] Christopher David: Other fields on our step model.

[28:32] Christopher David: And let's give it the entire step model.

[28:35] Christopher David: What are we doing?

[28:45] Christopher David: Thank you.

[29:15] Christopher David: This is awesome.

[29:25] Christopher David: Let's go back here.

[29:36] Christopher David: Okay.

[29:58] Christopher David: This needs to be tweaked, but like,

[30:01] Christopher David: wow.

[30:15] Christopher David: That looks great, but we need to populate the node with the node data from the database.

[30:29] Christopher David: And we need to create a node.

[30:39] Christopher David: Thank you.

[31:09] Christopher David: Thank you.

[31:34] Christopher David: Thank you.

[31:49] Christopher David: Did not work.

[31:50] Christopher David: Let's see what's going on here.

[31:55] Christopher David: Do I even have this?

[31:56] Christopher David: Yeah, I've got the information.

[32:01] Christopher David: I can see the name behind there.

[32:21] Christopher David: Okay, can I change the node title?

[32:31] Christopher David: I have curiosity.

[32:36] Christopher David: Does that do something?

[32:42] Christopher David: Oops.

[32:46] Christopher David: There we go.

[32:57] Christopher David: First the order.

[33:06] Christopher David: Yeah.

[33:11] Christopher David: I don't need it to do that.

[33:33] Christopher David: I want to offset this way instead.

[33:48] Christopher David: I don't need it to work.

[34:10] Christopher David: I don't need it to work.

[34:22] Christopher David: I see.

[34:31] Christopher David: How do I do full screen?

[34:44] Christopher David: Like graph.

[34:50] Christopher David: I don't need that.

[35:02] Christopher David: I've seen it to beat bigger.

[35:11] Christopher David: Oops.

[35:25] Christopher David: All right.

[35:26] Christopher David: I'll just hard get that for now.

[35:27] Christopher David: See that out later.

[35:41] Christopher David: Okay.

[36:06] Christopher David: I'm positioning looks better.

[36:16] Christopher David: Now why is it not filling in the properties?

[36:26] Christopher David: I don't need it to fill in the properties.

[36:36] Christopher David: I don't need it to fill in the properties.

[36:41] Christopher David: Okay.

[36:51] Christopher David: Okay.

[37:01] Christopher David: Thank you.

[37:31] Christopher David: I'm going to guess if this might not be the best way to do it.

[38:23] Christopher David: Thank you.

[38:53] Christopher David: There we go.

[39:08] Christopher David: Okay.

[39:18] Christopher David: Let me make that look nicer.

[39:48] Christopher David: Okay.

[39:59] Christopher David: and

[40:14] Christopher David: I don't know, like, care. Cool.

[40:18] Christopher David: Some of these people might not want to be editable, like maybe order is not something you type in, maybe it's just something you drag around.

[40:26] Christopher David: Yeah, add a toggle.

[40:30] Christopher David: Yeah, toggle the, uh, the thing for the things.

[40:35] Christopher David: Well, as you can see, we can basically specify the behavior that we want feed our docs and code into

[40:43] Christopher David: the chatty for tea and it will do it for us.

[40:47] Christopher David: I think that is a good stopping point for this video. Clearly, this is, uh, seems pretty perfect for what we want.

[40:55] Christopher David: And next, I guess we'll, uh, maybe I'll do some code clean up offline and then let's get it so that we can run a flow in this editor and see the output.

[41:13] Christopher David: Cool.

[41:14] Christopher David: Thanks for watching.