Transcription: OpenAgents Episode 018 - Connecting the UI, Part 2

Source: https://twitter.com/OpenAgentsInc/status/1725246583623590158 Wiki source: https://raw.githubusercontent.com/wiki/OpenAgentsInc/openagents/Video-Series.md Media title: OpenAgents - Episode 018: Connecting the UI, Part 2 We use our new embeddings to s... Upload date: 20231116 Transcription model: mlx-whisper/mlx-community/whisper-tiny Generated at: 2026-06-01T20:42:42Z

Machine-generated transcript. Review speaker labels and wording before using this as quote-grade source material.

[00:00] Christopher David: Okay, that is now working on the live production site, openagents.app.

[00:07] Christopher David: I'm going to throw in a PDF processing and you can see from previous.

[00:18] Christopher David: I've uploaded this document twice before it six pages each.

[00:25] Christopher David: If it finishes processing after five to ten seconds, go to the chat page.

[00:29] Christopher David: If I refresh, there is the new six pages with the embedding and the page text.

[00:36] Christopher David: We're going to obviously chunk that up more granularly than page, but that's a great start.

[00:41] Christopher David: It has not worked with longer PDFs.

[00:44] Christopher David: I don't know if there's like a file size limit either with Lairofl or with the Queen B handling,

[00:50] Christopher David: not as many embeddings.

[00:52] Christopher David: We've got some more stuff to do with the debug for larger documents, but that's great.

[00:56] Christopher David: Let's connect the chat to this to be able to ask questions about it.

[01:04] Christopher David: So let's do the same thing that we did in the last video.

[01:09] Christopher David: Kind of swapping out our VectorA integration for.

[01:17] Christopher David: What would you do?

[01:20] Christopher David: Let's trace the lifecycle of sending a message.

[01:28] Christopher David: So right now it's post and the chat component.

[01:36] Christopher David: It's post API query with query being what the user says.

[01:48] Christopher David: And corpus ID, which we are going to replace with file ID.

[02:00] Christopher David: I think that's one thing that we're not right now.

[02:02] Christopher David: We're just saving direct embeddings, but we're not associating with these with a file.

[02:07] Christopher David: So that's one area now that we recognize this is where our file model comes in.

[02:14] Christopher David: So we'll do unit tests for that once we get to it.

[02:22] Christopher David: So what our query controller does, and are we testing this anywhere?

[02:30] Christopher David: Yes, we have this old test from VectorA.

[02:34] Christopher David: Anyone can query corpus via API.

[02:38] Christopher David: I'm going to change this test we're skipping over.

[02:43] Christopher David: I'm going to call this query doc test.

[02:47] Christopher David: This should be pretty straightforward to test.

[02:51] Christopher David: Anyone can query their uploaded file via API.

[03:01] Christopher David: So given there is an uploaded file.

[03:13] Christopher David: No, I just want to mark the file in the embeddings.

[03:23] Christopher David: So let's do this.

[03:25] Christopher David: File equals file.

[03:29] Christopher David: Factory create with embeddings.

[03:33] Christopher David: I don't know if that's the right syntax.

[03:41] Christopher David: We've got some L.A.

[03:45] Christopher David: Model like that.

[03:49] Christopher David: So we're going to post to query doc store with file.

[03:53] Christopher David: We have a query and the file ID.

[04:09] Christopher David: All right.

[04:11] Christopher David: And then it's starting that we've got some kind of response.

[04:17] Christopher David: That's the response we got back from VectorA.

[04:19] Christopher David: That's what I was looking for.

[04:21] Christopher David: Let's do this.

[04:23] Christopher David: Pf query.

[04:25] Christopher David: File.

[04:27] Christopher David: Factory's exist.

[04:29] Christopher David: Does it know what file we mean?

[04:31] Christopher David: We mean app.

[04:33] Christopher David: Models.

[04:35] Christopher David: File.

[04:37] Christopher David: Does not know what with embeddings means.

[04:43] Christopher David: Here's where we want to do a little.

[04:49] Christopher David: Let's go ask chat to be to the proper syntax for this.

[04:53] Christopher David: In L.A. Val, how do we make this with embeddings thing work on the factory?

[05:07] Christopher David: Well, has many.

[05:09] Christopher David: A file has many embeddings.

[05:13] Christopher David: Factory so far.

[05:29] Christopher David: Define a custom state function.

[05:31] Christopher David: Let's ensure you have a factory for your embeddings.

[05:37] Christopher David: We do.

[05:39] Christopher David: Although we haven't defined anything there.

[05:41] Christopher David: To establish the relationship, we have not done yet.

[05:57] Christopher David: That's a good time to do it.

[05:59] Christopher David: Isn't that nice?

[06:13] Christopher David: So.

[06:15] Christopher David: In file factory, we'll drop that there.

[06:21] Christopher David: There.

[06:33] Christopher David: Column file ID of relationship embeddings is not exist.

[06:36] Christopher David: Let's drop to our unit test and set up that relationship.

[06:45] Christopher David: And what did we say in the first principles doc that?

[06:57] Christopher David: So we previously did it as file has many chunks chunk has one embedding.

[07:03] Christopher David: But we're actually storing the chunk data next to.

[07:08] Christopher David: Like the chunk is actually stored with the embedding.

[07:10] Christopher David: So we're just going to change this to say file has many embeddings.

[07:14] Christopher David: So we have no unit test for those.

[07:29] Christopher David: So it's due to file test.

[07:35] Christopher David: It has many embeddings.

[07:38] Christopher David: Okay.

[07:48] Christopher David: Okay.

[07:58] Christopher David: We're getting the wrong tests.

[08:04] Christopher David: So we're going to have to do that.

[08:14] Christopher David: So we're going to have to do that.

[08:20] Christopher David: So we're going to do that.

[08:29] Christopher David: All right.

[08:33] Christopher David: Call to undefined method file embeddings.

[08:57] Christopher David: That's a new one.

[09:27] Christopher David: Well, I know that the embedding stable doesn't have that.

[09:45] Christopher David: Or does it factor embedding?

[09:55] Christopher David: Okay.

[10:08] Christopher David: Now we've got undefined column.

[10:17] Christopher David: And then we've got embeddings.

[10:27] Christopher David: We're getting closer.

[10:37] Christopher David: That's interesting.

[11:07] Christopher David: I can't even really tell what's going on there.

[11:27] Christopher David: But.

[12:18] Christopher David: Okay.

[12:28] Christopher David: Okay.

[12:57] Christopher David: Okay.

[13:07] Christopher David: Okay.

[13:34] Christopher David: Okay.

[13:44] Christopher David: Okay.

[14:04] Christopher David: Okay.

[14:21] Christopher David: Okay.

[14:31] Christopher David: Okay.

[14:49] Christopher David: Okay.

[15:09] Christopher David: Okay.

[15:39] Christopher David: Okay.

[15:52] Christopher David: It's failing on this line embedding save.

[16:08] Christopher David: This is failing on the save line with.

[16:29] Christopher David: Embedding factory is.

[16:39] Christopher David: And embedding embeddings is not nullable.

[17:04] Christopher David: Does that work?

[17:12] Christopher David: Yeah, uninformed vector.

[17:18] Christopher David: That's.

[17:34] Christopher David: Does that work?

[17:50] Christopher David: Okay.

[18:07] Christopher David: There we go.

[18:10] Christopher David: Okay.

[18:11] Christopher David: I did always missing the required stuff from the.

[18:15] Christopher David: Okay.

[18:23] Christopher David: Okay.

[18:31] Christopher David: We do not have as many embeddings.

[19:01] Christopher David: Where am I doing that.

[19:11] Christopher David: Oh.

[19:18] Christopher David: Okay.

[19:28] Christopher David: Okay.

[19:38] Christopher David: Come on.

[19:43] Christopher David: That's right.

[19:48] Christopher David: That's what I want.

[19:50] Christopher David: Okay.

[20:59] Christopher David: and

[21:14] Christopher David: there we go

[21:17] Christopher David: and

[22:02] Christopher David: we're going to

[22:09] Christopher David: test see what we're

[22:14] Christopher David: getting corpus ideas required

[22:17] Christopher David: and it's going to query controller

[22:20] Christopher David: and we want to say that

[22:22] Christopher David: corpus ideas not required file ideas required

[22:27] Christopher David: and

[22:42] Christopher David: and

[22:57] Christopher David: we're going to

[23:01] Christopher David: say

[23:12] Christopher David: to

[23:27] Christopher David: to

[23:42] Christopher David: to

[23:57] Christopher David: to

[24:12] Christopher David: to

[24:27] Christopher David: to

[25:12] Christopher David: to

[25:19] Christopher David: to

[25:34] Christopher David: to

[25:49] Christopher David: to

[26:04] Christopher David: to

[26:19] Christopher David: to

[26:34] Christopher David: to

[26:49] Christopher David: to

[27:04] Christopher David: to

[27:19] Christopher David: to

[27:34] Christopher David: to

[27:49] Christopher David: to

[27:56] Christopher David: to

[28:03] Christopher David: to

[28:06] Christopher David: to

[28:11] Christopher David: to

[28:14] Christopher David: to

[28:19] Christopher David: to

[28:26] Christopher David: to

[28:34] Christopher David: to

[28:44] Christopher David: to

[28:54] Christopher David: to

[28:59] Christopher David: to

[29:10] Christopher David: to

[29:16] Christopher David: to

[29:19] Christopher David: to

[29:24] Christopher David: to

[29:34] Christopher David: to

[29:38] Christopher David: to

[29:43] Christopher David: to

[29:44] Christopher David: to

[29:49] Christopher David: to

[29:55] Christopher David: to

[29:59] Christopher David: to

[30:06] Christopher David: to

[30:11] Christopher David: to

[30:14] Christopher David: to

[30:19] Christopher David: to

[30:24] Christopher David: to

[30:31] Christopher David: to

[30:34] Christopher David: to

[30:39] Christopher David: to

[30:44] Christopher David: to

[30:49] Christopher David: to

[30:56] Christopher David: to

[31:01] Christopher David: to

[31:09] Christopher David: to

[31:14] Christopher David: to

[31:19] Christopher David: to

[31:24] Christopher David: to

[31:29] Christopher David: to

[31:33] Christopher David: to

[31:37] Christopher David: to

[31:42] Christopher David: to

[31:47] Christopher David: to

[31:52] Christopher David: to

[31:56] Christopher David: to

[32:00] Christopher David: to

[32:05] Christopher David: to

[32:10] Christopher David: to

[32:14] Christopher David: to

[32:17] Christopher David: to

[32:20] Christopher David: to

[32:23] Christopher David: to

[32:28] Christopher David: to

[32:29] Christopher David: to

[32:30] Christopher David: to

[32:34] Christopher David: to

[32:37] Christopher David: to

[32:41] Christopher David: to

[32:42] Christopher David: to

[32:44] Christopher David: to

[32:46] Christopher David: to

[32:50] Christopher David: to

[32:54] Christopher David: to

[32:56] Christopher David: to

[32:59] Christopher David: to

[33:03] Christopher David: to

[33:06] Christopher David: to

[33:10] Christopher David: to

[33:12] Christopher David: to

[33:14] Christopher David: to

[33:17] Christopher David: to

[33:21] Christopher David: to

[33:26] Christopher David: to

[33:31] Christopher David: to

[33:36] Christopher David: to

[33:42] Christopher David: to

[33:46] Christopher David: to

[33:51] Christopher David: to

[33:55] Christopher David: to

[34:00] Christopher David: to

[34:05] Christopher David: to

[34:09] Christopher David: to

[34:15] Christopher David: to

[34:20] Christopher David: to

[34:24] Christopher David: to

[34:29] Christopher David: to

[34:34] Christopher David: to

[34:36] Christopher David: to

[34:39] Christopher David: to

[34:42] Christopher David: to

[34:44] Christopher David: to

[34:47] Christopher David: to

[34:51] Christopher David: to

[34:56] Christopher David: to

[35:00] Christopher David: to

[35:02] Christopher David: to

[35:04] Christopher David: to

[35:05] Christopher David: to

[35:07] Christopher David: to

[35:09] Christopher David: to

[35:11] Christopher David: to

[35:14] Christopher David: to

[35:18] Christopher David: to

[35:20] Christopher David: to

[35:23] Christopher David: to

[35:25] Christopher David: to

[35:27] Christopher David: to

[35:28] Christopher David: to

[35:30] Christopher David: to

[35:31] Christopher David: to

[35:32] Christopher David: to

[35:34] Christopher David: to

[35:36] Christopher David: to

[35:38] Christopher David: to

[35:40] Christopher David: to

[35:43] Christopher David: to

[35:47] Christopher David: to

[35:49] Christopher David: to

[35:52] Christopher David: to

[35:54] Christopher David: to

[35:57] Christopher David: to

[35:58] Christopher David: to

[35:59] Christopher David: to

[36:00] Christopher David: to

[36:02] Christopher David: to

[36:03] Christopher David: to

[36:05] Christopher David: to

[36:07] Christopher David: to

[36:09] Christopher David: to

[36:12] Christopher David: to

[36:13] Christopher David: to

[36:14] Christopher David: to

[36:15] Christopher David: to

[36:16] Christopher David: to

[36:17] Christopher David: to

[36:18] Christopher David: to

[36:19] Christopher David: to

[36:20] Christopher David: to

[36:21] Christopher David: to

[36:22] Christopher David: to

[36:23] Christopher David: to

[36:24] Christopher David: to

[36:25] Christopher David: to

[36:26] Christopher David: to

[36:27] Christopher David: to

[36:28] Christopher David: to

[36:29] Christopher David: to

[36:30] Christopher David: to

[36:31] Christopher David: to

[36:32] Christopher David: to

[36:33] Christopher David: to

[36:34] Christopher David: to

[36:35] Christopher David: to

[36:36] Christopher David: to

[36:37] Christopher David: to

[36:38] Christopher David: to

[36:39] Christopher David: to

[36:42] Christopher David: to

[36:44] Christopher David: to

[36:46] Christopher David: to

[36:48] Christopher David: to

[36:50] Christopher David: to

[36:51] Christopher David: to

[36:53] Christopher David: to

[36:55] Christopher David: to

[36:56] Christopher David: to

[36:57] Christopher David: to

[36:58] Christopher David: to

[36:59] Christopher David: to

[37:00] Christopher David: to

[37:01] Christopher David: to

[37:02] Christopher David: to

[37:03] Christopher David: to

[37:04] Christopher David: to

[37:06] Christopher David: to

[37:08] Christopher David: to

[37:11] Christopher David: to

[37:12] Christopher David: to

[37:13] Christopher David: to

[37:15] Christopher David: to

[37:17] Christopher David: to

[37:18] Christopher David: to

[37:19] Christopher David: to

[37:20] Christopher David: to

[37:22] Christopher David: to

[37:24] Christopher David: to

[37:25] Christopher David: to

[37:26] Christopher David: to

[37:27] Christopher David: to

[37:28] Christopher David: to

[37:29] Christopher David: to

[37:30] Christopher David: to

[37:31] Christopher David: to

[37:32] Christopher David: to

[37:33] Christopher David: to

[37:34] Christopher David: to

[37:35] Christopher David: to

[37:36] Christopher David: to

[37:37] Christopher David: to

[37:38] Christopher David: to

[37:40] Christopher David: to

[37:41] Christopher David: to

[37:42] Christopher David: to

[37:43] Christopher David: to

[37:44] Christopher David: to

[37:45] Christopher David: to

[37:46] Christopher David: to

[37:47] Christopher David: to

[37:48] Christopher David: to

[37:49] Christopher David: to

[37:50] Christopher David: to

[37:51] Christopher David: to

[37:52] Christopher David: to

[37:53] Christopher David: to

[37:54] Christopher David: to

[37:55] Christopher David: to

[37:56] Christopher David: to

[37:57] Christopher David: to

[37:58] Christopher David: to

[37:59] Christopher David: to

[38:00] Christopher David: to

[38:01] Christopher David: to

[38:02] Christopher David: to

[38:03] Christopher David: to

[38:04] Christopher David: to

[38:05] Christopher David: to

[38:06] Christopher David: to

[38:07] Christopher David: to

[38:09] Christopher David: to

[38:10] Christopher David: to

[38:11] Christopher David: to

[38:12] Christopher David: to

[38:39] Christopher David: to

[38:40] Christopher David: to

[38:41] Christopher David: to

[38:42] Christopher David: to

[38:43] Christopher David: to

[38:44] Christopher David: to

[38:45] Christopher David: to

[38:46] Christopher David: to

[38:47] Christopher David: to

[38:48] Christopher David: to

[38:49] Christopher David: to

[38:50] Christopher David: to

[38:51] Christopher David: to

[38:52] Christopher David: to

[38:53] Christopher David: to

[38:54] Christopher David: to

[38:55] Christopher David: to

[38:56] Christopher David: to

[38:57] Christopher David: to

[38:58] Christopher David: to

[38:59] Christopher David: to

[39:00] Christopher David: to

[39:01] Christopher David: to

[39:02] Christopher David: to

[39:03] Christopher David: to

[39:04] Christopher David: to

[39:05] Christopher David: to

[39:06] Christopher David: to

[39:08] Christopher David: to

[39:09] Christopher David: to

[39:10] Christopher David: to

[39:11] Christopher David: to

[39:12] Christopher David: to

[39:13] Christopher David: to

[39:14] Christopher David: to

[39:15] Christopher David: to

[39:16] Christopher David: to

[39:17] Christopher David: to

[39:18] Christopher David: to

[39:19] Christopher David: to

[39:20] Christopher David: to

[39:21] Christopher David: to

[39:22] Christopher David: to

[39:23] Christopher David: to

[39:24] Christopher David: to

[39:25] Christopher David: to

[39:26] Christopher David: to

[39:27] Christopher David: to

[39:28] Christopher David: to

[39:29] Christopher David: to

[39:30] Christopher David: to

[39:31] Christopher David: to

[39:32] Christopher David: to

[39:33] Christopher David: to

[39:34] Christopher David: to

[39:35] Christopher David: to

[39:38] Christopher David: to

[39:39] Christopher David: to

[39:40] Christopher David: to

[39:41] Christopher David: to

[39:42] Christopher David: to

[39:43] Christopher David: to

[39:44] Christopher David: to

[39:45] Christopher David: to

[39:46] Christopher David: to

[39:47] Christopher David: to

[39:48] Christopher David: to

[39:49] Christopher David: to

[39:50] Christopher David: to

[39:51] Christopher David: to

[39:52] Christopher David: to

[39:53] Christopher David: to

[39:54] Christopher David: to

[39:55] Christopher David: to

[39:56] Christopher David: to

[39:57] Christopher David: to

[39:58] Christopher David: to

[39:59] Christopher David: to

[40:00] Christopher David: to

[40:01] Christopher David: to

[40:02] Christopher David: to

[40:03] Christopher David: to

[40:04] Christopher David: to

[40:05] Christopher David: to

[39:59] Christopher David: worked via the test.

[40:03] Christopher David: What?

[40:21] Christopher David: Come on.

[40:25] Christopher David: I currently have zero embeddings.

[40:28] Christopher David: I currently have six embeddings with file IDs.

[40:35] Christopher David: Now we are not currently.

[40:38] Christopher David: We need to pass the file ID back to the client for now.

[40:42] Christopher David: Does it work if I hard code it?

[40:46] Christopher David: So instead of corpus ID, we're going to do file ID too.

[40:54] Christopher David: Now is this also going to now D-D?

[41:01] Christopher David: What is this going to do with the response?

[41:07] Christopher David: The shape is of this response, so I can be correct.

[41:10] Christopher David: What is a soft spot?

[41:17] Christopher David: Oh, look.

[41:19] Christopher David: We're just returning all this print.

[41:25] Christopher David: So we say embedding length.

[41:27] Christopher David: Let's remove that.

[41:29] Christopher David: And then it's returning some kind of weird shape D thing here.

[41:40] Christopher David: Look at that results.

[41:43] Christopher David: Text appears.

[41:44] Christopher David: The internet soft spot is.

[41:50] Christopher David: This is returning us the correct results that we would expect.

[41:58] Christopher David: It's not yet formatted via an inference.

[42:02] Christopher David: So what I'd like to do now is that, at least, just show that maybe on the show that I'm a

[42:26] Christopher David: on the show that here, show it on the UI.

[42:38] Christopher David: If response.data and response data, okay, then response.data.

[42:50] Christopher David: Let me try to get them without that print out there.

[42:56] Christopher David: What is a soft spot?

[43:00] Christopher David: It's fast.

[43:03] Christopher David: Data.

[43:04] Christopher David: Okay, true results.

[43:05] Christopher David: array of three.

[43:06] Christopher David: It's almost ready to respond.

[43:12] Christopher David: Instead of response.data.summering, I remember when we used to have the code that would append

[43:18] Christopher David: together, all those things.

[43:19] Christopher David: Let's just do this response.data.

[43:27] Christopher David: Results0.text.

[43:37] Christopher David: Now, what is a soft spot?

[43:44] Christopher David: Yes!

[43:46] Christopher David: Okay, so what it's doing now is it's doing the search and it's finding the context.

[44:02] Christopher David: It's returning the page that's most accurate.

[44:08] Christopher David: It's returning three results because we asked it to return three results, but it would sort

[44:13] Christopher David: of be a good one.

[44:24] Christopher David: Let's just keep playing with this.

[44:33] Christopher David: Let's just keep playing with this.

[44:37] Christopher David: Let's just test.

[44:40] Christopher David: If I say what is a soft spot?

[44:43] Christopher David: It says the internet soft spot is a blah blah.

[44:45] Christopher David: But if I ask it something about, let's just say that I say describe the request form for sending

[45:01] Christopher David: a document.

[45:03] Christopher David: Hopefully, if this is working correctly, it should return the content of the second page.

[45:09] Christopher David: It does.

[45:10] Christopher David: Yes, I think you're one.

[45:12] Christopher David: There are a quest for a presenting a document.

[45:23] Christopher David: Action schema.

[45:26] Christopher David: I'm coding that find.

[45:35] Christopher David: Okay, we have connected our retrieval.

[45:48] Christopher David: We are now querying our PDF for the most accurate context.

[45:56] Christopher David: The last step here, which we will do in the next video, is to pass this context to an LLM,

[46:04] Christopher David: which will be the, I think it's a mystery that we're using, the swarm inference power by the

[46:10] Christopher David: GB token network to just kind of convince us down into a cleaner summary.

[46:16] Christopher David: We'll do that in the next video.