Transcription: OpenAgents Episode 014 - Embeddings 101

Source: https://twitter.com/OpenAgentsInc/status/1724509783086989333 Wiki source: https://raw.githubusercontent.com/wiki/OpenAgentsInc/openagents/Video-Series.md Media title: OpenAgents - Episode 014: Embeddings 101 We learn about vector embeddings and writ... Upload date: 20231114 Transcription model: gpt-4o-transcribe-diarize Generated at: 2026-06-01T00:21:38Z

Machine-generated transcript. Review speaker labels and wording before using this as quote-grade source material.

[00:00] Christopher David: Now that we have our Rag roadmap built, let's learn about embeddings.

[00:07] Christopher David: So in this video we wanna learn what are embeddings, we're gonna see what we have so far for our embeddings A_P_I_ uh we're gonna build out these data models, file chunk embeddings and uh work on the feature test. Let's see how far we can get.

[00:22] Christopher David: Um what the hell are embeddings?

[00:25] Christopher David: Let's learn from this Hugging Face blog.

[00:28] Christopher David: This is from a while ago,

[00:29] Christopher David: but... An embedding is a numerical representation of a piece of information,

[00:34] Christopher David: for example text,

[00:35] Christopher David: documents,

[00:36] Christopher David: images,

[00:37] Christopher David: audio,

[00:37] Christopher David: etc.

[00:38] Christopher David: etc.

[00:38] Christopher David: The representation captures the semantic meaning of what is being embedded,

[00:43] Christopher David: making it robust for many industry applications.

[00:46] Christopher David: Given the text,

[00:47] Christopher David: what is the main benefit of voting,

[00:49] Christopher David: an embedding of the sentence could be represented in a vector space,

[00:52] Christopher David: for example with a list of three hundred eighty four numbers.

[00:55] Christopher David: For example, array of floats. Since this list captures the meaning we can do exciting things like calculating the distance between different embeddings to determine how well the meaning of two sentences matches.

[01:08] Christopher David: Embeddings are not limited to text. You can also create an embedding of an image,

[01:13] Christopher David: for example a list of 384 numbers,

[01:15] Christopher David: and compare it with a text embedding to determine if a sentence describes the image.

[01:19] Christopher David: This concept is under powerful systems for image search, classification,

[01:25] Christopher David: description,

[01:25] Christopher David: and more.

[01:27] Christopher David: How are they generated?

[01:29] Christopher David: The GPUTopium API.

[01:31] Christopher David: What are embeddings for?

[01:33] Christopher David: Once you understand this machine learning multi-tool embedding, you'll be able to build everything from search engines to recommendation systems to chatbots and a whole lot more.

[01:42] Christopher David: You don't have to be a data scientist with ML expertise to use them, nor do you need a huge label dataset.

[01:48] Christopher David: Once a piece of information,

[01:49] Christopher David: a sentence,

[01:50] Christopher David: a document,

[01:50] Christopher David: an image is embedded,

[01:52] Christopher David: the creativity starts.

[01:53] Christopher David: Several interesting industrial applications use embeddings.

[01:57] Christopher David: For example, Google search uses embeddings to match text to text and text images,

[02:01] Christopher David: Snapchat uses them to serve the right ad to the right user at the right time,

[02:05] Christopher David: and Meta Facebook uses them for their social search.

[02:08] Christopher David: Before they could get intelligence from embeddings, these companies had to embed their pieces of information.

[02:14] Christopher David: An embedded data set allows algorithms to search quickly,

[02:16] Christopher David: sort,

[02:17] Christopher David: group,

[02:17] Christopher David: and more.

[02:17] Christopher David: However, it can be expensive and technically complicated.

[02:21] Christopher David: Blah, blah, blah.

[02:22] Christopher David: Okay, that's a great introduction.

[02:23] Christopher David: Thank you.

[02:25] Christopher David: So we're going to do that for this.

[02:33] Christopher David: We set it up so a file has chunks. The chunks are embedded. And we're chunking up the file.

[02:41] Christopher David: Uh,

[02:42] Christopher David: because we need to pass these, you know, small amounts of text, the most relevant amounts of text into a large language model prompt so it has to fit within the context window.

[02:56] Christopher David: So let's take a look at what we have already.

[02:59] Christopher David: I have not used this yet,

[03:01] Christopher David: but the Queen Bee API,

[03:04] Christopher David: which is our kind of coordinator.

[03:09] Christopher David: worker or a coordinator that sends requests out to our worker bees the software that are running on people's computers all around the world here's the API it is compatible with the open AI API there's probably a few things that we're missing but we do have embeddings I have not used this before so we're going to be testing this probably in this video

[03:35] Christopher David: But that's the embedding. So theoretically if we pass it, let's see,

[03:40] Christopher David: example value.

[03:44] Christopher David: Okay, so there's the schema.

[03:47] Christopher David: Input has to be a string or array of strings.

[03:50] Christopher David: The model, I

[03:56] Christopher David: think the model is the...

[04:02] Christopher David: Alright, examine that to me. Might need to pass the model in. Uh and then encoding format, user, GPU filter, I'm not sure

[04:10] Christopher David: what those are, but if you get what do you get back? You get back the embedding.

[04:16] Christopher David: The list of arrays. Then then you store that.

[04:18] Christopher David: Okay.

[04:20] Christopher David: Cool. So we know that we're gonna be building in that direction.

[04:27] Christopher David: So let's start

[04:32] Christopher David: Instead of starting with the unit tests we did that last time, this time I'm going to start with a feature test.

[04:39] Christopher David: And then we'll just do the unit tests that are only needed to solve the

[04:50] Christopher David: the embeddings. So let's first just run our whole test suite,

[04:52] Christopher David: make sure this is all passing.

[05:09] Christopher David: Takes a while because we have now third-party um

[05:13] Christopher David: integration tests for Vektara that I will probably remove once we get our rag in place. Okay, sixty-seven tests passed, that's great.

[05:25] Christopher David: Let's create a new feature test called embedding test.

[05:34] Christopher David: And let's start with the most important thing which is creating an embedding.

[05:42] Christopher David: Is that what we should start with?

[05:46] Christopher David: Or do we want to start with the file and chunk? Yeah, I guess let's do the feature that we care about.

[05:53] Christopher David: Um...

[05:56] Christopher David: Can...

[05:59] Christopher David: embed

[06:02] Christopher David: Can create embedding from

[06:07] Christopher David: text.

[06:10] Christopher David: Let's see if Copilot can suggest something good.

[06:14] Christopher David: Vektara.

[06:14] Christopher David: No, we're not using Vektara.

[06:17] Christopher David: I don't think they support that at all.

[06:22] Christopher David: But this general idea of like,

[06:24] Christopher David: maybe we should create a class called Embetter.

[06:37] Christopher David: Embedding equals.

[06:39] Christopher David: better create embedding and

[06:45] Christopher David: then I'm going to expect embedding

[06:52] Christopher David: to be a

[06:58] Christopher David: I don't know what the shape is it going to be oh does that work

[07:13] Christopher David: Okay. Well, let's

[07:15] @: uh

[07:19] Christopher David: let's do this.

[07:39] Christopher David: Should be an em better

[07:44] Christopher David: So the embedding, so what are we going to be doing?

[07:47] Christopher David: What is the purpose of the class?

[07:49] Christopher David: I think the purpose of the class is going to be to interact with the queen bee.

[07:55] Christopher David: And so rather than embedder, I'm thinking we're

[08:05] Christopher David: going to want like a queen

[08:09] Christopher David: bee gateway.

[08:11] Christopher David: We call it a queen bee gateway.

[08:15] Christopher David: It's going to be our way of interacting with the queen bee.

[08:21] Christopher David: Okay?

[08:22] Christopher David: All right, queen bee gateway not found.

[08:28] Christopher David: So let's do queen bee gateway. This is going to be a class.

[08:43] Christopher David: put this in the name space

[08:46] Christopher David: app services. What's that do?

[08:51] Christopher David: Got to import it.

[08:59] Christopher David: Uh okay, embedder doesn't know what that is. Oh wait a second, embedding

[09:05] Christopher David: equals gateway.

[09:08] Christopher David: There we go.

[09:10] Christopher David: Alright, call to undefined method, create embedding public function

[09:17] Christopher David: create embedding. Wait, did I just call it should be public method?

[09:22] Christopher David: Am I being a p-h-p-monger uh JavaScript monger? No, it's public function.

[09:27] Christopher David: Public function create embedding.

[09:34] Speaker B: Oh.

[09:36] Christopher David: Call to a member function shape.

[09:40] Christopher David: Well, I don't know what that collect thing does.

[09:46] Christopher David: I don't know what the collect thing does.

[09:49] Christopher David: Let's try removing

[10:00] Christopher David: Yeah, what is this shape thing?

[10:05] Christopher David: Let's ask ChatGPT.

[10:18] Christopher David: What does the

[10:25] Christopher David: Oh.

[10:29] Christopher David: Oh.

[10:30] Christopher David: We've updated I don't shut up. What does the embedding

[10:37] Christopher David: need to be to have that shape work?

[11:18] Christopher David: Figure it out, ChatGPT.

[11:28] Christopher David: Hurry up.

[11:42] Christopher David: Just tell me how to fix it.

[12:30] Christopher David: Uh there's a bug that apparently we have a PR for when you create a new chat.

[12:34] Christopher David: Black.

[12:43] Christopher David: Zero. Okay, that's not helpful.

[13:12] Christopher David: I don't want it to be multi-dimensional.

[13:17] Christopher David: Just a single array of seven sixty eight.

[13:39] Christopher David: Alright, yeah, some placeholder code for now is fine.

[13:51] Christopher David: Let's at least just see if these match each other.

[14:02] Christopher David: Thumb passing test.

[14:05] Christopher David: Now, what

[14:08] Christopher David: I actually need to do is connect to the Queen

[14:17] Christopher David: Bee Gateway.

[14:23] Christopher David: And make a post request.

[14:26] Christopher David: Now I think I might have some

[14:31] Christopher David: of this code already done in terms of connecting with a...

[14:38] Christopher David: Yeah.

[14:44] Christopher David: I'm going to steal this code from myself.

[15:20] Christopher David: Let's see.

[16:00] Christopher David: can I just copy this whole thing?

[16:16] Christopher David: Post V1 embeddings.

[17:04] Christopher David: Let's try this.

[17:15] Christopher David: Okay, the default model is going to be this.

[17:21] Christopher David: I don't know what user it needs to be,

[17:26] Christopher David: but let's see what happens.

[17:39] Christopher David: First of all, this is the successful response.

[18:45] Christopher David: Let's go

[18:47] Christopher David: Well, in a second we'll go grab our A_P_I_ token.

[19:04] Christopher David: And this needs to be slash embeddings.

[19:13] Christopher David: content type application JSON

[19:21] Christopher David: And this just needs to be data, I think.

[19:34] Christopher David: Let's see what that is.

[19:36] Christopher David: Okay, app services open AI. Where, what the hell? Oh.

[19:43] Christopher David: This doesn't even...

[19:49] Christopher David: This client...

[19:52] Christopher David: I don't think we're using that there.

[19:59] Christopher David: services http, that's solvable.

[20:05] Christopher David: Think I was experimenting with two different approaches not last code base.

[20:15] Christopher David: Okay, well count embedding is not the right

[20:19] Christopher David: thing, but here's what I wanna do. Let's just die and dump the response, see if we get anything meaningful back.

[20:26] Christopher David: That doesn't look right.

[20:29] Christopher David: How about response.

[20:32] Christopher David: JSON.

[20:35] Christopher David: Ooh-hoo.

[20:37] Christopher David: Request validation error.

[20:42] Christopher David: Input should be a valid dictionary.

[20:56] Christopher David: pi-dantic.

[21:08] Christopher David: Okay, so what's the problem here?

[21:13] Christopher David: First it's diandub the data.

[21:19] Christopher David: Okay, so input looks good, model looks good.

[21:23] Christopher David: encoding format that does not look good

[21:36] Christopher David: default is float okay that does look good

[22:12] Christopher David: I don't know what user is supposed to be.

[22:23] Christopher David: Body GPU filter. Input should be a valid dictionary.

[22:31] Christopher David: Can I make it no?

[22:39] Christopher David: Input none.

[23:13] Christopher David: Uh what happens if I just don't pass that?

[23:18] Christopher David: Ha-ha.

[23:20] Christopher David: Insufficient funds in account or incorrect auth token.

[23:35] Christopher David: For now let's hard code my token, usually a no-no, but

[23:40] Christopher David: Error.

[24:03] Christopher David: Error not sizable.

[24:13] Christopher David: Se ze abo.

[24:16] Christopher David: Now out of curiosity let me pop up for the queen bee.

[24:20] Christopher David: I don't know that anyone has reliably used this before. Could still be a little buggy.

[24:30] Christopher David: I don't see se ze abo here, so I don't know where that error is coming from.

[25:11] Christopher David: I may need to append it with fast embed.

[25:43] Christopher David: Oh snap,

[25:47] Christopher David: yes, there's

[25:54] Christopher David: an embedding and it's got 768 looks like

[25:59] Christopher David: Wow, why did we assume correctly or why did copilot know I guess because other people are using this stuff, okay

[26:05] Christopher David: Fucking cool.

[26:06] Christopher David: There's our embedding

[26:16] Christopher David: I think it gave me the answer previously.

[26:18] Christopher David: Yeah.

[26:35] Christopher David: Wait, wait, wait.

[26:36] Christopher David: What does this need to be?

[26:38] Christopher David: Body, JCode, JCode, blah, blah, blah.

[26:45] Christopher David: Does that now work?

[26:51] Christopher David: I need to remove the die and dump.

[26:58] Christopher David: Holy shit! Holy shit.

[27:03] Christopher David: We're generating and embedding. Oh my goodness. I don't know what this what is this thing doing? Can I remove that?

[27:13] Christopher David: Yeah. Okay.

[27:36] Christopher David: Wow.

[27:42] Christopher David: Just gonna hard-code my token for now. I don't give a shit.

[27:54] Christopher David: Bruh. That's fucking cool.

[27:58] Christopher David: Can create embedding from text.

[28:00] Christopher David: We are.

[28:06] Christopher David: We can create an embedding. We can create an embedding.

[28:09] Christopher David: Fuck, that's cool.

[28:12] Christopher David: No third party dependencies.

[28:15] Christopher David: Takes about, okay, so that took 1.22 seconds in total. So about one second to generate and embedding from our swarm network.

[28:32] Christopher David: And that'll get lower over time.

[28:35] Christopher David: as we add more powerful computers or do different network jism jazzing, but uh we have a decentralized network providing embeddings via an open A_I_ compatible end point. That's pretty cool. Okay, so what else do we need to do on So we we we created

[28:56] Christopher David: and uh we we can create an embedding. We have not yet uh saved the embedding.

[29:17] Christopher David: Okay, I think this is a good starting point for this video and we will do the kind of like data model connections and stuff in the next video.

[29:26] Christopher David: But what did we do?

[29:28] Christopher David: We reviewed what is embeddings.

[29:33] Christopher David: We took a look at the GPTopia embedding API and implemented it. We'll do this in the next video.

[29:42] Christopher David: We did our first feature test for creating an embedding.

[29:48] Christopher David: So, again, we are initializing a connection to our Queen Bee gateway, the like work the job orchestrator of our GP Utopia network of GPUs, that's the list of them, there's probably 150 of them there,

[30:10] Christopher David: 30 of them.

[30:12] Christopher David: have a capability called fast embed and

[30:17] Christopher David: so when you do this it's somehow choosing between them using the default embeddings model and

[30:32] Christopher David: we're getting now we can do cool stuff with it let's do some cool stuff with it