Transcription: OpenAgents Episode 015 - Similarity Search

Source: https://twitter.com/OpenAgentsInc/status/1724568957598708192 Wiki source: https://raw.githubusercontent.com/wiki/OpenAgentsInc/openagents/Video-Series.md Media title: OpenAgents - Episode 015: Similarity Search We configure our local Postgres databa... Upload date: 20231114 Transcription model: gpt-4o-transcribe-diarize Generated at: 2026-06-01T00:36:15Z

Machine-generated transcript. Review speaker labels and wording before using this as quote-grade source material.

[00:00] Christopher David: So we are generating embeddings via our swarm GPU network.

[00:06] Christopher David: Our feature test passes.

[00:14] Christopher David: So next, before we do a bunch of the different data models and relationships, I want to just validate that we can do what we think that we can do in interaction with our Postgres database. Now...

[00:27] Christopher David: In video two,

[00:28] Christopher David: when choosing between MySQL and Postgres, I said we would probably prefer to use Postgres assuming how we envision supporting this hosting-wise would support that because Postgres has support for pgVector and MySQL does not.

[00:49] Christopher David: enabling the data belt the database like the SQL database itself to do vector similarity search as part of queries that's huge so having that support locally means that we don't need to integrate with some third-party vector database

[01:09] Christopher David: We might want to at some point.

[01:12] Christopher David: I'm not really clear at what point that would become necessary if we're able to just throw it alongside the rest of our data in our Postgres database,

[01:22] Christopher David: which is orchestrated by Laravel Forge, like a wrapper on top of AWS RDS.

[01:28] Christopher David: So, you know, it should be fine and scale to the size that we need.

[01:35] Christopher David: But I don't quite know how to set up PG Vector, but we're going to learn that in this video.

[01:44] Christopher David: So the idea is we have the ability now to create embeddings and now we need to save them on our database.

[01:54] Christopher David: And we're going to want to do a feature test for running the...

[02:00] Christopher David: cosine similarity search basically querying our database

[02:11] Christopher David: via the pg vector postgres ql because like we could probably figure out how to do just comparing if like given an array of five different embeddings compare one to those five that's probably simple or that's like something else that we have to figure out but right now the question is can we store database um store the embeddings in the database so let's get that working and then we'll start building out the rest of this um so you get to

[02:38] Christopher David: First watch me struggle with pgVector um

[02:45] Christopher David: support. So first of all

[02:49] Christopher David: does the dbEngin so we're using locally we're using dbEngin.

[02:53] Christopher David: So the questions we have to answer is um is pgVector supported via dbEngin?

[03:02] Christopher David: If not, how do we add support?

[03:06] Christopher David: Is pgVector supported out of the box?

[03:11] Christopher David: And then this exact same question for the Laravel Forge AWS RDS. I think it's the RDS.

[03:25] Christopher David: Okay.

[03:25] Christopher David: And then also,

[03:27] Christopher David: what Laravel package do we need?

[03:30] Christopher David: I think we need to extend the Laravel ORM support to be able to...

[03:36] Christopher David: know what vectors how they should be represented in the database like there's a package for that data type what layer about package do we need to add support for vectors in databases okay that's probably the easiest one to answer because i think i've seen one this

[03:59] Christopher David: is interesting i've seen this before um

[04:04] Christopher David: But no, Qt Drent, that's a third party service.

[04:09] Christopher David: But they might have some interesting

[04:15] Christopher David: points struct. Oh no, that's from Qt Drent.

[04:23] Christopher David: Um, Laravel,

[04:24] Christopher David: vector emb

[04:28] Christopher David: embedding.

[04:36] Christopher David: Using pgvector embeddings search in Laravel.

[04:40] Christopher David: When you want to find search results that are relevant to the meaning of the query slash question rather than the exact words terms used you need to use semantic search.

[04:47] Christopher David: Semantic search requires the meaning to be somehow numerically represented in the data that you are searching.

[04:51] Christopher David: This is accomplished using embeddings which are vector representations of tokens,

[04:54] Christopher David: words,

[04:54] Christopher David: characters,

[04:55] Christopher David: etc.

[04:56] Christopher David: Blah blah blah.

[04:57] Christopher David: The embeddings data needs to be stored somewhere to perform semantic search.

[05:00] Christopher David: There are quite a few options out there by now,

[05:02] Christopher David: but generally what you're looking for is a vector database.

[05:05] Christopher David: It allows storage,

[05:06] Christopher David: indexing, and fast lookups of embeddings.

[05:09] Christopher David: What if you didn't want to use a hosted managed provider to store your embeddings, but somehow keep them in your own database?

[05:15] Christopher David: Well, you're in luck.

[05:16] Christopher David: Here, I'm going to show you how to use pgvector for semantic search in Laravel eloquent queries.

[05:22] Christopher David: That's exactly what we want.

[05:23] Christopher David: pgvector is an open source vector similarity search for Postgres.

[05:27] Christopher David: First, you'll need a Postgres installation.

[05:29] Christopher David: We do.

[05:33] Christopher David: Okay, now this thing here. They're using Docker and doing something that has PG vector enabled.

[05:43] Christopher David: Okay,

[05:44] Christopher David: so I'm going to skip over that and then just get as far as we can get and

[05:51] Christopher David: then we'll solve the rest from there.

[05:53] Christopher David: Okay, so.

[06:04] Speaker B: Does our test pass? What's going on?

[06:31] Christopher David: In your Laravel app, add the pgvector Pucket package and publish its extension-enabling migration.

[06:40] Christopher David: Yeah, that's not good.

[06:42] Christopher David: I'm guessing a model wasn't loaded.

[06:44] Christopher David: Oh, there we go.

[06:48] Christopher David: Now, what does this say?

[06:54] Christopher David: Okay.

[06:56] Christopher David: Extension vector is not available.

[07:02] Christopher David: Cannot open extension control file.

[07:03] Christopher David: No such file or directory.

[07:05] Christopher David: Hint: the extension must first be installed on the system where PostgreSQL is running.

[07:15] Christopher David: So first of all, db engine,

[07:17] Christopher David: how to add PostgreSQL extension.

[07:36] Christopher David: Um

[07:39] Christopher David: Oh.

[07:48] Christopher David: Many hosted providers. That's not what we're doing, is it?

[07:59] Christopher David: DB.

[08:00] Christopher David: You know what? Let's just try asking ChatDB team.

[08:04] Christopher David: How do I have a Mac and need to install a Postgres extension called pgVector and get it to work with my Postgres database running via DB engine, running for the software DB engine.

[08:30] Christopher David: How do I do it?

[08:45] Christopher David: You'll need to build PG vector from source.

[08:48] Christopher David: Well, that's clear.

[09:08] Christopher David: Make and make install.

[09:17] Christopher David: Make PG Config.

[09:20] Christopher David: Command not found.

[09:32] Christopher David: Okay, if your machine has multiple Postgres... Okay, so I need the DB Engine PG Config.

[09:54] Christopher David: So,

[09:56] Christopher David: step two.

[10:26] Christopher David: I need to do this.

[10:30] Christopher David: No.

[10:45] Christopher David: Step two gives me this. I need to do this.

[10:50] Christopher David: How do I find the PG config path?

[10:54] Christopher David: for DB engine.

[11:38] Speaker B: Searching my entire system doesn't seem

[11:48] Speaker B: Okay, I'm guessing it's in let me just try snooping through s applications folder.

[11:56] Speaker B: CD applications CD DB engine.

[12:02] Speaker B: CD

[12:05] Speaker B: See the contents well,

[12:12] Speaker B: what if I now search through here do

[12:18] Speaker B: this isn't knowing okay cancel that wait

[12:22] Speaker B: there it is

[12:28] Speaker B: That worked thank you okay, so export PG config equals that

[12:38] Speaker B: Then before I rerun it, there's a particular command, it is

[12:41] Speaker B: uh

[12:44] Speaker B: What I go

[12:48] Speaker B: Here.

[12:51] Speaker B: Oh.

[12:53] Speaker B: Looks like first I just do make.

[13:12] Speaker B: Okay, it looks like that worked. Then we'll do

[13:18] Speaker B: this.

[13:26] Speaker B: Maybe that worked.

[13:30] Speaker B: Okay.

[13:32] Speaker B: Let's try

[13:42] Speaker B: Running what was it that I needed to run next?

[14:01] Speaker B: Okay, that's good.

[14:02] Speaker B: Now what did that do actually?

[14:05] Speaker B: DB statement create extension if not exist vector.

[14:09] Speaker B: Okay.

[14:11] Speaker B: Alright.

[14:12] Speaker B: Having some trouble with my embedding test.

[14:20] Speaker B: Because I'm getting a timeout. So let me just try running a test.

[14:27] Speaker B: Okay, my test run,

[14:29] Speaker B: which means I think...

[14:37] Christopher David: The vector extension is running locally.

[14:40] Christopher David: That's very good.

[14:48] Christopher David: Now this says, which I think I want to do too,

[14:50] Christopher David: let's treat our embeddings as first class models. So let's create an eloquent model and migration. I didn't know you could pass dash dash factory.

[14:59] Christopher David: That's awesome.

[15:00] Christopher David: Okay, didn't we say we wanted to create an embedding model?

[15:03] Christopher David: Yes, we did.

[15:03] Christopher David: So that's actually perfect.

[15:05] Christopher David: PHP artisan make model migration factory embedding booyah

[15:11] Christopher David: let's save

[15:15] Christopher David: that and let's take a look at the factory empty cool

[15:33] Christopher David: okay what's next

[15:35] Christopher David: In the migration, let's create a vector column to store the raw embeddings as well as a JSON column for metadata to return along with the results.

[15:47] Christopher David: Okay,

[15:48] Christopher David: copy.

[15:56] Christopher David: 1536 dimensions for OpenAI's Ada,

[15:59] Christopher David: but we're not using that. We're using...

[16:02] Speaker B: The other one, which I believe was 768, yeah.

[16:10] Speaker B: And then what is this?

[16:12] Christopher David: This is a Postgres specific index that allows us to do fast nearest neighbor searches when there are a lot of high dimensional embeddings in the database. Yeah,

[16:21] Christopher David: this is where we just copy paste and cross our fingers.

[16:28] Christopher David: Okay.

[16:31] Christopher David: Just make sure that our tests run, which means that the migrations are successful.

[16:39] Christopher David: And then let's

[16:51] Christopher David: What's this? Functions.

[16:52] Christopher David: Let's connect to the local

[16:57] Speaker B: database called Postgres functions I don't care about function here tables we

[17:07] Speaker B: have the table has been created well

[17:19] Speaker B: the table has been created

[17:25] Speaker B: Let's just copy pasta what they're doing here in the model and the relevant casts.

[17:39] Speaker B: Nice. Time to run our migrations.

[17:42] Speaker B: We already did,

[17:43] Speaker B: I think.

[17:45] Speaker B: Nothing to migrate.

[17:59] Christopher David: Composer require open AI PHP.

[18:04] Christopher David: Now, but I might copy this.

[18:13] Christopher David: What's going on with our test?

[18:15] Christopher David: Oops.

[19:12] Speaker B: Inference failed.

[19:13] Speaker B: No workers available.

[19:24] Speaker B: So this is a live stream of the network and errors on our network.

[19:32] Speaker B: It's not super optimized for readability.

[19:40] Speaker B: Someone's doing some chat completions, that's cool.

[19:42] Speaker B: Now, um...

[19:57] Speaker B: I'm trying to figure out why the embedding...

[20:00] Christopher David: It's timing out.

[20:01] Christopher David: There it is.

[20:03] Christopher David: I'm guessing that the workers,

[20:05] Christopher David: some workers just didn't have the embedding model loaded.

[20:08] Christopher David: So that took 1.41 seconds.

[20:10] Christopher David: Try it again.

[20:13] Christopher David: It might have just required maybe

[20:19] Christopher David: no models had it loaded or the worker that it's being assigned to isn't responding.

[20:25] Christopher David: Okay, so it's at least working.

[20:28] Christopher David: Um,

[20:34] Christopher David: okay,

[20:35] Christopher David: we've installed pgVector.

[20:42] Christopher David: Let's see. Now with that set up out of the way,

[20:44] Christopher David: let's create a simple console command to populate our database with the settings.

[20:50] Christopher David: This looks fun.

[20:51] Christopher David: So in routes slash console.php.

[20:56] Christopher David: Yeah, another timeout. I think it it probably depends on the worker that it's um.

[21:00] Christopher David: Oh, there we go. Failed asserting to z that zero that embedding didn't return.

[21:08] Christopher David: Alright, it's looking like some workers are giving us good answers, some are not.

[21:11] Christopher David: So we've gotta yeah punish some of them. Routes console.

[21:57] Christopher David: Okay, so here

[22:11] Christopher David: I forgot you can pass in an array of embeddings. Does that work with ours?

[22:20] Christopher David: So let's first

[22:24] Christopher David: emulate the code from here.

[22:33] Christopher David: Will this work?

[22:36] Christopher David: Well, let's do this.

[22:40] Christopher David: I know that our function that we did last episode does not support - we got to like change the syntax to be able to handle an array of answers.

[22:50] Christopher David: Or is it just returning the first?

[22:53] Christopher David: So first of all, let's see what this does.

[22:57] Christopher David: PHP Artisan.

[23:00] Christopher David: Insert.

[23:03] Christopher David: Class Queen Bee Gateway not found.

[23:14] Christopher David: Undefined variable result.

[23:17] Christopher David: Okay, so for now just die and dump the embeddings.

[23:30] Christopher David: And have to do it again with a different worker.

[23:37] Christopher David: It should automatically punish whoever doesn't give it give me an answer.

[23:46] Christopher David: And if I try it in a second window, it should run a second query that gives it to me from a different worker.

[23:56] Christopher David: There we go. There we go.

[23:59] Christopher David: It's so beautiful.

[24:02] Christopher David: Now, if

[24:06] Christopher David: I pass it the entire one,

[24:16] Christopher David: I see. Yeah,

[24:17] Christopher David: so it is just giving me the first.

[24:26] Christopher David: So what I need to have it do basically is just return the entire body

[24:35] Christopher David: of data.

[24:39] Christopher David: And then that should be an

[24:45] Christopher David: array that says data at the top.

[24:48] Christopher David: Oh yeah.

[24:55] Christopher David: object embedding well that's awesome that's awesome it's returning exactly what we expect still got some like slow workers or some workers that are not processing the job time out i don't know why but um

[25:12] Christopher David: at least it's working some of the times we can debug that later okay so i know we'll need to change our test

[25:24] Christopher David: like this will need to be by the way this was doing 700 assertions i don't want to assert that every single one is numeric that's just kind of crazy um let's just expect that the first

[26:02] Christopher David: Check if some elements are numeric.

[26:12] Christopher David: Okay, let's see if I have that right and this would not be that, it would be

[26:22] Christopher David: Should be results and then embedding would be result

[26:28] Christopher David: zero.

[26:32] Christopher David: embedding I think

[26:42] Christopher David: Okay let's see what PF embedding does now passes

[27:01] Christopher David: Okay, so we'll call this

[27:04] Christopher David: um

[27:06] Christopher David: refactor embedding test

[27:11] Christopher David: and

[27:13] Christopher David: gateway to support multi

[27:17] Christopher David: um embeds.

[27:21] Christopher David: And then we'll call this

[27:24] Christopher David: um

[27:26] Christopher David: set up embedding model.

[27:29] Christopher David: and migration.

[27:33] Christopher David: Now let's go back to our artisan command.

[27:56] Christopher David: I don't know if this is the right response syntax. Let's print R_ the sayings.

[28:18] Christopher David: If it's too slow, we know it's one of the non-working workers. I'm just gonna do it again. Undefined variable results.

[28:26] Christopher David: Oops.

[28:29] Christopher David: What did that say originally? Result was the result of the embeddings. So

[28:41] Christopher David: Oh.

[28:44] Christopher David: Oops, I'm redefining sayings. Results is that

[28:56] Christopher David: I see, it's using that as the key. Alright, so result equals that for each saying says blah blah blah.

[29:07] Christopher David: Maybe.

[29:09] Christopher David: Attempt to read property embeddings on an array.

[29:28] Christopher David: results

[29:34] Christopher David: okay first of all hang on what is the result again

[29:48] Christopher David: okay the results so I just need the key

[29:54] Christopher David: Or do I? What's the structure of this? I can't tell, it's too big.

[30:11] Christopher David: Alright, let me just go take a look at Queen B to see what it is.

[30:54] Christopher David: Wow. I don't quite know how to decode that.

[30:58] Christopher David: Let's try some process of elimination.

[31:06] Christopher David: Actually, can I get the keys like that?

[31:21] Christopher David: of the array.

[31:23] Christopher David: That's not right.

[31:28] Christopher David: What?

[31:34] Christopher David: O one two three.

[31:36] Christopher David: It's just an array.

[31:38] Christopher David: So I just need the result,

[31:40] Christopher David: the key,

[31:42] Christopher David: and then inside that I need the embedding. Is that right?

[31:56] Christopher David: Is no news good news?

[32:09] Christopher David: Okay,

[32:09] Christopher David: that's good.

[32:10] Christopher David: And it's good because I was like,

[32:11] Christopher David: oh no,

[32:12] Christopher David: do we need to send each string,

[32:14] Christopher David: each embedding to the server separately? No, we sent it one array of strings and it came back with every single.

[32:23] Christopher David: holy shit okay so we now and that i was like we also need demo data we

[32:29] Christopher David: have demo data so that's freaking amazing okay so ladies and gentlemen we have our first actual embeddings

[32:39] Christopher David: of these sayings feline same meow canine say well blah blah blah that we got from our queen bee we passed a list of four sayings

[32:48] Christopher David: It vector embedded them and gave us it back to us, gave them back to us within a second or so.

[32:56] Christopher David: And then we looped through them and we created the database entries for embeddings in our local database.

[33:04] Christopher David: And there they are.

[33:05] Christopher David: Now let's see if this tutorial guide was already querying of it.

[33:10] Christopher David: Now let's run a query.

[33:12] Christopher David: Oh yeah,

[33:13] Christopher David: oh yeah.

[33:14] Christopher David: Now let's run a query.

[33:16] Christopher David: So let's do one more command. This one's called search. Actually let me save that first. Um

[33:24] Christopher David: demo embeddings work and see dB.

[33:34] Christopher David: So artisan command search.

[33:46] Christopher David: No, I don't want to use the open

[33:51] Christopher David: AI blah blah blah blah.

[33:54] Christopher David: Remove, what do dogs say?

[34:08] Christopher David: Embedding is, and this is not the correct.

[34:14] Christopher David: It's just going to be, okay,

[34:23] Christopher David: first,

[34:24] Christopher David: DD the result,

[34:27] Christopher David: because I need to remember what the syntax or shape of that response object is.

[34:33] Christopher David: Gateway,

[34:34] Christopher David: I need the gateway.

[34:44] Christopher David: Embedding. No, result

[34:49] Christopher David: It's an array. So result zero

[35:07] Christopher David: This table saying

[35:10] Christopher David: Saying I don't know what that is.

[35:17] Christopher David: Oops.

[35:20] Christopher David: What do dogs say?

[35:23] Christopher David: Class vector not found.

[35:34] Christopher David: Now, what do dogs say?

[35:40] Christopher David: Canines say woof. Why is it also telling us that felines say meow?

[35:48] Christopher David: Take two.

[35:49] Christopher David: What is this?

[35:52] Christopher David: Canines say woof, everybody.

[35:55] Christopher David: Today, we have learned something truly important.

[36:01] Christopher David: Now, now, now, now, now, now.

[36:03] Christopher David: What do cats say?

[36:09] Christopher David: Canines say wolf.

[36:10] Christopher David: God motherfuckers.

[36:19] Christopher David: This is a cop-out. You're taking two?

[36:30] Christopher David: Okay, there you go.

[36:31] Christopher David: What do cats say?

[36:33] Christopher David: Felines say meow.

[36:35] Christopher David: Oh, you know what?

[36:37] Christopher David: I'm guessing, you know how sometimes when we ran it earlier,

[36:41] Christopher David: sometimes we return an empty result.

[36:43] Christopher David: I need to probably add some kind of, let's make sure that we actually have the result.

[36:48] Christopher David: Because yes,

[36:49] Christopher David: this is the correct answer.

[36:50] Christopher David: Feline, same, yeah.

[36:51] Christopher David: Let's take the top one.

[36:58] Christopher David: Okay,

[36:58] Christopher David: so when we have an actual embedding.

[37:02] Christopher David: What do cats say? Yes.

[37:05] Christopher David: What do birds say?

[37:13] Christopher David: What the fuck? Why does it now say feline say meow?

[37:19] Christopher David: Wait, wait, wait. What were our sayings?

[37:23] Christopher David: Birds say tweet.

[37:31] Christopher David: It worked on the second time. Is someone giving us back bad data or something that's queued?

[37:39] Christopher David: Because these might be different workers.

[37:41] Christopher David: Bird say tweet,

[37:42] Christopher David: bird say tweet.

[37:44] Christopher David: What do,

[37:45] Christopher David: out of curiosity,

[37:46] Christopher David: what does it do for what do wolves say?

[37:51] Christopher David: That one gave me the previous one.

[37:54] Christopher David: We've got to make sure that there's no caching.

[37:58] Christopher David: There you go.

[37:59] Christopher David: Canines say wolf.

[38:01] Christopher David: Oh, you know what?

[38:04] Christopher David: I think that's actually an artisan caching.

[38:09] Christopher David: Oh, yeah,

[38:10] Christopher David: yeah, yeah. It's because the artisan is caching. What's the command to?

[38:18] Christopher David: What is the command to clear artisan command?

[38:23] Christopher David: Clear Laravel artisan command cache.

[38:30] Christopher David: Actually, what I'm going to do is make this take a parameter.

[38:36] Christopher David: Refactor this to take the query as a command line parameter.

[38:42] Christopher David: Yeah,

[38:43] Christopher David: it wasn't working on the next one because it was cached for a second, but we're going to remove that as a problem.

[38:59] Christopher David: Nice.

[39:02] Christopher David: Do this.

[39:09] Christopher David: Create the embedding of query

[39:13] Christopher David: and then um that results. What's the structure of this?

[39:35] Christopher David: Does this work?

[39:40] Christopher David: P_H_P_ array length

[39:42] Christopher David: count

[39:46] Christopher David: maybe?

[39:49] Christopher David: Oops. What happened?

[39:54] Christopher David: P_H_P_ artisan search what do dogs say?

[40:01] Christopher David: Canines say woof, woof.

[40:08] Christopher David: Okay, everybody.

[40:09] Christopher David: Canines say woof.

[40:11] Christopher David: What do cats say?

[40:14] Christopher David: What the f...

[40:17] Christopher David: Canines do not say woof.

[40:19] Christopher David: Felines say meow.

[40:22] Christopher David: Oh my gosh.

[40:24] Christopher David: Now, question,

[40:25] Christopher David: was that...

[40:27] Christopher David: Somehow, still an issue of cache, this being cached.

[40:39] Christopher David: Dogs.

[40:53] Christopher David: That time it timed out.

[41:00] Christopher David: Canine say woof.

[41:02] Christopher David: Canine say woof.

[41:09] Christopher David: Canine say woof.

[41:13] Christopher David: Now, what do birds say?

[41:17] Christopher David: Birds say tweets.

[41:19] Christopher David: What do dogs say? Canine say woof. What do birds say?

[41:28] Christopher David: Canines say wolf, this is timing out.

[41:31] Christopher David: What do birds say? Why is it returning canine say wolf right there?

[41:39] Christopher David: Okay, I'm guessing there might be just a little,

[41:41] Christopher David: maybe like an error in our implementation where two of them are kind of stepping on top of each other.

[41:50] Christopher David: But...

[41:59] Christopher David: Alright we got birds saying tweet if I come over here and I say what do cats say feline

[42:09] Christopher David: say meow birds say tweet that's not good feline

[42:19] Christopher David: say meow

[42:21] Christopher David: Felines say meow.

[42:27] Christopher David: Felines say meow.

[42:30] Christopher David: Birds say tweet.

[42:37] Christopher David: Birds say tweet.

[42:42] Christopher David: Birds say tweet.

[42:43] Christopher David: Awesome.

[42:44] Christopher David: Okay, we've discovered some probably minor bug with um...

[42:49] Christopher David: embedding jobs that's probably a problem with worker being that should be easy enough to debug but the cool thing here is that we have vector embeddings working we've set up PG vector locally we have the ability to do cosine similarity searches that's what's being run here this is the cosine similarity search

[43:19] Christopher David: It's done in the SQL query that we get the little Laravel version of thanks to this pg vector package.

[43:29] Christopher David: Okay,

[43:30] Christopher David: that was the kind of last missing piece for us to be able to know that we are able to do embeddings in Laravel. And so next video we will complete setting up our data model so we can start manipulating this via API and via our interface.

[43:48] Christopher David: See you soon.