Transcription: OpenAgents Episode 024 - Faerie Makes a Plan

Source: https://twitter.com/OpenAgentsInc/status/1725969687102534110 Wiki source: https://raw.githubusercontent.com/wiki/OpenAgentsInc/openagents/Video-Series.md Media title: OpenAgents - Episode 024: Faerie Makes a Plan We give our agent all relevant code ... Upload date: 20231118 Transcription model: mlx-whisper/mlx-community/whisper-tiny Generated at: 2026-06-01T20:38:23Z

Machine-generated transcript. Review speaker labels and wording before using this as quote-grade source material.

[00:00] Christopher David: Let's keep going. We now have a whole bunch of embeddings and notes. We're storing file IDs for each of them. Each of them is its own file.

[00:14] Christopher David: We're using a factory, a sign of it, and we're using a factory.

[00:22] Christopher David: We'll deal with that later. But basically, right now, in the past, when we had done querying, we would use this query controller, which would create a searcher class,

[00:43] Christopher David: and run query, we would pass it a file ID, and it would search the embedding dissociated only with that file ID. In this case, we want to search it by all the files. Later on, we'll figure out some other way of segmenting.

[01:00] Christopher David: If we have a bunch of files, we don't want to query all of them. But for now, we're just going to query all the embeddings that we have, because we have the ones there that we want.

[01:10] Christopher David: So, how should we approach this?

[01:29] Christopher David: Let's go take a look at our commands.

[01:35] Christopher David: So right now, we have the helper function to format the GitHub issue in comments as messages.

[01:47] Christopher David: And I think I want some similar helper for generating context.

[02:10] Christopher David: To generate the context, the question is, what do we pass in as the query?

[02:19] Christopher David: So we know we're going to query all of the embeddings, but what do we pass in?

[02:27] Christopher David: So one thing I definitely want to do is decide where in a prompt to put the context. I want to definitely use the same prompt structure that we used for this previous query conversation.

[02:52] Christopher David: Yes, we'll just use this phrase here.

[03:12] Christopher David: Wait a second.

[03:22] Christopher David: We didn't use the prompt at all in that last comment.

[03:42] Christopher David: Interesting.

[03:46] Christopher David: Maybe it sounds a little bit more like a chat GPT unless like our senior developer.

[03:57] Christopher David: It's basically what this to be the system prompt.

[04:23] Christopher David: Okay, let's do this.

[04:26] Christopher David: Messages combine.

[04:52] Christopher David: Okay.

[05:13] Christopher David: And then for the content of this, we're going to call them system prompt.

[05:21] Christopher David: You are fairy and AI agent specializing in analyzing code.

[05:35] Christopher David: The issue body is the first message below.

[05:42] Christopher David: Please respond with the comment you like to add to the issue.

[06:12] Christopher David: Fix missing prompt.

[06:29] Christopher David: And then what I also want to do is build.

[06:39] Christopher David: Context from the combination.

[06:51] Christopher David: I want to use the content.

[07:01] Christopher David: Okay, so I want to do this first.

[07:09] Christopher David: And then I want to build.

[07:11] Christopher David: Context.

[07:18] Christopher David: Build context from those messages.

[07:31] Christopher David: And then we'll say.

[07:35] Christopher David: For additional context.

[07:44] Christopher David: Consult.

[07:48] Christopher David: Following code snippets.

[08:02] Christopher David: Yeah.

[08:09] Christopher David: And then this build context from.

[08:18] Christopher David: Okay, so first of all.

[08:22] Christopher David: Just step through this with some dies and dumps.

[08:25] Christopher David: So I'm going to do.

[08:56] Christopher David: Yeah, here's where like I think we're just going to start by generating and betting for this whole thing.

[09:24] Christopher David: But this is probably where we would want some sort of like re ranking based on the importance like some intermediary LLM to like.

[09:33] Christopher David: Give me based on this huge text the just the text that's most relevant in passing to the query to find the appropriate files for an hour.

[09:44] Christopher David: And then we're going to see how far we can get just doing the base just embed any entire thing.

[09:54] Christopher David: Okay, so context so far.

[10:30] Christopher David: Let's just try passing that is the query.

[10:32] Christopher David: It's really long.

[10:50] Christopher David: I think we'll do one.

[11:00] Christopher David: Let's do one LLM call to.

[11:10] Christopher David: Samarize.

[11:14] Christopher David: And this is not context. This is query.

[11:22] Christopher David: Call this query input.

[11:31] Christopher David: Let's do one LLM call to Samarize query and query input.

[11:37] Christopher David: And then we're going to go to the same file.

[11:47] Christopher David: And then we're going to go to the same file.

[11:52] Christopher David: And then we're going to go to the same file.

[11:58] Christopher David: And then we're going to go to the same file.

[12:03] Christopher David: And then we're going to go to the same file.

[12:15] Christopher David: Samarize this conversation into a.

[12:44] Christopher David: And to three to five sentences.

[12:49] Christopher David: Explaining the types of.

[12:53] Christopher David: Files.

[12:56] Christopher David: We should search for.

[12:58] Christopher David: To.

[13:02] Christopher David: Provide the next.

[13:04] Christopher David: Response.

[13:07] Christopher David: Does that make sense.

[13:10] Christopher David: And then answer equals.

[13:15] Christopher David: The response choices.

[13:17] Christopher David: And then let's.

[13:19] Christopher David: Diant them that.

[13:21] Christopher David: And then just make sure we have this correct.

[13:27] Christopher David: Okay.

[13:30] Christopher David: We want something small that we can pass in as a query.

[13:33] Christopher David: Not that huge.

[13:35] Christopher David: Conversation history.

[13:41] Christopher David: This conversation focuses on implementing a memory stream architecture from the jetter to the ages paper into an AI structure making the AI more efficient and precise in its responses.

[13:57] Christopher David: The first step is creating a basic.

[13:59] Christopher David: Create retrieve update delete memory objects.

[14:01] Christopher David: Then a retrieval mechanism will be added selected according to the AI's.

[14:05] Christopher David: Finally, this process is to be optimized according to the principles of the residency and board development files that will provide pertinent information for the next response include the projects.

[14:14] Christopher David: GitHub repository commit history read me documentation.

[14:17] Christopher David: Okay.

[14:18] Christopher David: No.

[14:35] Christopher David: Close.

[14:40] Christopher David: Focus only on the next step.

[14:51] Christopher David: Not future optimizations.

[14:55] Christopher David: Ignore mentions of video transcriptions.

[15:01] Christopher David: I don't have the next.

[15:04] Christopher David: Let's just see if that's any better.

[15:10] Christopher David: This is fun.

[15:15] Christopher David: The conversation revolves around enhancing the AI's assistant's abilities.

[15:25] Christopher David: I actually don't want it to be summarized.

[15:29] Christopher David: Right.

[15:33] Christopher David: Two to three sentences explain the types of files we just searched for in our code base.

[15:46] Christopher David: There is no requirement to follow for other types of files.

[15:49] Christopher David: For the next step, which is creating a basic.

[15:52] Christopher David: For memory objects, which you'd focus on files related to data management, so it's such as those concerning the implementation of databases or any data persistence they are already in place.

[15:58] Christopher David: We just look at files related to API interactions, particularly with the GitHub API, since these might continue to function.

[16:03] Christopher David: Okay.

[16:04] Christopher David: This is close and might be close enough to give us.

[16:08] Christopher David: What we need.

[16:11] Christopher David: So we're going to just straight up use that as the query.

[16:16] Christopher David: So then.

[16:22] Christopher David: We're going to say the context equals searcher.

[16:27] Christopher David: Search files. Search all files for.

[16:32] Christopher David: We're going to call this the query for the query.

[16:37] Christopher David: We're going to say, searcher equals new.

[16:42] Christopher David: Searcher.

[16:55] Christopher David: Now we don't have search all files.

[16:58] Christopher David: We're just going to.

[17:00] Christopher David: I'll call it query all files.

[17:05] Christopher David: Now we're just going to copy paste this.

[17:10] Christopher David: To not.

[17:13] Christopher David: Take a file ID.

[17:21] Christopher David: Just a query.

[17:25] Christopher David: We do need to embed.

[17:28] Christopher David: Get the embedding.

[17:30] Christopher David: Searcher embeddings do not pull in file ID.

[17:34] Christopher David: We do want to take more than one file.

[17:38] Christopher David: Let's. How I should we take.

[17:40] Christopher David: Maybe we're going to are going to fit.

[17:41] Christopher David: I don't know.

[17:44] Christopher David: Eight.

[17:46] Christopher David: Do we want to summarize the results?

[17:50] Christopher David: No.

[17:53] Christopher David: No.

[17:56] Christopher David: So.

[18:03] Christopher David: That will return results.

[18:07] Christopher David: Theoretically.

[18:19] Christopher David: Which we then need to loop through.

[18:27] Christopher David: The results.

[18:39] Christopher David: The first three lines of the file.

[18:42] Christopher David: Let's just say the first twenty lines of the file.

[18:49] Christopher David: I don't know if that's the correct.

[18:53] Christopher David: I don't know if that's correct.

[18:54] Christopher David: But.

[18:56] Christopher David: Let's see if.

[18:58] Christopher David: Copilot was able to nail that.

[19:25] Christopher David: Okay.

[19:29] Christopher David: It's trying to.

[19:34] Christopher David: Let's go through the results.

[19:38] Christopher David: Not the whole query response.

[19:45] Christopher David: It's returning.

[19:48] Christopher David: Oh, look at this.

[19:50] Christopher David: Is this the rankings?

[19:52] Christopher David: Is this the ranking?

[19:54] Christopher David: Is this the ranking?

[19:56] Christopher David: Is this the ranking?

[19:58] Christopher David: Is this the ranking?

[20:00] Christopher David: Is this the ranking?

[20:04] Christopher David: Is this the ranking?

[20:06] Christopher David: Is this the ranking?

[20:08] Christopher David: Is this the ranking?

[20:10] Christopher David: Is this the ranking?

[20:12] Christopher David: Is this the ranking?

[20:14] Christopher David: Is this the ranking?

[20:16] Christopher David: Is this the ranking?

[20:18] Christopher David: Is this the ranking?

[19:59] Christopher David: a reasonable thing to add.

[20:03] Christopher David: I need the content, not the path.

[20:13] Christopher David: Alright, let's just see what this gives.

[20:15] Christopher David: We'll need a polyp, maybe do a new lookup of the...

[20:24] Christopher David: It's not result file.

[20:27] Christopher David: It result path.

[20:30] Christopher David: And then for result content, we'll need to do content equals,

[20:36] Christopher David: like this get file content from the result of the path.

[20:42] Christopher David: I don't really know what's going on here.

[21:02] Christopher David: But for now, I'm just going to go with it.

[21:09] Christopher David: What happened there?

[21:12] Christopher David: What was that way of opening files?

[21:42] Christopher David: Maybe?

[22:12] Christopher David: Why is this giving this error message?

[22:42] Christopher David: Well, then it was trying to...

[22:58] Christopher David: For each result results, as result.

[23:12] Christopher David: How else?

[24:12] Christopher David: And guessing I should put this in three dashes,

[24:22] Christopher David: and then put the actual code in...

[24:33] Christopher David: Where was I?

[24:43] Christopher David: Put the actual code...

[24:59] Christopher David: New line...

[25:01] Christopher David: Boom...

[25:02] Christopher David: New line...

[25:13] Christopher David: New line...

[25:14] Christopher David: Okay.

[25:16] Christopher David: Maybe?

[25:28] Christopher David: Now assuming those new lines are just kind of...

[25:33] Christopher David: Not duplicative, this looks about right.

[25:43] Christopher David: Except...

[25:52] Christopher David: Change this to...

[25:55] Christopher David: Use the entire file not first 20 lines.

[26:12] Christopher David: I'll keep that code just in case I want to use it another time.

[26:18] Christopher David: But I want to play with that first.

[26:33] Christopher David: Okay, now that might be too big.

[26:43] Christopher David: We're turning that context...

[26:51] Christopher David: To there...

[26:55] Christopher David: Combining with the prompt...

[27:00] Christopher David: This message is used...

[27:02] Christopher David: This message is GB4...

[27:13] Christopher David: You can't just use the code...

[27:16] Christopher David: And then you can just use the code...

[27:20] Christopher David: And then you can just use the code...

[27:24] Christopher David: And then you can just use the code...

[27:27] Christopher David: And then you can just use the code...

[27:30] Christopher David: And then you can just use the code...

[27:33] Christopher David: And then you can just use the code...

[27:35] Christopher David: And then you can just use the code...

[27:38] Christopher David: And then you can just use the code...

[27:41] Christopher David: And then you can just use the code...

[27:44] Christopher David: And then you can just use the code...

[27:47] Christopher David: And then you can just use the code...

[27:50] Christopher David: And then you can just use the code...

[27:53] Christopher David: And then you can just use the code...

[27:56] Christopher David: And then you can just use the code...

[27:58] Christopher David: And then you can just use the code...

[28:01] Christopher David: And then you can just use the code...

[28:04] Christopher David: And then you can just use the code...

[28:06] Christopher David: And then you can just use the code...

[28:08] Christopher David: And then you can just use the code...

[28:10] Christopher David: And then you can just use the code...

[28:13] Christopher David: And then you can just use the code...

[28:16] Christopher David: And then you can just use the code...

[28:19] Christopher David: And then you can just use the code...

[28:22] Christopher David: And then you can just use the code...

[28:25] Christopher David: And then you can just use the code...

[28:28] Christopher David: And then you can just use the code...

[28:31] Christopher David: And then you can just use the code...

[28:33] Christopher David: And then you can just use the code...

[28:36] Christopher David: And then you can just use the code...

[28:39] Christopher David: And then you can just use the code...

[28:42] Christopher David: And then you can just use the code...

[28:45] Christopher David: And then you can just use the code...

[28:48] Christopher David: And then you can just use the code...

[28:51] Christopher David: And then you can just use the code...

[28:54] Christopher David: And then you can just use the code...

[28:56] Christopher David: And then you can just use the code...

[28:58] Christopher David: And then you can just use the code...

[29:00] Christopher David: And then you can just use the code...

[29:03] Christopher David: And then you can just use the code...

[29:05] Christopher David: And then you can just use the code...

[29:07] Christopher David: And then you can just use the code...

[29:10] Christopher David: And then you can just use the code...

[29:11] Christopher David: And then you can just use the code...

[29:13] Christopher David: And then you can just use the code...

[29:15] Christopher David: And then you can just use the code...

[29:16] Christopher David: And then you can just use the code...

[29:18] Christopher David: And then you can just use the code...

[29:20] Christopher David: And then you can just use the code...

[29:22] Christopher David: And then you can just use the code...

[29:24] Christopher David: And then you can just use the code...

[29:26] Christopher David: And then you can just use the code...

[29:28] Christopher David: And then you can just use the code...

[29:30] Christopher David: And then you can just use the code...

[29:32] Christopher David: And then you can just use the code...

[29:34] Christopher David: And then you can just use the code...

[29:37] Christopher David: And then you can just use the code...

[29:39] Christopher David: And then you can just use the code...

[29:42] Christopher David: And then you can just use the code...

[29:44] Christopher David: And then you can just use the code...

[29:46] Christopher David: And then you can just use the code...

[29:48] Christopher David: And then you can just use the code...

[29:50] Christopher David: And then you can just use the code...

[29:52] Christopher David: And then you can just use the code...

[29:54] Christopher David: And then you can just use the code...

[29:56] Christopher David: And then you can just use the code...

[29:58] Christopher David: And then you can just use the code...

[30:00] Christopher David: And then you can just use the code...

[30:02] Christopher David: And then you can just use the code...

[30:05] Christopher David: And then you can just use the code...

[30:07] Christopher David: And then you can just use the code...

[30:10] Christopher David: And then you can just use the code...

[30:12] Christopher David: And then you can just use the code...

[30:14] Christopher David: And then you can just use the code...

[30:16] Christopher David: And then you can just use the code...

[30:18] Christopher David: And then you can just use the code...

[30:20] Christopher David: And then you can just use the code...

[30:22] Christopher David: And then you can just use the code...

[30:24] Christopher David: And then you can just use the code...

[30:25] Christopher David: And then you can just use the code...

[30:27] Christopher David: And then you can just use the code...

[30:29] Christopher David: And then you can just use the code...

[30:32] Christopher David: Well, I'm not even using that client...

[30:34] Christopher David: And I do in the bare PHP...

[30:39] Christopher David: Oh, yeah...

[30:47] Christopher David: Of 30 seconds...

[30:50] Christopher David: Change the...

[30:52] Christopher David: Guzzle call to use a custom timeout of 90 seconds...

[31:01] Christopher David: Ah...

[31:06] Christopher David: Timeout before the post...

[31:17] Christopher David: Okay, let's try that...

[31:31] Christopher David: Yeah, a whole bunch of optimization possible...

[31:35] Christopher David: With prompt engineering...

[31:36] Christopher David: And trying out different ways of passing in things...

[31:39] Christopher David: And lots of we can learn from...

[31:41] Christopher David: People that have been doing this already...

[31:43] Christopher David: Like, like...

[31:44] Christopher David: Like, like...

[31:45] Christopher David: Like, like...

[31:46] Christopher David: Like, like...

[31:47] Christopher David: Like, like...

[31:48] Christopher David: Like, like...

[31:49] Christopher David: Like, like...

[31:50] Christopher David: Like, like...

[31:51] Christopher David: Like, like...

[31:52] Christopher David: Like, like...

[31:53] Christopher David: Like, like...

[31:54] Christopher David: Like, like...

[31:55] Christopher David: Like, like...

[31:56] Christopher David: Like, like...

[31:57] Christopher David: Like, like...

[31:58] Christopher David: Like, like...

[31:59] Christopher David: Like, like...

[32:00] Christopher David: Like, like...

[32:01] Christopher David: Like, like...

[32:02] Christopher David: Like, like...

[32:03] Christopher David: Like, like...

[32:04] Christopher David: Like, like...

[32:05] Christopher David: Like, like...

[32:06] Christopher David: Like, like...

[32:07] Christopher David: Like, like...

[32:08] Christopher David: Like, like...

[32:09] Christopher David: Like, like...

[32:10] Christopher David: Like, like...

[32:11] Christopher David: Like, like...

[32:12] Christopher David: Like, like...

[32:13] Christopher David: Like, like...

[32:14] Christopher David: Like, like...

[32:15] Christopher David: Like, like...

[32:16] Christopher David: Like, like...

[32:17] Christopher David: Like, like...

[32:18] Christopher David: Like, like...

[32:19] Christopher David: Like, like...

[32:20] Christopher David: Like, like...

[32:21] Christopher David: Like, like...

[32:22] Christopher David: Like, like...

[32:23] Christopher David: Like, like...

[32:24] Christopher David: Like, like...

[32:25] Christopher David: Like, like...

[32:26] Christopher David: Like, like...

[32:27] Christopher David: Like, like...

[32:28] Christopher David: Like, like...

[32:30] Christopher David: Like, like...

[32:31] Christopher David: Like, like...

[32:32] Christopher David: Like, like...

[32:33] Christopher David: Like...

[32:34] Christopher David: Like...

[32:35] Christopher David: Like...

[32:36] Christopher David: Like...

[32:37] Christopher David: Like...

[32:38] Christopher David: Like...

[32:39] Christopher David: Like...

[32:40] Christopher David: Like...

[32:41] Christopher David: Like...

[32:42] Christopher David: Like...

[32:43] Christopher David: Like...

[32:44] Christopher David: Like...

[32:45] Christopher David: Like...

[32:46] Christopher David: Like...

[32:47] Christopher David: Like...

[32:48] Christopher David: Like...

[32:49] Christopher David: Like...

[32:50] Christopher David: Like...

[32:51] Christopher David: Like...

[32:52] Christopher David: Like...

[32:53] Christopher David: Like...

[32:54] Christopher David: Like...

[32:55] Christopher David: Like...

[32:56] Christopher David: Like...

[32:57] Christopher David: Like...

[32:58] Christopher David: Like...

[32:59] Christopher David: Like...

[33:00] Christopher David: Like...

[33:01] Christopher David: Like...

[33:02] Christopher David: Like...

[33:03] Christopher David: Like...

[33:04] Christopher David: Like...

[33:05] Christopher David: Like...

[33:06] Christopher David: Like...

[33:07] Christopher David: Like...

[33:08] Christopher David: Like...

[33:09] Christopher David: Like...

[33:10] Christopher David: Like...

[33:11] Christopher David: Like...

[33:12] Christopher David: Like...

[33:13] Christopher David: Like...

[33:14] Christopher David: Like...

[33:15] Christopher David: Like...

[33:16] Christopher David: Like...

[33:17] Christopher David: Like...

[33:18] Christopher David: Like...

[33:19] Christopher David: Like...

[33:20] Christopher David: Like...

[33:21] Christopher David: Like...

[33:22] Christopher David: Like...

[33:23] Christopher David: Like...

[33:24] Christopher David: Like...

[33:25] Christopher David: Like...

[33:26] Christopher David: Like...

[33:27] Christopher David: Like...

[33:28] Christopher David: Like...

[33:29] Christopher David: Like...

[33:30] Christopher David: Like...

[33:31] Christopher David: Like...

[33:32] Christopher David: Like...

[33:33] Christopher David: Like...

[33:34] Christopher David: Like...

[33:35] Christopher David: Like...

[33:36] Christopher David: Like...

[33:37] Christopher David: Like...

[33:38] Christopher David: Like...

[33:39] Christopher David: Like...

[33:40] Christopher David: Like...

[33:41] Christopher David: Like...

[33:42] Christopher David: Like...

[33:43] Christopher David: Like...

[33:44] Christopher David: Like...

[33:45] Christopher David: Like...

[33:46] Christopher David: Like...

[33:47] Christopher David: Like...

[33:48] Christopher David: Like...

[33:49] Christopher David: Like...

[33:50] Christopher David: Like...

[33:51] Christopher David: Like...

[33:52] Christopher David: Like...

[33:53] Christopher David: Like...

[33:54] Christopher David: Like...

[33:55] Christopher David: Like...

[33:56] Christopher David: Like...

[33:57] Christopher David: Like...

[33:58] Christopher David: Like...

[33:59] Christopher David: Like...

[34:00] Christopher David: Like...

[34:01] Christopher David: Like...

[34:02] Christopher David: Like...

[34:03] Christopher David: Like...

[34:04] Christopher David: Like...

[34:05] Christopher David: Like...

[34:06] Christopher David: Like...

[34:07] Christopher David: Like...

[34:08] Christopher David: Like...

[34:09] Christopher David: Like...

[34:10] Christopher David: Like...

[34:11] Christopher David: Like...

[34:12] Christopher David: Like...

[34:13] Christopher David: Like...

[34:14] Christopher David: Like...

[34:15] Christopher David: Like...

[34:16] Christopher David: Like...

[34:17] Christopher David: Like...

[34:18] Christopher David: Like...

[34:19] Christopher David: Like...

[34:20] Christopher David: Like...

[34:21] Christopher David: Like...

[34:22] Christopher David: Like...

[34:23] Christopher David: Like...

[34:24] Christopher David: Like...

[34:25] Christopher David: Like...

[34:26] Christopher David: Like...

[34:27] Christopher David: Like...

[34:28] Christopher David: Like...

[34:29] Christopher David: Like...

[34:30] Christopher David: Like...

[34:31] Christopher David: Like...

[34:32] Christopher David: Like...

[34:33] Christopher David: Like...

[34:34] Christopher David: Like...

[34:35] Christopher David: Like...

[34:36] Christopher David: Like...

[34:37] Christopher David: Like...

[34:38] Christopher David: Like...

[34:39] Christopher David: Like...

[34:40] Christopher David: Like...

[34:41] Christopher David: Like...

[34:42] Christopher David: Like...

[34:43] Christopher David: Like...

[34:44] Christopher David: Like...

[34:45] Christopher David: Like...

[34:46] Christopher David: Like...

[34:47] Christopher David: Like...

[34:48] Christopher David: Like...

[34:49] Christopher David: Like...

[34:50] Christopher David: Like...

[34:51] Christopher David: Like...

[34:52] Christopher David: Like...

[34:53] Christopher David: Like...

[34:54] Christopher David: Like...

[34:56] Christopher David: Alright, let's have Fairy start implementing this plan.

[35:00] Christopher David: Season.