Transcription: OpenAgents Episode 113 - Agent Memories & Reflections

Source: https://x.com/OpenAgentsInc/status/1816269923414327630 Wiki source: https://raw.githubusercontent.com/wiki/OpenAgentsInc/openagents/Video-Series.md Media title: OpenAgents - Episode 113: Agent Memories & Reflections We use AutoDev to impl... Upload date: 20240725 Transcription model: gpt-4o-mini-transcribe Generated at: 2026-06-01T17:44:26Z

Machine-generated transcript. Review speaker labels and wording before using this as quote-grade source material.

[00:00] Speaker: So that's working. We've got a canvas of AI-generated insights represented spatially based on their vectors. You can click on it and it'll kind of put it down there. And this is cool. You got a little basic tags. But we're now running into this issue of like, what's actually the best way of structuring this for retrieval? And we're not going to kind of just spitball here. We're going to use some papers from Archive. I think there are more recent papers that touch on this, but we're going to start with one that I have been familiar with in the past called the Generative Agents paper. And it would be fun, I think, for you to watch me use AutoDev to implement the relevant pieces of this paper all in this video. I specifically did not do any prep for this video. I want you to watch me implement this from scratch. So just to review, we're building a knowledge graph representing visually and spatially the knowledge that this AutoDev agent has about a particular code base. And so offline between yesterday's video and today, I made a tool in our code base called RecordObservation. I've been using Observation and Insights interchangeably, but you can record an observation and remember the insights. So we can save and retrieve insights. And then we put that into a Claude 3.5 Sonnet chat using the Vercel AI SDK and a system prompt that says, basically, you are AutoDev. You have access to these tools. The tools that I added were RememberInsights, so perform a similarity search on previously recorded insights and query files by vector. Query files by vector similarity based on their summaries. And so I had a different version of this down here where I had it go analyze, step through the code base to generate these insights. You can record observations, basically telling it to loop through and use that record tool to save things to the database. And so we've got now this database of 50 or so observations, type, architecture, the project uses Zust for state management, blah, blah, blah, as well as the vector embedding. And then we've got a basic algorithm that collapses the 1,536 dimension array of the vector to a three dimensional array to visualize it here. So if we did nothing but connect this database to retrieval, like do basic cosine similarity search using that embedding to like get a few pieces of relevant context, that could like, you know, probably improve the queries. But I'm really interested in identifying what the proper architecture for the code base knowledge is because, you know, having seen people say, hey, you know, I'd be willing to pay thousands of dollars per month for something that can just like kind of take over my coding, resolve PRs automatically. And to do that, you're going to need like a level of insight over the code base that I just haven't seen people really master the depth of knowledge necessary. So for things like every single commit that you make as a developer, it should be able to kind of study those commits and like see how you do things, match up errors that you've had and like problems that you've had or chats that you've had with an LLM, how that matches up against commits that resolved it. There's so much data in both an individual GitHub repository as well as the associated chats from that that ideally an LLM should have access to all that data so it can kind of like learn over time. And my speculation is that we'll get at least an initial approximation of a good kind of memory and retrieval and reflection systems based on this generative agents paper that I remember reading like nine months ago. But aside from that, so I just basically have this idea that this is like a thing that I think is next logical for us to implement as a starting point to having a more structured approach to building a really deep, deep knowledge over the code base. And we're just going to start from here. So let's try it. Generative agents. Let's grab the PDF. They set up this whole little simulated environment and they had agents kind of learn over time and remember their interactions with each other. You know, the simulation itself was pretty cool, but more important was the like memory module stuff that they actually did. So here's what I'm gonna do. I'm gonna set up a new project for this. I think we'll just go into the same one that we have here. So the first thing I want to do is grab this and have Claude summarize this in a markdown doc. So let's say summarize this PDF in a markdown document focusing on the memory system. Memory reflection systems. These will be instructions to a junior dev LLM that will implement it. We want to understand the concepts, the general concepts of the paper and details about memory, etc. such that we can implement it in our app. All right, let's start with this. This is a thing that I would like to pull into the auto dev codebase working with PDFs. Not hard to do. We just need to take the time to do it, but not the priority right now. Okay, so we'll start by just reading this. Generative agents, key concepts and implementation guide. Overview. Generative agents are AI powered agents that simulate believable human behavior. They use a novel architecture that combines large language models with mechanisms for memory, reflection and planning to create coherent, context aware behavior over time. Yeah, we want that for our coding agents. So there will be some percentage of this that's relevant to what we're doing and we'll play with it. So core components. Memory stream, a comprehensive record of the agent's experiences stored as natural language descriptions. Each memory object contains a natural language description, creation timestamp, most recent access timestamp. Retrieval functions. Select relevant memories based on recency. More recent memories score higher. Implement as exponential decay function. Importance distinguishes mundane from core memories. Use LLM to rate importance on a scale of one to 10. Prompt rate the poignancy of this memory from one mundane to 10 extremely poignant. Poignant. Relevance related to current situation. Use LLM to generate embedding vectors from memories. Calculate cosine similarity between memory and current situation. Score equals recency. Dude, I need to learn some math. I don't know what that thing does. This recency times recency plus that importance times importance plus that relevance times relevance. Okay. Reflection. Generate higher level thoughts periodically. Yeah. Trigger reflection when some of importance scores exceeds a threshold, e.g. 150. Process. Identify reflection questions based on recent experiences. Use retrieval function to gather relevant memories. Prompt LLM to extract insights citing specific memories as evidence. Store reflections as memories. Okay. So for here, so we've got 50 different things here ish. And probably a bunch of this is either like duplicative or there should be sort of like higher level things like this app uses the latest Next.js libraries. It uses Tailwind. It uses MDX. It uses ChatUI components or whatever equivalent we're using. And then to just have that as like a one liner that can then be linked to other memories. So if the LLM wants to dive into that, it can. But, you know, being smart about what context gets passed, there's probably some like distillation that should be done here while preserving the additional data in sort of linked data. Okay. Planning. Create high level daily plans. Recursively decompose plans into finer grained actions. Store plans in memory stream. Update plans based on new events or interactions. Yeah. So yeah, I want all of these things. Implementation steps. Set up a database to store the memory stream. To implement the retrieval function with recency, importance and relevance scoring. Create a reflection system that periodically generates high level thoughts. Develop a planning system for creating and updating agent plans. Integrate with an LLM GPT 3.5 turbo for generating responses and processing natural language. Implement a main loop that perceives the environment, retrieves relevant memories, generates actions or responses, updates the memory stream, triggers reflection and planning as needed. Key considerations. Balance between short-term reactivity and long-term coherence. Efficient memory retrieval and summarization for large memory streams. Handling conflicting memories or plans. Tuning hyperparameters, e.g. reflection threshold, retrieval weight, function weights. By implementing these components, you can create generative agents that exhibit believable context-aware behavior. Okay. So this is great. Now what I'm going to do is I'm going to drop some of the docs that we have already. Like database knowledge, repo map. I'm not really doing that so much, but maybe that's relevant. Let's just try dumping in these. Let's do this. Oops. Let's do the readme. And one, two, three, four. I think I can do a maximum of five files at a time. Write an updated markdown doc as an artifact with pretty much all of that data, but also contextualizing this in terms of our AutoDev project. We need to apply those four things to semi-automated. No, I'm just going to call it automated software development. We do not have that repo map implemented. That was an idea to take of like, do the same kind of things that ADER is doing, like using tree sitter to build an abstract syntax tree of a code base. I'm actually interested now to see how far we can get without doing that. My speculation is that just putting into place this memory system with different insights being vector embedded and then letting it like retrieve the proper context and feed it the proper context will get us far enough. We can experiment with ASTs later. Okay. I put way too much stuff in here. Okay. But I don't want a general document. I want you to give me specifically the generative agents paper summary again, just knowing that we will be applying it to this AutoDev thing. We want to use those memory reflection plan, etc. and extend our knowledge implementation. Extend slash replace our knowledge implementation with that. First, give me a comprehensive markdown doc for that, but don't duplicate what's in the other markdown docs I gave you

[19:59] Speaker: The system will be capable of building a deep understanding of code bases over time and providing increasingly valuable insights and assistance to developers. OK, so I like this document. It's not perfect, but we'll use it as a starting point. So I'm going to throw this into docs. What am I calling this? Generative Agents. We'll call that initial, and then do I want to preserve anything from that initial artifact? I think I'm just going to keep this whole thing here. Still learning Neovim. Not a wizard at it, but it feels nice. OK. So let's see what that looks like in our repo. Yeah, I just want something that if I can later just feed into an LLM, it'll be somewhat comprehensive. OK, that's pretty cool. We could probably just feed that into Claude or Autodev and have it start building stuff. But it's probably worth consulting the code, because I think they implemented this in Python, maybe. AI Town LOL. Inspired by Generative Agents. That's cool. So this looks like it's in React. That's good. Here's a question. Can I just add this repo to Autodev? Hang on, hang on, hang on. I don't know. That's a little risky, add repo. Let's try it. I don't know if it'll work for repos that aren't mine. So let's just try it. A16Z. Thank you, Andreessen Horowitz. AI Town. Let's just see. Let's just see. Is it main branch? I've got to fix the zindex of this canvas. It's overwriting everything. OK. Summarize the project based on the readme. Be pretty cool if this works. No, I think my token's expired. Hang on. I haven't put the GitHub OAuth refresh token stuff in there yet. OK. Summarize this repo based on the readme. Maybe. Mwahaha. Uh-oh. Do my table here reference a repo? Oh, it does. OK, thank God. Can you traverse the hierarchy and tell me what files implement relevant code from the generative agents paper, like the memory, reflection, planning modules? Let's see if it can answer this. OK, Convex slash Agent has conversation, embeddings cache, memory schema. Might be able to just have some code that we yoink from this. Oh, they got memories. So I'm happy that this is all React. I'm mostly wondering now, like, how faithful of an implementation of the generative agents paper does this have? Does it matter? Rank and touch memories. I don't know what an internal mutation is. So they got some proprietary Convex-y thing. I don't know. Based on the contents of the memory.ts file, I can confirm that this file implements several key aspects of the memory module as described in the generative agents paper. Interesting that I did not pass in the context of the generative agents paper. Maybe that's in its training data now. This is Sonnet 3.5. Here's a summary of the relevant implementations. Memory storage and retrieval. This file defines a memory type and functions to insert and query memories. Memories are stored with attributes like description, importance, and last access time. The insert memory function is used to add new memories to the database. Memory embedding. Memories are associated with embeddings, which are likely vector representations of the memory content. Duh. The fetch embedding function is used to generate these embeddings. Memory search implements a vector search to find relevant memories based on a query embedding. It uses a combination of relevance, recency, and importance to rank memories. Memory importance. The calculate importance function assigns an importance score to memories, similar to the concept of memory salience in the paper. Salience? Does it use the word salience? It does not use the word salience. Okay. Does not use the word salience. The reflect on memories function implements a reflection mechanism where the agent generates insights based on its recent memories, similar to the reflection process where agents periodically review and synthesize their experiences. Conversation memory. Memory ranking. Okay. These implementations closely align with the memory and reflection modules described in the generative agents paper. The code shows a sophisticated system for storing, retrieving, and reasoning over memories, which is crucial for creating agents with persistent and evolving knowledge. To find implementations related to the planning module, we might need to look at the files. Okay. I don't know if this will work. I have not tried this yet, but ideally I can pop over to Autodev, loads the knowledge in there. Nice. Hopefully I can continue the same chat without breaking anything. Continue that. Compare that to the write-up we have in our code base I just connected you to. Analyze the docs slash generative agents paper file and see what's missing. I don't know. Please, please, please, please, please. No. Did I not push it up? Docs generative agents.md. Underscore. Motherfucker. Okay. I apologize. Stop. I gave you. Wait, is it going to traverse it? I don't have a stop button. Docs slash generative. No, no, no, no, no. Thank you for providing the correct path. Okay. There's the file. Okay. It's still yapping about that previous one. Now, did I confuse it by saying that? Continue. Certainly. I'll compare the implementation we reviewed earlier in the memory.ts file with the contents and plans outlined in the docs generative agents file. I'll analyze what's already implemented, what's missing, and what could be improved. Memory stream. Implemented. The memory's table in the document aligns well with the memory type in the code. Both include content, importance, and timestamps. Partially implemented. The code uses embeddings for memories, which is mentioned in the document. Missing. The document suggests a memory stream class, which isn't present in the code. Class, that's more of a Python way of doing things. No, this seems fine. If it's got all the right functionality, it's good. Retrieval function. Implemented. The search memories function in the code implements a vector search, which aligns with the relevant scoring method mentioned in the document. Partially implemented. The code considers recency and importance in ranking memories, but the exact formula differs from the document suggestion. Missing. The document suggests using an exponential decay function for recency, which isn't... Okay. It doesn't use exponential decay function for recency. Maybe we don't need it. I don't know. Reflection. Implemented. The reflect on memories function in the code aligns with the reflection process. Partially implemented. The code triggers reflection based on an importance score threshold. The document suggests generating reflection questions based on recent experiences, which isn't explicit. And then this is just our thing. Okay. Planning might just might be in a different file. Areas for improvement. Implement the memory stream class. No, I don't think we need that. Refine the memory retrieval function to more closely match the formula suggested in the document. Okay, so it sounds like there's just not much missing, at least in the functionality that we're looking at. Okay, well, let's just see if there's... Let's go back to traversing this. And then to maybe save some tokens, let's see if I can point it at the right file. So it was looking in what? Convex? Okay, so this is good to know. Let me... I'll save that as a path. Embeddings cache. Conversation. Okay, the structure of conversations. That's cool. And I'll definitely want that. Okay, let's see if there's any planning module in here. Okay, so they are calling it thinking, not planning. This is good. So what I'm going to say is... Check out the file path at architecture.md to see if you can learn anything, like where we might find a planning thinking modules. Okay, planning slash thinking. While there isn't an explicit mention of a long-running task of a planning module, the agent's decision-making process is described in the agent loop. Agent.tick method can read and modify game state as time progresses. When the agent needs to perform a long-running task, like talking to an LLM, it calls start operation with a reference to a convex function. The agent can observe the new game state and continue to make decisions after an operation is complete. Okay. Memory system. Okay, while there isn't a dedicated planning module as described in the generative agents paper, the decision-making process seems to be distributed across the agent loop, the conversation system, and the memory system. Should there be a dedicated planning? Let me go back to our document. Core components. The memory stream, retrieval function, reflection, planning. Okay, so this is good to know. I was mainly interested in this because I think that the original code base is in Python. That was in React. So, like, if there's probably just some code there to, like, harvest, but I also want to be careful not to, like, adopt their opinions on things, which it seems kind of opinionated in particular directions. Let's go back to the archive paper and take a look. Where's the official one? Is it this? Reverie. Yeah, this sounds right. Reverie from Westworld. Okay, you know what we're going to do? We're going to take this and we're going to add this as a repo. I'm liking this. Okay, so add repository. Generative agents. Man. Now we are in the main implementation of the generative, like the main official slash reference implementation of the generative agents paper. It's in Python. I want to see how they implement planning and generally the other four.

[39:59] Speaker: The four main modules. Start in the readme, and then look through the hierarchy and summarize relevant stuff. Based on the readme, the main simulation code is located in the reverie slash backend server directory with the main file being reverie.py. Let's examine the directory structure to find the relevant modules for planning and the four main components. Now we're getting closer to the core implementation. Let's examine the cognitive modules. Ooh, cognitive modules. Converse, execute, perceive, plan, reflect, retrieve. plan.py, this file likely contains the implementation of the planning module, which is responsible for creating and managing the agent's plan. Perceive, probably implements the perception module, handling how the agent perceives its environment and events around it. Yeah, some of this might be relevant to like being active in an actual world, which is less relevant. Maybe some of it's relevant to like the GitHub environment. I don't know. Retrieve, memory retrieval, reflect, execution, converse, plan. Long, can we handle all this? That's messed up. Based on the contents, here's a summary of how the planning module is implemented and how it relates to the other main modules. Long-term planning. The long-term planning function handles daily planning. It generates a wake-up hour and creates a daily plan using this, an hourly schedule, yeah. Determine action, reactive planning. Okay. The implementation seems more focused on daily planning and reactive behavior rather than explicit long-term goal setting. Less structured than described in the paper. Okay. Now, again, read our docs slash generativeagents.md file and add sections to it. Again, read our docs slash generativeagents.md file and add sections to it for... Okay, I'm gonna take a break. I'll be right back. Okay, let's try to simplify this. I wanna get to implementing. Let's try edit our docs slash generativeagents.md paper with doc to add a more specific plan to implement it in our code base, referencing existing files by their file paths. Something that we can feed back in and it can just start kind of doing it. I might need to log out and in. Looking forward to releasing this. You know, I think we're not loading tool result history. I'm gonna do a new chat. Actually, let's just see if this works. I don't think we're properly loading the messages into the used chat. Dang it. Okay. Let's try a new chat here. Edit our generativeagents.md doc to add a more specific plan to implement it in our code base, referencing existing files. Use hierarchy to traverse. Yeah, we lost the context of the previous conversation because even though I'm populating it from the database to the screen, I don't think I'm loading it into the used chat properly. I know I'm not loading the results of tools. There's just some stuff we gotta add to get this fully thorough. But let's see what this does. Now that we have the content of the file, I'll suggest some edits. Here's a suggested update. So set up the memories table in the database. Okay, good. So consult with the hierarchy. Set up the memories table. Implement the memory stream class. Yeah, even though I wanna learn from the AI town thing from A16Z, I think maybe we'll just kind of keep it simpler and have it just do what it thinks makes sense based on this, and then we can harvest their code later if we want to. So this is a pretty good plan. Okay, so here's what I'm gonna try doing. I'm gonna go into the generative agents doc. At the bottom, I'm gonna put the implementation plan. I'm gonna save that and commit that. Update implementation. Okay. Now, in a fresh conversation, I wanna say what? I wanna say, consult our docs slash generativeagents.md file. We are implementing the plan you find at the bottom. First, let's implement, do the first step. Out of curiosity, what happens if I just do this? Do the plan. Certainly, I'll start by consulting the docs slash generativeagents file to understand the plan and then proceed with implementing the first step. Failed to find contents. Oh, I'm at the wrong repo. Let's try that again. New chat. Auto dev. Okay, let's go. Thank you for providing the contents. I have reviewed the implementation plan and I'll proceed with the first step as requested. The first step is set up the memories migration, the memories table in the database. Let's start by updating the migration file. There is no init.sql. Where's it getting that from? That is, I think, a hallucination. Oh, look. It repaired. It was like, let me check the actual folder. Oh, wait, there is no init. Since we're adding a new table, we'll add a new migration file. We'll name it using the current date and blah, blah. Let's create a new migration file called blah, blah, blah. Okay, so I'm actually going to use the Supabase migration new create memories table. Yeah. Okay. And then here I'll put this. Does that look great? UUID, repository ID, type, content, importance, embedding, created at last accessed. Okay, Supabase, what, DB push, maybe? Yeah, done. And we see it right there. Memories. Okay, so let's go into our readme. Oops, not our readme. Let's go into our generative agents. Actions taken. Created a new table memories in the database schema. The migration file is located at Supabase slash migrations slash. Still learning my way around NeoVim. I need to paste. Okay. So I'm going to edit my prompt and I'm going to say consult our docs generative agents. We're implementing the plan you find at the bottom. Note the actions taken at the bottom. Do the next needed step, giving me specific instructions here when files need to be edited. Then include a line we should add to actions taken once I do those things. Okay, so let's. And eventually this is going to be fully automated. Update doc. Okay. So without me even knowing what the next step is. No. I guess I'll keep it going in this conversation. Let's see what happens. Oh, I should put .SQL there. Based on the implementation plan and the actions taken section, the next step we need to take is to implement the memory stream class. Here are the specific instructions for this step. Create a new file, lib memory stream in your project directory. All right, I will do so. Lib memory stream. Move this up a little bit. In memory stream, we're going to paste this. Although that's not the kind of client. I guess it actually doesn't matter. I think I can just pull this in from. Is it a browser client that I want? All right, I think I'm just going to leave that as it is for now. Let's see what this does. Okay, after implementing this class, you should add the following line to actions taken. Note that this is a basic implementation. This is like a thing that I would like to have added to observations or insights. This stuff here. Okay. You need to enhance it with more specific scoring based on recency, blah, blah, blah. Okay. Okay, so what I'm going to do is I'm going to put both of those things. I'll put the comment just next to it. So I'll delete that and then I'll put this here. And then let's see what that looks like. Memory stream, blah, step two. Okay, let's go here. Make sure this looks nice. Yeah. Okay, now out of curiosity, let me go to a new chat starting fresh, picking the right repo. Okay, it's giving some incorrect syntax for the OpenAI usage. That's okay to tweak. Based on the implementation steps and actions taken so far, the next step is to enhance the memory retrieval function in the memory stream class. Is that what it, what actually was the next step? Oh, develop the memory retrieval. I see. So it did a basic one. Now it's iterating it. Use the existing lib slash OpenAI. I don't have a lib slash OpenAI. I don't think. No. So this is not the right way to create embeddings. This is the way to create multiple embeddings. And then where did we create singular embeddings? We did that somewhere else. Maybe in one of the tools. Although I could probably have it do this. Now I'm going to say, thanks, but that is not the right syntax for creating embeddings or using OpenAI. Please consult tools slash record observation dot TS. What am I not doing here? I'm not copy pasting it in.

[59:59] Speaker: and give me instructions again. I apologize for the confusion. You're right. First, let's view record observation. Now let's update the instructions. Okay, ordinarily, I would want it to give me the full file, or I'm so excited for this to be done in patches that I can just review and approve in a PR. Okay, for now, I'll do some copy pasta. Open the file, memory stream. These imports at the top. Oh, look, it's even giving me the right import, I think. Update the memory stream class constructor. Nice. Classes, huh? Replace the getEmbedding method with the following. I don't see getEmbedding. There's no getEmbedding here. Update the retrieve memories as follows. Create a new file, Supabase functions, retrieve memories. I'm guessing this needs to be this.supabase. I'm guessing this needs to be this.supabase. I'm guessing this needs to be... Promise memory cannot find memory. Where is that even defined? What? Create and replace function memories. Okay, so let's do... What did I call these in here? Functions. Create the name of the function. Okay. Supabase new migration. Create retrieve memories function. Supabase migration new. Okay. Drop that there. Let's push that. Supabase DB push. Yeah. Okay, update the actions taken section with this. Except that's not the right path. Although I can probably tell it to give me the right path. Three and four. Watch this. I'll have it do two things. Look at lib memory stream and fix the missing memory type error. And then look at the Supabase migrations folder to get the correct path to that function migration and tell me that so I can paste it into the actions taken doc. Gotta automate a bunch more of this stuff. Okay, throw this at the top. Lib memory stream. Looks good. Additionally, you should add another action. Okay, I can do that. I don't know if I will do that. Okay. Nice. That, I'm not going to do that. Okay, so I'll push that up and then I'm going to say the exact same prompt in a new chat. This is a loop that I will be automating. Consult our doc slash generative agents file. We're implementing the plan you find at the bottom. Note the actions taken. Do the next needed step giving me specific instructions here when files need to be edited. Then include a line we should add to actions taken once I do those things. I'm again in the wrong repo. I gotta persist the chosen repo. Let's try that again. Where did my knowledge go? There it is. I'm having fun. This is fun. It feels good. I'm going to add more like cool visualizations here. Like see things light up as it's working. Update lib slash files. Man, it's just making shit up. Is it? Lib slash files. There is no lib slash files. Okay, there is no lib slash files or file tree. So you need to give me the entire code that should be in there. Consult other code in my project if you need to learn more about how stuff is structured. That kind of stuff I could be putting in the system prompt. It assumes that it's an SRC. Actually, there's something in my system prompt that says SRC. That's why it's assuming it. Gotta remove that. Source lit. No, no, no, no, no. Okay, so let's actually fix this now. Let me go to my prompts. Give me back the same file without any reference to the SRC folder because that's confusing. Assume I have a Next.js app router project with app and components, etc. in the root folder and use that for any examples. Rewrite the entire thing accordingly. Don't comment anything out, you lazy bastard. Okay, so for the creation of the new file, it gave me the wrong path, but I'm just gonna put this here. The content looks like it'll be the same. So I'm gonna call that file utils.ts. Let's modify the component that handles file viewing. What is it doing? Creating memories from file content and metadata. Modify components slash file tree to trigger memory creation when files are viewed or modified. Okay, so this plan isn't so good because it's referencing stuff that doesn't exist. It's like mostly accurate. Like that should exist, but it does not. Should we modify the plan? Since we don't have a file tree component, let's assume we have a workspace component that handles file interactions. This wants to create a memory from file on file click. Let's think about this. Okay, we may need to adjust the plan a little bit manually here. Let's just take a look at what we've done. We've created a new table called memories. Implemented a basic memory stream class with methods for adding memories, updating access timestamps, and retrieving memories. Note this is a basic implementation. You'll need to enhance the retrieve memories with more sophisticated scoring. Then we updated memory stream to use the correct index for creating embeddings and using OpenAI based on the implementation and tools record observation. Created a new SQL function retrieve memories for sophisticated memory retrieval based on recency, importance, and relevance. Okay, I'm gonna try this. I'm gonna say we need to update our plan in doc slash generative agents.md. Step four mistakenly assumes we have an existing lib files.ts and components slash file tree.tsx, but we do not. We eventually want to create or update memories or create memories based on updates of files, but for now, we mainly want to generate memories of files, at least one memory of per file, which can begin as a one to two line summary of the file contents. Later we'll be having the agent reflect on these memories to derive higher level insights. Rewrite the steps four plus of the plan accordingly. The idea is we want to have the agent develop an understanding of the codebase based on memories slash observations of all files plus reflection on those, factoring in user intent as derived from README and other documentation. Give me back the entire updated plan in a markdown document here, preserving steps we've already completed. Okay, that might do it. Let's do a new conversation. I'm gonna refresh my login token just for good measure. Right now that expires every hour because I didn't implement something. All right, we got a new chat. Auto dev. Let's go. Certainly, I'll update the plan focusing on rewriting steps four and beyond to reflect our current goals. Here's the current content. Step five still references files.ts that doesn't exist and components file tree that doesn't exist. You need to reimagine steps five plus to be logical based on step four. Don't feel compelled to include any of the rest of it. Think through it newly. Rewrite it and don't say completed on stuff we already did. That's handled in actions taken. Okay, let's see. Generate memories for existing files. Implement file change detection. Automate memory generation for file changes. Develop a basic reflection system. Implement project understanding. Okay, decent, but let's simplify that slash move more advanced things later in the flow. For example, file change detection isn't needed now. I want to get the simplest version implemented ASAP. So step four is good. Postpone five and six. I do want reflection system, then query system. Postpone everything else. Rewrite that plan.

[01:19:59] Speaker: You're absolutely right. Okay, this is better. Set up the memories table, implement the memory stream class. Add retrieval, recency, and importance relevance. This doesn't exist. Generate memories for existing files. Create a script to iterate through all files in the repository. For each file, generate a one to two line summary using an LLM. Store these summaries as memories in the memories table. Implement this in a new file. Develop a basic reflection system. Alternately, I could skip past memories of existing files and just reuse, at least for now, the insights here. Let's just see where else it's going with this. Develop a basic reflection system. Implement a method to analyze the existing file memories. Generate high level insights about the codebase structure, patterns, potential implements. Store these reflections as new memories with a different type. Implement a simple query system. Implement functions to query the accumulated memories and reflections. Okay, I think the simple query system needs to use a variation of the tool query file by vector. Okay, do I want to keep refining that plan or do that cross that bridge when we come to it? Do I want to do this as scripts? Okay, let's just do the next step. New chat, autodev, do the next needed step, specifically the script to make memories from files. That's not what I wanted to use. I wanted to... Use this syntax here. Wrong syntax for embedding. Please use what you see in tools slash record observation file, specifically generate embedding at the top there. Pull that code in. And this is generating... How is this getting the files? Read dir sync the root directory. And how does that traverse the directory? Are you assuming the repo is local? Repos are hosted on GitHub. This is not using the same patterns that we're using like that SDK, but I am just kind of curious to see if this will work. All right, so generate file memories. We'll just try it. Ah, shit. One thing I can't figure out with Neovim is occasionally when I'm pasting something, it'll just like fart. I think it might be a console memory thing. Like it might be an iTerm problem. Fuck. Okay. Superbase. All right, let's just try some stuff. Lib. Utilities. Superbase clients, maybe? I don't know. I don't know. Let's assume there's a generate embedding. That's annoying. But yeah, I mean, it makes sense. Let's make a generating embedding file and pull that from what? Oh, I've said generating embedding at the top there. Okay, it's like taking me literally. Okay, record observation. Give me this file, this thing here. Although that's using that freaking... Fuck. There we go. This is the generate embedding syntax. I just copy pasted something. Use the... What the hell? I'm not running the actual application. Damn it. Almost ready for a break time. You made me use my human thinking. All right, let's do this. Let's go old school. What the fuck? Here we go. All right, folks, I'm going to disentangle this and eat lunch and resume in a bit. See ya. Okay, we're back. We have run our script and generated memories for files with just one line summaries. Right now I'm saving both to the file table, the vector of the summary, as well as the content, and then also memories. I don't know. We're probably not going to keep the vector in both places, but we've got a good starting point here. So let's go back to our... What was it? Generative agents. And we can say that we did created scripts to generate file memories and reflections and generate file memories. Reflections, no. Do we want to do a script for reflections? Created scripts to generate file memories and ran it. We've got all of the files of our current code base populating the memories and files tables with 200 plus file memories, one for each file in this repo. Okay, so let's go push this up and then do the next step. 225. Okay, do this, do this, and come back to here. Let's log in, log out. Create a new chat.

[01:39:59] Speaker: What's even next? The reflection system and then the querying. Based on the information provided, the next step we need to implement is the basic reflection system. Here are the specific instructions. Create a new file, lib/reflectionengine, and implement the reflection engine class. Okay, we got to change the syntax of that. Text DaVinci 002. LOL. And app router. So... In our memory stream, we made it so you can pass in a different kind of Supabase client because we needed a different one whether it was being called on the server or via the script. Do we need that here? How are we going to be generating these reflections? The question is, like, should it be a tool or should it be a script? Tools are going to be like, you want the agent to be able to access that during a conversation. Yesterday I had a tool for creating memories or creating observations and it was cool to be like, generate 10 observations for a particular part of the code base so it would use the other tools it had to, like, navigate there and then, like, it was really cool to have it kind of, like, choose what to create observations for. And I like that. However... We want to get to the point where these things are going to be running at the back end. So an agent, like, thinking, maybe that should be something that happens when you're AFK and it just kind of, like, generates higher level insights so it can, like, learn overnight or something. But maybe it should be a tool too. Presumably, we're going to be having, like, a lib or a function defined in one place and then have that be able to be used in a tool or in a script. But for now, I just want to get something kind of basic in place. So I need to say, that seems fine, but you used the wrong syntax. Please emulate lib slash memory stream. Let's just see what that does. No, don't use that OpenAI syntax. Use this. I'm looking for an example of where we made a OpenAI chat call. Or for text, use the generate text syntax you see in the script slash generate file memories.ts. All right, so we're going to be creating reflection engine. This looks better. Is it response.text? Looks like it. Query and limit. This has repository ID file and 10. Your memory stream method calls don't use the right parameters. Okay, this seems decent. Okay, so here's what I want to do. I don't know what this agent.ts is trying to give me is, but what I want to do is create a script. I did. I'm actually commit this. And let's go put that in the created reflection. Okay, so I'm going to do a new chat. Now I just want this script. Write script slash generate reflections.ts based on script slash generate file memories.ts. Okay, let's try this. Still not the right OpenAI syntax. But let's try this. Generate reflections. Wrong syntax of reflection engine. Look closer. Okay, no errors. Let's see what this does. Does generate one reflection. Okay. Add a. Okay, look at package.json and add a reflect script that calls that. Man, if cloud artifacts had nothing but like being able to tell it to look at files instead of having to paste in, that would that itself would be nice. Okay, let's see what this does. And then let's go take a look at our database. So in our memories, all of our current memories are tied to files. And then this should, without even looking at the code, this should create reflections. I'm sure there'll be some error. Okay, missing Supabase URL. What are we missing here? I don't know what it's actually doing here. Generating reflection for repository AI town. Don't reflect on them. Reflect on my stuff. What are you doing? What are you doing? Reflection. Upon reviewing. Oh, my gosh. Oh, my gosh. Oh, my gosh. Okay, sort by type. Reflection. Upon reviewing the recent observations about the code base. Now, is this hallucination or is it actually consistencies among the name? That's true. Design patterns. Singleton. No. Okay, this is just some slop. Okay. Okay, so I'm going to call and commit this as reflection, but slop, it's not using our actual. This is pulling all of the repos. That's hilarious. Generate reflection. What's actually going on there? Memory stream retrieve memories. Huh. Okay, let's do this. Let me see if that actually works. We're also using 4.0. That might be shit. And then go back here. If repo name is not auto dev. I don't give a shit. Okay, let's try this again. It's kind of ignoring that. It's looping through, but that's okay. Memory descriptions. None. Okay, so it's not retrieving anything. And then it just comes up with some slop. So I'm going to delete the slop. 4.0 slop. I also need to embed this. Let's think. It looks like recent memories is not being called or doesn't work. Yeah, there's no memories. This type memory stream retrieve memories. Why not? Okay, let's see if I can just tell auto dev to debug this. First, let's make sure you've got the most recent stuff up. And I'm going to say, figure out why my lib reflection engine.ts and memory stream.ts consults and scripts generate reflections.ts to see why my why no memories are being returned for my auto dev repo, which I see has many memories. We're probably just querying off the wrong thing or something. Let's try putting it here. Oh. Retrieve memories. Recency importance relevance. Match threshold. Okay. Okay, no errors. Let's see what this does. Does generate one reflection? Okay. Add it. Okay, look at package.json and add a reflect script that calls that. Man, if cloud artifacts had nothing but like being able to tell it to look at files instead of having to paste in, that would that itself would be nice. Okay, let's see what this does. And then let's go take a look at our database. So in our memories, all of our current memories are tied to files. And then this should, without even looking at the code, this should create reflections. I'm sure there'll be some error. Okay, missing Supabase URL. What are we missing here? I don't know what it's actually doing here. Generating reflection for repository AI town. Don't reflect on them. Reflect on my stuff. What are you doing? What are you doing? Reflection. Upon reviewing. Oh, my gosh. Oh, my gosh. Oh, my gosh. Okay. Sort by type. Reflection. Upon reviewing the recent observations about the code base. Now, is this hallucination or is it actually consistencies among the name? That's true. Design patterns. Singleton. No. Okay, this is just some slop. Okay. Okay, so I'm going to call and commit this as reflection, but slop. It's not using our actual. This is pulling all of the repos. That's hilarious. Generate reflection. What's actually going on there? Memory stream retrieve memories. Huh. Okay, let's do this. Let me see if that actually works. We're also using 4.0. That might be shit. And then go back here. If repo name is not auto dev, I don't give a shit. Okay. Let's try this again. It's kind of ignoring that. It's looping through, but that's okay. Memory descriptions. None. Okay, so it's not retrieving anything. And then it just comes up with some slop. So I'm going to delete the slop. 4.0 slop. I also need to embed this. Let's think. It looks like recent memories is not being called or doesn't work. Yeah, there's no memories. This type memory stream retrieve memories. Why not? Okay, let's see if I can just tell auto dev to debug this. First, let's make sure you got the most recent stuff up. And I'm going to say, figure out why my lib reflection engine.ts and memory stream.ts consult and scripts generate reflections.ts to see why my why no memories are being returned for my auto dev repo, which I see has many memories. We're probably just querying off the wrong thing or something. Let's try putting it here. Oh. Retrieve memories. Recency importance relevance. Match threshold. Okay. Okay, no errors. Let's see what this does. Does generate one reflection. Okay. Add it. Okay, look at package.json and add a reflect script that calls that. Man, if cloud artifacts had nothing but like being able to tell it to look at files instead of having to paste in, that would that itself would be nice. Okay, let's see what this does. And then let's go take a look at our database. So in our memories, all of our current memories are tied to files. And then this should, without even looking at the code, this should create reflections. I'm sure there'll be some error. Okay, missing Supabase URL. What are we missing here? I don't know what it's actually doing here. Generating reflection for repository AI town. Don't reflect on them. Reflect on my stuff. What are you doing? What are you doing? Reflection. Upon reviewing. Oh, my gosh. Oh, my gosh. Oh, my gosh. Okay. Sort by type. Reflection. Upon reviewing the recent observations about the code base. Now, is this hallucination or is it actually consistencies among the name? That's true. Design patterns. Singleton. No. Okay, this is just some slop. Okay. Okay, so I'm going to call and commit this as reflection, but slop. It's not using our actual. This is pulling all of the repos. That's hilarious. Generate reflection. What's actually going on there? Memory stream retrieve memories. Huh. Okay

[01:59:59] Speaker: Was there any error? Out of curiosity, if I drop the match threshold, what does that do? Anything? How does this update it? Repository ID, is that needed? I'm just gonna hard code this for now. I may need to take a look at that function definition. Oh. Okay. Have I forgotten to push something up to the database? No. Query underscore embedding. It's clear now why the appear returned. The issue is a mismatch between the function parameters. Yeah. No, I don't want that. Query embedding. No, for now, I don't want to add repository ID. I did a retrieval yesterday that worked fine. Query files by vector. For now, let's replace retrieve memories. Fuck. Out of curiosity, what if instead of recent file memories, I said something else? Why do I say that? Reflection engine. Package.json. Damn it. The retrieve memories part of memory stream file of lib memory stream is not working. Always returns empty array. But compare it to tools query files by vector. And supabase slash migration slash 2024-07-24-029-23 create query files by vector function.sql which does retrieval fine. Modify a memory stream implementation to work more like that does. We already have a file on migration for retrieving memories. It's at supabase migrations 2024-07-24-164956 create retrieve memories function.sql. Adjust your advice based on that, whether we should modify it or not or start fresh. Let's update retrieve memories to use this function correctly. All right, I will try what it says. Lib memory stream. Update generate reflection in reflection engine. Whoa! Ooh! Looks like that worked. Do I have an actual reflection here? No, that's the same slop bullshit. Okay, hang on a second, hang on a second. So let me commit this first as actual retrieval. Oh, I see. So the memory descriptions are not being generated even though the memories are fine. Memory dot repository. Yeah, there's no... Okay, so there we got memories back, but the description part didn't work. Change that code to reflect what actually comes back. For now, just assume all files are from the right repository. Then we need to generate a vector of the reflection. Does that help? No, it's the same slop bullshit. Parse the content. Oh, this fucking filter. Get out of here. Okay. There we go. There we go. Okay. It worked, but it feels a little like slop, and that's because we're using this garbage OpenAI model when we should be using the amazing Anthropic model. What is it? Where else are we using that? Chat route. There we go. Where is that? Memory stream. OpenAI, get the fuck out of here. Okay. So, let's delete this slop bullshit there and try it again. Based on the provided observations, here's a high-level reflection on the project structure. Okay. Some generic stuff. Okay, this is better. Non-horrible reflection via Sonic. Okay. Now, this is asking for recent file memories. It should be able to also reflect on reflections, should it not? Now, there's no embedding here. We have to add the embedding. Where's an example of getting the embedding? Let me just ask. This is working, but we need to save an embedding of that reflection along with it.

[02:19:59] Speaker: They edited the add memory. Hang on. No, look at lib memory stream.ts. That is expecting an embedding to be passed in. Maybe it makes sense to fold it in, but can I copy this? What's the syntax for that? MyY. Damn it. All right. Use import generate embedding from lib generate embedding and read lib generate embedding TS to see how it works. See this thing where it like, it can kind of keep up with the context of the conversation, but it like loses some context where I have to like keep reminding it of different things. If it can be aware that it's currently like making use of certain systems that need to be done in a certain way, it should be able to have that context, like it automatically added as reminders in the context. So I don't have to keep manually reminding it. And that's what we're building this memories and reflection thing to enable. Smooth chat. Okay, so in reflection engine, I'm gonna add this. And after the cost reflection, we're gonna do that. OK. And then the other thing is right now we're just saying reflect on. Maybe we can do something like authentication. Just see how that affects it. OK, let's see if we're able to generate a reflection with an embedding. Well, you can tell the cosine similarity search is working because these are all the auth related files. OK, good. But. Yeah, high level. No, no, no. Based on the following context, generate a high level. That's not right. Generate a unique observation worth remembering about this code base. Let's just start with that. Because we're gonna want this to be then turned into a tool where the an agent can choose to reflect on different topics. If it wants to, like, learn more or think more about a particular topic. Actually, you know, it's cool. We can have the agent study past conversations, identify weak spots. And choose to reflect on those topics to identify things that are missing, new things it wants to learn. Learning. OK, let's do this one more time. That time, because I had put authentication in the query, it pulled up files related to query, but the system prompt was like, generate some bullshit. So now hopefully this observation will be more tied to the actual content. Let's just see what it says. One unique observation worth remembering about this code base is its comprehensive approach to authentication and user management across different parts of the application. The code base demonstrates a well-structured integration of authentication flows, including GitHub OAuth, Supabase authentication, and protected routes. What's particularly noteworthy is how the authentication logic is distributed and applied consistently throughout various components and API routes. For instance, the auth button component adapts its display based on the user's authentication status. This distributed yet consistent approach to authentication suggests that the developers have prioritized... OK. This is much better. I wanted to add at least one, like identify one potential area of improvement. Be concise. Include at least one area of improvement. Let's just start with that. I'll go with authentication again. It's not the newest one up there, is it? OK. Oops. I need to sort it by type and then by created at descending, right? No, ascending. Mm, ascending. Descending. OK. Here we go. OK, so observation. The code base demonstrates a well-structured authentication flow using Supabase and GitHub OAuth with protected routes and user-specific functionality. However, there's an opportunity for improvement in error handling and user feedback. Area for improvement. Implement more robust error handling and user-friendly error messages throughout the application, especially at API routes. We do need that. OK. Now this is cool because, so we have memories and reflections. Some of these reflections, like areas for improvements, should feed into plans. So maybe we'll do plans in the next episode. Amazing. So just to recap, we implemented basically two of the four modules of the generative agents paper. We did the memory stream. We're saving memories. Those are of different types. We have both file-specific, like summaries of files, memories or observations about files, and then reflections based on those files. We can also make reflections of other reflections just by putting that in. What else? So retrieval, recency, importance, relevance. We're doing this actually. That is in our retrieval function in Postgres here. Retrieve memories, query embedding, match threshold, match count, recency weight, importance weight, relevance weight. This might not be like the most mature implementation of the algorithm, but this all can be tweaked by agents, probably. I don't want to think about it. I'll make them upgrade it as they want to. OK, and then what else? Reflection. We're generating reflection. We haven't automated it, but it's very easy to, like, create a tool around this. We just got to figure out how to slot that into a workflow. OK, so planning. I think this one is probably worth doing in its own video. Store the plans in the memory. So this ties more into sort of like the agentic loop, but we got three of the four components done and we'll do planning in the next video. Amazing. See you soon.