Transcription: OpenAgents Episode 163 - Issue Reasoning Demo

Source: https://x.com/OpenAgentsInc/status/1890616487536111878 Wiki source: https://raw.githubusercontent.com/wiki/OpenAgentsInc/openagents/Video-Series.md Media title: OpenAgents - Episode 163: Issue Reasoning Demo We demo at @PlebLab the Onyx issue... Upload date: 20250215 Transcription model: gpt-4o-transcribe-diarize Generated at: 2026-06-01T05:17:15Z

Machine-generated transcript. Review speaker labels and wording before using this as quote-grade source material.

[00:00] Christopher David: Okay, hello from Pleblab, we are

[00:01] Speaker B: Woo!

[00:02] Christopher David: at Pleblab, woo!

[00:03] Christopher David: Got a massive crowd of three or four hundred people here. Uh you can't see them all, but they are behind me, trust me.

[00:11] Christopher David: Uh we are here at the uh Pleblab Hack and Tell evening, and um we're hacking and telling.

[00:17] Christopher David: So, cool,

[00:18] Christopher David: kind of fun to just continue from yesterday's video.

[00:21] Christopher David: I've got a little demo here,

[00:22] Christopher David: and we'll get some crowd participation now that I've verified that it kind of sort of works. So, recap of the last video.

[00:29] Christopher David: We built this basically this little coding loop.

[00:32] Christopher David: Actually, we'll just kind of do the from the top what we're doing here.

[00:36] Christopher David: We're on the quest for the Holy Grail of agentic software development.

[00:41] Christopher David: aka an agent that can reliably convert GitHub issues into mergeable pull requests in an automated loop.

[00:48] Christopher David: Here is the thing I want to do, do it,

[00:51] Christopher David: it's tested,

[00:51] Christopher David: I merge it, I don't have to worry about it,

[00:53] Christopher David: repeat.

[00:55] Christopher David: This is a goal for which some closed AI unicorns have raised hundreds of millions of dollars, blah, blah, blah, let's do it in the next 10 videos,

[01:00] Christopher David: okay?

[01:01] Christopher David: Open source,

[01:01] Christopher David: yada, yada, yada.

[01:02] Christopher David: So we're building repo maps,

[01:04] Christopher David: we are finding the relevant files, we've kind of identified this.

[01:08] Christopher David: kind of six-step plan and we're currently kind of doing

[01:14] Christopher David: Well, we've got basically three through five that we just did today and yesterday that I'm gonna be demoing here. Not yet creating the po because they're still not good enough. But we're gonna kind of together evaluate, we're gonna step through um me just spinning this up. Uh a second ago I pointed it at some random issue on the Bitcoin uh official Github repo.

[01:34] Christopher David: Uh we'll kind of evaluate it to see if it sounds like bullshit or not. And then uh we'll get some issue recommendations from the crowds. If anyone has a

[01:42] Christopher David: Uh GitHub repo that you have or you just know of one you want to throw an issue at this and we'll kind of evaluate together how good this is.

[01:51] Christopher David: So do do do.

[01:53] Christopher David: Let's see.

[01:54] Christopher David: So we did a basic coding loop.

[01:56] Christopher David: Um we basically take an issue, we map the repo, basically we clone it down. We use I think either tree set or just a basic algorithm of getting kind of the file folder hierarchy as well as the like main functions. We pass that along with the issue, and I think is the issue comments to an L_L_M_ and we say hey, what are the relevant files. We have it loop through the al relevant files, identify what changes need to be made

[02:20] Christopher David: and make them and then m issue a P_R_ from there. In this video we're kind of gonna be demonstrating the reasoning and we're gonna be kind of like evaluating the reasoning, maybe not so much the specific code changes, um but I wanted to kind of get things hooked up to the U_I_ so uh we can just actually use it.

[02:35] Christopher David: So, uh I did this about three seconds ago.

[02:38] Christopher David: Basically just put in a little uh repo down here that kicked off the process. And let's um let's take a look. So I pointed this at

[02:48] Christopher David: A Bitcoin issue,

[02:51] Christopher David: 31852. I just kind of picked this randomly, I didn't even read this. So two days ago someone on the official Bitcoin repo said, I am using a passphrase for my encrypted wallets, eight in total. This passphrase is space the keys backslash okay. I use this passphrase for all my wallets, total 2.4 BTC okay. But now it doesn't work as usual. I tried creating a new wallet with the same passphrase and I encountered the same issue.

[03:18] Christopher David: You can review my question in all the comments here,

[03:20] Christopher David: where I describe the bug in your software,

[03:22] Christopher David: Satoshi.

[03:24] Christopher David: I hope this is something important because I feel like I've lost all my money.

[03:28] Christopher David: Feel free to email me.

[03:29] Christopher David: Wow.

[03:31] Christopher David: Expected behavior,

[03:31] Christopher David: blah, blah, blah.

[03:35] Christopher David: Windows. Okay.

[03:37] Christopher David: I think StackExchange... Okay,

[03:38] Christopher David: so this is just interesting.

[03:39] Christopher David: Some dude can't access,

[03:41] Christopher David: I guess,

[03:41] Christopher David: 2.4 Bitcoin and is...

[03:43] Christopher David: I don't know that this is actually should be an issue. But let's see what uh let's see what open agent's uh issue has to

[03:48] Speaker C: So

[03:48] Christopher David: say about it.

[03:48] Speaker C: so somebody posted that in GitHub? Is that a thing that happens all the time?

[03:51] Speaker D: This looks like voltage port.

[03:55] Christopher David: I we're gonna be charging for this stuff here. Okay.

[04:00] Christopher David: Starting open router analysis. Can make this go

[04:02] Speaker C: It's like this dumpster in uh Rhode Island that they're missing a laptop.

[04:06] Speaker C: Oh

[04:06] Christopher David: Okay, this looks like a serious issue. A user is reporting that their passphrase which has leading and trailing spaces is no longer working with version twenty eight. This could indicate a regression in how passphrases with white space are handled. Here's my plan to address this. One, analyze the Bitcoin core code, focus on the parts that handle password entry, storage and specifically the wallet passphrase, worth first saying that this agent here has no background information aside what it

[04:34] Christopher David: it s been trained on, I think this part of the analysis uh was done by Gemini Flash two point oh. So, you know, it's gonna crap-shoot as to what its actual knowledge is, but at least try to learn from from analyse what it it says here. Uh I need to analyse relevant code related to key derivation from the passphrase to reproduce the issue. I'll attempt to create a wallet with the reported passphrase or a similar one containing leading trailing spaces in a testing environment to confirm the bug.

[05:00] Christopher David: it's crucial to test this thoroughly. Three, identify the root cause. Use debugging and code review to pinpoint where the past phrase processing goes wrong. Is it a trimming issue encoding problem, something related to key derivation or perhaps something more obscure. Alright, so I'll just kinda breeze through this little bit. Propose a fix, blah blah blah, test the fix, document the change.

[05:19] Christopher David: Okay, files to investigate, likely candidates. Uh so this is its selection based on the repo map that we fed it. Um I don't know how comprehensive this is, given that Bitcoin repo is very large and I don't know if like all of the files even fit in the context or just stuff I need to evaluate separately. But uh let's see. So source wallet wallet CPP, this is the core wallet code. It definitely handles passphrase setting, encryption, decryption, wallet management. Uh crypter dot CPP and any related header files.

[05:46] Christopher David: Blah blah blah.

[05:47] Christopher David: Is anyone here familiar with the Bitcoin core code base? Does do these seem like files that are actually

[05:51] Speaker D: I have like skipped a like cryptor.

[05:55] Christopher David: Is that even a thing?

[05:56] Speaker D: CPP I'm looking up right now.

[05:57] Speaker C: Did you take the base for the eight core?

[06:00] Speaker C: You took the base for the eight core.

[06:01] Speaker D: We don't read the whole damn Bitcoin core code.

[06:04] Christopher David: You don't read the whole core code, my agent's gonna read the whole core code. So let's see. Source, wallet, crypto.

[06:12] Christopher David: That C_P_P_ okay, that is an actual file. And what does this file do? This mimics the behaviour of open S_S_L_'s bytes to key, cipher, message digest. Yeah, this is nothing that I would touch. But let's learn more about it. Okay, script. S_P_P_ Um alright, assumptions. This describe problem happens with the C_L_I_ and not just the G_U_I_ Step by step analysis and proposed code changes. I will start by looking for string manipulation functions that trim white space.

[06:40] Christopher David: Potential area of interest,

[06:41] Christopher David: Bool set wallet flag.

[06:43] Christopher David: There's nothing explicitly trimming white space here, but the set wallet flag function and others in wallet.cpp potentially manipulate wallet data. I will search for any usage of boost trim or any similar functions like std trim or some of the tokens are missing spacing here.

[06:57] Christopher David: String erase with arguments used to eliminate white space.

[06:59] Christopher David: If I find that any of those functions is ever called on the passphrase, then that is the place where the bug is.

[07:04] Christopher David: Wallets. This handles JSON RPC commands related to wallets.

[07:09] Christopher David: Let's examine the encrypt wallet RPC command handler. I'll be surprised if it stumbles on anything that's like

[07:16] Christopher David: like very accurate, uh given that it does not actually have the ability right now to like uh it said something about I will search for any usage. Um but here's where over the next few videos we're gonna be extending this sort of process of analysis analysing a repo with the kinds of tools that, for example, cursor uses. So if you are is anyone here use cursor at day to day?

[07:41] Speaker C: Yep.

[07:41] Christopher David: Yeah. Um cursors are awesome.

[07:43] Christopher David: I

[07:43] Christopher David: Uh, it's agent flow is actually extremely similar to the open agents agent flow, um the sort of version three, um as I've kind of shown in the previous videos of this series, I think that when we get this process in place and I know I'll get in my old white theories, but my my my what we're trying to do with open agents now is kind of build what comes after

[08:04] Christopher David: after cursor like after the IDE it's just you say the thing that you want and the agent kind of solves it for you but one of the things that they do really well is one of their tools is they can have it very smartly you know search through your code base based on keywords they're obviously using some kind of combination of you know vector embeddings and whatever kind of repo map or indexing they've got going on under the hoods so giving basically this agent here a tool to similarly search through

[08:32] Christopher David: The repo um it's it's very easy for us to do like in a single video we'll do that probably in a couple days. Anyways, um this is so this is basically not to say that we're expecting that this analysis is a hundred percent accurate. I'm more interested in like do we have a good process of it somewhat knowing kind of where to look and then which we can then augment by kind of niching down on some of the individual component parts here.

[08:54] Speaker D: I I think

[08:58] Christopher David: Okay, he thinks he found everybody. Okay, so let's just keep going. Crucial observation. The pass phrase is read directly from args zero as a standard string and transferred to a secure string using args zero C-ser. There aren't any explicit trimming operations in the snippet. However, if this string is somehow further manipulated before being used for the encryption issues might arise. Let's analyse where str passphrase is used.

[09:22] Christopher David: This file contains functions related to key derivation which only happen after blah blah blah, here I just

[09:29] Christopher David: messed up the styling of this. I need to investigate the something crypt key, store and crypt keys function to see how the pass phrase interacts with the key derivation.

[09:39] Christopher David: uh boo hypothesis. Since the issue is happening after upgrading to version twenty eight, a patch in the library might have introduced a function to trim the input fast ways. Since the encrypt keys is a vital function I will look for get commands related to it or the cryptofunc again it doesn't have the ability to do that. But um one thing I'm very excited about particularly for these very large repos is

[09:57] Christopher David: You have all of the context that you need to solve things essentially in the repo paired with like a knowledge of the language.

[10:04] Christopher David: And so to be able to have an agent digest,

[10:07] Christopher David: wow,

[10:08] Christopher David: here's the entire history of commits and from which committer for the entire life cycle of this repo,

[10:14] Christopher David: like all the data is right there. We just need to like parse it and index it.

[10:18] Christopher David: And even if this can't by itself produce code change, I certainly wouldn't trust particularly Bitcoin core to make any.

[10:24] Christopher David: changes based on this, but it may provide at least some helpful pointers of where to look and so might help developers uh you know, maybe hopefully. Uh let's see so might be in the f blah blah. I need to find out the get history for this function. You can't, sorry. Let's check some recent get blame histories. Yeah, it would like to be able to run this command, but you can't.

[10:45] Christopher David: After further research I can't find any evident commit, that's 'cause you couldn't do it.

[10:48] Christopher David: I can propose a change that could fix the issue by ensuring no undesired trimming happens. Proposed solution: the safest approach is to add an explicit test case to make sure that the passphrase is not trimmed.

[10:58] Christopher David: Add a unit test. Add a unit test to create a wallet,

[11:01] Christopher David: encrypt it with a passphrase containing leading and trailing spaces, and then unlock it.

[11:04] Christopher David: This test will fail if the spaces are trimmed.

[11:06] Christopher David: Here's how the test could look, blah blah blah.

[11:08] Christopher David: Okay, uh, Austin,

[11:10] Christopher David: what is your kind of analysis of the smell test of this, and ha w did you find something relevant?

[11:48] Christopher David: What line did you say?

[12:24] Christopher David: And then here also is where I want um you know will very soon just add chat with this so you can respond to this um thing. And here's my plan to address it, analyse the co okay they already did that stuff.

[12:42] Christopher David: Hypothesis since the issue is happening after upgrading to version twenty eight, a patch in the library may have introduced a function to trim the input passphrase.

[12:50] Christopher David: Um, okay. Alright so, you know good, you know better than nothing probably. Okay, let's um someone give me another repo issue uh please and we'll just test this out kinda from scratch, I'll show you the process and then uh we'll wrap up.

[13:06] Speaker C: Um there's a lot of stuff on stacker news.

[13:09] Christopher David: Stack your news.

[13:11] Christopher David: It's countering.

[13:16] Speaker C: Issues you want to

[13:18] Christopher David: Any GitHub issue you want us to take a look at?

[13:26] Christopher David: So let's try this uh

[13:28] Speaker D: I need you on as well.

[13:29] Christopher David: Okay. The first okay, so let's do let's do the first issue on Stack Overflow. I'm assuming you mean the most latest one. So Stack Overflow, stack or dot news. Uh issue not one nine oh five.

[13:41] Christopher David: one nine oh five. Alright, fingers crossed, demo gods.

[13:46] Christopher David: Oh, I didn't restart the.

[13:49] Christopher David: I actually have no idea what's gonna happen uh if it's gonna work 'cause I it's like in the same session. But let's see what happens. Alright so blah blah blah. Okay, I understand the issue.

[14:00] Christopher David: Uh let me blow this up.

[14:01] Speaker C: Oh, so it pulled it all up, Christopher?

[14:03] Speaker C: Uh

[14:03] Christopher David: Uh

[14:03] Speaker C: blank?

[14:04] Christopher David: it um yep, it it looks up the issue.

[14:07] Christopher David: Um if it's public, I don't need any special permissions. And then it it using my GitHub auth token so I can also look at my repo.

[14:16] Christopher David: Alright.

[14:18] Christopher David: Okay.

[14:21] Christopher David: Okay, so we're getting the

[14:25] Christopher David: I think we're using um

[14:27] Christopher David: deep seek for some of this. So we've the reasoning trace as well as

[14:33] Christopher David: the

[14:35] Christopher David: Let's see.

[14:38] Christopher David: Okay, I understand the issue. The item popover okay for for what is the issue. Item popover does not consider comment ID param.

[14:45] Christopher David: Good first issue alright. Easy hopefully.

[14:49] Christopher David: Bug if you paste a S_ and link like this, we render the comment I_D_ since that's where you will get scrolled to.

[14:56] Christopher David: However the popover does not show the comment.

[14:59] Christopher David: Steps to reproduce, paste this into a comment, hover over it, popover shows item.

[15:03] Christopher David: Expected behaviour, it should. And it doesn't show the comment. Let's see what it says.

[15:13] Christopher David: Okay, I understand the issue. The item popover currently only considers the main item I_D_ from a stacker news link, but it should also respect the comment I_D_ U_R_L_ parameter if present, displaying the popover for the specified comment instead of the parent item.

[15:25] Christopher David: Here's my plan to address this issue. Identify the relevant code, blah blah blah. Modify link parsing. Update the link parsing logic to extract the common ID parameter from the URL, if it is exists. Adjust power of data fetching. Test thoroughly. Blah blah blah. Okay, now let's assume we have the following

[15:43] Christopher David: simplified files. Uh-oh, I don't like the word simple files. Is it is it bullshitting?

[15:48] Speaker C: Uh

[15:55] Christopher David: Okay.

[15:55] Speaker C: It's on.

[15:56] Christopher David: Okay, so so so

[15:59] Christopher David: I'm gonna guess that something got messed up. Let me actually do something. I wanna reboot the server.

[16:05] Speaker C: It's why it's called hack and tell.

[16:06] Christopher David: Yeah, we're hacking. Doing it live. Okay, let's try this again.

[16:29] Christopher David: Yeah, I'm I'm gonna guess that I just there was something that got set in like some server-side state uh from the original

[16:38] Christopher David: or n or not let's see.

[16:40] Christopher David: Oh yeah yeah. So so here it's not it's not it's here it's bullshitting.

[16:45] Christopher David: 'Cause it says hypothetical.

[17:01] Christopher David: Alright, so here's um here this might be actually fun to do. So I'll just give you an example of my debugging workflow.

[17:09] Speaker E: Mm-hmm.

[17:09] Christopher David: Uh sometimes I use sometimes when I have like a massive log, I'll dump that into ChatGPT Pro. Let's see if ChatGPT can help me here.

[17:18] Christopher David: I just re-signed back up for this today because uh

[17:22] Christopher David: I had some massive massive massive log and I was like

[17:26] Christopher David: Let's see if we can identify. So I'm going to paste the whole server log.

[17:34] Christopher David: Goes up to here. No.

[17:47] Christopher David: That's the C++ stuff.

[17:51] Christopher David: Still C++ stuff.

[17:54] Christopher David: Stacken is

[18:06] Christopher David: Uh okay, I'll just start it from here. So I'm gonna say um analyse this log and tell me why it's not getting the actual

[18:20] Christopher David: GitHub issue. It's response includes some hallucinated REST files that don't exist.

[18:29] Christopher David: But I tried this a second ago

[18:34] Christopher David: with a different repo and it got the right files. Tell me what's wrong. Now hopefully Chai TPT doesn't complain about the length. It usually is good with super long stuff.

[18:51] Christopher David: Reasoning

[18:54] Christopher David: Based on the log, it appears that while the repository was cloned and processed without error, the solver's analysis prompt ended up including file names mostly rust files that don't exist in this particular repo.

[19:06] Christopher David: This contrasts with your previous run on another repo where the correct files were identified.

[19:11] Christopher David: Possible reasons include

[19:14] Christopher David: Oh I know what happened.

[19:16] Christopher David: Um yeah earlier the agent then when I was working like read one of the a test okay this is funny a test issue that I gave it which was to create a repo map index the agent like thought I wanted it to do that so it built part of a repo map indexer that I guess got accidentally triggered here so anyways we're getting just some kind of some hallucination but hey

[19:44] Speaker C: But anyway,

[19:45] Speaker C: First one?

[19:45] Christopher David: It did work on the first one, um and I don't know

[19:49] Speaker C: The second one.

[19:50] Christopher David: I don't I don't know the difference between the two.

[19:52] Christopher David: Um okay so we'll end there. This is obviously a work in progress, but you can kind of see where we're going with this, which is I want a single mobile app

[19:59] Christopher David: i can just point at any repo so like while i'm at the grocery store like cool stuff can happen and i want to um i want to kind of just end with a comment about there's a bunch of people out in the world that kind of talk about what their agents can do or oh i pointed my agent at and the cognition that raised 200 million dollars like oh here's it solving an upwork job here's it doing a pr here's it doing you know there's a lot of demos

[20:28] Christopher David: But if anyone actually had this working,

[20:31] Christopher David: where are the 50 pull requests where the agent has actually solved?

[20:39] Christopher David: Because if you can do this for a few dollars worth of compute,

[20:42] Christopher David: isn't it going to be worth the demo of like,

[20:44] Christopher David: hey, open source projects that I care about,

[20:46] Christopher David: the 10 of them, I've solved three issues each or I go find.

[20:51] Christopher David: People that have signed up with lightning bounties or tiny grad or any of these repo people that are paying per issue I mean, I hope a week or two from now. I can start scooping up some of these bounties for these agents

[21:02] Christopher David: The compute costs are coming down way down way down particularly as some of the stuff can actually run locally.

[21:07] Christopher David: So I think

[21:09] Christopher David: The fact that no one has done this, and I think that we're potentially on the verge of being able to do this where I'm not going to start with the Bitcoin Core repo because it's so sensitive,

[21:18] Christopher David: but hopefully we can come over to the Stacker News GitHub and find some good first issue and like,

[21:25] Christopher David: hey, you might see OpenAgent slash me popping up here with some comments in the next week or so. And yeah.

[21:30] Speaker B: Christopher, so do you think that's, I find it interesting that

[21:34] Speaker B: Everybody's trying to solve for GitHub issues right now and it seems like everybody's trying to figure that out first.

[21:40] Speaker B: Why do you think that is? Like why do you think everybody's headed for that kind of?

[21:44] Speaker B: Crown Jewel, yes.

[21:46] Christopher David: I think part of it is, you know, going after what is the common denominator that people are already using.

[21:51] Christopher David: So cognition and some of the more like enterprise facing AI companies have gone after Slack as an integration where they may have started with kind of a generic web interface.

[22:01] Christopher David: It's kind of like they want to go to where companies they have like, you know, they can click one or two things and then like the agent can just kind of like run wild on that.

[22:10] Christopher David: I am really looking forward to getting away from GitHub and like.

[22:13] Christopher David: and like every Microsoft product and so I'm hoping this when we get this properly done this might even help people get off of GitHub because fundamentally the things that we're doing are

[22:26] Christopher David: native to Git um you know traversing a repo having all the history there's no reason that we need to be using GitHub it's just what the majority of people do uh as well as because it's a public platform that everyone's familiar with like as soon as you start to see coding agents going into open source product projects and like solving issues in mass you know I've got my little at the bottom commented by open agents uh it should be fun for kind of lead gen and like oh shit there actually are agents now that are able to make meaningful reliable pull requests that I can look at oh my god that's really

[22:54] Christopher David: really good beautiful documentation, click merge, um and then people can

[22:59] Speaker B: Interesting.

[22:59] Christopher David: pay us for more. All right we'll leave it there, thanks so much and uh I'll see you soon.