Transcription: OpenAgents Episode 044 - Sleuth Agent Planning

Source: https://twitter.com/OpenAgentsInc/status/1742992606785622272 Wiki source: https://raw.githubusercontent.com/wiki/OpenAgentsInc/openagents/Video-Series.md Media title: OpenAgents - Episode 044: Sleuth Agent Planning We plan our second agent, the Sleu... Upload date: 20240104 Transcription model: mlx-whisper/mlx-community/whisper-tiny Generated at: 2026-06-01T20:21:38Z

Machine-generated transcript. Review speaker labels and wording before using this as quote-grade source material.

[00:00] Christopher David: All right, let's get into it.

[00:01] Christopher David: What do we want from our sleuth agent?

[00:04] Christopher David: The first thing is just the exact thing

[00:08] Christopher David: that the Epstein GPT was intended to do

[00:12] Christopher David: enabling the public to ask questions about the corporate trip.

[00:15] Christopher David: VIA and Easy Web interface.

[00:16] Christopher David: That's what OpenAI did not allow questions

[00:21] Christopher David: and follow-up questions being able to dig into the source material

[00:25] Christopher David: and have it be able to reference all of that.

[00:27] Christopher David: So that's probably just like a nice weekend hack project.

[00:34] Christopher David: So that's probably easily enough easy to do in the next couple days.

[00:38] Christopher David: But the cooler thing will be if we can all

[00:39] Christopher David: search a reason about how we supplement the knowledge

[00:43] Christopher David: with other information collected from around the internet.

[00:46] Christopher David: And so you can envision that the sort of rag

[00:51] Christopher David: retrieval augmented generation is handled by what we already have here.

[00:56] Christopher David: We have the embedding of the inputs to compare to our knowledge base,

[01:00] Christopher David: which is a series of vectorized strings.

[01:02] Christopher David: We can very easily take the PDF and a PDFs and represent it as strings.

[01:07] Christopher David: We already implemented the functionality to parse a PDF,

[01:10] Christopher David: which is maybe connected into this new brain structure here.

[01:15] Christopher David: That's something I'll be able to do quickly offline.

[01:18] Christopher David: So it's basically just retrieval augmented generation.

[01:25] Christopher David: And then, you know, for the collective from around the internet,

[01:28] Christopher David: part, that's where we want to start building out the APIs.

[01:32] Christopher David: We're going to do that anyways for our auto dev agent building

[01:37] Christopher David: on the GitHub APIs.

[01:38] Christopher David: But this will be just a nice way to start reasoning about what are

[01:41] Christopher David: more generic building blocks for API integration

[01:46] Christopher David: that we can represent in this same graph flow.

[01:51] Christopher David: So just like here we have the flow that powers our current

[01:56] Christopher David: Consier's agent, let's live there.

[01:59] Christopher David: You have four different categories of notes.

[02:01] Christopher David: We have validation embedding similarity search and inference.

[02:07] Christopher David: The other tool that we need, not for that one,

[02:11] Christopher David: that's we have enough to do that.

[02:13] Christopher David: The other tool or node type that we need,

[02:16] Christopher David: is an API.

[02:17] Christopher David: As a people call them tool, like the fact that an agent will then use

[02:21] Christopher David: something, an API call would be one sort of tool.

[02:24] Christopher David: But basically we need the agent to be able to make an API call.

[02:28] Christopher David: To something like, for example, a search engine.

[02:30] Christopher David: There are APIs that are just kind of programatically,

[02:36] Christopher David: like you programmatically access the Google search results, for example.

[02:40] Christopher David: So if we find some like negative information,

[02:43] Christopher David: that's relevant that the user asks a question about.

[02:47] Christopher David: And if the agent can look into its knowledge base and see that it has

[02:51] Christopher David: some information but not enough to really answer the question,

[02:54] Christopher David: then it could sort of opt to search.

[02:57] Christopher David: And the important thing for our purposes,

[02:59] Christopher David: particularly with this sensitive information where you're going to want

[03:02] Christopher David: things to be like super factual, is being able to see the information

[03:06] Christopher David: fully sourced.

[03:07] Christopher David: That's one thing that you just do not get from open AI.

[03:11] Christopher David: And maybe a few of the services that are out there that do this kind of

[03:17] Christopher David: search augmented retrieval will tell you exactly what keyword search

[03:22] Christopher David: terms they use, what sources they use, how they compile it.

[03:26] Christopher David: All of that needs to be publicly inspeckedable here.

[03:30] Christopher David: And because it's inspeckedable, it'll make it easy for a bunch of people to

[03:33] Christopher David: kind of look at it, suggest improvements and kind of collectively

[03:36] Christopher David: iterate and improve these agent workflows or time.

[03:41] Christopher David: Okay, so when people help to improve these algorithms,

[03:47] Christopher David: we want to pay them for their efforts.

[03:49] Christopher David: For now, I do not envision there being like a revenue stream

[03:52] Christopher David: associated with this agent.

[03:55] Christopher David: I wanted to be super just completely free to use.

[03:58] Christopher David: And so the payments will come just in this case,

[04:01] Christopher David: the agent author is going to be putting up like a bounty to kind of

[04:05] Christopher David: incentivize people to help upgrade it.

[04:08] Christopher David: So the main deliverables that we want a public chat agent that anyone can

[04:11] Christopher David: interact with with no registration needed and wanted to have these

[04:14] Christopher David: functionalities.

[04:15] Christopher David: But deliverable number one will just be like the basic kind of

[04:18] Christopher David: exact same thing as the FC and GPT except actually enabling that.

[04:23] Christopher David: So I may kind of record certain pieces of that process.

[04:26] Christopher David: I probably do most of it offline.

[04:28] Christopher David: It's just kind of taken stuff we've already built and kind of

[04:30] Christopher David: sewing it together.

[04:34] Christopher David: So we will do that first and deploy that.

[04:39] Christopher David: So you know, probably this week under so we'll be able to deploy that.

[04:42] Christopher David: And then we'll start enabling effective follow-up on any information that we

[04:48] Christopher David: find enabling people to surface tidbits that should be acted on in different ways

[04:55] Christopher David: and paying them.

[04:57] Christopher David: Okay, see you soon.