Transcription: OpenAgents Episode 033 - Agent Inspectability Planning

Source: https://twitter.com/OpenAgentsInc/status/1731156734335398303 Wiki source: https://raw.githubusercontent.com/wiki/OpenAgentsInc/openagents/Video-Series.md Media title: OpenAgents - Episode 033: Agent Inspectability Planning We think about agent inspe... Upload date: 20231203 Transcription model: mlx-whisper/mlx-community/whisper-tiny Generated at: 2026-06-01T20:31:27Z

Machine-generated transcript. Review speaker labels and wording before using this as quote-grade source material.

[00:00] Christopher David: So we're building open agents.

[00:02] Christopher David: Open must mean inspectable.

[00:05] Christopher David: It's one thing for code to be open,

[00:07] Christopher David: but for us to really see exactly what's going on.

[00:12] Christopher David: We need to make it as easy as possible for the most amount of people

[00:17] Christopher David: to be able to easily inspect what the hell agents are doing.

[00:21] Christopher David: So they can see how to improve them,

[00:24] Christopher David: how to make them safer transparency matters.

[00:29] Christopher David: So we're going to kind of go back into first principle mode

[00:33] Christopher David: and think about this kind of from scratch.

[00:36] Christopher David: I like doing this because it just kind of fun

[00:38] Christopher David: to pretend to be dumb and just like,

[00:41] Christopher David: oh, like, take nothing as a given basically.

[00:44] Christopher David: So let's try this.

[00:47] Christopher David: What do we want?

[00:49] Christopher David: What do we want?

[00:51] Christopher David: We want to see, we want full transparency

[00:56] Christopher David: into what agents do and why.

[01:02] Christopher David: The things that an agent does is composed

[01:07] Christopher David: of a number of steps.

[01:15] Christopher David: Let's kind of categorize the types of tasks

[01:17] Christopher David: that our current agent does.

[01:20] Christopher David: It will make an LLM call.

[01:25] Christopher David: It will query a vector database.

[01:31] Christopher David: What else?

[01:36] Christopher David: It will make an API call.

[01:40] Christopher David: Probably to get up.

[01:43] Christopher David: So every time one of these three things happens,

[01:47] Christopher David: we want an easy way to see what the input, the output,

[02:07] Christopher David: what else do you want to know?

[02:08] Christopher David: We want to know what model was used.

[02:09] Christopher David: What the...

[02:19] Christopher David: What third party, what API was used?

[02:23] Christopher David: If it's a vector database, we're used, like, what vector database,

[02:26] Christopher David: we're using?

[02:27] Christopher David: What third party, what API was used?

[02:31] Christopher David: If it's a vector database, we're used, like, what vector database,

[02:34] Christopher David: we're using?

[02:43] Christopher David: How many tokens or how much did it cost?

[02:49] Christopher David: Maybe how long did it take?

[02:58] Christopher David: I have curiosity. Let's go back to here and say,

[03:03] Christopher David: let's write some documentation.

[03:09] Christopher David: Documentation.

[03:12] Christopher David: Context Purpose.

[03:15] Christopher David: Maybe, like, description, like, what's going on?

[03:41] Christopher David: What's, you know, agents, user ID?

[04:11] Christopher David: We want to be able to see...

[04:26] Christopher David: This should be in an easily accessible web UI,

[04:39] Christopher David: allowing for public inspection.

[04:48] Christopher David: And ease of developer upgrading, developer iteration.

[05:08] Christopher David: What we're going to do is we're going to turn these into feature tests.

[05:35] Christopher David: Kind of like we did for the agent first principles.

[05:45] Christopher David: We kind of said what would happen,

[05:49] Christopher David: turn to the data models, then data model relationships,

[05:52] Christopher David: and then we made tests for those things.

[05:56] Christopher David: Now, I think that a number of these things will slot pretty well

[05:59] Christopher David: into what we already previously envisioned for, like,

[06:02] Christopher David: agent has many tasks, tasks, many steps.

[06:08] Christopher David: These won't be tasks, these will be types of agent steps.

[06:15] Christopher David: The task right now is basically,

[06:20] Christopher David: this is a GitHub agent.

[06:23] Christopher David: So, I know we're going to add auditing a GitHub repo,

[06:28] Christopher David: and then making code commits pairs.

[06:58] Christopher David: So, let's open both of these and see if this maps nicely to our tasks.

[07:06] Christopher David: Because we already had here, task has many steps.

[07:11] Christopher David: Task has one output.

[07:28] Christopher David: So, I think that's the first step.

[07:38] Christopher David: Okay, awesome.

[07:43] Christopher David: Before we stop with do some pseudo coding,

[07:48] Christopher David: I kind of write in our test suite.

[07:55] Christopher David: Let's make sure we're up to date here.

[08:08] Christopher David: Let's do a feature test, call it inspection test.

[08:17] Christopher David: And let's just make some

[08:19] Christopher David: comments and comment placeholders.

[08:26] Christopher David: So, what do we want to be able to do?

[08:32] Christopher David: We want a guest to be able to visit, so let's just say,

[08:37] Christopher David: guest can visit inspection dashboard and see all agent.

[08:49] Christopher David: All recent agent tasks and steps.

[08:57] Christopher David: All agent tasks and steps.

[09:11] Christopher David: Can visit task.

[09:16] Christopher David: We may want to distinguish between tasks and task running.

[09:22] Christopher David: Like a run is like one attempt to go through the task.

[09:31] Christopher David: Can visit task run page and see all steps taken.

[09:45] Christopher David: Can click on any step to see full details of input output metadata.

[10:16] Christopher David: Eventually we may want the owner of the agent to be able to.

[10:26] Christopher David: Agent owner can modify prompts use that.

[10:30] Christopher David: I'll take a bunch of refactoring to get it.

[10:32] Christopher David: Make pull from database.

[10:38] Christopher David: So, a guest like anyone can visit the inspection dashboard and see all agents.

[10:50] Christopher David: Task and steps.

[10:52] Christopher David: They can visit the task run page and see all the steps taken by an agent in one attempt to go through the task.

[10:59] Christopher David: Can click on any step to see the full details of the input output and metadata.

[11:04] Christopher David: Okay cool.

[11:05] Christopher David: All right so I'm going to implement a bunch of this offline in the board of view it in the next video.

[11:10] Christopher David: We're going to be using probably blade components rather than inertia for now because blade is going to be more testable for having a feature test being able to visit pages.

[11:23] Christopher David: Yeah so the idea is to just get the minimum viable UI that accomplishes this and so I want to get it such that our the next time that I run the script to kind of further debug will keep going with this PR.

[11:41] Christopher David: I want to be able to see on a link that I can give you.

[11:46] Christopher David: Hope at agents.com slash inspect or whatever.

[11:49] Christopher David: So you'll be able to see and full details the next run that I make here.

[11:55] Christopher David: So we'll do that next.