Transcription: OpenAgents Episode 085 - API Design

Source: https://twitter.com/OpenAgentsInc/status/1762596179643371596 Wiki source: https://raw.githubusercontent.com/wiki/OpenAgentsInc/openagents/Video-Series.md Media title: OpenAgents - Episode 085: API Design We introduce the OpenAgents API and compare i... Upload date: 20240227 Transcription model: mlx-whisper/mlx-community/whisper-tiny Generated at: 2026-06-01T18:25:07Z

Machine-generated transcript. Review speaker labels and wording before using this as quote-grade source material.

[00:00] Christopher David: Let's talk about API design.

[00:02] Christopher David: Want to unpack this tweet and share thoughts about how we're approaching the API.

[00:09] Christopher David: The series began as let's build our own version of the GPT store.

[00:15] Christopher David: I had a hunch back in November when we started this that when they did launch the GPT

[00:21] Christopher David: store, open AI, the monetization would be half-assed.

[00:26] Christopher David: They'd probably do maybe weekly payouts via Stripe Connect to some limited set of providers.

[00:34] Christopher David: Kind of like X does now, I guess.

[00:37] Christopher David: Not global.

[00:39] Christopher David: If you're in India, if you're in the Philippines, you're not eligible or they'll add support

[00:43] Christopher David: for you maybe way down the road.

[00:47] Christopher David: But that seems like the perfect opportunity to demonstrate the power of a global payment

[00:53] Christopher David: that we're calling Bitcoin and our willingness to say, hey, if you provide value to an ecosystem,

[00:58] Christopher David: you deserve to be paid permissionlessly.

[01:01] Christopher David: Like, just take the money proportional to the value that you create.

[01:06] Christopher David: Have that be accessible.

[01:08] Christopher David: And that's what we've been building toward for 85 videos.

[01:11] Christopher David: We're pretty close now to a product rollout.

[01:14] Christopher David: I think I always think that we're two weeks away, but that keeps extending down.

[01:20] Christopher David: We've made some real progress.

[01:24] Christopher David: We've built basically all of the functionality that we need in kind of quick and dirty ways.

[01:33] Christopher David: And as we've been circling back to think about what this needs to look like in production,

[01:38] Christopher David: it became clear that there's an opportunity here to kind of go after both the Open AI,

[01:46] Christopher David: GPT store, as well as there are assistance API at the same time.

[01:50] Christopher David: And that while we are getting our services production ready, kind of building up the back end

[01:57] Christopher David: that our front end will consume, we may as well just take the little bit of extra effort

[02:03] Christopher David: to think through how to make that same back end available to other people, other application developers.

[02:13] Christopher David: So there's this weird dichotomy, and I don't understand why.

[02:18] Christopher David: But Open AI has released both an assistance API as well as their GPT store,

[02:28] Christopher David: which is like apparently completely separate.

[02:32] Christopher David: Assistance that you create via the API do not show up in their GPT store.

[02:37] Christopher David: The GPT store has to be if they make it through their interface.

[02:41] Christopher David: So there's this split between what developers can do and what low code,

[02:46] Christopher David: no code developers to do.

[02:48] Christopher David: Our idea here now is let's combine these two.

[02:53] Christopher David: Let's let developers create assistance or agents via an API and have those show up on the website.

[03:03] Christopher David: Eleigible for the same sorts of monetization.

[03:10] Christopher David: I took a close look at this last few days because got to the point where we're using,

[03:17] Christopher David: we were using GPT4 for some of this chat.

[03:24] Christopher David: I have since switched that over to Mr. Olarge, and this is working great.

[03:29] Christopher David: But one of the things I need to solve next is how to handle long context.

[03:35] Christopher David: If you're just passing chat messages each time to the GPT4 API,

[03:41] Christopher David: you read to point where you're just passing too much context in,

[03:46] Christopher David: and GPT4 will just fail saying too large of context window.

[03:55] Christopher David: And you're supposed to, as the developer, kind of do whatever, rag or truncation you need to do,

[04:03] Christopher David: so that you've passed to open AI only what's needed.

[04:06] Christopher David: So there's additional work on how that should be done.

[04:12] Christopher David: One of the selling points of the assistance API is they handle that for you,

[04:20] Christopher David: as well as offer things like function calling the code execution environment,

[04:26] Christopher David: and I wanted to say image generation, there's some other thing that you have access to.

[04:37] Christopher David: So one thought I had was, well, should we use the assistance API in the back end?

[04:42] Christopher David: Kind of optional, let people swap that out.

[04:48] Christopher David: But, kind of quickly rule that out for a few reasons.

[04:56] Christopher David: So number one is, and I think part of why, you know, I hear a number of folks complain on Twitter,

[05:03] Christopher David: why the assistance API sucks or one of the limitations is the strategy of fitting your message history into the context window is very naive, very simplistic.

[05:18] Christopher David: This says once, so it automatically manages the context window touch that you never exceed the model's context link.

[05:26] Christopher David: Once the size of the messages in a thread exceeds the context window of the model,

[05:30] Christopher David: the thread will attempt to include as many messages possible if it the context window and drop the oldest messages.

[05:36] Christopher David: So like, no special magic there, anyone could do that, that kind of sucks.

[05:43] Christopher David: Chaggy PT on the other hand probably definitely is using more sophisticated strategies than that,

[05:51] Christopher David: but you don't know. They're constantly changing in.

[05:56] Christopher David: And I just one of the downsides of using Chaggy PT, you could come to depend on it or get accustomed to it working in a particular way.

[06:04] Christopher David: They make an update. You don't know what that update was. You have no transparency into that.

[06:10] Christopher David: Now you're getting different results. It's very frustrating. It's been frustrating to me.

[06:14] Christopher David: So this type of extremely basic context shrinking is like gosh, if it's going to be limited to that,

[06:30] Christopher David: maybe I don't want to use this. And then you go down here and you look at the other limitations.

[06:37] Christopher David: There's no streaming. That sucks. I love streaming. I love streaming.

[06:54] Christopher David: If you're an interested in streaming, no, no, no. I mean, I love that you are streaming this response to me.

[07:02] Christopher David: How shit would it be if you're a hundred billion dollar company? And your API does not support streaming.

[07:21] Christopher David: Regardless of your question, it would certainly be a missed opportunity for a hundred billion dollar company to not support streaming in the API.

[07:31] Christopher David: So there's no streaming. There's no notifications. You have to pull. It's like a developer thing.

[07:42] Christopher David: You can't get updated when the run is done or when steps advance. Watch this.

[07:48] Christopher David: Analyze this. Watch this. Execute the step. There it is. What did I not need to do?

[08:00] Christopher David: Pull. What are you doing now? API? What are you doing now? Like one second polling like no. Notifying me.

[08:10] Christopher David: We already have that. Support for Dolly or browsing as a tool.

[08:13] Christopher David: I just gave it a URL. We wrote plugins to do this. Not only do we support browsing. We support

[08:27] Christopher David: browsing via arbitrary wasom plugins. If you don't like our URL scraper plugin, which is open source when

[08:34] Christopher David: I get up, right your own plugin in. Okay. It's more for Dolly. They could very easily, easily.

[08:43] Christopher David: Add support to their existing APIs, but they don't. That's something that like a developer,

[08:50] Christopher David: any developer in the world could take a weekend and add support there. But they've got one

[08:55] Christopher David: with reason for having things be the way that they are and you can't change it and fuck you.

[09:01] Christopher David: Support for user message creation with images. I'm guessing this means image input.

[09:07] Christopher David: We have a image input. I don't have it still working. I made a budget changes since we added a few videos ago.

[09:21] Christopher David: Oops. Let's just see if I broke it or not.

[09:25] Christopher David: Anyway, we got that working. Check two videos ago. We're actively mobile.

[09:47] Christopher David: So this assistance API has, I would say, nothing special about it. Would you say that?

[09:55] Christopher David: They've organized business logic in a particular way. They've got a pretty well thought through

[10:02] Christopher David: structure of assistance, threads, messages, runs. And this all works. It works well kind of

[10:11] Christopher David: sort of. Underwhelming has been the consensus of what I've seen from people talking about this on Twitter.

[10:19] Christopher David: Maybe some people are having amazing success with this and aren't sharing about it.

[10:23] Christopher David: But lots of limitations there. So for things like file handling, which is one of the reasons why someone

[10:35] Christopher David: would use this is because, oh, I can upload a file and I can attach it to an assistant and it'll

[10:41] Christopher David: consult that file. Well, we've done that already in this series.

[10:51] Christopher David: We built our own rag from scratch. And taking the idea of the user

[10:57] Christopher David: extensibility, let's say our rag algorithm sucks. Whereas now what you want, you can just

[11:04] Christopher David: swap that out with your own because we have a soon to be market place of these plugins.

[11:10] Christopher David: You could already currently, I mean, you have to fork our code and

[11:16] Christopher David: smooth it out. But executing arbitrary business logic in user compiled,

[11:23] Christopher David: web assembly plugins via ex-dism, that's already working.

[11:31] Christopher David: So, well, like you know, let's do our own version of this.

[11:40] Christopher David: Hence the tweet, the open agents API helps developers define and interact with agents

[11:48] Christopher David: programmatically. If taken this stuff from the open AI assistance API, we're calling them agents.

[11:57] Christopher David: We're not pussy footing around with these are just assistance. Maybe agents will come later. These are agents.

[12:04] Christopher David: We'll share our thoughts more on definitions of that particular word in a different video.

[12:14] Christopher David: An AI entity executing defined tasks.

[12:19] Christopher David: Thread a message chain between a user and an agent, a message, a single communication file,

[12:25] Christopher David: self-exponatory active execution. In our data models thus far, we've broken this into task

[12:35] Christopher David: and tax execute, executed step and step executed that correlates to basically.

[12:43] Christopher David: The flow is the task, the run is the task executed. Node is basically the step.

[12:50] Christopher David: And a type of node could be a plugin. Exxysm web assembly binder. We've got a bunch of videos on

[13:00] Christopher David: ex-dism, but if you're newly tuning in, the idea that you can build AI agent logic in these

[13:09] Christopher David: composable reusable plugins, which can be monetized out the box with a fee.

[13:15] Christopher David: So when any time anyone uses a plugin you write in their agent, you get paid. How fucking cool is that?

[13:25] Christopher David: That's how it should be. So the concept. Agents are built from interchangeable building blocks called nodes,

[13:33] Christopher David: which are organized into sequences called flows. Here's an example, flow. This is the flow used by

[13:44] Christopher David: the current chat. Not if you send it just a basic hello, if there's no URL in it, it goes straight

[13:56] Christopher David: to inference. I must have broke something, but read this. Okay, if you pass it to URL,

[14:09] Christopher David: it runs through these three things. Extract URL, scrape URLs, LLM inference. There's first two things

[14:16] Christopher David: are ex-ism plugins. The third thing we had as an ex-ism plugin, but that didn't support streaming,

[14:23] Christopher David: so we were placed it with our own kind of native version. But you could do inference also via plugins.

[14:28] Christopher David: So here's the flow. And right now it's kind of specified in this database migration's file,

[14:39] Christopher David: but you soon will be able to build these little nodes. This is a flow of nodes. These two being

[14:48] Christopher David: plugins, so they'll have a different color on it. And yeah, plenty of ideas we're going to be borrowing

[14:56] Christopher David: from flow based programming. This is a whole, cool, fun rabbit hole. I recommend learning about,

[15:02] Christopher David: but there's already plenty of best practices around representing software execution,

[15:11] Christopher David: flows, visually, Unreal Engine, blue prints is a good example of this kind of thing. We'll do more

[15:20] Christopher David: future videos on that. So example nodes include API endpoints, conditional logic, data parsing,

[15:26] Christopher David: and third party integrations. Nodes can be created by community developers by uploading

[15:31] Christopher David: Wazom plugins. Plugins are a special type of node that allows for a bespoke operations and

[15:37] Christopher David: enhance capabilities. You can kind of tell which sentences like ChagitBT Road,

[15:42] Christopher David: really just that one. Each node may have an associated fee payable to its creator upon use.

[15:50] Christopher David: How are we going to pay you? Are we going to pay you in government money that's losing purchasing

[15:58] Christopher David: power every day? Or the native currency of the internet, which can be sent in extremely small

[16:04] Christopher David: denominations anywhere in the world instantly for essentially no fee. Yeah, we're going to use

[16:10] Christopher David: that second one. It's called Bitcoin, except those substitute Bitcoin only. Thanks. All right,

[16:18] Christopher David: nodes can reference an agent's files. Users can reverse with agents in conversations that are

[16:24] Christopher David: threads of messages. A run is an instance of executing a flow.

[16:31] Christopher David: Somewhat clear, hopefully, we're taking the basics of the assistance API.

[16:39] Christopher David: And instead of having some secret magic behind the scenes that you can't edit or even know about,

[16:45] Christopher David: we're saying that every, every, agentic flow will be open, will be definable in this

[16:58] Christopher David: visual scripting language that we are creating, borrowing from flow-based programming and other

[17:04] Christopher David: people who've done somewhat similar things over the years. We're working on the docs for this.

[17:12] Christopher David: Flows of nodes and some of those nodes will be things that we just offer at the system or platform

[17:21] Christopher David: level like a branching logic thing. That's simple enough. We'll just have our own version of it, but

[17:28] Christopher David: anything can be done as a plugin and upload it. We can run any arbitrary

[17:36] Christopher David: wasm plugin via this ex-tism framework. So, it's a little bit more context between our

[17:44] Christopher David: assertion that we are launching the OpenAI assistance API plus open source

[17:50] Christopher David: extensibility via ex-tism and monetization via Bitcoin payouts. What else? These API created agents

[18:01] Christopher David: will also be visible, listed, discoverable via the OpenAgence agent store.

[18:14] Christopher David: We'll do a nice big reveal of that when it's ready, but you heard it here first.

[18:19] Christopher David: So, you can make money by writing plugins or just composing agents out of other plugins or

[18:32] Christopher David: augmenting an agent with some unique data set you have access to. It's a marketplace for everything

[18:40] Christopher David: related to agent at workflows. So, that's it for now. I'll be traveling for the next week.

[18:51] Christopher David: Any conference called Bitcoin Atlantis. I just have to go to. That is have to go to. So, I'll be there.

[18:57] Christopher David: And I'm going to work it on the API docs and we'll continue soon. See ya.