Transcription: OpenAgents Episode 083 - Multimodal Chat

Source: https://twitter.com/OpenAgentsInc/status/1761082689095090180 Wiki source: https://raw.githubusercontent.com/wiki/OpenAgentsInc/openagents/Video-Series.md Media title: OpenAgents - Episode 083: Multimodal Chat We upgrade our agent with the ability to... Upload date: 20240223 Transcription model: mlx-whisper/mlx-community/whisper-tiny Generated at: 2026-06-01T18:35:23Z

Machine-generated transcript. Review speaker labels and wording before using this as quote-grade source material.

[00:00] Christopher David: Welcome back. Our mission, these past few videos, has been to build an agent that can

[00:06] Christopher David: replace my day-to-day usage of chatGVT. Most of what I do is paste code and documentation

[00:13] Christopher David: in along with a request. So here's what we have. I can do things like summarized this.

[00:23] Christopher David: Extract URLs, scrape URLs, all of inference.

[00:29] Christopher David: And I should be able to do things like...

[00:36] Christopher David: Right, an example, Lair of L controller that could implement this.

[00:51] Christopher David: Got a few minor things still to do, like make this auto scroll down.

[01:00] Christopher David: The syntax highlighting doesn't kick in until after it finishes streaming.

[01:07] Christopher David: So there's some minor quality of life things.

[01:12] Christopher David: But...

[01:12] Christopher David: This lets me productive for a bunch of the things that I use

[01:23] Christopher David: chatGVT for instead in this interface.

[01:30] Christopher David: So we accomplished this couple areas for improvement, but this works as a

[01:38] Christopher David: of pull request 142. But before we declare victory here and bid a due to chatGVT,

[01:49] Christopher David: there is one more thing that I rely on chatGVT for, which is multimodal or image input.

[02:00] Christopher David: I like dropping it in a screenshot of a Figma document or the CSS that I have,

[02:11] Christopher David: the styling here and say, change it so it does something else.

[02:16] Christopher David: So we're going to try in this video to add image input.

[02:22] Christopher David: We got this nice little paperclip thingy for file input.

[02:29] Christopher David: And let's try it. I want to be able to drop in an image.

[02:38] Christopher David: Files, we had to done a previous version of that in the past.

[02:44] Christopher David: And we'll get back to PDFs, kind of revisiting how we want to do that. But adding images now

[02:55] Christopher David: would be absolutely huge. So we will do kind of what we did for this, which is start with the

[03:03] Christopher David: open AI. If you are, as long as we're building it in a way that we can easily swap in other

[03:11] Christopher David: from now, open AI, compatible APIs, which are plug-in and either supports or can be very easily

[03:18] Christopher David: extended to support. And we'll kind of branch out from there once we get the basic UI UX around

[03:29] Christopher David: industry-leading standard API implemented. We can add support for all sorts of other things.

[03:34] Christopher David: And the cool thing is that I'm not going to need to do all that myself because we have this plug-in

[03:38] Christopher David: system. We'll have community developers having the opportunity and financial incentive to build a lot

[03:45] Christopher David: of that themselves. So for multimodal, this is really just image input to the same chat completions

[03:57] Christopher David: endpoint that we are using currently. And you may note from the issue here, the GPT4 call

[04:08] Christopher David: was working fine as an XSM plug-in, but it seems like XSM plug-in's don't currently support

[04:15] Christopher David: streaming. So because we watch streaming, we implemented the third plug-in there with our own

[04:21] Christopher David: implementation, which we can see in the inferenceer.

[04:33] Christopher David: Yeah, just basically taking the history of the conversation, I think that the last 15 messages

[04:42] Christopher David: formatting that would pass in them to the GPT4 API.

[04:53] Christopher David: We want to extend this now with image input. And let's try using our own interface for driving this out.

[05:03] Christopher David: First let's clean this up. This has been a really nice passing into the system prompt the name of

[05:12] Christopher David: the agent and the description. So if I do something like say introduce yourself.

[05:22] Christopher David: Yeah, it's the same as our Connie agent defined previously. I just changed the name to junior

[05:35] Christopher David: developer. I'm junior developer, a highly advanced AI agent with expertise in providing information,

[05:40] Christopher David: guidance and understanding things in context. Yeah, junior developers not right, not like the most

[05:47] Christopher David: accurate name for this contextual inferenceer, but the idea is for it to become a drop-in,

[05:53] Christopher David: replace before a junior developer.

[05:58] Christopher David: I'll move these comments because it's kind of self-explanatory.

[06:07] Christopher David: I don't know if we need to do this.

[06:10] Christopher David: All the way in curious if we're duplicating anything here.

[06:25] Christopher David: Yeah, we are duplicating something.

[06:40] Christopher David: Great.

[06:45] Christopher David: Oh, let's make sure that still works with this.

[07:15] Christopher David: Okay.

[07:45] Christopher David: Okay.

[08:08] Christopher David: How do this on a branch called multi-modal?

[08:23] Christopher David: So for now, to more easily pass context to this agent assistant,

[08:43] Christopher David: I'm going to copy the URL of the current branch code.

[08:54] Christopher David: And this will at some point, like as soon as we have plugins for it,

[09:01] Christopher David: allow you to just specify a GitHub repo or a branch and they can smartly pull all of this for you.

[09:08] Christopher David: But for now.

[09:14] Christopher David: Now, I want your help to image input to this way the second, and I'll give you the

[09:44] Christopher David: hopefully I can image input docs.

[09:59] Christopher David: No, no, no, no, no, no.

[10:14] Christopher David: You're information is outdated.

[10:30] Christopher David: This is from the book and I gots.

[10:44] Christopher David: Pretty soon I'll be able to drop a link like this, but right now there's a bunch of HTML stuff we haven't cleaned.

[10:59] Christopher David: It's right.

[11:10] Christopher David: What?

[11:14] Christopher David: No, I was giving you an example of how you call the open AI input with an API with image input.

[11:44] Christopher David: You want to send an image URL along with a text for analysis to the open AI API.

[11:54] Christopher David: Here is how you can modify the influencer.php for multi-modal support.

[12:14] Christopher David: So I think one downside of our current implementation of this is we are not adding the

[12:31] Christopher David: file contents of these links to the chat history. It's only being passed to the current message.

[12:45] Christopher David: Let's easy enough to refactor. I might just need to do an additional prompt here.

[13:01] Christopher David: Now rewrite my influencer class accordingly.

[13:16] Christopher David: Here is the modified version of inference or class according to your demand.

[14:16] Christopher David: Thanks, except please refactor that controller to split into multiple methods.

[14:39] Christopher David: So LL inference method there will be more like a router checking to see if input has an image URL.

[14:53] Christopher David: If so using a different user, the appropriate controller method.

[15:03] Christopher David: Also extract logic, reused in both of those handled things to their own controller methods to

[15:21] Christopher David: reused.

[15:33] Christopher David: So we'll get this working with a demo image and then we'll hook up the image upload form.

[16:33] Christopher David: There will be a string or an array.

[16:40] Christopher David: I don't know if that's how I want it but we can try this.

[16:45] Christopher David: Can you get a copy button?

[16:55] Christopher David: Copy paste.

[17:02] Christopher David: Let's first see if we might have broken this conversation.

[17:15] Christopher David: We can come back to this I think. Let's start a new conversation.

[17:23] Christopher David: Make sure this works. Make sure it still works with this.

[17:32] Christopher David: I said one sentence.

[17:47] Christopher David: Okay. I'm going to commit this as

[17:53] Christopher David: inference or image input.

[18:24] Christopher David: I want to do a demo test of the multi-modal piece.

[18:41] Christopher David: I need an image URL. I think I'm going to use this one.

[18:53] Christopher David: Now we want to test this function.

[19:15] Christopher David: Test this multi-modal function.

[19:26] Christopher David: Now let's rewrite elementference at the top there.

[19:35] Christopher David: To take the user's string and create a new input object with that string as the text key and

[19:54] Christopher David: image URL of.

[19:59] Christopher David: of

[20:04] Christopher David: of

[20:07] Christopher David: a

[20:11] Christopher David: of

[20:14] Christopher David: a

[20:19] Christopher David: of

[20:24] Christopher David: of

[20:34] Christopher David: of

[20:44] Christopher David: a

[20:46] Christopher David: of

[20:49] Christopher David: a

[20:54] Christopher David: of

[20:59] Christopher David: of

[21:09] Christopher David: of

[21:13] Christopher David: of

[21:17] Christopher David: of

[21:19] Christopher David: of

[21:24] Christopher David: of

[21:31] Christopher David: the

[21:36] Christopher David: of

[21:39] Christopher David: of

[21:42] Christopher David: of

[21:44] Christopher David: of

[21:49] Christopher David: of

[21:56] Christopher David: of

[21:59] Christopher David: of

[22:03] Christopher David: of

[22:07] Christopher David: of

[22:09] Christopher David: of

[22:14] Christopher David: of

[22:21] Christopher David: of

[22:26] Christopher David: of

[22:30] Christopher David: of

[22:32] Christopher David: of

[22:34] Christopher David: of

[22:37] Christopher David: of

[22:42] Christopher David: of

[22:46] Christopher David: of

[22:51] Christopher David: of

[22:55] Christopher David: of

[22:59] Christopher David: of

[23:01] Christopher David: of

[23:06] Christopher David: of

[23:11] Christopher David: of

[23:15] Christopher David: of

[23:18] Christopher David: of

[23:21] Christopher David: of

[23:24] Christopher David: of

[23:26] Christopher David: of

[23:29] Christopher David: of

[23:31] Christopher David: of

[23:35] Christopher David: of

[23:39] Christopher David: of

[23:44] Christopher David: of

[23:49] Christopher David: of

[23:54] Christopher David: of

[23:59] Christopher David: of

[24:04] Christopher David: of

[24:08] Christopher David: of

[24:12] Christopher David: of

[24:15] Christopher David: of

[24:21] Christopher David: of

[24:28] Christopher David: of

[24:33] Christopher David: of

[24:37] Christopher David: of

[24:42] Christopher David: of

[24:47] Christopher David: of

[24:52] Christopher David: of

[24:57] Christopher David: of

[25:02] Christopher David: of

[25:06] Christopher David: of

[25:08] Christopher David: of

[25:11] Christopher David: of

[25:16] Christopher David: of

[25:19] Christopher David: of

[25:21] Christopher David: of

[25:24] Christopher David: of

[25:26] Christopher David: of

[25:28] Christopher David: of

[25:30] Christopher David: of

[25:35] Christopher David: of

[25:40] Christopher David: of

[25:45] Christopher David: of

[25:50] Christopher David: of

[25:53] Christopher David: of

[25:57] Christopher David: of

[25:58] Christopher David: of

[26:01] Christopher David: of

[26:03] Christopher David: of

[26:05] Christopher David: of

[26:08] Christopher David: of

[26:13] Christopher David: of

[26:16] Christopher David: of

[26:18] Christopher David: of

[26:21] Christopher David: of

[26:23] Christopher David: of

[26:25] Christopher David: of

[26:27] Christopher David: of

[26:31] Christopher David: of

[26:33] Christopher David: of

[26:36] Christopher David: of

[26:41] Christopher David: of

[26:44] Christopher David: of

[26:46] Christopher David: of

[26:49] Christopher David: of

[26:51] Christopher David: of

[26:53] Christopher David: of

[26:55] Christopher David: of

[27:00] Christopher David: of

[27:05] Christopher David: of

[27:10] Christopher David: of

[27:15] Christopher David: of

[27:20] Christopher David: of

[27:21] Christopher David: of

[27:24] Christopher David: of

[27:29] Christopher David: of

[27:34] Christopher David: of

[27:39] Christopher David: of

[27:44] Christopher David: of

[27:49] Christopher David: of

[27:53] Christopher David: of

[27:58] Christopher David: of

[28:03] Christopher David: of

[28:08] Christopher David: of

[28:11] Christopher David: of

[28:13] Christopher David: of

[28:16] Christopher David: of

[28:18] Christopher David: of

[28:20] Christopher David: of

[28:22] Christopher David: of

[28:26] Christopher David: of

[28:31] Christopher David: of

[28:35] Christopher David: of

[28:39] Christopher David: of

[28:41] Christopher David: of

[28:44] Christopher David: of

[28:46] Christopher David: of

[28:48] Christopher David: of

[28:50] Christopher David: of

[28:55] Christopher David: of

[28:57] Christopher David: of

[29:00] Christopher David: of

[29:02] Christopher David: of

[29:05] Christopher David: of

[29:08] Christopher David: of

[29:10] Christopher David: of

[29:13] Christopher David: of

[29:15] Christopher David: of

[29:17] Christopher David: of

[29:19] Christopher David: of

[29:24] Christopher David: of

[29:26] Christopher David: of

[29:29] Christopher David: of

[29:31] Christopher David: of

[29:34] Christopher David: of

[29:37] Christopher David: of

[29:39] Christopher David: of

[29:42] Christopher David: of

[29:44] Christopher David: of

[29:46] Christopher David: of

[29:48] Christopher David: of

[29:53] Christopher David: of

[29:58] Christopher David: of

[30:03] Christopher David: of

[30:08] Christopher David: of

[30:13] Christopher David: of

[30:17] Christopher David: of

[30:22] Christopher David: of

[30:27] Christopher David: of

[30:30] Christopher David: of

[30:35] Christopher David: of

[30:37] Christopher David: of

[30:42] Christopher David: of

[30:44] Christopher David: of

[30:46] Christopher David: of

[30:51] Christopher David: of

[30:56] Christopher David: of

[31:01] Christopher David: of

[31:06] Christopher David: of

[31:11] Christopher David: of

[31:15] Christopher David: of

[31:20] Christopher David: of

[31:25] Christopher David: of

[31:30] Christopher David: of

[31:35] Christopher David: of

[31:40] Christopher David: of

[31:44] Christopher David: of

[31:49] Christopher David: of

[31:54] Christopher David: of

[31:58] Christopher David: of

[32:02] Christopher David: of

[32:04] Christopher David: of

[32:07] Christopher David: of

[32:10] Christopher David: of

[32:13] Christopher David: of

[32:17] Christopher David: of

[32:22] Christopher David: of

[32:27] Christopher David: of

[32:32] Christopher David: of

[32:37] Christopher David: of

[32:38] Christopher David: of

[32:41] Christopher David: of

[32:59] Christopher David: of

[33:01] Christopher David: of

[33:04] Christopher David: of

[33:06] Christopher David: of

[33:08] Christopher David: of

[33:10] Christopher David: of

[33:15] Christopher David: of

[33:20] Christopher David: of

[33:25] Christopher David: of

[33:30] Christopher David: of

[33:34] Christopher David: of

[33:35] Christopher David: of

[33:37] Christopher David: of

[33:40] Christopher David: of

[33:44] Christopher David: of

[33:49] Christopher David: of

[33:53] Christopher David: of

[33:55] Christopher David: of

[33:57] Christopher David: of

[33:59] Christopher David: of

[34:01] Christopher David: of

[34:03] Christopher David: of

[34:05] Christopher David: of

[34:07] Christopher David: of

[34:12] Christopher David: of

[34:17] Christopher David: of

[34:22] Christopher David: of

[34:25] Christopher David: of

[34:27] Christopher David: of

[34:32] Christopher David: of

[34:34] Christopher David: of

[34:37] Christopher David: of

[34:41] Christopher David: of

[34:46] Christopher David: of

[34:51] Christopher David: of

[34:54] Christopher David: of

[34:56] Christopher David: of

[34:59] Christopher David: of

[35:02] Christopher David: of

[35:03] Christopher David: of

[35:06] Christopher David: of

[35:10] Christopher David: of

[35:12] Christopher David: of

[35:15] Christopher David: of

[35:17] Christopher David: of

[35:20] Christopher David: of

[35:21] Christopher David: of

[35:23] Christopher David: of

[35:25] Christopher David: of

[35:26] Christopher David: of

[35:28] Christopher David: of

[35:30] Christopher David: of

[35:32] Christopher David: of

[35:35] Christopher David: of

[35:39] Christopher David: of

[35:44] Christopher David: of

[35:49] Christopher David: of

[35:54] Christopher David: of

[35:59] Christopher David: of

[36:03] Christopher David: of

[36:08] Christopher David: of

[36:13] Christopher David: of

[36:18] Christopher David: of

[36:23] Christopher David: of

[36:28] Christopher David: of

[36:30] Christopher David: of

[36:32] Christopher David: of

[36:37] Christopher David: of

[36:42] Christopher David: of

[36:52] Christopher David: of

[36:57] Christopher David: of

[37:01] Christopher David: of

[37:06] Christopher David: of

[37:11] Christopher David: of

[37:16] Christopher David: of

[37:19] Christopher David: of

[37:21] Christopher David: of

[37:24] Christopher David: of

[37:26] Christopher David: of

[37:29] Christopher David: of

[37:34] Christopher David: of

[37:39] Christopher David: of

[37:43] Christopher David: of

[37:44] Christopher David: of

[37:47] Christopher David: of

[37:48] Christopher David: of

[37:52] Christopher David: of

[37:54] Christopher David: of

[37:56] Christopher David: of

[37:59] Christopher David: of

[38:03] Christopher David: of

[38:05] Christopher David: of

[38:08] Christopher David: of

[38:10] Christopher David: of

[38:13] Christopher David: of

[38:14] Christopher David: of

[38:16] Christopher David: of

[38:18] Christopher David: of

[38:21] Christopher David: of

[38:23] Christopher David: of

[38:25] Christopher David: of

[38:27] Christopher David: of

[38:32] Christopher David: of

[38:37] Christopher David: of

[38:42] Christopher David: of

[38:47] Christopher David: of

[38:52] Christopher David: of

[38:54] Christopher David: of

[38:55] Christopher David: of

[38:56] Christopher David: of

[39:01] Christopher David: of

[39:03] Christopher David: of

[39:06] Christopher David: of

[39:08] Christopher David: of

[39:11] Christopher David: of

[39:12] Christopher David: of

[39:14] Christopher David: of

[39:16] Christopher David: of

[39:18] Christopher David: of

[39:19] Christopher David: of

[39:21] Christopher David: of

[39:23] Christopher David: of

[39:25] Christopher David: of

[39:30] Christopher David: of

[39:35] Christopher David: of

[39:40] Christopher David: of

[39:45] Christopher David: of

[39:50] Christopher David: of

[39:52] Christopher David: of

[39:54] Christopher David: of

[39:59] Christopher David: of

[40:03] Christopher David: of

[40:08] Christopher David: of

[40:12] Christopher David: of

[40:16] Christopher David: of

[40:19] Christopher David: of

[39:59] Christopher David: We won't have streaming here.

[40:24] Christopher David: We have our response.

[40:43] Christopher David: This image depicts a beautiful natural landscape.

[40:48] Christopher David: In the foreground, there is a wooden boardwalk that provides blah blah blah.

[40:56] Christopher David: Okay, that's nice.

[41:14] Christopher David: Well, now can we stream that?

[41:20] Christopher David: Please.

[41:27] Christopher David: Oh yeah.

[41:44] Christopher David: Okay, we have streaming.

[41:45] Christopher David: That's nice.

[41:55] Christopher David: Streaming works with hard coded image.

[42:03] Christopher David: Cool.

[42:11] Christopher David: So can we pass it a base 64 or what?

[42:42] Christopher David: Not seen by her.

[42:55] Christopher David: You can pass multiple images.

[43:05] Christopher David: Either a URL of the image or the base 64 encoded image data.

[43:34] Christopher David: Well, because we can do the basics before encoded data, I think that makes our implementation easier

[43:45] Christopher David: because we don't need to manage file uploads.

[43:47] Christopher David: We're not going to think right now about saving files.

[43:51] Christopher David: Just take what the user uploads.

[43:55] Christopher David: Get the base 64.

[43:57] Christopher David: Throw that to the API call here.

[44:03] Christopher David: Let's try something.

[44:12] Christopher David: We'll try asking.

[44:30] Christopher David: Do you develop here?

[44:36] Christopher David: Even though it's going to append an image, do you?

[44:39] Christopher David: Now help me figure out how many I support for image uploads to our interface.

[44:59] Christopher David: So instead of that hard coded image, we pass it in.

[45:05] Christopher David: Don't say much until I also add code for the chat input field, etc.

[45:35] Christopher David: I may need to learn about best practices for image uploads.

[46:30] Christopher David: I'm analyzing both the code and also the image I submitted.

[47:00] Christopher David: Open AI wrapper.

[47:10] Christopher David: What files are you to paste in here?

[47:15] Christopher David: Chat.

[47:23] Christopher David: Let's do this first.

[47:36] Christopher David: Let's make two of our tokens.

[48:06] Christopher David: Let's do this.

[48:19] Christopher David: Let's do this.

[48:49] Christopher David: Let's do this.

[48:59] Christopher David: Let's do this.

[49:22] Christopher David: Live wire with file uploads.

[49:34] Christopher David: That's cool.

[49:43] Christopher David: Let's start with that.

[50:13] Christopher David: Let's do this.

[50:34] Christopher David: Let's do this.

[51:04] Christopher David: Let's do this.

[51:34] Christopher David: Let's do this.

[51:54] Christopher David: Let's do this.

[52:24] Christopher David: Let's do this.

[52:39] Christopher David: Let's do this.

[52:59] Christopher David: Let's do this.

[53:29] Christopher David: Let's do this.

[53:44] Christopher David: Let's do this.

[54:14] Christopher David: Let's do this.

[54:32] Christopher David: Let's do this.

[55:00] Christopher David: Let's do this.

[55:25] Christopher David: Let's download that one image.

[55:51] Christopher David: Let's do this.

[56:19] Christopher David: Let's do this.

[56:44] Christopher David: Let's do this.

[57:14] Christopher David: Let's do this.

[57:42] Christopher David: Let's do this.

[58:09] Christopher David: Let's do this.

[58:39] Christopher David: Let's do this.

[59:09] Christopher David: Let's do this.

[59:34] Christopher David: Let's do this.

[59:59] Christopher David: here where I later pass input to

[01:00:06] Christopher David: inferencer element for instance

[01:00:11] Christopher David: that's where I need to have the base 64 image

[01:00:19] Christopher David: base 64 of the image passed as image URL

[01:00:28] Christopher David: to the original function I gave

[01:00:32] Christopher David: and remove where I hard-coded

[01:00:36] Christopher David: an image

[01:00:40] Christopher David: note there could be multiple images

[01:00:44] Christopher David: and there can be multiple images

[01:00:48] Christopher David: in that same message

[01:00:51] Christopher David: object

[01:00:57] Christopher David: let's see

[01:01:28] Christopher David: let's see

[01:01:48] Christopher David: let's try

[01:01:57] Christopher David: let's see

[01:02:18] Christopher David: let's see

[01:02:22] Christopher David: what this looks like

[01:02:32] Christopher David: okay

[01:02:40] Christopher David: no

[01:02:42] Christopher David: I must have another dd

[01:02:52] Christopher David: text

[01:02:59] Christopher David: as df

[01:03:04] Christopher David: images

[01:03:07] Christopher David: let's see

[01:03:12] Christopher David: I don't know if that's correct

[01:03:19] Christopher David: first let's remove the dd

[01:03:27] Christopher David: first let's remove the dd

[01:03:34] Christopher David: first let's remove the dd

[01:03:42] Christopher David: first let's see what it looks like at this point

[01:03:52] Christopher David: input

[01:04:02] Christopher David: input is this

[01:04:05] Christopher David: thing

[01:04:52] Christopher David: you ready

[01:04:59] Christopher David: make it work

[01:05:22] Christopher David: let's assume it's simplified scenario

[01:05:38] Christopher David: don't

[01:05:40] Christopher David: but we'll try

[01:05:53] Christopher David: make it this first

[01:06:08] Christopher David: give me the whole thing

[01:06:11] Christopher David: this try

[01:06:22] Christopher David: there's no way this will work

[01:06:44] Christopher David: let's see

[01:06:51] Christopher David: let's see

[01:06:58] Christopher David: let's see

[01:07:16] Christopher David: let's see

[01:07:23] Christopher David: I'm guessing this just needs some sort of base 64 prefix

[01:07:49] Christopher David: let's see

[01:07:56] Christopher David: let's see

[01:08:04] Christopher David: let's see

[01:08:12] Christopher David: let's see

[01:08:19] Christopher David: let's see

[01:08:26] Christopher David: let's see

[01:08:33] Christopher David: let's see

[01:08:40] Christopher David: first let's see

[01:08:55] Christopher David: let's see

[01:09:02] Christopher David: if

[01:09:08] Christopher David: let's see

[01:09:15] Christopher David: if

[01:09:23] Christopher David: let's see

[01:09:30] Christopher David: let's see

[01:09:37] Christopher David: let's see

[01:09:43] Christopher David: let's see

[01:09:51] Christopher David: what are we doing wrong

[01:09:59] Christopher David: let's see

[01:10:06] Christopher David: first let's see

[01:10:13] Christopher David: what is that

[01:10:20] Christopher David: do

[01:10:33] Christopher David: no

[01:10:38] Christopher David: let's see

[01:10:45] Christopher David: let's see

[01:10:55] Christopher David: let's see

[01:11:02] Christopher David: let's see

[01:11:32] Christopher David: let's see

[01:11:54] Christopher David: let's see

[01:12:01] Christopher David: let's see

[01:12:08] Christopher David: let's see

[01:12:15] Christopher David: let's see

[01:12:21] Christopher David: let's see

[01:12:26] Christopher David: let's see

[01:12:34] Christopher David: let's see

[01:12:41] Christopher David: let's see

[01:12:47] Christopher David: let's see

[01:12:54] Christopher David: let's see

[01:13:00] Christopher David: let's see

[01:13:09] Christopher David: let's see

[01:13:19] Christopher David: let's see

[01:13:24] Christopher David: let's see

[01:13:30] Christopher David: let's see

[01:13:37] Christopher David: let's see

[01:13:43] Christopher David: let's see

[01:13:51] Christopher David: let's see

[01:13:58] Christopher David: let's see

[01:14:04] Christopher David: let's see

[01:14:13] Christopher David: let's see

[01:14:23] Christopher David: let's see

[01:14:28] Christopher David: let's see

[01:14:34] Christopher David: let's see

[01:14:55] Christopher David: let's see

[01:15:03] Christopher David: let's see

[01:15:08] Christopher David: let's see

[01:15:16] Christopher David: let's see

[01:15:23] Christopher David: let's see

[01:15:29] Christopher David: let's see

[01:15:37] Christopher David: let's see

[01:16:04] Christopher David: let's see

[01:16:11] Christopher David: let's see

[01:16:18] Christopher David: let's see

[01:16:23] Christopher David: let's see

[01:16:33] Christopher David: let's see

[01:16:38] Christopher David: let's see

[01:16:44] Christopher David: let's see

[01:16:54] Christopher David: let's see

[01:17:03] Christopher David: let's see

[01:17:10] Christopher David: let's see

[01:17:19] Christopher David: let's see

[01:17:24] Christopher David: let's see

[01:17:28] Christopher David: let's see

[01:17:32] Christopher David: let's see

[01:17:39] Christopher David: let's see

[01:17:45] Christopher David: let's see

[01:17:55] Christopher David: let's see

[01:18:00] Christopher David: let's see

[01:18:06] Christopher David: let's see

[01:18:16] Christopher David: let's see

[01:18:25] Christopher David: let's see

[01:18:30] Christopher David: let's see

[01:18:40] Christopher David: let's see

[01:18:45] Christopher David: let's see

[01:18:51] Christopher David: let's see

[01:19:01] Christopher David: let's see

[01:19:06] Christopher David: let's see

[01:19:12] Christopher David: let's see

[01:19:35] Christopher David: let's see

[01:19:44] Christopher David: let's see

[01:19:49] Christopher David: let's see

[01:19:57] Christopher David: let's see

[01:19:59] Christopher David: let's see

[01:20:59] Christopher David: Okay.

[01:21:20] Christopher David: Okay.

[01:21:27] Christopher David: Image URL.

[01:21:42] Christopher David: Maybe.

[01:21:49] Christopher David: Okay.

[01:21:50] Christopher David: Looks like that might have made some progress.

[01:21:57] Christopher David: Oops.

[01:21:58] Christopher David: Okay.

[01:22:07] Christopher David: Okay.

[01:22:11] Christopher David: A photograph is being passed in.

[01:22:25] Christopher David: Now I want the bytes.

[01:22:33] Christopher David: Let's see if this works.

[01:22:50] Christopher David: Okay.

[01:22:52] Christopher David: Moment of truth, maybe.

[01:22:54] Christopher David: Upload.

[01:22:55] Christopher David: Image.

[01:22:56] Christopher David: What is this?

[01:23:01] Christopher David: You're reckoning.

[01:23:04] Christopher David: Invalid image.

[01:23:05] Christopher David: Okay.

[01:23:07] Christopher David: Probably we just need a basic C4.

[01:23:17] Christopher David: Watch.

[01:23:18] Christopher David: Okay.

[01:23:35] Christopher David: What was that example we had previously?

[01:23:45] Christopher David: Okay.

[01:23:52] Christopher David: Passing an image URL works.

[01:23:56] Christopher David: But not the base 64 bytes.

[01:24:00] Christopher David: I think we need to prefix it.

[01:24:04] Christopher David: Open AI wants this.

[01:24:07] Christopher David: How do we pass the bytes?

[01:24:12] Christopher David: Okay.

[01:24:27] Christopher David: Yeah.

[01:24:28] Christopher David: I have to extract the mine type and pass that in and we can get that from the image upload.

[01:24:37] Christopher David: Okay.

[01:24:47] Christopher David: Okay.

[01:25:04] Christopher David: Okay.

[01:25:14] Christopher David: Please.

[01:25:24] Christopher David: What is this?

[01:25:36] Christopher David: Okay.

[01:25:38] Christopher David: You uploaded an unsupported image.

[01:25:40] Christopher David: Please make sure.

[01:25:42] Christopher David: All right.

[01:25:49] Christopher David: What do I upload?

[01:26:01] Christopher David: This is a PNG.

[01:26:04] Christopher David: 315KB.

[01:26:30] Christopher David: Is it because I did this?

[01:26:37] Christopher David: Was it already encoded?

[01:26:47] Christopher David: What is this?

[01:26:56] Christopher David: What is this?

[01:27:06] Christopher David: You provided an image of a screen that seems to show a conversation between a user and an AI agent named

[01:27:11] Christopher David: Do your developer within a chat interface.

[01:27:16] Christopher David: It's reading the code.

[01:27:22] Christopher David: Game over.