dsh-plugin-vision

August 14, 2026 · View on GitHub

Give DeepSeek Harness (dsh) the ability to see and create images. Two tools:

image_analyze: reads an image file through dsh's sandboxed fs service (workspace and permission policy apply), sends it to an OpenAI-compatible vision endpoint (chat/completions with image_url content), and returns a text description or answer.
image_generate: sends a text prompt to an OpenAI-compatible image endpoint (images/generations), saves the generated image into the workspace, and returns the file path. Before generating, an automatic prompt-engineering pass runs (see below).

The endpoint, models, and credential are all configuration — official OpenAI, relay/proxy services, or self-hosted OpenAI-compatible gateways all work.

Why this plugin exists: dsh's built-in DeepSeek adapter rejects image content (UNSUPPORTED_CONTENT). This plugin delegates "image → text" and "text → image" to dedicated models, so your main agent model needs no image support at all.

Features

Pluggable like every dsh plugin: appears in the Settings → Plugins → Plugin list, config changes hot-reload
image_analyze reads through the sandboxed filesystem — workspace and permission policy apply
OpenAI-compatible: configure baseUrl, models, and API key freely
image_analyze supports png / jpeg / gif / webp, with configurable size limits and timeouts
image_generate saves into the workspace (default generated-<timestamp>.png, or a path you choose) and refuses writes outside the workspace root
Automatic prompt engineering before generation: no extra model is configured or billed — the session's own model (the one dsh is already using, e.g. DeepSeek) understands the request, rewrites it into a structured image prompt (Goal / Main subject / Composition / Visual style / Lighting and color / Details / Text / Constraints), and picks an official output size. If the pass fails for any reason, it degrades gracefully to the original prompt and default size — generation is never blocked.

Requirements

A running DeepSeek Harness Web profile (install Node.js first, then npx -y @deepseek-ai/dsh web)
Recognition: a vision-capable chat model behind an OpenAI-compatible API — e.g. gpt-4o, gpt-4o-mini, gpt-4.1
Generation: an image-generation model — e.g. gpt-image-1, gpt-image-2, dall-e-3
⚠️ Generation models cannot read images, and vision chat models usually cannot generate them — configure model and generateModel separately

🚀 Don't want to do it yourself? Let your dsh agent install it

Copy the whole block below to your agent (the one you are chatting with right now) — it will do the installation for you:

Please install the dsh-plugin-vision image plugin for me:
Run git clone https://github.com/WUDIFIVE/dsh-plugin-vision.git (clone into a temp directory)

Copy the whole dsh-plugin-vision folder into ~/.dsh/profiles/web/dsh-plugin-vision (if the target exists, remove the old copy first)
Append the following to ~/.dsh/profiles/web/cordis.patch.yml (put the key in apiKey, not apiKeyEnv):
- insert:
    - id: vision
      name: './dsh-plugin-vision/lib/index.js'
      config:
        apiKey: sk-your-key
        baseUrl: https://api.openai.com/v1
        model: gpt-4o
        generateModel: gpt-image-2
        imageSize: 1024x1024
        detail: auto
        timeoutMs: 120000
        maxImageBytes: 15728640
        maxTokens: 1024
Check Settings → Plugins → Plugin list shows vision enabled

If writing to ~/.dsh needs permission approval, prompt me to allow it

💡 The agent may ask for permission to write outside the workspace — just approve it. Config changes usually hot-apply; if not, restart dsh web.

Or, if pnpm is already set up, give your agent this one-liner:

dsh plugin --profile web add github:WUDIFIVE/dsh-plugin-vision

Then have the agent append an entry with the short name name: dsh-plugin-vision to ~/.dsh/profiles/web/cordis.patch.yml (see "Standard install" below).

Installation (manual)

macOS / Linux

1. Install the plugin package into the profile (standard path, requires pnpm):

dsh plugin --profile web add github:WUDIFIVE/dsh-plugin-vision

Once published to npm: dsh plugin --profile web add dsh-plugin-vision. Enable pnpm with corepack enable, or npm install -g pnpm.

2. Add the entry to your profile patch — append to ~/.dsh/profiles/web/cordis.patch.yml:

⚠️ Common mistake (many people get this wrong): apiKey holds the key itself; apiKeyEnv holds an environment-variable NAME (e.g. OPENAI_API_KEY). Use one of the two: either hard-code the key in apiKey: sk-xxx, or leave apiKey empty and put the key into the environment / the credential store (web Models page). Do not put the key string into apiKeyEnv — the plugin will treat it as an environment-variable name, fail to find it, and report VISION_CREDENTIAL_MISSING.

- insert:
    - id: vision
      name: dsh-plugin-vision
      config:
        apiKey: sk-your-key        # ← the key goes here (or leave empty and use env/credentials)
        # apiKeyEnv: OPENAI_API_KEY   # ← this is an env-var NAME, not the key! pick one
        baseUrl: https://api.openai.com/v1
        model: gpt-4o              # recognition model (vision chat model)
        generateModel: gpt-image-2 # generation model (text-to-image model)
        imageSize: 1024x1024       # default generation size
        detail: auto
        timeoutMs: 120000
        maxImageBytes: 15728640
        maxTokens: 1024

3. Restart the Web UI (or let config hot-reload apply):

dsh web

Open Settings → Plugins → Plugin list: you should see vision as an enabled entry.

Windows (PowerShell)

Same three steps, with Windows paths:

# 1. install (requires pnpm: corepack enable, or npm install -g pnpm)
dsh plugin --profile web add github:WUDIFIVE/dsh-plugin-vision

# 2. append the entry above to:
#    $HOME\.dsh\profiles\web\cordis.patch.yml
#    (put the key in apiKey, not apiKeyEnv)

# 3. restart
dsh web

No-pnpm alternative (manual copy)

If you cannot install pnpm, copy the plugin folder into the profile directory and reference it by relative path:

# macOS / Linux
cp -R dsh-plugin-vision ~/.dsh/profiles/web/

# Windows PowerShell
Copy-Item dsh-plugin-vision -Recurse $HOME\.dsh\profiles\web\dsh-plugin-vision

Then use name: './dsh-plugin-vision/lib/index.js' in the patch entry above instead of dsh-plugin-vision. The plugin's dependencies resolve from the profile's hoisted node_modules, so no extra install is needed.

Configuration

Field	Default	Description
`apiKey`	`""`	The key itself. Leave empty to use the credential store / environment instead.
`apiKeyEnv`	`OPENAI_API_KEY`	An environment-variable NAME (not the key!). The plugin looks up the value under this name.
`baseUrl`	`https://api.openai.com/v1`	OpenAI-compatible API base URL (`/chat/completions` and `/images/generations` are appended).
`model`	`gpt-4o`	Recognition model id (vision chat model).
`generateModel`	`gpt-image-1`	Generation model id (text-to-image model).
`imageSize`	`1024x1024`	Default generation size (fallback when the prompt engineer picks nothing valid).
`promptEnhance`	`true`	Run the prompt-engineering pass before generation (rewrites the request and picks a size via the session's own model).
`promptEngineerMaxTokens`	`1600`	`max_tokens` for the prompt-engineering pass.
`allowedSizes`	official size list	Whitelist the prompt engineer may pick from: `1024x1024`, `1536x1024`, `1024x1536`, `2048x2048`, `2048x1152`, `3840x2160`, `2160x3840`.
`detail`	`auto`	Recognition `auto` \| `low` \| `high` — image sampling detail hint.
`timeoutMs`	`120000`	Hard cap for one tool call, including upload/generation.
`maxImageBytes`	`15728640` (15 MiB)	Maximum accepted image size for recognition.
`maxTokens`	`1024`	`max_tokens` for recognition answers.
`defaultPrompt`	`Describe this image in detail...`	Recognition prompt used when no `question` is given.

API key resolution order: apiKey config → credential service (what the web Models page writes) → apiKeyEnv environment variable.

Usage

Recognition (`image_analyze`)

The agent automatically calls it when it needs to understand an image. Tool arguments:

path (required): image file path — absolute, or relative to the workspace root. Supported formats: png, jpeg, gif, webp.
question (optional): a specific question about the image; when omitted, the image is described in detail.

Example prompts for your agent:

Look at screenshots/login-bug.png — what error is shown, and what is the state of the form fields?

Read scans/receipt.jpg and extract the total amount and date.

看一下工作区里的 docs/架构图.png，用中文总结里面的模块关系

Note: images uploaded directly in chat are not visible to the tool (and dsh's DeepSeek adapter rejects image uploads anyway). Save the image to disk first, then pass its path.

Generation (`image_generate`)

The agent automatically calls it when you ask for an image. Tool arguments:

prompt (required): the user's image request in natural language — no need to hand-write an English prompt yourself.
output_path (optional): where to save the image (relative to the workspace; must stay inside the workspace). Defaults to generated-<timestamp>.png.
size (optional): output size for this call, overriding automatic selection; valid values are the allowedSizes (official size list).

Built-in prompt engineering: before generation, the plugin asks the session's own model (the one dsh is already using, e.g. DeepSeek — no extra configuration or billing) to rewrite prompt into the eight-section structured image prompt and to pick an aspect ratio from allowedSizes — 1:1 for avatars/icons, 1536×1024-class sizes for horizontal banners, 1024×1536 or 2160×3840 for vertical posters/wallpapers, 2048-class sizes for high-resolution work. The result carries an enhanced flag; on any failure the pass degrades to the original prompt and the default size.

Example prompts for your agent:

Generate an image: a yellow cartoon cat wearing a pink tie, standing in a sunny office, save it to docs/logo.png

Draw a Chinese solar-term poster about "末伏" (last dog days of summer) with sunflowers and a setting sun, vertical layout

The generated file lands in the workspace; ask the agent for the path. You can then run image_analyze on your own generated image — recognition and generation close the loop.

How it works

image_analyze(path, question?)                     image_generate(prompt, output_path?, size?)
   │                                                       │
   ├─ ctx.fs.resolve/stat/readBytes                         ├─ [prompt engineering] ctx.llm.stream()
   ├─ detect media type (png/jpeg/gif/webp magic bytes)     │   session's own provider/model (the one dsh uses)
   ├─ data:image/...;base64                                 │   rewrites the request into a structured prompt
   └─ POST {baseUrl}/chat/completions                       │   and picks an official size (allowedSizes)
        messages: [{role:user, content:[                     │   ↓ on failure: original prompt + default size
          {type:text, text:question},                        ├─ POST {baseUrl}/images/generations
          {type:image_url, image_url:{url:dataUrl, detail}}  │    {model, prompt(enhanced), n:1, size, response_format:"b64_json"}
        ]}]                                                  ├─ decode base64 → detect type → pick extension
        → choices[0].message.content                         └─ write into the workspace (default generated-<timestamp>.png)

Development & testing

git clone https://github.com/WUDIFIVE/dsh-plugin-vision.git
cd dsh-plugin-vision
npm install        # installs the @deepseek-ai/* dependencies for local testing
node test/smoke.mjs

The smoke test runs 25 checks — module shape, both tool registrations, recognition request body (base64 data URL, model, question), the prompt-engineering pass (session-model driven, structured rewrite, automatic size selection, explicit-size override, illegal-size fallback, failure degradation, no-session-model degradation), generation request body and file write, out-of-workspace path rejection, credential resolution order, and error paths — with a stubbed fetch and ctx.llm, so no real API key is needed.

Troubleshooting

Error code	Meaning
`VISION_FS_UNAVAILABLE`	The deployment has no `fs` service.
`VISION_IMAGE_NOT_FOUND`	Recognition: file does not exist or is not a regular file.
`VISION_IMAGE_TOO_LARGE`	Recognition: exceeds `maxImageBytes`; resize or crop first.
`VISION_UNSUPPORTED_IMAGE`	Recognition: not a png/jpeg/gif/webp file.
`VISION_CREDENTIAL_MISSING`	No API key found — most common cause: the key was put into `apiKeyEnv` instead of `apiKey`.
`VISION_PROVIDER_ERROR`	Endpoint error — includes the upstream message (bad key, unknown model, model not supported on this interface, etc.).
`VISION_EMPTY_RESULT`	Recognition: the model returned no usable content.
`VISION_GENERATE_INVALID_PROMPT`	Generation: prompt is empty.
`VISION_GENERATE_EMPTY_RESULT`	Generation: the endpoint returned no valid image.
`VISION_GENERATE_PATH_OUTSIDE_WORKSPACE`	Generation: save path escapes the workspace root — write refused.
`VISION_ABORTED`	The tool call was cancelled.

Common symptoms and fixes:

"The interface does not currently support this model" / model not found: wrong model id, or the model does not support the endpoint you are using (e.g. a text-to-image model like gpt-image-2 placed in the recognition model). Recognition needs a vision chat model (e.g. gpt-4o); generation needs a text-to-image model (e.g. gpt-image-2). Don't mix the two fields.
VISION_CREDENTIAL_MISSING with an sk-... value in the message: the key was written into apiKeyEnv — move it to apiKey (see the warning above).

License

MIT