dsh-surf-web

Web tools plugin for DeepSeek Harness: web search, page fetching and online-PDF reading, with your own local HTTP proxy support and built-in rate limiting.

中文文档

What it does

Registers three model tools into the host tools registry, so every session of your dsh deployment can use them (not just one conversation), and they survive restarts:

Tool	Description
surf_search	Web search. Engine chain: the deployment's own web.search (DeepSeek) → Bing → DuckDuckGo, tried strictly in order with automatic fallback.
surf_fetch	Fetch one URL and extract readable content (auto text, markdown, or raw html). Detects charset (GBK/UTF-8/…), strips nav/footer junk, decodes entities.
surf_pdf	Download one online PDF and extract its text page by page (supports pages="1-5" etc.). Pure-JS parser: xref tables & xref streams, object streams, FlateDecode/ASCII85/RLE/LZW, ToUnicode CMaps (incl. Chrome/Edge "print to PDF"), TeX ligatures and interword kerns.

Proxy support (your own "ladder"/VPN):

• proxy: auto (default) — reads HTTP_PROXY / HTTPS_PROXY / ALL_PROXY from the environment (Clash, V2Ray, …). If the proxy fails at the network level, it retries once directly (works with TUN mode too).
• proxy: none — direct connection.
• proxy: http://127.0.0.1:7897 — explicit proxy URL.

Politeness (anti-abuse, by design):

• Single-flight: at most one request in flight globally; ≥1.2 s between any two requests; ≥2 s between consecutive hits on the same host.
• One URL per tool call; max 5 redirects; one retry only on network-level failure; search engines tried sequentially, never in parallel.
• No background crawling, no link harvesting. Stopping the plugin stops everything.

Requirements

• Windows 10+ (uses the built-in curl.exe)
• DeepSeek Harness — works with both deployment styles:
  • npx @deepseek-ai/dsh web (npx cache install), or
  • git clone https://github.com/deepseek-ai/deepseek-harness (source install)
• The plugin package has no runtime npm dependencies beyond what dsh already ships (@deepseek-ai/dsh-tools).

Install (one command)

Copy this repository (or just install-surf.ps1 + the package files) to the target machine, then run:

# install into the default DSH_HOME (~/.dsh) web profile
powershell -NoProfile -ExecutionPolicy Bypass -File install-surf.ps1

# install into a different DSH_HOME / profile (e.g. a headless profile)
powershell -NoProfile -ExecutionPolicy Bypass -File install-surf.ps1 -DshHome D:\other\.dsh -ProfileName headless

# custom bundles when creating a brand-new profile
powershell -NoProfile -ExecutionPolicy Bypass -File install-surf.ps1 -Bundles '@deepseek-ai/dsh-base,@deepseek-ai/dsh-tui'

The script:

1. Copies the plugin package into every DSH package store it can find (the npx cache store, plus the <DshHome>/profiles/node_modules shared store — the flat fallback that makes the plugin work for git-clone deployments too).
2. Creates a complete profile if it does not exist yet (manifest with standard bundles + empty cordis.yml + the patch row), so a brand-new dsh environment works out of the box.
3. Inserts the surf-web row into <DshHome>/profiles/<ProfileName>/cordis.patch.yml (idempotent — safe to re-run).

Then restart dsh (or boot the profile). The three surf_* tools become available to every session.

Uninstall

powershell -NoProfile -ExecutionPolicy Bypass -File install-surf.ps1 -Remove

Removes the package from all stores and the patch row. Restart dsh to apply.

Manual install (no script)

For people who prefer to do it by hand:

1. Copy the package into <DshHome>/profiles/node_modules/@deepseek-ai/dsh-surf-web/ (create the path if needed).
2. Append to <DshHome>/profiles/<name>/cordis.patch.yml:

- insert:
    - id: surf-web
      name: '@deepseek-ai/dsh-surf-web'

3. Restart dsh.

Verified in a clean sandbox

The plugin was installed and tested end-to-end in brand-new, isolated dsh environments (fresh DSH_HOME, no pre-existing profiles) using the official one-shot runner dsh --profile headless "task":

• Install script on a fresh DSH_HOME (web + headless profiles auto-created) — OK
• No-npx-cache scenario (git-clone style deployment, empty LOCALAPPDATA) — OK
• surf_search (Bing engine, via curl + Clash proxy) — returned real results
• surf_fetch (example.com) — title + body extracted
• surf_pdf (arXiv 1706.03762, page 1) — title + abstract extracted
• Environment probe: curl, %TEMP%, proxy env vars auto-detected — OK

Architecture

install-surf.ps1        one-command install/uninstall (idempotent, UTF-8 safe, works on brand-new DSH_HOME)
package.json            standard ESM package (@deepseek-ai/dsh-surf-web)
lib/index.js            Cordis plugin body: export { apply, inject, name }; inject: subprocess/fs/tools;
                        uses the official @deepseek-ai/dsh-tools defineTool + ctx.tools.register
lib/surf-core.js        pure-JS core, zero dependencies: DEFLATE inflate, PDF object/xref/CMap parser,
                        HTML text/markdown extraction, Bing/DuckDuckGo result parsing

Known limitations:

• DuckDuckGo often blocks headless requests; the auto chain falls back to Bing.
• Scanned/image-only PDFs have no text layer and report "no extractable text".
• Windows only (uses curl.exe; the installer is PowerShell).

License

MIT — see LICENSE.