DESIGN.md

August 29, 2026 · View on GitHub

English · 中文

Status: maintained — this document describes the current implementation and repository structure. It is the reference for the README.

1. Why

Agentic coding tools run long tasks (builds, tests, migrations, batch edits) while you work on something else. When a reply finally lands you have to keep checking the screen. dsh-speak reads the final reply aloud through system speech synthesis (Windows SAPI5 / macOS say) so you know without looking that a long task finished — and what its outcome was.

The original implementation was built and proven in a local DSH (DeepSeek Harness) setup. This repository generalizes that working implementation into:

  • a harness-agnostic engine (PowerShell + Windows SAPI5 / bash + macOS say) that any process can call,
  • adapter layers that turn harness-specific events into engine calls (DSH session events, Claude Code Stop hooks, ...).

2. Goals / non-goals

Goals:

  • One-command install for DSH users (engine + plugin + registration).
  • Engine callable from any harness via a trivial command line.
  • Best-effort speech: never throws, never blocks a harness, never breaks a session.
  • Natural-sounding voices: Windows 11 built-in natural voice packs, or NaturalVoiceSAPIAdapter on Windows 10; graceful fallback to stock voices.

Non-goals (for now):

  • Linux/headless TTS is not supported (Windows uses speak.ps1 + SAPI5; macOS uses speak.sh + the built-in say, shipped in the npm package since 1.2.0).
  • In-repo packaging of NaturalVoiceSAPIAdapter (Windows 10 only) or voice data — they are prerequisites, not bundled.
  • Per-voice audio files, non-Chinese voice curation (the speech playback queue is already implemented in 1.7.0 — see §3.2's host FIFO queue).

3. Architecture

            +--------------------------------------------------------------+
            |                         harness                               |
            |   (DSH web app  |  Claude Code  |  anything with a shell)     |
            +--------+-----------------------------+-----------------------+
                     |                             |
                     | session events              | Stop hook JSON (stdin)
                     v                             v
            +------------------+         +--------------------------+
            |  adapters/dsh/   |         | adapters/claude-code/    |
            |  speech-hook.js  |         | stop-hook.ps1            |
            |  (event filter,  |         | (transcript extraction)  |
            |   throttle,      |         +------------+-------------+
            |   cancel)        |                      |
            +--------+---------+                      |
                     | text                           | text
                     v                                v
            +---------------------------------------------------------------+
            |              engine/speak.ps1 / speak.sh (harness-agnostic)   |
            |   text -> clean (markdown/emoji/length) -> system speech      |
            +--------------------+------------------------------------------+
                     |                                   |
                     v                                   v
            Windows SAPI5 (System.Speech) —   macOS say (system voice):
              voices:                           * default follows the system
              * preferred: a natural voice —      voice (may be a Siri voice;
                 Windows 11 built-in pack, or      not listed by `say -v '?'`,
                 one registered by                 not selectable by name)
                 NaturalVoiceSAPIAdapter on      * or -v forces a classic voice
                 Windows 10 (e.g. "Microsoft      (Eddy / Tingting / Flo ...)
                 Xiaoxiao")                      * no volume flag (follows
              * fallback:  any zh voice (e.g.      the system output)
                 "Microsoft Huihui")

3.1 Engine — engine/speak.ps1 (+ engine/speak.sh on macOS)

The only file a new adapter needs. Two input modes: -Text "..." inline, or -File C:\path\msg.txt (UTF-8). Also -Volume, -Rate, -MaxChars, -LongTextMessage (see §5). On macOS the plugin auto-picks speak.sh (the say command; default voice follows the system — the Siri voices "声音 1-4" are not exposed to say, use -v to force a name; no volume flag).

Processing pipeline (in order):

  1. Read text (file read is always UTF-8).
  2. Strip markdown — code blocks, inline code, links, bare URLs, emphasis chars.
  3. Strip emoji / non-printable — keep CJK, CJK punctuation, full-width ranges, ASCII printable (regex [^一-龥　-〿＀-￯ - -~]).
  4. Collapse whitespace.
  5. Length guard — if cleaned text exceeds MaxChars (default 300), replace with LongTextMessage (default: 本次播报内容较长，请自行阅读。).
  6. Speak — System.Speech.Synthesis.SpeechSynthesizer, volume/rate applied, best zh natural voice selected, then Speak().

Engine contract for adapters:

  • exit 0 always; never writes to stdout/stderr on failure paths;
  • synchronous (returns when the utterance finishes, or immediately on any failure);
  • safe to call from a sandboxed process provided the caller does not need to nest another powershell.exe inside a harness sandbox (see §6.3).

3.2 DSH adapter — adapters/dsh/speech-hook.js

A DSH web-profile plugin (Cordis plugin) registered via cordis.patch.yml. DSH has no "reply finished" hook, so the plugin observes the session event stream:

  • listens to session/event;
  • filters assistant/message events with surfaceOp == 'append';
  • extracts only text content blocks (reasoning / tool_use blocks are skipped);
  • default mode: buffers the text and starts a throttle timer (default 1500 ms) to merge multi-step messages of one reply; a tool/call event cancels the pending announcement (that round's assistant text is process narration), but turn/end fallback-announces the final reply (tool-calling replies are still heard);
  • host speech queue (1.7.0, from PR #2): every announcement (final reply, approvals, questions, optional events, manual replay) goes through a FIFO queue — only one native speech process runs at a time, queued items continue automatically. A /dsh-speak/control POST route (play/stop/status) and a /dsh-speak/ws WebSocket broadcast the authoritative speech state (which message is speaking, queue length).
  • Final-reply replay (1.7.0): the 🔊 button in the turn-tail (final reply) action bar calls the control route to replay that final message; speech execution stays fully owned by the host (keeps speaking even with the browser closed).
  • queueAllMessages switch (1.7.0, default off): off = throttled final reply + optional events (as before); on = every assistant message is enqueued immediately (intermediate messages spoken too).
  • Optional event announcements (1.6.0, all off by default): turn/end, command/done, goal/change, tool/result (on error), and todo/write each have an independent toggle and announce a fixed phrase on fire (see §5).
  • Settings namespace registration (1.6.0): one timer tick after apply the plugin calls installSettingsSection(ctx, 'dsh-speak', schema, patchConfig, hooks), resolving config as schema default → patch config → UI user layer. onChange re-derives cfg from a saved settingsSource() thunk (note: installSettingsSection only calls setSource on attach/detach, so changes must be re-read in onChange). On hosts without a settings service (dsh < 0.1.0-rc.7 or no provider mounted) the registration is skipped silently and the plugin works purely from the patch config — backward compatible.

Registration snippet (also automated by install.ps1; npm installs use the bare package name 'dsh-speak' — this is the file-install path):

# ~/.dsh/profiles/web/cordis.patch.yml
- insert:
    - id: speech-hook
      # replace <your-username> with your Windows username
      name: 'file:///C:/Users/<your-username>/.dsh/profiles/web/plugins/speech-hook.js'

Node's ESM loader does not accept Windows absolute paths as plugin names — the file:///C:/... URL form is required.

3.4 DSH browser half — client/client.js

A DSH client bundle (window.__ModuleLoader__.load({ id: 'dsh-speak', factory })) that registers two pieces of UI:

  • Turn-tail Speak button (1.7.0, from PR #2): registered into the conversation.chat.assistant-actions slot (the turn's final-reply action bar). Clicking 🔊 POSTs to /dsh-speak/control to replay that final message; clicking again stops; clicking another switches. The button's speaking/paused state is derived from the authoritative host state over the /dsh-speak/ws WebSocket (matched by session + turn identity).

  • Settings → dsh-speak settings page (1.7.0): registered into the settings.section slot, drawn with @deepseek-ai/dsh-client-ui-primitives (Button / DisclosureRow / Input; Toggle / Options / SettingInput helpers). Every option (master switch, automatic speech, queueAllMessages, Markdown cleaning, code blocks, maxChars, longTextMode, fixed prompt, approvals/questions, the five optional events) is read/written through settingsScope.bind({ namespace: 'dsh-speak' }).

  • The package declares its browser half via package.json dsh.client: { platform: 'web' } + exports['./client']; DSH's client-modules scanner picks it up and loads it automatically.

  • Deliberately handwritten, zero build: it only uses platform seed modules and official primitives (the bundle-purity gate allows primitives but forbids importing official package internals), matching the built bundles' contract.

3.3 Claude Code adapter — adapters/claude-code/stop-hook.ps1

Claude Code does have a Stop hook. The hook JSON (with transcript_path) arrives on stdin; the script scans the transcript backwards for the last assistant message that contains text (the final entry is often a pure tool call), writes it to a temp file and launches the engine in its own hidden powershell process, so the hook returns immediately. (Async spawning is safe here — the nested-spawn restriction in §6.3 is specific to DSH's sandbox.)

4. Event-flow truth table (DSH)

assistant round / eventannounced?
final text reply, no tool call✅ after throttle
text + tool/call(s)🟡 throttle cancelled (intermediate); fallback-announced at turn end
text + ask_user_question call✅ each question announced separately: "问题N" prefix (when several) + "选项N" numbering, questionGapMs pause between questions
approval/asked✅ immediately (reason, else a fixed prompt)
reasoning only, no text❌ (no text block)
streaming chunks❌ (filtered)
turn/end🟡 off by default; announces "第 N 轮对话完成/中断/异常结束"
command/done🟡 off by default; announces "命令执行完成/失败"
goal/change🟡 off by default; announces "已创建目标/目标已完成…" (head)
tool/result🟡 off by default; announces "工具调用出错" only when error or an isError content block is present (English details / technical codes are dropped, Chinese details kept)
todo/write🟡 off by default; announces "待办已更新：n/m 完成"
assistant/message (queueAllMessages on)✅ every message enqueued immediately (intermediate spoken too)
manual replay (per-message 🔊)✅ clear queue → stop current → speak that turn

5. Configuration reference

Engine (speak.ps1 parameters)

paramdefaultmeaning
-Text''inline text (used when -File is empty)
-File''UTF-8 file to read
-Volume500–100
-Rate1speech rate (SAPI scale)
-MaxCharsplatformbeyond this, replaced by LongTextMessage (macOS default 0 = unlimited)
-LongTextMessage本次播报内容较长，请自行阅读。spoken instead of over-long text
-LongTextModemessagemessage (fixed prompt) | heading (speak the largest markdown heading)
-CleanMarkdownFormattingtrueconvert Markdown to natural speech (link labels kept, URLs stripped)
-ReadInlineCodetrueread inline code without backtick markers
-CodeBlockssmartall | smart | replace (fenced code blocks)
-CodeBlockMaxChars300smart-mode code block character limit
-CodeBlockReplacementTextYou can see the code in our history.spoken in replace mode

DSH plugin (profile config; since 1.7.0 also editable in the Settings → dsh-speak settings page)

config:
  enabled: true                # master switch
  automaticSpeech: true        # auto-speak final replies
  queueAllMessages: false      # true = enqueue every assistant message
  replayFullRead: false        # true = manual replay skips the long-text truncation
  cleanMarkdownFormatting: true
  readInlineCode: true
  codeBlocks: smart            # all | smart | replace
  codeBlockMaxChars: 300
  codeBlockReplacementText: 'You can see the code in our history.'
  throttleMs: 1500
  engine: ''                   # '' = auto-resolve
  announceApprovals: true
  announceQuestions: true
  stripApprovalPrefix: true
  questionGapMs: 2000          # pause between multiple question announcements (ms)
  longTextMode: message        # message | heading
  longTextMessage: '本次播报内容较长，请自行阅读。'
  maxChars: 300                # macOS default 0 = unlimited
  volume: 50                   # Windows only
  rate: 0                      # rate: Win SAPI scale -10..10 (0=normal) / mac wpm (175)
  # —— optional event announcements (off by default) ——
  announceTurnEnd: false     # turn/end
  announceCommandDone: false # command/done
  announceGoalChange: false  # goal/change
  announceToolErrors: false  # tool/result with error or isError block
  announceTodoWrite: false   # todo/write

Resolution order: schema default → patch config (base) → UI user layer. The browser dsh-speak settings page (client/client.js) and the patch YAML read/write the same settings document. Platform note: maxChars defaults to 0 on macOS (say has no ceiling) and 300 on Windows (SAPI safe limit).

Full configuration guide: the README's Configuration section.

6. Pitfalls (hard-won; do not "fix" casually)

#pitfallsymptomfix / rule
6.1Emoji / surrogate pairs reach Speak()silent — no audio, no errorstrip non-CJK/ASCII before speaking (engine step 3)
6.2Text longer than the adapter's per-Speak ceiling (~375–470 chars)silent — the whole utterance is dropped, not truncatedlength guard at 300 chars (engine step 5)
6.3Nested Start-Process powershell inside a DSH-sandboxed processsilent failure, no exceptionkeep the DSH chain synchronous at the adapter boundary (spawn once from the plugin; speech-summary.ps1 calls speak.ps1 synchronously)
6.4Plugin name with a raw Windows path in cordis.patch.ymlplugin fails to loadfile:///C:/... URL form
6.5Matching adapter voices by name onlyfalls back to robotic stock voicematch Name + Description against Natural|Online
6.6Reading/writing speech text as ANSImojibake or empty speechalways UTF-8 ([System.IO.File]::ReadAllText(..., UTF8))
6.7A repo .sh checked out as CRLF by core.autocrlf=true; npm pack bundles the working-tree filethe published speak.sh dies in bash on macOS (command not found, syntax error near {), silent failure.gitattributes pins *.sh text eol=lf (check file engine/speak.sh for CRLF before publishing)
6.8Log path hard-coded as /tmpon macOS os.tmpdir() is /var/folders/.../T, the log is not at /tmplook for the log at os.tmpdir() (= $TMPDIR)

7. Extending

New engine backend

The engine is the single seam for TTS backends. A future speak-edge.ps1 could wrap edge-tts, or a speak-piper.ps1 a local offline model — same parameter contract, same cleaning pipeline, swap the Speak() step. Adapters never change.

New harness adapter

Implement: capture the final reply text → call the engine. DSH (event stream), Claude Code (Stop hook), and any shell-based harness (speech-summary.ps1 called by the agent) are the three reference patterns.

8. Scope

This repository stays small and self-contained: a small engine plus the two adapter patterns (event-stream and stop-hook), and it is actively maintained. If you need more (voice management UI, more backends, cross-platform), treat the engine as the seam and build on top.

9. Publishing as an npm plugin (appendix)

The DSH plugin mechanism is Cordis-based, and the official install path for out-of-tree plugins is dsh plugin --profile web add <package> (pnpm-managed dependencies in the profile). This repository is prepared for that path:

Package layout

  • package.json — name: dsh-speak, main: adapters/dsh/speech-hook.js, files whitelists exactly what ships (plugin, engine/*.ps1, install.ps1, docs, license). prepublishOnly runs node --check on the plugin.
  • The plugin entry is the same CJS module (module.exports = { apply(ctx) }) already used by the file install — no code change is needed to publish.

Engine resolution (npm vs file install)

speech-hook.js locates engine/speak.ps1 in this order:

  1. config.engine override;
  2. <package>/engine/speak.ps1 resolved relative to the plugin file — covers both a repo checkout and node_modules/dsh-speak/ after npm install;
  3. legacy %USERPROFILE%\.dsh\hooks\speak.ps1 (the file-install location).

Because the engine rides inside the npm package, dsh plugin --profile web add dsh-speak alone is sufficient — no separate copying step.

Publish steps (maintainer)

npm login --registry=https://registry.npmjs.org   # official registry, 2FA required
npm publish                                       # publishConfig.registry pins the official registry
# bump "version" in package.json before every subsequent publish

China note: if your global .npmrc points at a mirror (registry.npmmirror.com etc.), npm login/npm publish would target the mirror, which does not accept publishes. The package's publishConfig.registry pins publishing to the official registry; just make sure the login used the official registry too.

Install steps (DSH user)

dsh plugin --profile web add dsh-speak
# then register in ~/.dsh/profiles/web/cordis.patch.yml:
#   - insert:
#       - id: speech-hook
#         name: 'dsh-speak'
# restart the DSH web app

No pnpm installed? dsh plugin forwards to pnpm; the npm equivalent is (same result: package lands in the profile's dependencies + node_modules):

  • Windows (PowerShell): npm install --prefix "$env:USERPROFILE\.dsh\profiles\web" dsh-speak
  • macOS (bash): npm install --prefix "$HOME/.dsh/profiles/web" dsh-speak

The patch watcher hot-reloads the plugin tree on cordis.patch.yml changes — verified: the plugin re-applies with the npm-bundled engine path, no restart needed for the registration switch itself (verified on macOS with 1.2.0).