DSH-WM
August 18, 2026 Β· View on GitHub
A playable world-model toolkit for DeepSeek Harness β look at a strip, name the route, score the run, and iterate the research loop.
Point the agent at a rollout (or just fixtures/sunset) and ask: did the second half melt, is Sora even a world simulator, and which memory recipe is allowed to win.
π One command to install | Play sunset with no GPU | Built-in WM map | RSI on skills and evals
π English | δΈζ
World-model work inside DeepSeek Harness is more fun when the agent can see the strip, name the lineage, and measure the claim. DSH-WM is the profile bundle for that: contact-sheet inspect, a compare page (side-by-side / swipe / diff heat + action HUD), three-route knowledge (3D display / pixel video-gen / latent prediction), run scoring, and an RSI loop on skills and wm.yaml.
dsh plugin --profile wm add github:WayneJin0918/dsh-wm
dsh --profile wm
Then try: Triage fixtures/sunset. Look at first, mid, last. Is this late-horizon?
DeepSeekβs product mainline can skip world models. Harness is still the research OS β this plugin is the WM lab on top of it.
Runtime: deepseek-ai/deepseek-harness
Table of contents
Play it in 30 seconds
fixtures/sunset is an 8-frame toy strip. Early pred frames stay warm and close to GT; the second half is wiped to cool blue so late-horizon collapse is obvious. No checkpoint, cluster, or GPU.
node cli.js inspect fixtures/sunset --indices first,mid,last
node cli.js view fixtures/sunset
node cli.js diff --pred fixtures/sunset/pred --gt fixtures/sunset/gt
node cli.js diagnose "is Sora a world simulator"
node cli.js knowledge --id wm-routes
wm_inspect prints a luma sketch you can read in a terminal:
pred #0 luma=148.7 low contrast, warm / orange
****************
**##************
pred #7 luma=62 near-uniform, cool / blue
::::::::::::::::
gt #7 luma=160.4 low contrast, warm / orange
***#############
wm_rollout_diff on the same strip reports a second-half SSIM drop and names frames 4β6 as the worst window. wm_view writes a local compare page if you want to scrub pred vs GT. Look, score, then open a card.
Highlights
- Install and play. Official DSH bundle, pure JavaScript, no
prepare/allowBuilds. Sunset works fromnode cli.jsbefore you even open Harness. - Look at the frames in-repo.
wm_inspectsamples first / mid / last (or named indices), writes a contact sheet, and returns a luma sketch plus a color/contrast look. - Compare the page yourself.
wm_viewwrites a self-contained HTML page: side-by-side, swipe overlay, abs-diff heatmap, SSIM timeline, and an action HUD whenactions.jsonis present. - Name the route first. 3D display, pixel / video-gen WM, and latent prediction are three exams.
wm-routesthendisplay-3d/pixel-wm/latent-wm. - A run is a directory. Optional
wm.yamldeclares pred / gt / log / metrics / actions. No manifest β heuristics. Cannot tell β candidates and warnings, never invented paths. - Measure when you have a run.
wm_discoverβwm_summarizeβwm_rollout_diffβwm_inspectβwm_viewfor layout, logs, numbers, a look, and a page you can scrub. - Built-in WM knowledge. Technique cards for chunk-AR, memory, KV, exposure bias, revisit, ablation, action following, cache eviction, and RSI-in-Harness.
wm_knowledge/wm_diagnosebefore a new architecture. - RSI on the harness layer. Skill
wm-rsiuses DSH trajectory, fork, Creator, and sunset to evolve skills,wm.yaml, and eval notes. - Skills that keep the game honest. Triage, knowledge, RSI, fair ablation, revisit.
- No GPU required to start. Rollout scores are luminance SSIM + MSE in pure JS. Videos need
ffmpeg; PNG/PPM folders do not.
Three routes
The word βworld modelβ is three research games. Cards follow the map in Awesome World Models. Open wm-routes before you design a backbone.
| Route | Card | What it predicts | What βgoodβ looks like | Field tropes |
|---|---|---|---|---|
| 3D display | display-3d | Geometry you can fly / occupy (mesh, Gaussian, occupancy, 4D) | Spatial consistency, explorable scene | Consistency is bought, not painted; a fly-through is a display until the stick does something |
| Pixel / video-gen WM | pixel-wm | The next pixels, often action-conditioned | An interactive strip that still obeys the stick | βIs Sora a world simulator?β; pretty clip, wrong joystick; Self-Forcing / late melt |
| Latent prediction | latent-wm | The next compact state (RSSM, JEPA, DINO) | Planning / control in the dream | Do not pay the loss on every pixel; a decoded video is a projector |
A forgotten room is revisit-eval on a pixel strip, a pose / occupancy check on a 3D scene, and a latent-state mismatch on JEPA / Dreamer. Name the route, then measure.
node cli.js knowledge --id display-3d
node cli.js knowledge --id pixel-wm
node cli.js knowledge --id latent-wm
node cli.js diagnose "Gaussian explorable 3D"
node cli.js diagnose "JEPA latent Dreamer"
Who it is for
| You want to⦠| DSH-WM gives you |
|---|---|
| Play a rollout without a cluster | Sunset + wm_inspect / wm_rollout_diff on a laptop |
| See what a run directory actually contains | wm_discover β layout, paths, frame counts, warnings |
| Turn a log tail into a next test | wm_summarize β last loss / NaN / early-stop plus three hypotheses |
| Put a number on βlooks worseβ | wm_rollout_diff β mean/min SSIM, curve, worst frames, diagnosis |
| Look at those worst frames | wm_inspect β contact sheet, luma sketches, per-tile look |
| Scrub pred vs GT and the stick | wm_view β side-by-side / swipe / heat, action arrow, followed / dropped |
| Place a paper on the map | wm-routes β display-3d / pixel-wm / latent-wm |
| Keep an ablation honest | wm-ablation β paired (scene, protocol, seed) and failure rate first |
| Talk about coming home | wm-revisit β geometric vs frame-similarity proxy |
| Tighten how the agent debugs WM | wm-rsi β one claim, one card, one measurement, one skill / wm.yaml delta |
Quick start: three steps
1. Install
Into a dedicated research profile:
dsh plugin --profile wm add github:WayneJin0918/dsh-wm
Web or Headless also work:
dsh plugin --profile web add github:WayneJin0918/dsh-wm
dsh plugin --profile headless add github:WayneJin0918/dsh-wm
From a local checkout (path install does not need GitHub access):
dsh plugin --profile wm add /path/to/dsh-wm
The package is pure JS. Git installs do not need pnpm allowBuilds. Pin a commit if you want a frozen default: github:WayneJin0918/dsh-wm#<sha>.
2. Restart and check it
dsh --profile wm --dump-config # look for "# == dsh-wm"
dsh --profile wm
Restart a running Web profile after adding the bundle, then start a new session so the skill catalog reloads.
3. Ask something you would actually say
Triage fixtures/sunset. What failed, and is it late-horizon?
Look at first, mid, last β what do the pixels do in the second half?
Is Sora a world simulator, or a pixel WM that still has to pass the stick?
The return trip forgot the room β which memory recipe is even allowed?
These two runs claim a memory win β are they paired on scene/protocol/seed?
Use Harness RSI to tighten the revisit skill; keep sunset as the gate.
Common workflows
| Task | Recommended workflow |
|---|---|
| First five minutes / no GPU | inspect sunset β diff β diagnose a question you care about |
| A training or eval run looks wrong | wm-run-triage β discover β summarize β diff β inspect β view |
| Which WM route is this paper? | wm-routes β display-3d / pixel-wm / latent-wm |
| βWhat kind of memory should we use?β | wm-knowledge β chunk-ar / memory-types / kv-memory β then measure |
| Late-horizon melt, train loss fine | wm_diagnose β exposure-bias β scheduled sampling |
| Which cache / memory config won? | wm-ablation β paired n and failure rate β mean delta |
| Did the camera come back? | wm-revisit β full-strip diff β first/last only if no poses |
| Improve the research loop itself | wm-rsi β Creator / trajectory β one skill or wm.yaml change β sunset gate |
| Offline CI / no API key | node cli.js knowledge, diagnose, discover, diff, inspect, view |
Toolbox
Three families you can compose in one session:
| Family | Tools | Job |
|---|---|---|
| Measure | wm_discover, wm_summarize, wm_rollout_diff, wm_inspect, wm_view | Layout, logs, pred vs GT numbers, look, compare page |
| Know | wm_knowledge, wm_diagnose | Route + technique cards, symptom β next step |
| Iterate | skills wm-run-triage, wm-knowledge, wm-rsi, wm-ablation, wm-revisit | Honest eval and harness-layer RSI |
| Tool | Best question to ask | Main result |
|---|---|---|
wm_discover | βWhat is in this run directory?β | layout, pred/gt/log/metrics, frame counts, warnings |
wm_summarize | βDid training actually finish, and what should I test?β | last loss / NaN / early-stop, metric keys, 3 hypotheses |
wm_rollout_diff | βWhere does pred drift from GT?β | mean/min SSIM, curve, worst 3 frames, diagnosis |
wm_inspect | βWhat do first / mid / last / the worst frames look like?β | contact sheet, luma sketch, color/contrast look |
wm_view | βLet me scrub pred vs GT and see whether the action was followed.β | HTML page + pred / gt / heat sheet |
wm_knowledge | βWhich route / what is chunk-AR / KV / RSI?β | catalog or a full technique card |
wm_diagnose | βIt forgets when we come back β now what?β | card ids + next tool / skill |
Rollout scores are luminance SSIM + MSE. wm_inspect is the built-in way to look at the strip; wm_view is the page you scrub.
Knowledge cards
Routes: wm-routes Β· display-3d Β· pixel-wm Β· latent-wm
Technique: chunk-ar Β· memory-types Β· kv-memory Β· exposure-bias Β· revisit-eval Β· ablation-protocol Β· action-following Β· cache-eviction Β· rsi-harness Β· diagnosis-map
node cli.js knowledge
node cli.js knowledge --id wm-routes
node cli.js knowledge kv memory
node cli.js knowledge --id rsi-harness
node cli.js diagnose "is Sora a world simulator"
node cli.js diagnose "late collapse after the first chunk"
Skills
- wm-run-triage β walk a run: discover β summarize β diff β inspect β view, then name the failure
- wm-knowledge β open a route or technique card before designing
- wm-rsi β one claim, one card, one measurement, one skill /
wm.yamlchange, sunset gate - wm-ablation β paired scene / protocol / seed before any mean
- wm-revisit β geometric loop vs frame-similarity proxy
RSI with Harness
DeepSeek Harness already gives you append-only trajectories, fork/replay, and Creator mode (inspect the live plugin tree). DSH-WM points that at world-model process:
- Write a falsifiable claim.
- Open
wm_knowledge(rsi-harness+ the technique, afterwm-routesif the lineage is unclear). - Measure (
wm_summarize/wm_rollout_diff) and look (wm_inspect/wm_view). - Change one skill,
wm.yamlfield, or eval note. - Gate on
fixtures/sunset(must still report late-horizon drop) and a paired user scene. - Solidify or roll back; keep the session.
The repeatable core is numbers plus cards.
wm.yaml
name: sunset-revisit
pred: outputs/pred # frame directory or mp4
gt: outputs/gt
log: logs/train.log
metrics: metrics.json # any JSON; keys are summarized, not schema-validated
actions: actions.json # optional per-frame control track for wm_view
Without the file, the plugin looks for pred|preds|recon, gt|target|ref, train.log / logs/*.log, metrics.json / *eval*.json, and actions.json.
How it works
flowchart LR play[Ask or point at a run] --> know[wm_knowledge / wm_diagnose] know --> measure[wm_discover / summarize / diff / inspect / view] measure --> rsi[wm-rsi on skills and wm.yaml] rsi --> gate[sunset fixture plus paired scene]
Three layers, one session:
- Knowledge β name the route, then open a technique card.
- Measure β filesystem tools plus
wm_inspectandwm_view. - RSI β evolve the research loop and pass the sunset gate.
Offline fixture
fixtures/sunset is the built-in playground. Pred frames 0β3 stay close to GT; 4β7 are wiped so second-half SSIM drops.
npm test
npm run check
node scripts/generate-fixtures.js # regenerate after changing the painter
Configuration and limits
Requirements
- DeepSeek Harness
0.1.0-rc.6or compatible, withpnpmon PATH fordsh plugin. - Node.js 18+.
- Optional
ffmpegfor JPEG or video inputs. PNG/PPM frame directories work offline.
Install, upgrade, disable, and uninstall
dsh plugin --profile wm update github:WayneJin0918/dsh-wm
dsh plugin --profile wm remove dsh-wm
To disable the bundle temporarily, set this in the profile patch:
- id: dsh-wm
disabled: true
Restart the profile after enabling or upgrading.
Troubleshooting
| Problem | What to do |
|---|---|
--dump-config has no # == dsh-wm layer | Re-run dsh plugin --profile wm add from the checkout or github:WayneJin0918/dsh-wm; confirm pnpm is on PATH |
| Git install 404s or asks for credentials | Confirm the repo is public at github:WayneJin0918/dsh-wm, or install from a local path |
pred not found | Add a wm.yaml or pass explicit --pred / --gt to wm_rollout_diff |
| Video / JPEG rejected | Install ffmpeg, or extract PNG frames first |
| Agent concludes without tools | Load wm-run-triage or wm-knowledge first; no layout / no card, no verdict |
| Agent invents a KV design from chat | wm_knowledge --id kv-memory then wm-rsi; open the card first |
| First-last SSIM treated as loop closure | Load wm-revisit; without poses that number is a proxy only |
| βRSIβ started rewriting training code | Pause. wm-rsi changes skills / wm.yaml / eval notes unless the user opened a train job |
Development
npm test
npm run check
- See CHANGELOG.md for releases.
- Use GitHub Issues on this repository for bugs and focused requests.
Acknowledgements
DSH-WM stands on these upstream projects. Thank you to their authors and the maps they made reusable.
- DeepSeek Harness (
dsh) β the official runtime this bundle installs into. Docs: deepseek.com/harness. - DSH Vision Toolkit by Anionex, with agent-vision-toolkit β thanks for the open plugin and the homepage this README learned from.
- Awesome World Models β the map of 3D / pixel / latent lineages the built-in route cards follow.