Munder Difflin

May 31, 2026 · View on GitHub

A desktop control room for the Claude Code agents you already run in terminals. Each agent is a Sims-style avatar in a shared 2D workspace; you can watch them work, send them commands, and configure their goals/skills/MCP from one place.

1. Product shape

What it is

  • Electron desktop app, macOS-first.
  • Renders a single 2D canvas ("the floor") populated by avatars, one per registered Claude Code session.
  • Each avatar represents a real claude process running in a tmux pane somewhere on the machine.
  • Avatars walk around, visit stations (file shelf, web portal, terminal station, etc.) based on what tool the underlying agent is using.
  • A side panel shows the raw terminal stream for whichever avatar is selected. A command bar lets you type into that agent.

What it explicitly is NOT (for MVP)

  • Not a replacement for the claude CLI. The CLI is the runtime; this app is a viewer/controller.
  • Not an agent-to-agent message bus. Avatars don't hand things off to each other yet. (Deferred to v2.)
  • Not a remote dashboard. Local sessions only. No web access, no auth.
  • Not a code editor. We show terminal output, not source files.

2. Architecture: the two data planes

This is the load-bearing design decision. Everything else follows from it.

┌───────────────────────────────────────────────────────────────┐
│                     Electron Renderer                          │
│   ┌──────────────────┐    ┌──────────────────────────────┐    │
│   │ Avatar Canvas    │    │ Terminal View + Command Bar  │    │
│   │ (Pixi.js)        │    │ (xterm.js, read-only-ish)   │    │
│   └─────────▲────────┘    └────────────▲─────────────────┘    │
│             │                          │                       │
│             │ avatar state             │ pty bytes             │
└─────────────┼──────────────────────────┼───────────────────────┘
              │                          │
       ┌──────┴──────────┐        ┌──────┴─────────────┐
       │  Event Plane    │        │  Terminal Plane    │
       │  (hooks → IPC)  │        │  (tmux pipe-pane)  │
       └──────▲──────────┘        └──────▲─────────────┘
              │                          │
              │ JSON events              │ raw bytes
       ┌──────┴──────────────────────────┴─────────────┐
       │           Claude Code processes               │
       │  (running in tmux panes the user already has) │
       └───────────────────────────────────────────────┘

Terminal Plane (raw pty)

  • Powered by tmux. Each registered agent = one tmux {session}:{window}.{pane} triple.
  • Read terminal output: tmux pipe-pane -O -t <pane> 'cat >> /tmp/cth/<id>.log' (or stream via an open file watcher).
  • Send input: tmux send-keys -t <pane> "..." Enter.
  • The xterm.js view in the app tails the log; user typing in the command bar becomes send-keys.

Event Plane (structured agent state)

  • Powered by Claude Code hooks, configured in each agent's settings.json (or via env var override).
  • Hooks the app installs:
    • UserPromptSubmit → avatar "wakes up"
    • PreToolUse → avatar starts walking to the station matching the tool
    • PostToolUse → avatar walks back carrying an artifact
    • Notification → avatar waves; toast in the UI
    • Stop → avatar idle at desk; badge if there's unread output
    • SubagentStop → spawns/despawns child avatar (v2)
  • Each hook runs a tiny shim: cth-hook (a Node CLI we ship). It reads the hook's JSON from stdin, tags it with the session id, and POSTs to a Unix domain socket at ~/.cth/events.sock that the Electron main process owns.

Why both planes? Hooks alone don't give you the raw stream the user expects to see. The tmux pipe alone doesn't tell you which tool is running without fragile output parsing. Together: the canvas is event-driven; the terminal view is byte-for-byte authentic.

3. Registering an agent

How an avatar comes to exist.

Option A — User adds existing pane (MVP path)

  1. User runs claude somewhere in tmux as usual.
  2. In the app: "Add agent" button → app calls tmux list-panes -a -F '#{session_name}:#{window_index}.#{pane_index} #{pane_current_command} #{pane_current_path}'.
  3. App shows candidate panes (filtered to those running claude or node).
  4. User picks one, names it, picks a sprite/color.
  5. App:
    • Writes a per-project hook config to <cwd>/.claude/settings.local.json merging in the cth-hook shims.
    • Starts pipe-pane on the tmux pane.
    • Reads existing scrollback with tmux capture-pane -p -S -1000.
    • Avatar appears on the floor.

Option B — Spawn from app (later)

  • "New agent" button → app spawns a new tmux pane and starts claude in it with the right hooks pre-wired.
  • Simpler UX, but requires us to own session lifecycle. Defer.

What if the user closes the tmux pane?

  • pipe-pane file goes silent; periodic tmux list-panes probe detects the pane is gone; avatar enters a "ghost" state for 30s, then is archived.

4. The Sims metaphor — mapping events to behavior

The floor

  • One large 2D scene. Background is a stylized "workshop" with stations:
    • Desk (per avatar) — home base, idle position
    • File shelf — for Read/Write/Edit
    • Terminal station — for Bash
    • Web portal — for WebFetch/WebSearch
    • MCP corner — for any mcp__* tool (sub-stations per server)
    • Task board — for TodoWrite
    • Mailbox — for Notification (visible when something needs the user)

Avatar state machine

idle ──UserPromptSubmit──▶ alert
alert ──PreToolUse──▶ thinking(→station) ──arrival──▶ working(station)
working ──PostToolUse──▶ thinking(→desk) ──arrival──▶ idle (or next tool)
thinking ──PreToolUse──▶ thinking(→new station)   (loop)
working ──Stop──▶ success ──250ms──▶ idle
any ──Notification──▶ blocked (waving at mailbox)
any ──tmux pane gone──▶ ghost ──30s──▶ archived

thinking is the in-motion + reasoning state. The avatar walks (physical animation) while the badge reads "thinking" (cognitive state). One state, two readings.

Movement

  • Pixi.js scene; avatars are sprite-based, 32×32 or 64×64.
  • Pathing: simple A* on a coarse grid, or just lerp to target — overkill for MVP, plain lerp is fine.
  • Multiple agents in the same project share a "room" (visual zone). Different cwds → different rooms with named doors.

What the user sees at a glance

  • Avatar at the file shelf reaching up → "agent is reading or writing a file"
  • Avatar at the terminal station with motion lines → "agent is running a bash command"
  • Avatar standing at mailbox waving → "agent is blocked, wants your input"
  • Avatar sitting at desk → "done, awaiting next prompt"

This is the demo. If this is fun to watch, the product works.

5. Configuration per agent

Each avatar has a config drawer (right-click → Configure):

  • Name & sprite (cosmetic, persisted in app DB)
  • Working directory (read-only; derived from tmux pane)
  • Skills enabled — toggles that edit the project's .claude/settings.json enabledMcpjsonServers / skill allowlist
  • MCP servers — list of MCP servers attached for this session; toggle on/off
  • Hooks — show which hooks the app has installed; option to add custom ones
  • Goal / system prompt prefix — see §6
  • Model — Opus / Sonnet / Haiku selector that writes to the project's settings
  • Permission mode — default / acceptEdits / bypassPermissions / plan

All of this is just a GUI over settings.json. The app is not inventing config; it's editing the files Claude Code already reads.

6. The "/goal" question

You mentioned /goal — there's no built-in slash command by that name in Claude Code as of early 2026. Three possibilities:

  1. You meant a custom slash command (user-defined skill in ~/.claude/skills/goal/). If so: the app should let you pick which skills are exposed per agent and surface them as quick-action buttons next to the avatar.
  2. You meant a persistent goal/objective, like a long-running directive an agent should pursue across prompts. This isn't a CC feature but is easy to implement here: the app maintains a "goal" string per avatar and prepends it to each user prompt before send-keys, or installs it as a UserPromptSubmit hook that injects context.
  3. You saw it somewhere recent. If so, point me at it and we'll wire to the real thing.

Spec'd behavior for now: each avatar has a goal field (free text). On send, the app prefixes the user's command with <goal>...</goal> context (or injects via UserPromptSubmit hook — cleaner because it doesn't pollute the visible transcript). User toggles whether goal is active per-send.

Also surface, per avatar:

  • Skills picker (anything in ~/.claude/skills/ or <cwd>/.claude/skills/)
  • Subagents picker (anything in ~/.claude/agents/)
  • MCP picker

7. Sending commands from the app

Command bar at the bottom of the selected agent's panel. Three input modes:

  • Free prompt: types into the pane via tmux send-keys, then Enter.
  • Slash command: same path, but autocomplete from the agent's available skills.
  • Quick actions: buttons for things like "/clear", "Stop", "Continue", "Run goal".

Edge cases:

  • If the agent is mid-tool, app warns: "agent is busy — queue or interrupt?"
  • Interrupt = send-keys Escape (Claude Code's cancel key).
  • Queue = wait for Stop event, then send.

8. Tech stack & key dependencies

LayerPickNotes
ShellElectronRenderer + Node main
UI frameworkReact + TypeScript
Avatar canvasPixi.jsSprite scene, 60fps, easy lerp/tweening. Phaser is overkill.
Terminal viewxterm.jsTails pipe-pane log; supports ANSI properly
Process controlshelling out to tmuxNo node-pty needed since we attach
Event ingestUnix domain socket at ~/.cth/events.sockNode net.createServer; hook shim is a tiny CLI that writes JSON+newline
PersistenceSQLite via better-sqlite3Agents, layouts, command history, goals
SpritesCustom or itch.io asset pack8 directions × idle/walk/work animations
Packagingelectron-builder.dmg for Mac, defer Linux/Windows

External binaries the user must have: tmux. App detects on first run and refuses to start without it (with install instructions).

9. Data model (SQLite)

CREATE TABLE agents (
  id            TEXT PRIMARY KEY,        -- uuid
  name          TEXT NOT NULL,
  sprite        TEXT NOT NULL,           -- key into sprite pack
  color         TEXT NOT NULL,
  tmux_target   TEXT NOT NULL,           -- 'session:window.pane'
  cwd           TEXT NOT NULL,
  goal          TEXT,
  created_at    INTEGER NOT NULL,
  archived_at   INTEGER
);

CREATE TABLE events (
  id            INTEGER PRIMARY KEY,
  agent_id      TEXT NOT NULL,
  ts            INTEGER NOT NULL,
  hook          TEXT NOT NULL,           -- 'PreToolUse', etc
  tool          TEXT,                    -- 'Edit', 'Bash', etc
  payload_json  TEXT NOT NULL
);

CREATE TABLE commands (
  id            INTEGER PRIMARY KEY,
  agent_id      TEXT NOT NULL,
  ts            INTEGER NOT NULL,
  text          TEXT NOT NULL,
  source        TEXT NOT NULL            -- 'user', 'quick_action'
);

CREATE TABLE layout (                    -- avatar positions/rooms
  agent_id      TEXT PRIMARY KEY,
  x             REAL NOT NULL,
  y             REAL NOT NULL,
  room          TEXT NOT NULL
);

10. Risks & open questions

Risks

  • Hook spec drift. Claude Code's hook JSON format may evolve. Mitigation: pin the hook shim to a version, surface schema errors in the UI, lazy-fallback to text parsing of tool calls if hooks break.
  • Writing to user's settings.json. Touching their config is intrusive. Mitigation: only ever write to settings.local.json (gitignored), back up first, show a diff before applying, and an "uninstall hooks" button.
  • tmux not available / user uses iTerm/kitty/wezterm directly. No tmux = no app. Mitigation v1: require tmux. v2: shim for iTerm via AppleScript; long-tail other terminals never.
  • The Sims metaphor lands as gimmicky. Real risk. The mitigation is to make the metaphor informational — every animation should actually tell you something you didn't know. If walking-to-the-shelf doesn't convey "is reading a file" faster than a text label, we built a toy.

Open questions (decide before/while building)

  1. Hook installation scope — ~/.claude/settings.json (global, all sessions get hooks) vs per-project .claude/settings.local.json (only registered projects). Recommend per-project to keep blast radius small.
  2. Sprites: build or buy? Buy an asset pack for the prototype, replace later if the product sticks.
  3. What does "goal" actually mean for you? Long-running directive (item #2 in §6) is my best guess — confirm before we build.
  4. Multi-machine? If you have agents running on a remote box, do we need a remote daemon? Defer past MVP.
  5. iTerm support deadline. When (if ever) does this become a blocker?

11. Milestones

M0 — Skeleton (1–2 days)

  • Electron + React + Pixi shell
  • Renders one hardcoded avatar that walks between two fixed positions
  • xterm.js wired to a static log file

M1 — One real agent end-to-end (3–5 days)

  • "Add agent" picks a tmux pane
  • Installs hook shim into .claude/settings.local.json
  • Hook events flow over UDS into the app
  • Avatar state machine driven by real events
  • Command bar sends keys to the pane
  • Terminal view tails pipe-pane log
  • Success criterion: you run claude in tmux, register it, ask it to edit a file, watch the avatar walk to the shelf, walk back, and finish at its desk.

M2 — Multi-agent (3–5 days)

  • N agents simultaneously, each in their own room (grouped by cwd)
  • Per-agent config drawer (name, sprite, goal, model, permission mode)
  • Quick actions, command queueing
  • Goal injection via UserPromptSubmit hook
  • Persistence: avatars and positions survive app restart

M3 — Polish (1 week)

  • Sprite pack with proper animations
  • Sounds (optional, off by default)
  • Notification toasts + dock badge
  • Onboarding for first-time users (tmux check, hook install consent)
  • DMG packaging, code signing

Out of scope for v1

  • Inter-agent handoffs ("agent A passes a file to agent B")
  • Remote agents over SSH
  • Web/mobile companion
  • Non-tmux terminals
  • Subagent visualization (parent spawns child avatars)

12. Decisions captured so far

  • Avatar metaphor: active Sims-like (agents walk, visit stations).
  • Execution model: attach to existing terminal sessions (tmux), do not spawn or own the claude process.
  • Tech stack: Electron + React + Pixi.js + xterm.js + SQLite.
  • MVP scope: N independent agents in a shared workspace; no inter-agent coordination.

13. Decisions still needed from you

  1. "/goal" — which of the three interpretations in §6?
  2. Hooks scope: per-project (recommended) or global?
  3. How important is iTerm direct support? (If not now, we require tmux.)
  4. Sprite art: stand-in asset pack OK, or do you want custom?
  5. Anything in §10 ("Risks & open questions") you'd answer differently?