Margin Analyst

August 5, 2026 · View on GitHub

A demo of AG-UI × Claude Managed Agents × CopilotKit, built on the official @ag-ui/claude-managed-agents integration.

Runs two ways. Replay mode is the default and needs no API key — it drives the real published integration against a recorded session, with nothing faked above the client seam. Live mode is verified against a real Anthropic account: one command provisions the agent, and the demo runs on a real cloud sandbox. See Two modes.

You ask one question — "Why did our gross margin drop in Q3?" — and watch an agent do the work: pull the order ledger into its cloud sandbox, write a Python script, hit a traceback on real dirty data, read the error, fix its own script, re-run, and render the finding as live React components.

It runs with no API key. See Two modes.

npm install
npm run dev          # http://localhost:3000

Watch it

Margin Analyst: the agent's script fails on a dirty column, it reads the traceback, patches itself, and renders the finding

▶ Full 49-second recording — unedited, one uninterrupted pass through all three beats.

The clip above is the middle of it: the agent's first script dies on a column literally named freight_cost — with a trailing space, which you can read in the traceback — and it recovers without being told. The full video adds the workspace surviving a loss of server state, and a consequential action stopping to wait for a human.

Both are produced by npm run record, which drives the same Playwright beats the test suite asserts. The video cannot show something the tests do not prove.

What it demonstrates

Three things, in order, each hard to show any other way.

1. An agent doing real work in a sandbox, and recovering from failure

The dataset in data/orders-2026.csv is deliberately dirty in ways that are common in real exports:

the freight column is named freight_cost — with a trailing space
370 EMEA rows write European decimal commas ("1.234,56")
215 marketplace rows write 07/03/2026 while 830 write ISO dates

The agent's first script fails on that trailing space. You watch it read the traceback, inspect the raw header, patch the script, and re-run. That loop is the point: it needs a real filesystem and a real shell, which is what a Managed Agents environment is.

The Files and Terminal panes are derived from the AG-UI event stream — every line and every file comes from an observed tool call. Nothing is staged and no path is invented.

2. The workspace outlives the process

Each AG-UI thread maps to one managed session, and each session has its own sandbox. Swap the integration's default in-memory SessionStore for a persistent one and a thread's session — with everything the agent wrote in it — survives a restart.

Press Restart in the status bar to drop the server's in-process state, then ask a follow-up: same session, same files.

The button drops the server's in-process state — the agent bundle and its run gate — while the on-disk store at .managed-agents.json survives. That is what the next run reads to resume the same managed session.

Do not demo this with Ctrl-C + npm run dev in replay mode. Killing the process also loses the replay transcript's position, which lives in memory by nature — the store still holds the mapping, so the thread resumes, but the script restarts from Beat 1 and you will watch the analysis run twice. The app logs a warning when it detects this. In live mode a full process restart is the stronger proof, because the managed session and its sandbox live at Anthropic and have no local position to lose.

A note on what doesn't prove this: opening a new thread creates a new session and therefore a new sandbox. That demonstrates the opposite of persistence.

3. Human approval on a consequential action

send_email is a frontend tool, so when the agent calls it the managed session parks in requires_action and stops. An approval card renders the actual drafted email, and nothing resumes until a person clicks.

Reject it and the agent acknowledges the decision instead of retrying. That branch matters more than the approval one.

Two modes

`replay` — the default, no credentials

A ReplayClient implements the five Anthropic methods the integration actually calls (beta.agents.retrieve, beta.sessions.create / .update, beta.sessions.events.send / .stream) and streams a recorded session transcript. Everything above it is real: the published ManagedAgentsAgent, its turn loop, its event translation, its park/resume.

This is the same seam the integration's own test suite uses. It means you can clone this repo and see the whole demo without an API key or Managed Agents beta access.

Replay mode is always labelled REPLAY in the status bar. A demo that silently fakes an API call is not a demo.

`live` — against the real API

Verified end to end against a real account: the agent fetches the dataset into a real cloud sandbox, writes and runs Python, recovers from its own errors, and drives the same generative-UI tools you see in the video.

cp .env.example .env.local     # then paste your key into ANTHROPIC_API_KEY
npm run setup:live             # provisions the environment + agent, writes the ids back
# set MODE=live in .env.local
npm run dev

setup:live is idempotent — it finds the environment and agent by name and reuses them, so running it twice costs nothing. It never updates an existing agent (a prompt change needs --recreate), and it never touches your key line when it writes the ids.

Live mode refuses to start if ANTHROPIC_API_KEY, MA_AGENT_ID or MA_ENV_ID is missing, rather than falling back to replay. Your key needs Managed Agents beta access; check with:

curl -sS -o /dev/null -w '%{http_code}\n' https://api.anthropic.com/v1/agents -H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01" -H "anthropic-beta: managed-agents-2026-04-01"

200 means you're clear; 403/404 means the beta isn't enabled on that account.

Live mode needs the dataset reachable from the sandbox. The agent fetches it with curl -sS -o orders-2026.csv "$ORDERS_CSV_URL", and that command runs inside Anthropic's cloud environment — not on your machine — so http://localhost:3000/... will not resolve. Publish data/orders-2026.csv somewhere the sandbox can reach (a raw file URL on your fork works well) and set ORDERS_CSV_URL on the managed agent's environment. Replay mode does not need this: the fetch is part of the recorded transcript.

Live sessions and cloud environments are billable and outlive the process. Run npm run teardown when you're finished (dry-run by default; pass --delete to actually remove them). Stop the dev server first, so it doesn't resurrect a session you deleted.

How live mode differs from the video

The video is replay, and replay is deterministic by design. A live model is not, and the honest differences matter more than the similarities:

The numbers move between runs. A live agent picks its own method. Across verification runs it put the Q3 counterfactual at 38.63% and then 38.57%, and excess freight at $107,526 and then $105,901 — both defensible readings of the same data, neither identical to the other. It reliably lands the headline (34.6%, -4.07pp vs Q2, the same three SKUs), because that signal is genuinely in the data. Do not expect a live run to match data/README.md to the cent.

The self-correction beat is not guaranteed. It is a real model deciding how careful to be. In one run it inspected the header first and stripped the trailing space pre-emptively — no failure to watch. In another it hit a genuine Python error mid-analysis and recovered on screen. Both are correct behaviour; only one makes a good hero shot. That asymmetry is exactly why the recording is replay.

Two bugs only live mode could find. Worth stating because it justifies the effort: a real model sent a chart with a compound unit ("% / $", mixing percentages and dollars in one chart) which clipped every Y-axis label down to the shared substring 0% / $ — six identical, all wrong. And because it works via cd /mnt/session && python3 analyze.py, the file tree listed every file twice, once relative and once absolute. Neither was reachable from the recorded transcript, which sends tidy hand-authored payloads. Both are fixed and regression-tested.

Paths differ. The real sandbox working directory is /mnt/session; the transcript uses /workspace. Cosmetic, but do not be surprised by it.

Live tooling

Command	What
`npm run setup:live`	Provision or reuse the environment + agent; write ids to `.env.local`
`npm run setup:live -- --print`	Same, but print the ids instead of writing them
`npm run setup:live -- --recreate`	Archive the existing agent and rebuild it from `scripts/agent-spec.ts`
`npm run probe:live`	Drive one live turn headlessly and dump every AG-UI event to `.probe-live.jsonl`
`npm run e2e:live`	One Playwright smoke test against a live server (needs `npm run dev` already running)
`npm run teardown -- --delete`	Delete this demo's sessions (dry-run without `--delete`)

probe:live is the one to reach for when something looks wrong: it bypasses the browser and CopilotKit entirely, so what it prints is the raw truth of which tools the agent called and with what arguments.

The system prompt lives in scripts/agent-spec.ts, not in the app. The integration forwards only user-message text and tool results into a session — it never sends system or context content — so CopilotKit's instructions prop cannot reach a managed agent. If you want the agent to behave differently, edit that file and re-run with --recreate.

Configuration

Variable	Default	Purpose
`MODE`	`replay`	`replay` or `live`. Anything else throws.
`ANTHROPIC_API_KEY`	—	Live mode only.
`MA_AGENT_ID`	—	Live mode only. The managed agent (`agent_…`).
`MA_ENV_ID`	—	Live mode only. The environment (`env_…`).
`SESSION_STORE_PATH`	`.managed-agents.json`	Where thread↔session mappings persist.
`REPLAY_DELAY_SCALE`	`1`	Multiplies transcript delays. `0` = instant, `0.25` = fast.
`COPILOTKIT_TELEMETRY_DISABLED`	unset	CopilotKit's runtime reports anonymous telemetry by default. Set to `true` to opt out.

See .env.example. Replay mode needs none of it.

Scripts

Command	What
`npm run dev`	The demo, on :3000
`npm run verify`	typecheck + unit tests + build
`npm test`	vitest
`npm run e2e`	Playwright: asserts all three beats
`npm run record`	Playwright: records the hero video into `recordings/`
`npm run teardown`	Deletes live managed sessions (dry-run by default)

/preview renders the shell from static fixtures — empty, mid-stream and completed states, in light and dark — with no backend. It's the fastest way to iterate on the UI.

Presenting it

Wait for each reply before typing the next message. CopilotKit's composer accepts the keystroke, leaves Send enabled, and silently drops the submit while a turn is still streaming — so an early keypress does nothing and looks like the app hanging. The e2e suite works around this by retrying until the composer clears; a human has to just wait.
Don't reload mid-demo. A reload mints a new thread, which means a new session and a fresh sandbox, and the replay script starts over from Beat 1. The session badge shows a -t2 marker so you can tell this happened.
The three beats are a script: opening question → follow-up → email. Asking them out of order in replay mode plays the wrong turn.

How it fits together

browser (Next.js, React 19)
  CopilotKit chat + generative UI ──┐
  Files / Terminal / Canvas panes   │
                                    ▼
  /api/copilotkit → CopilotRuntime → ManagedAgentsAgent → Managed Agents API
                                          │                 (session + sandbox)
                                          └─ client seam ──→ ReplayClient

Path	Purpose
`lib/managed-agents/`	the client seam: replay client, transcript format, persistent session store
`lib/sandbox/`	pure reducer turning observed tool calls into the file tree and terminal
`lib/copilot/`	agent construction, mode selection, zod tool schemas, client store
`components/shell/`	the IDE shell — pure and presentational, no CopilotKit imports
`components/tools/`	generative-UI renderers and the approval card
`components/SandboxObserver.tsx`	`useDefaultTool` catch-all → `ObservedToolCall` → `derive()`
`data/`	the dirty dataset, its generator, and the documented ground truth

The shell components take props and render — they never import CopilotKit or the Anthropic SDK. That's what makes /preview possible and the UI cheap to iterate on.

data/README.md documents the schema, every piece of intentional dirt, and the ground-truth answer. lib/managed-agents/__tests__/ground-truth.test.ts re-parses the committed CSV and asserts every number in the transcript's chart, KPIs, memo and email against it — so the scripted session cannot drift from the data.

Security

This is a demo, not a deployment.

There is no authentication. AG-UI thread IDs come from the client and the integration keys session state by thread ID, so a thread ID is effectively a bearer token: anyone who presents one resumes that thread's session. Do not expose this beyond localhost.
A real multi-tenant deployment needs a SessionStore that partitions records by the authenticated caller, derived from your auth layer and never from the request body. The integration's README carries a per-language recipe.
Memo and tool-result text is rendered, never executed, and treated as untrusted.

Credits

The integration is ag-ui#2247 by CJ Avilla, which ships TypeScript, Python and .NET ports of the same event mapping.