Subagent Module
September 2, 2026 · View on GitHub
Overview
The subagent module (kiro_crew/subagent.py) spawns isolated background agents for parallel task execution. Each subagent gets its own LLM session via SessionManager, runs a focused task, and announces the result via callback.
Supports on_tool_approval callback for interactive tool approval (routed through gateway's approval system in Normal/Trust modes).
Constants
| Constant | Value | Purpose |
|---|---|---|
_MAX_CONCURRENT | 3 | Legacy fallback / auto-size floor. agent.max_subagents now defaults to 0 = auto-size the cap at startup (floor 3, ceiling agent.subagent_auto_max, default 32); a positive value pins a fixed cap. Session-shared subagents are cost-sampled as the runtime's measured RSS/CPU divided by the live shared-session count on that PID (_live_shared_count), so the memory term no longer binds and the cap rises to the provider-concurrency ceiling. |
_TIMEOUT_SECS | 1800 | Hard timeout per subagent (30 minutes) |
_ON_DONE_TIMEOUT | 1200 | Outer cap: max total seconds for semaphore wait + injection (20 minutes) |
INJECTION_TIMEOUT | 900 | Inner cap: max seconds for a single stream_and_collect call (15 minutes); default _DEFAULT_INJECTION_TIMEOUT = 900.0, tunable via KIROCREW_INJECTION_TIMEOUT (float seconds, clamped to _ON_DONE_TIMEOUT) |
_RESET_TIMEOUT | 30 | Max seconds for session reset in finally block |
_TURN_LIMIT | 100 | Default tool-call budget per subagent (configurable via agent.subagent_max_turns, per-spawn via max_turns) |
_STALL_IDLE_SECS | 120 | Seconds with no stream activity before a running subagent is surfaced as stalled in the running-card (configurable via agent.subagent_stall_idle_secs). Surface-only — a stalled subagent is never auto-terminated; the user closes it from the UX (per-row stop / Stop-all) and the 30-min _TIMEOUT_SECS ceiling still applies. |
_SYSTEM_PREFIX | (string) | Injected before task text to prevent spawn recursion |
COMPLETION_KEEP_DEFAULT_CHARS | 3000 | Default character cap for the completion event injected into the parent session (configurable via agent.completion_keep_chars). Lives in context_management.py alongside the helper. |
Turn Limit Resolution Chain
Priority (highest wins): per-spawn max_turns → config agent.subagent_max_turns → hardcoded default (100)
A value of 0 means "not set" and falls through to the next level. Implemented as SubagentManager._effective_turn_limit(), shared by the enforcement path in _run_inner() and the timeout/reap error strings (_timeout_context()).
Concurrency Auto-Sizing — Memory Probe (per platform)
When agent.max_subagents == 0, compute_max_subagents() sizes the cap from
host memory and CPU, clamped to [3, agent.subagent_auto_max]. The
available-memory term is read by _available_memory_gb(), which is dispatched
per operating system (see dynamic-subagent-sizing.md):
- Linux —
/proc/meminfoMemAvailable, then clamped by cgroup headroom. - macOS — reclaimable memory (free + inactive + speculative + purgeable
pages) via the Mach
host_statistics64syscall throughctypes/libSystem(_macos_vm_reclaimable_pages), combined with theos.sysconfpage size. This is in-process, non-blocking, no subprocess — required because the probe runs on the gateway event loop at startup and the spawn-audit guard rejects unrouted subprocess spawns. - Other (e.g. Windows) — no probe yet; returns
-1.0and the cap fails open to the legacy floor of 3.
Hard floor: the auto-sized cap is always ≥ 3 — compute_max_subagents clamps to
[3, hard_cap] and the config loader clamps subagent_auto_max UP to 3 (with a
warning + config_bounds_clamped SEL event, mirroring the > 64 ceiling clamp).
Applies only to auto-sizing (max_subagents=0); an explicit max_subagents pin
is unrestricted (any 0..64).
Limitation: the per-spawn spawn_min_memory_gb admission gate
(check_memory_available) still reads /proc/meminfo and so remains inert
(fails open) on non-Linux hosts. Auto-sizing and the runtime gate are
independent guards; unifying them is out of scope for the sizing probe.
APIs
SubagentManager.__init__(sessions, ctx_builder, on_done, max_concurrent)
sessions: SessionManager— provides isolated LLM sessionsctx_builder: ContextBuilder— builds context with memory/skills/hookson_done: AnnounceCallback | None— called withSubagentInfowhen donemax_concurrent: int— capacity limit (default 3)
spawn(task, parent_session_key="") -> SubagentInfo | None
Spawns a background agent. Returns SubagentInfo or None if at capacity. Uses atomic _running_count to prevent race conditions. parent_session_key tracks the originating session for completion injection.
Spawn flow:
- YOLO mode: skips approval, runs immediately
- Parent trusted: parent session has
approval_policy="auto"(set by dashboard trust toggle) → skips approval, runs immediately - Non-YOLO, non-trusted: enters
_spawn_with_approval, which re-checks YOLO (defense-in-depth against toggle race), then requests interactive approval with a 2-minute timeout. Timeout or rejection frees the concurrency slot.
Tool Approval Cascade
When a subagent's tool call triggers EVENT_PERMISSION_REQUEST, approval
is decided in strict priority order:
- Hook deny —
hooks.on_tool_call()returnsTOOL_DENY→ reject - Hook auto-approve —
hooks.on_tool_call()returnsTOOL_AUTO_APPROVE(theauto_approve_toolsglobs / read-only allowlist — a grant made by program NAME), honoured only aftername_grant.refusal_for_event(event)confirms each program name in the shell command still resolves to the program it appears to name. A refusal DOWNGRADES to rungs 3–5 (never a hard block) and is audited asoutcome=auto_approve_declinedwithreason=name_grant, the refusal code, andtier=hook_auto_approve. This matters most here: the subagent surface runs unattended, so an unverified shadowed name would be honoured with nobody watching. On Windows the check cannot model the shell's lookup at all, so it declines every name-based shell grant there — a headless subagent (no parentautopolicy, no interactive approver) then rejects shell tools its allowlist used to grant. - Parent policy —
parent_policy == "auto"→ auto-approve. Resolved once at_run_innerstart (see the chain below); an active global YOLO folds into this snapshot rather than being re-read per event. - Interactive callback —
on_tool_approval(races dashboard + Slack, 2h timeout) - Deny by default — none of the above matched → reject
parent_policy is resolved once when _run_inner starts, using this chain:
- Read from parent session via
get_approval_policy(parent_session_key) - If empty and YOLO mode active →
"auto" - If still empty and subagent has no parent session key → use the cached
KiroCrewConfig.agent.approval_mode(snapshotted atSubagentManagerinit); if"auto"→"auto"
Step 3 ensures parentless subagents (e.g. cron jobs) respect the user's global approval mode instead of falling through to interactive approval.
Child-fidelity gate. A child-origin permission event whose SECURITY context
is absent (AcpEvent.child_low_fidelity: structured params never reached the
tool_call cache, unresolved shell classification, or a shell without a
recoverable command) skips steps 2–3 and is handed to the interactive callback
with an "UNVERIFIED child request" annotation (headless: rejected), because
every field a shortcut would judge is agent-authored. One carve-out: when the
event's canonical MCP identity IS verified (child_mcp_identity_trusted — the
_meta.kiro server/tool pair resolved from the tool_call cache, carrying the
explicit mcp_identity_trusted provenance flag those cache hits set, resolved
non-shell; the shape a remote MCP server produces by streaming empty
rawInput), the unconditional parent_policy == "auto" grant still
auto-approves — the call site reads the hoisted
AcpEvent.child_unconditional_grant_eligible property: its decision consumes
no agent-authored event data, only the
arguments remain unverified. The hook auto-approve (title-pattern-matched) and
every content-matching path stay fail-closed on the composite fidelity.
The is_yolo() read happens once, when parent_policy is resolved at
_run_inner start — a YOLO toggle mid-execution takes effect on the next
subagent run, not on the current run's remaining tools.
cancel_all() -> None
Cancels all running subagents, stops the reaper loop, and awaits their cleanup. Handles CancelledError gracefully — sessions released, count decremented.
steer_run(agent_id, message) -> (ok, detail) / follow_up_run(agent_id, message) -> (ok, detail)
Two delivery modes for spawn_steer (REST POST /api/spawn/{id}/steer, body mode: "interrupt" default / "follow_up"). steer_run injects into the RUNNING turn via the provider's steer, with a bounded startup-grace poll for a live run whose session has not registered yet (#1113). follow_up_run never interrupts: it queues the message on SubagentInfo.pending_followups and arms a one-per-run watcher (_deliver_followups, registered in the manager-owned _followup_watchers dict — NOT the global _safe_fire set — because a watcher can spawn a brand-new run and must therefore be reachable by cancel_all(), per the same containment contract as _schedule_cancel_recovery; cancel_all cancels watchers BEFORE the run tasks so none can dispatch into a shutting-down gateway, and the watcher re-checks _shutting_down before dispatch). The watcher waits for the run to complete (info.done AND its task popped from _tasks, so teardown is finished), then dispatches the whole queue as ONE continue_conversation on the run's own conversation (messages joined in arrival order — three corrections cost one continuation, not three). The continuation is a normal new run on the same parent session, so its result arrives as a separate completion event. OUTCOME-AWARE: a run the user explicitly STOPPED (user_stopped) suppresses dispatch (followup_suppressed audit) — resurrecting killed work is the opposite of "the correction can wait"; error/timeout terminals still dispatch (the continuation carries the conversation's context, so "fix what broke" is legitimate). NEVER SILENT: every undeliverable path (suppressed, watcher expiry, dispatch failure) announces a SYNTHETIC failure completion event through the normal _on_done path, because the spawn_steer reply promised the parent an event — followup_expired/followup_failed/followup_suppressed SEL audits alone would leave the parent blocked on an event that never comes. Deliberately a per-run poller, NOT a hook in _run's 3-guard finalization: completion is reached from many terminal paths (normal/error/timeout/cancel-recovery/reaper) and a watcher observes the outcome without adding an obligation to any of them. Bounded everywhere: poll cadence 2s, hard deadline default_timeout + 300s, and residual conversation_busy after done gets a bounded retry. Typed refusals mirror steer: not_found, and not_running (use spawn_continue directly on a finished run).
Properties
running -> list[SubagentInfo]— currently running agentscount -> int— number of running agentsmax_concurrent -> int— capacity limit
SubagentInfo
@dataclass
class SubagentInfo:
id: str # 8-char hex UUID
task: str # original task text
started: float # time.time() at spawn
done: bool # True when finished (success or error)
result: str # LLM response text (trimmed to completion_keep for the event)
result_path: str # ~/.kiro/crew/subagents/<id>/result.txt (full transcript)
result_truncated: bool # completion copy dropped content → event carries summary+path
error: str # error message if failed
elapsed: float # seconds from start to completion (set in _run finally)
tool_count: int # observed tool calls (incl. auto-approved); drives running-card progress
last_activity: float # time.time() of last stream event; reset to _exec_started; drives idle-stall
stalled: bool # reaper flagged this subagent as idle/stalled (UI signal)
_awaiting_approval: bool # blocked on a human approval prompt (spawn gate or mid-run tool) → exempt from idle-stall
Session Lifecycle
spawn()increments_running_count, creates asyncio task_spawn_with_approval()(non-YOLO): re-checks YOLO, requests approval with 2-min timeout_run()wraps_run_inner()withasyncio.wait_for(_TIMEOUT_SECS)_run_inner()resolvesparent_policy(parent session → YOLO fallback → config fallback), creates sessionsubagent:{id}viaSessionManager.get_or_create(approval_policy=parent_policy)— policy is persisted on the new session- Streams through ACP with context injection, tool approval cascade, and turn counting
- On completion (in
_runfinally block): firesubagent_doneWS event immediately (before slow reset + on_done), thensessions.release()→_running_count -= 1→sessions.reset()→ callon_donecallback - On timeout:
error = "Timed out after 30 minutes" - On turn limit:
error = "turn_limit:{turn_limit}"(default 100) - On
CancelledError: three-way, by cancellation source (see Terminal-State Contract below) — user stop → neutraluser_stoppedrecord (NO error); shutdown / spent one-shot →error = "cancelled"; any other (unexpected) cancel → one-shot auto-continue via_schedule_cancel_recovery
Early WS event firing: subagent_done WS event is fired in the _run finally block BEFORE the slow reset() + on_done() path. This ensures the dashboard receives completion status within seconds, not 30-90s later when stream_and_collect finishes processing.
Terminal-State Contract (stopped vs failed vs completed)
A record's terminal outcome is three-way, with a single canonical source: the SubagentInfo.outcome property ("stopped" | "failed" | "completed"). Every subagent_done emission (live, _run finally, _force_reap, WS reconnect replay managed + native), native_subagent_snapshots, the /api/spawn listing, and tombstones carry outcome explicitly. Consumers MUST use outcome — never re-derive from error-nullability (the legacy error ? failed : completed idiom misreports a stopped agent as completed). stopped/error remain on the wire for compatibility:
| Outcome | Record shape | UI/consumer meaning |
|---|---|---|
stopped | user_stopped=True, error unset | neutral: user killed it; partial result preserved; NOT a success, NOT a failure |
failed | error set | failure (tombstoned, counted in Stats) |
completed | neither | success |
- A user stop is neutral in the record itself:
cancel()setsuser_stopped=Trueand neither it nor_force_reapever synthesizes anerrorfor it. - Every emission carries the flag explicitly: live
subagent_doneevents, the_runfinally emit,_force_reap's emit, WS reconnect replay (managed and native),native_subagent_snapshots, and the/api/spawnlisting all includestopped. Cancelling a native card persistsstoppedon the slot tracker record so replay reconstructs it as stopped. - The gateway completion consumer (
_subagent_done) classifies three-way: a stopped agent is announced as "stopped by user ⏹" with partial output flagged, and in orchestrator mode records neitherrecord_successnorrecord_failure. - Intentional-cancel rule: every code path that cancels a subagent task on purpose MUST set a terminal marker first —
cancel()→user_stopped,cancel_all()→_shutting_down,_force_reap→reaped. An unmarked cancel is treated as unexpected and recovered once (below). Enforced MECHANICALLY, not by convention: all in-module intentional cancels route through the_cancel_task_intentionally(task, info, reason=...)chokepoint, which verifies a marker is visible before cancelling (a missing marker logs an error and consumes the recovery budget defensively so a mis-marked cancel can never zombie-respawn), and a source-scan test asserts no raw.cancel()on a managed run task exists outside the chokepoint.
Transient Retry (mid-stream 5xx)
_run_inner streams through _stream_with_transient_retry, mirroring the main path's retry ladder: transient backend errors (the -32603 class, per acp_error_is_transient) are retried with exponential backoff on the same live session; each retry fires a subagent_retrying WS event (chip shows ⟳ retrying) and a SEL audit record. Replay-safety: if ANY activity was observed (text chunk, approved tool turn, or auto-allowed tool call), the retry sends _TRANSIENT_CONTINUE_MSG instead of the original prompt — a mutating tool may have executed before the first text chunk, and replaying the full prompt would re-run it. Budget: TRANSIENT_RETRIES applies only while ZERO activity was observed (replaying the bare prompt is side-effect-free); after any activity, recovery is ONE-SHOT — exactly one continuation turn, matching the main path's _posttoken_retry_used rule, since each post-activity continuation is an independent opportunity to repeat a side effect. The two ladders (this one and dashboard/chat_runner.py's) are intentionally-identical copies cross-referenced in both sources; a change to either's predicate or budget must be mirrored. Non-transient errors and exhausted budgets propagate to the generic error arm.
Unexpected-Cancel Recovery (one-shot auto-continue)
An unmarked CancelledError (see intentional-cancel rule) triggers _schedule_cancel_recovery: exists for cancellations arriving from outside the manager's lifecycle (parent task-tree teardown around a live subagent), mirroring the main path's PR #173 recovery. Mechanics:
- Side-effect gate: recovery fires ONLY when
tool_count == 0. The respawn runs on a fresh session with no ledger of prior tool calls, so once any tool has executed the model cannot verify which side effects already happened — the run is finalized instead (error"cancelled (auto-continue suppressed: tools already executed …)", partial output preserved and delivered). Text-only activity is safe to resume. - One-shot: gated by
info._cancel_retry_used; the recovered run's own cancel is terminal. - Explicit handshake:
_resumeawaits the ORIGINAL task's full teardown (session release/reset, slot decrement, registry pop) before respawning — never a timed sleep. - Slot re-acquisition: waits (bounded,
_RECOVERY_SLOT_WAIT_SECS) for free capacity; the slot claim andcreate_taskare ATOMIC (no await between) so a concurrent_drain_queuecannot overshootmax_concurrent. - Shutdown-reachable: the pending
_resumetask is registered in_tasksunder"{id}:recovery"socancel_all()cancels it; a cancelled recovery finalizes the record terminally and never respawns. - Failed recovery (no slot / teardown timeout) still fully finalizes:
subagent_doneemitted, tombstoned, delivered viaon_done. - Replay-safety at respawn: when the first attempt streamed partial text, the respawned prompt is prefixed with
_CANCEL_RESUME_PREFIXso the model continues instead of restarting (the prefix also gates ontool_countas defense-in-depth, though the side-effect gate above means a tool-activity run never reaches respawn). A bare original prompt is re-sent only for a zero-activity first attempt.
Reaper Loop
start_reaper() launches a periodic loop (60s interval) that force-kills subagents exceeding the 30-minute timeout deadline. Defense-in-depth for cases where asyncio.wait_for fails to fire due to event-loop saturation or orphaned tasks.
_reaper_loop: sweeps every 60s, calls_force_reapon expired agents_force_reap: reset with 30s timeout → SIGKILL fallback → mark done → firesubagent_doneWS event- Terminal completion is arbitrated by FOUR separate guards, not by
reapedalone. Two paths can finish a subagent —_force_reapand_run'sfinally— and between them there are four distinct one-time concerns. Earlier revisions tried to arbitrate them withreapedplusdoneand every attempt satisfied two while breaking a third (duplicate delivery when the marker was set late; a lost outcome when it was set early and the reaper was cancelled; a lost outcome when the claim was handed back to a run that had already exited; and finally no reporter at all plus a leaked concurrency slot when the report claim was gated onnot info.done). The guards are now:info.reaped— classification. Was this a deliberate reap? The cancel-recovery scheduler reads it, and the marker MUST precede the intentional cancel (see the intentional-cancel rule above) or an unexpected-cancel respawn fires on the run being killed. Unchanged.if not info.done— the terminal RECORD. Error synthesis, failure stat, tombstone, cost. First-arrival-wins, so it is never written twice (pinned bytest_subagent.py::TestOnDoneTimeout::test_force_reap_skips_tombstone_when_already_done)._release_slot(info)— SLOT accounting. A one-shot token perSubagentInfo; the winner decrements_running_countonce and drains the queue. Deliberately independent of both flags above: inferring slot ownership fromdoneorreapedproduced a double decrement in one interleaving and none at all in another. A leaked slot permanently starves the spawn queue, which matters far more at the 60-100 concurrent agents the scale work targets. The cancel-recovery respawn re-arms this token when it re-admits a slot (_running_count += 1), because the respawned run occupies a fresh slot and needs its own release._claim_finalize(info)— REPORT ownership (subagent_done+ the_on_doneinjection, plus wave-digest settling and the result.txt TTL bookkeeping). Granted to exactly one caller; contains noawaitso the check-and-set is atomic on the loop. It does not consultinfo.done— that was the last defect. It returns False while_recoveringwithout consuming itself, so a pending respawn is not reported done and its respawned run can claim later.
- A claimed report is atomic, not merely exclusive. The claim alone still lost outcomes when the claimer was cancelled mid-report.
_report_terminaltherefore runs on a strongly-referenced task underasyncio.shield, spawned by_runbefore its teardown awaits so the task is already live wherever a cancellation lands; the caller still receivesCancelledErrorwhile the report completes.cancel_all()drains outstanding reports with a bounded timeout and then cancels and gathers any straggler, so none is left invoking_on_doneagainst tearing-down state or killed by a closing loop. Because the awaiter is shielded, shutdown is bounded by that drain rather than the_ON_DONE_TIMEOUTinjection cap. Enforced bytest_subagent_reap_race.py. - An undelivered report abandoned at shutdown is made RECOVERABLE, not silently dropped. The terminal record — including the tombstone — is written before delivery is attempted, and a tombstone is exactly what
list_orphans()uses to EXCLUDE a folder from the next start's reconciliation. So cancelling a still-pending report at the drain deadline would leave an outcome that was never injected and invisible to the only path that could still inject it.cancel_all()therefore callsclear_tombstone(id)for each report it cancels, re-admitting that agent to the next start's orphan reconciliation (which findsresult.txtand re-delivers). Extending the drain to_ON_DONE_TIMEOUTinstead was rejected: it would hold gateway shutdown for up to 20 minutes on one wedged injection, which is the exact failure the bounded drain exists to prevent. Only reports cancelled before_on_donereturned are re-admitted —info._reported_to_parentis set the moment the injection returns, so a cancellation in the later teardown/tombstone waits cannot cause a duplicate delivery on restart. - Every reporter goes through the claim — including cancel-recovery failure. There are more terminal paths than the two obvious ones: when a cancel-recovery respawn cannot happen, its
exceptarm also finalizes the agent. That site previously firedsubagent_doneand_on_donedirectly, gated only ondone/reaped, so a reaper racing a failed respawn delivered the outcome twice. It now takes_claim_finalizelike every other reporter and reports through the shielded helper (which matters because_force_reapcancels that very task)._resume_guarded's CancelledError arm writes only the RECORD and deliberately never reports — during shutdown the drain owns delivery. - The reaped marker and the recovery cancel precede every
awaitin_force_reap. Both used to sit after the session teardown, which yields for up to_RESET_TIMEOUT(longer on the SIGKILL path). A recovery task whose bounded handshake expired inside that window observedreaped == Falseand respawned the run being killed — tools executing after a user Stop, strictly worse than a duplicate report. - Delivery bookkeeping trails teardown. Spawning the report ahead of teardown opens a window the older ordering did not have: writing the "delivered" tombstone before the session is torn down would hide a surviving child from orphan reconciliation if the process died in between. The report therefore waits on a
teardown_doneevent (set in_run'sfinally, so it fires even under cancellation, and bounded so the report can never wedge) before marking delivery. A reaped or recovery-failed member still settles its siblings' digest holds, since those siblings' results did reach the parent even though this member's did not. On the dashboard routes the report's own settle andmark_deliveredare no-ops by design:_subagent_donedefers the delivery bookkeeping — the completed member's own tombstone AND any held wave siblings — to the parent's CONSUMPTION of the announce via_defer_queued_delivery(the #4839 content-keyed slot ledger +_delivery_queued), on the queue branch settled by the drain and on the direct-injection branch by_arm_queued_delivery_settlementarmed on the injection task (#2233). A bare_on_donereturn is a local routing success, not evidence the parent received anything; an unconfirmed hand-off leaves the debt parked and orphan-recoverable rather than tombstoned. - A synthesized reap error names only the cause the observed state supports. The wall clock fires at the 30-minute deadline, but a run parked on a never-answered spawn approval has reached no execution deadline:
turns == 0,_pid is None,_exec_started is None, and the dashboard's approval window is still open. So_force_reap's error synthesis tests_awaiting_approval and _exec_started is Nonefirst and reports the unanswered spawn approval, before thestartup_timeoutand generic-deadline arms. The predicate is captured above the intentional cancel, because the flag's owner clears it in afinallythe cancel schedules; reading it at the record site would hold only while noawaitsits in between. _sigkill_session: best-effort SIGKILL when graceful reset hangs- After decrementing
_running_count,_force_reapcalls_drain_queue()so the freed slot immediately starts a queued spawn. Normal completion pumps the queue via itsfinallyblock, but that block is gated onnot info.reaped; a reap setsreaped=Trueand decrements the count itself, so without this explicit drain a queued spawn would sit stranded until an unrelated agent finished or a new spawn arrived. - Wired up in
gateway.pyafterSubagentManagerinit
Idle-Stall Detection
The main-agent watchdog stack (tool_stall_suspect) does not govern subagents; _maybe_flag_stall(agent_id, info, now) (called from the reaper sweep) is their equivalent. It does, however, consult the same liveness oracle (acp/liveness.py) — see the attribution note below. Each session-scoped stream event calls _touch_activity(info), which updates info.last_activity, clears a prior stalled flag (re-emitting subagent_stalled {stalled: false} when work resumes), retires the agent's oracle and bumps info._stall_gen. info.last_activity is (re)initialised to _exec_started at the top of _run_inner so a queue / spawn-approval wait is never counted as idle.
Event kinds are NOT the discriminator: the same EVENT_SUBAGENT_LIST also reaches a session through the routed KAS sub-agent lifecycle path (_handle_kas_subagent, off a session/update frame), where it IS that session's own progress -- excluding by kind would falsely badge a working KAS agent. Provenance is carried instead: AcpRuntime._reader_loop sets JsonRpcMessage.fanout_no_owner when it fans an ownerless frame out to MORE THAN ONE registered session (a lone session is the sole owner, so it stays unmarked), the dispatch loop copies that onto AcpEvent.runtime_global, and _run_inner skips the refresh only for a runtime_global event. Everything else stays fail-open: an event kind the dispatch switch does not special-case still counts as activity, so a new session-scoped kind can never invent a false stall.
Why the exclusion exists: _kiro.dev/subagent/list_update carries no sessionId, so the runtime broadcasts it to every session queue, and under agent.session_sharing one roster notification lands on every co-tenant subagent's stream. Counting it as activity refreshed last_activity for a whole batch of wedged subagents at the same instant and cleared their badge, so the badge flapped and the reported idle_secs measured time since an unrelated agent's roster churn (#4841; the plateau measured in #2854).
Per sweep, for an agent that has actually started (turns > 0 or a live _pid) and is not blocked on a human approval prompt (_awaiting_approval):
idle > _stall_idle_secsand not already flagged → consult liveness (below), and unless the verdict clears it, setinfo.stalled = True, emitsubagent_stalled {stalled: true, idle_secs}(surface-only; the card shows a "no activity" warning), and append a record of the slow command to~/.kiro/crew/subagents/slow_commands.jsonlfor later analysis (rotated at 1 MiB keeping one previous generation,.jsonl.1, so total disk stays bounded at ~2 MiB; a reader wanting full available history must consume both generations).- Detection is surface-only:
_maybe_flag_stallnever terminates the agent. A genuinely-hung subagent is closed by the user from the UX (per-row stop →spawnDelete→SubagentManager.cancel(agent_id), or header Stop-all). The wall-clock reaper at_TIMEOUT_SECSremains the only automatic terminator; aDEADliveness verdict deliberately does not escalate to a kill, because that would be a change to reap semantics rather than to the signal.
Liveness attribution (why idle time alone is not the detector)
Idle time cannot separate a wedged tool call from a slow, silent one, so the flag is gated on a liveness verdict. Attribution is per-CHILD, not per-runtime-PID: the subagent event loop records the in-flight tool's dispatch snapshot (_inflight_tool — the trusted is_shell, the command, tool_name, dispatch time, taken from the same AcpEvent the main agent's ToolCallState is built from), and _stall_verdict hands it to a per-agent LivenessOracle. With is_shell set this takes the oracle's shell-child branch, which cmdline-matches a live descendant and then tracks that pid.
This is what makes the verdict meaningful even though session-sharing subagents share the parent's runtime PID: the match keys on the command, not the runtime. A whole-subtree aggregate would be useless here — it is dominated by kiro-cli's own background socket/keepalive traffic, so a sleep-only subagent reads as "working".
Verdict → action:
WORKING— an attributable live child, so the agent is progressing: not flagged (suspicion stays open so the badge appears as soon as that child stops).DEAD/STUCK_INPUT— positive evidence of a wedge, so it flags immediately, skipping the two-sweep confirmation that exists to dampen guesses. That skip is withdrawn whenever the runtime is session-shared (see the third bound below), because the trust it assumes is exactly what a shared runtime removes — the parent session is always a co-tenant of that process, so a lone subagent is no safer than one with siblings.UNKNOWN— no attributable evidence (no tool in flight, a non-shell tool with no child to match, unreadable/proc, or a refused executor): falls back to idle-time-only with two-sweep confirmation.
Four bounds keep this honest, and each exists for a failure that was observed rather than imagined:
_SUPPRESS_CEILING— aWORKINGverdict only suppresses whileidle < _stall_idle_secs * _SUPPRESS_CEILING. Attribution is not infallible: two siblings running similar commands undersession_sharingcan cmdline-match the same child, so a wedged agent could readWORKINGfor as long as its sibling's child lives. Unbounded that would convert a case the idle-time-only path DID badge into a permanent false negative — worse than a spurious badge, since the badge is self-clearing and a missing one is not. Past the ceiling the badge wins, so misattribution costs latency, not the signal.- The wedged skip is withdrawn under a shared runtime. The same fallible match runs in the other direction: a
DEADreading can describe another session's child that exited, and becauseDEAD/STUCK_INPUTnormally bypass the two-sweep confirmation, that would raise an immediate badge on a healthy agent — defeating the dampening that keeps the badge trustworthy at 60-100 agents. Granting one path immediate trust in a signal the ceiling exists because it is unreliable is incoherent, so wheninfo._session_sharingis set the wedged verdict earns its badge the same way a guess does: by holding across two sweeps (~60s). The gate keys on the flag, not on a sibling count._create_shared_sessionputs the subagent on the parent's AcpRuntime — one process hosts everything — soinfo._pidis the parent's process and the parent's own tool children are descendants of it too._live_shared_countiterates the subagent registry and therefore cannot see the parent, so an earlier_live_shared_count(pid) > 1form left a lone session-shared subagent on the fast path while it could still cmdline-match the parent's child and flag the instant that child exited. Since a shared runtime always contains the parent, "could this match belong to someone else?" holds for every session-sharing agent; only a dedicated-process subagent (session_sharingfalse, or a per-spawn model/effort override that forces its own process) keeps the immediate flag. - The walk is offloaded, never inline.
check_toolis a synchronous/procwalk (iter_descendants, plusos.readlinkon/proc/<pid>/fd/*, which can block on the very wedged fd being investigated) and the reaper runs on the same event loop that serves every chat turn. It is submitted throughconsult_offloaded(acp/liveness.py) — the one shared guard, not a local mirror of it: the same helper the main-agent watchdog reaches viaAcpSessionHandle._consult_oracle_offloaded, soSubagentInfosupplies the_consult_futureitsConsultFutureHolderprotocol requires and a fix to the guard lands on every caller at once. The guard owns submission-inside-the-guard, exception retrieval attached at submission, the bound (OFFLOADED_CONSULT_TIMEOUT_SECS, 10s) viawait_for(shield(...)), at most one outstanding walk per holder so a permanently wedged read cannot leave a new blocked worker behind on every sweep, and degrade-to-UNKNOWNon any failure. Failure mode to be aware of: consults are awaited serially within a sweep, so if many agents cross the idle threshold while their/procreads wedge, a single sweep can stretch toward N×10s and delay the wall-clock reap for the other agents in it. Bounded and unlikely (one walk per agent, later sweeps short-circuit on the in-flight guard), but it is the cost of doing this in the sweep rather than out of band. - Generation counter. The awaited verdict is discarded (
superseded mid-consult) wheninfo._stall_genmoved during the walk — i.e. activity, a final tool result, or the next dispatch retired the snapshot it was submitted for. Without this the walk's own latency is enough to flag an agent that has resumed working, andDEAD/STUCK_INPUTskip two-sweep dampening, so a stale one would flag instantly.
The snapshot is retired only on a tool_final result. EVENT_TOOL_RESULT is also emitted for non-completed progress updates (_dispatch sets tool_final = status == "completed"), and treating one of those as the end of the tool would drop attribution mid-command — degrading exactly the long silent command this detection exists to judge. acp.client gates on the same field. On each retirement the oracle is replaced via fresh() rather than mutated, so a walk still running against the previous command cannot write its late sample into the next tool's baseline.
The verdict and its evidence are recorded in the reaper's log line but are deliberately not on the subagent_stalled wire: no consumer reads them today (the frontend narrows the payload on arrival and the coalesced batch update forwards only stalled), and the event is app-sdk-forwarded, so unread keys would become semi-permanent surface.
The slow-command record (record_slow_command, subagent_persistence.py) is append-only and deliberately NOT a tombstone: a tombstone marks an agent dead and is consumed by orphan-reconciliation / TTL cleanup, whereas a stalled subagent is still running. Fields: id, flagged (ts), last_tool (redacted), tool_count, turns, idle_secs, elapsed_secs, parent_session, session_sharing.
_awaiting_approval is set around both human approval awaits — the mid-run tool approval in the EVENT_PERMISSION_REQUEST branch (reset in finally, which also refreshes last_activity) and the pre-execution spawn gate in _spawn_with_approval (also reset in finally) — so a slow approval never looks stalled. The two are told apart by _exec_started: it is set for the mid-run prompt and None at the spawn gate, which is what lets the reaper name the right cause (see Reaper Loop).
Running-card progress events
subagent_tool is fired on EVENT_TOOL_CALL (not only EVENT_PERMISSION_REQUEST) — kiro-auto-allowed tools surface only as informational tool_call updates, so this is the sole progress signal a simple/read-only task emits. Payload carries {tool, tool_kind, turns, tool_count}; info.tool_count increments per observed tool call. The subagent_snapshot reconnect payload (dashboard/ws.py, built by build_subagent_snapshot()) also carries tool_count, stalled, and — only while stalled — idle_secs, recomputed at replay time from last_activity (clamped at 0, omitted entirely for a healthy agent) so a reloading client recovers progress/stall state including the span that justifies the stall badge (a transition-only WS signal always needs a matching snapshot field).
Model Provenance (#3582)
Every subagent card names the model the run actually ran on, so a model-pinned
review's real model is auditable. SubagentInfo carries two fields: requested_model
— the EFFECTIVE pin, i.e. the per-spawn model OR, when empty, the
agent.role_models['subagent'] config pin (AGENTS.md's documented way to pin a
subagent model), resolved once at spawn; "auto" when completely unpinned (no
per-spawn model, no role pin) — and resolved_model, the id the live
session actually served, read via the provider's public served_model accessor
(_resolved_model_of, which normalizes the DEFAULT_MODEL "auto" sentinel to ""
= unknown). resolved_model is captured at spawn (ACP reports it immediately) and
refreshed on the first text chunk (covers the CC path); a known value is never
clobbered back to "".
The resolved id rides the wire as a model field on the subagent_spawn,
subagent_done, and reconnect subagent_snapshot payloads, and the requested pin
rides alongside it as a requested_model field on those same payloads (both are
_redact()-ed, since the pin is caller-supplied). The single-completion
meta (subagent_completion_meta.single_completion_meta, mirrored by
website/src/pages/chat/subagentCompletion.ts) additionally carries requestedModel
and resolvedModel. The frontend renders the resolved model as a chip beside the
agent pill and flags a downgrade — an amber chip plus a persistent
role="status" "Requested X, served Y" banner — when the two name different models,
on BOTH the completion card AND the live Subagents-panel row (ActivityViewer),
so a mis-pinned run is visible mid-flight, not only at completion. Because the pin
rides subagent_done (and its reconnect replay), a downgraded run that completes
before a reconnect rehydrates its card with the amber flag intact. For unpinned
spawns requested_model="auto" records the sentinel so the frontend shows a neutral
chip rather than hiding the column. "Same model" is decided by the shared
normalizeModelKey
(website/src/lib/model.ts, mirroring the backend _normalize_model_key): dotted vs
dashed spellings and case fold, and auto/default fold to "no pin", so an honored
pin whose wire spelling differs does not false-flag. Wave-digest completions
(wave_chunk_meta/wave_final_meta) carry no structured model field, but each
member's served model is surfaced inline in the digest body
(ok_lines/fail_lines) that both the parent LLM and the card already read —
— \id` ✅ task · model — so batch members are auditable for which model actually ran. Only the served id is shown (no requested/downgrade qualifier): a raw requested-vs-resolved inequality is not the card's downgrade fold and would false-amber every member of a normalauto`-pinned wave, so the
amber-downgrade signal stays a single-completion concern until this shares the
card's fold (or #5339's registry fold). The value is redacted through the
display context before it enters the broadcast digest text.
Completion Injection
Subagent results are routed back to the originating session via
_subagent_done in gateway.py. The parent_session_key on SubagentInfo
tracks which session spawned the subagent.
Two-Level Timeout
| Timeout | Location | Duration | Scope |
|---|---|---|---|
| Outer cap | subagent.py _run() | 1200s (20 min) | Semaphore wait + injection combined |
| Inner cap | gateway.py _subagent_done() | 900s (INJECTION_TIMEOUT, tunable via KIROCREW_INJECTION_TIMEOUT) | Single stream_and_collect call |
On timeout (inner or outer):
- Kill stuck kiro-cli process via
sessions.reset() - Queue failure event into
slot._pending_subagent_failures - Next
_run_chatdrains the queue into LLM context withresult_path - LLM reads result from disk if needed
Prompt-Busy Recovery
_inject_with_retry() in gateway.py makes up to 3 attempts (1 initial + 2 retries) of stream_and_collect on AcpError. Between retries: cancels orphaned prompt, exponential backoff. On PromptBusyExhaustedError: kills provider, queues failure event. Note: the 1200s outer cap (_ON_DONE_TIMEOUT) bounds total wall-clock time, so not all retries may fire if earlier attempts consume the budget.
Reconnect recovery: subscribe_subagents in ws.py restores both managed and native subagent cards. Managed subagents are authoritative in SubagentManager: running records replay as subagent_snapshot, and recently completed records replay as subagent_done. Managed results remain disk-backed and are not copied into inline Redux card payloads.
Native kiro-cli subagents run inside the parent ACP turn and are owned by the parent dashboard slot. DashboardState.native_subagent_snapshots() replays running native cards as subagent_snapshot and recent terminal cards as subagent_done. A native subagent_done payload may include optional task, agent, and result fields. result is a redacted output tail bounded to 8,000 characters, with an explicit truncation marker when earlier output was dropped. Running output retained for replay is bounded to 40,000 characters, with an 80,000-character hard accumulation ceiling. Terminal native records are retained globally up to 50 cards for at most one hour. The client treats done and error as monotonic terminal states, so a stale running snapshot interleaved after a live completion cannot demote the card.
Redaction: All subagent event payloads (running snapshots and done events) have the agent field redacted before sending to the dashboard. Task text is redacted before truncation to prevent credential patterns spanning the boundary.
| Parent Session | Backend Delivery | Client Follow-up | User Sees |
|---|---|---|---|
Dashboard (dashboard:*) | Append as user message + broadcast via WS | TUI/web re-injects via sendMessage → LLM round-trip | LLM's response summarizing the result |
| Slack (thread ts) | Post to Slack channel thread + dashboard notification | (none — raw result posted directly) | Raw subagent result text |
Non-Slack channel (telegram:*, discord:*, unified:*, …) | Inject into the parent ACP session, then send the synthesized reply through the governed cross-surface transport ladder (_deliver_channel_reply → _resolve_channel_target → MessagingTransport.send_message) + dashboard notification. Target resolution: origin link (recorded by Discord's inbound dispatch) → non-Slack mirror link (e.g. Telegram /link) → for direct (1:1) sessions only, the stored "{namespace}:{user_id}" channel value, resolved to a postable conversation via transport.resolve_configured_target. Group/forum sessions without an origin/mirror link, channels whose dispatcher records neither, and denied/unsupported egress all degrade to notification-only — never a cross-conversation send. | (none) | LLM's synthesized reply in the channel conversation |
| Cron/no parent | Dashboard notification only | (none) | Notification panel entry |
Post-fan-out Synthesis Turn
After a fan-out of sub-agents, a single dedicated synthesis turn produces the user-facing summary (restate goal → synthesize across all results → recommend next actions), instead of leaving the last visible message as a per-sub-agent completion note. Dashboard chat only (orchestrator mode has its own stage synthesis).
- Arm — in
_subagent_done(chat mode,not _is_orchestrator), when the last outstanding sub-agent for the parent completes (running_agents_for(parent_key) == []), setslot._pending_synthesis = True. - Fire — in
chat_runner._run_chat's drain/idle branch, once the queue is empty, no agents are running,_pending_synthesisis set, andslot._subagent_deliveries_inflight == 0, launch exactly one tracked synthesis task._synthesis_inflightprevents duplicates. There is no readiness wait: readiness is latched at gateway boot and refreshed only on explicit user action, so parking the arm on it would strand the synthesis indefinitely. The task clears the arm once the delivery guards pass, immediately before starting one timeout-bounded_run_chatturn withSUBAGENT_SYNTHESIS_PROMPT; a signed-out CLI surfaces as anAcpAuthRequirederror card from that turn. - Per-result turns kept — each completion is still processed in its own turn (no raw buffering) to avoid a context-window blowup; the synthesis works over the already-condensed per-result turns.
- Delivery-race guard —
_subagent_deliveries_inflightis incremented in_subagent_donefrom entry until the completion is queued/launched (try/finally). Because a concurrently-finishing sibling holds this count while it awaits the current turn (busy path), an earlier turn cannot fire synthesis before that sibling's result is delivered. - Cancellation — a real user message draining first clears
_pending_synthesis(user takes over); a newer in-flight batch defers synthesis until it too completes (only one synthesis fires, after all work). - Linked surfaces —
SUBAGENT_SYNTHESIS_PROMPTbegins withSUBAGENT_SYNTHESIS_PREFIX, marking it a synthetic continuation that is NOT mirrored to Slack/Telegram as a user message (only its reply is delivered).
Parent Session Discovery
The gateway sets the KIROCREW_SESSION_KEY env var when spawning kiro-cli,
and mcp_core.py reads it via os.environ.get(). If the env var is missing
(e.g. older gateway), it falls back to reading
~/.kiro/crew/session_pid_{getppid()}.txt for backward compatibility. The
session key flows through the /api/spawn endpoint as parent_session.
Scale Plumbing (60-100 concurrent agents)
Large waves must not flood the WS socket, the parent LLM's context, or the UI. Five mechanisms — WS coalescing/replay-batching and UI caps are inert below their thresholds; digest chunking applies uniformly to every multi-task wave (single-task spawns behave byte-identical to legacy):
-
Batch identity:
spawn(batch_id=..., batch_total=...)(threaded fromspawn_run tasks=[...]— one 12-hex id per multi-task call — viaPOST /api/spawntransport params; survives the stagger queue).spawn_batch_started {batch_id, count}fires once per batch on its first started member; the id rides every WS frame (base["batch_id"]). -
Event coalescing (
subagent_scale.SubagentEventCoalescer, wired in the gateway's_subagent_event): above 8 active agents,subagent_tool/subagent_stalled/subagent_retryingbuffer per-agent (latest state wins, merged) and flush every ~1s as ONEsubagent_batch_update {updates:[...]}frame to all clients;subagent_chunktext buffers append-concatenated (16KB/agent cap) and flushes assubagent_batch_chunks {chunks:[...]}to subagent subscribers only. Lifecycle events (spawn/done/recovering/injection_failed/batch_*) are NEVER coalesced, and adone/spawnflushes buffered state first so ordering is preserved. Non-int active-count fails open to pass-through. -
Chunked wave-digest completion injection (gateway
_subagent_done): every batch member is accounted perbatch_id(this is the single completion consumer for all terminal paths). Every multi-task wave (batch_total > 1) delivers results to the parent queue-style: completed members are HELD, and everySUBAGENT_DIGEST_CHUNK_SIZEcompletions (default 10, envKIROCREW_SUBAGENT_DIGEST_CHUNK_SIZE, clamped 1..1000) flush ONE[Subagent batch completion event]chunk digest — failures first with detail, successes as one-lineresult_pathpointers (60KB cap per chunk); the final member flushes the remaining partial chunk. A 60-agent wave = 6 digest turns spread across the wave's runtime — bounded chunk size, incremental signal, and no straggler-gated mega-digest. Chunk buffers (fail_lines/ok_lines/guard_msgs/held_ok_ids) reset per flush; cumulativeok/err/stoppedcounts ride the final chunk's summary. Spawn discipline: non-final chunks instruct the parent NOT to spawn new sub-agents while batches are still arriving; the final chunk releases the gate ("finish processing all results before spawning follow-ups") — mirrored by a line in thespawn_runtool description. Chunk order is FIFO: the injection busy-check (_injection_slot_busy) treats a liveslot.task— the claim assigned synchronously at dispatch — as busy in addition toslot.running, so a later chunk waits behind an injection that is dispatched but still insidebounded_chat_turn's off-loop timeout resolution, instead of racing ahead of it or assigningslot.taskover the earlier chunk's still-pending task. Single-task spawns have no batch identity and keep the plain per-agent injection. A batch member rejected at spawn (empty task, low memory, cwd, governance, bad agent) is counted as submitted AND announced through the done callback with its batch identity (_announce_rejection) — so a rejection that closes the wave still reaches the consumer and releases held sibling results (non-batch rejections do not announce; the caller gets the error synchronously).batch_finished {batch_id, total, ok, err, stopped}broadcasts for every batch regardless of size. Wave liveness (lost-submission backstop): a member rejected before reachingspawn()or lost during transport is counted in every sibling'sbatch_totalbut never insubmitted— un-reconciled, the count-drivenbatch_members_pending()wedges the wave forever. Three layers close it: (1)api_spawnmarks in-process rejections/capacity withcounted: true(preserved through the MCP client's error flattening); (2)spawn_runbest-effort POSTs/api/spawn/lostfor each explicit UNcounted rejection, which callsrecord_lost_submission— counts the member as submitted and announces a synthetic terminal failure through the completion consumer so the wave closes; uncertain transport failures are not immediately reconciled because the gateway may have accepted them; (3) the reaper's_sweep_stuck_waves(every sweep) force-reconciles uncertain or lost submissions whensubmitted < expected, all registered members are terminal, nothing is queued, and no submission progress occurred for_WAVE_STUCK_SECS(1800s / 30 minutes — deliberately generous, symmetric with the per-agent hard ceiling) — one lost member per sweep, converging across sweeps; this also bounds the_batch_submitted/_batch_progress_tsleak. Straggler-held partial chunks are bounded by the hold deadline (below), not by the member's 30-minute hard ceiling. -
Digest hold deadline (straggler escape hatch): both chunk triggers are event-driven — a COUNT trigger (
SUBAGENT_DIGEST_CHUNK_SIZEpending completions) and wave close — so neither can fire while a straggler is simply not finishing. With the default count (10) above any wave size the concurrency cap realistically produces (2–5), the count trigger is unreachable and wave close becomes the ONLY flush: every sibling's finished result is withheld for the slowest member's entire remaining runtime, and a member that HANGS rather than fails withholds them for the full_TIMEOUT_SECSreap — up to 30 minutes of total silence, indistinguishable from a dead session (issue #2215). The reaper's_sweep_digest_holdssupplies the LATENCY trigger the count lacks: when the OLDEST outstanding hold in a live wave ages pastDIGEST_HOLD_SECS(default 120s, envKIROCREW_SUBAGENT_DIGEST_HOLD_SECS, clamped to_TIMEOUT_SECS;0opts back out to count-trigger-only),force_digest_flushannounces a synthetic flush-only record through the single completion consumer — the same re-entry mechanismrecord_lost_submissionuses, so digest composition, routing, and the held-tombstone settle contract stay in one place. The record carries the wave'sbatch_idbut is NOT a member:_digest_flush_onlymakes the gateway skip every per-member side effect (terminal WS event, orchestration tracker accounting,done/ok/errcounters, digest lines) and only force the pending chunk out. One knob, two jobs, now split: the count keeps bounding digest SIZE for large waves; the deadline caps worst-case delivery LATENCY at every wave size. A wave whose members all finish within the deadline of each other still delivers ONE consolidated digest, so the deliberate small-wave behavior is unchanged. The forced chunk is labelled honestly as a PARTIAL release (k/k+1, "N of M delivered, R still running") and tells the parent to synthesize what it has rather than keep waiting. Hold bookkeeping: the gateway stamps_digest_held_atwhen it holds a member and clears it when that member's chunk fires — deliberately separate from_digest_held, which is the restart-safety flag the run loop reads and which the sweep must never mutate. The sweep is skipped entirely whenbatch_members_pending()is False, so it can never race the real wave-close digest into a duplicate delivery. -
Reconnect replay batching (
ws.py): more thanSUBAGENT_REPLAY_BATCH_THRESHOLD(8) replay frames collapse into ONEsubagent_snapshot_batch {items:[{type, data}]}frame; the client fans items into the per-frame reducers. -
Stall two-sweep confirmation (
_maybe_flag_stall): the first reaper sweep past_stall_idle_secsonly marks_stall_suspect_at; the second consecutive idle sweep flagsstalled(event + slow-command record). Any stream activity that BELONGS to the session (_touch_activity) resets the suspicion; aruntime_globalframe fanned out to co-tenants does not. Adds ≤1 sweep interval (~60s) latency; prevents alarm fatigue from healthy-slow agents ambering at scale.
Retry endpoint: POST /api/spawn/{agent_id}/retry re-spawns a terminal FAILED agent's original task (never running — would double work; never user-stopped — deliberately killed; native rejected). New id, no batch identity carried (a finished wave's digest is never reopened). Backs the UI's "Retry failed (N)" control.
Hook Integration
PostToolUse Firing
The subagent loop fires both PreToolUse (on EVENT_TOOL_CALL) and
PostToolUse (on EVENT_TOOL_RESULT), mirroring chat_runner.py. The
tool name is cached on EVENT_TOOL_CALL by tool_call_id and looked up
when the result arrives. The Running: prefix is stripped so both hooks
receive identical tool_name strings. Hook errors are caught at debug level
to prevent misbehaving hooks from breaking the subagent loop.
Hook Payload Metadata
Three optional fields are passed to ScriptHookStore.fire() and the
fire_tool_hooks() wrapper when called from subagent context:
| Field | Source | Description |
|---|---|---|
subagent_id | SubagentInfo.id | 8-char hex ID of the firing subagent (None for parent) |
parent_session_key | SubagentInfo.parent_session_key | Session key of the parent that spawned this subagent |
agent_role | SubagentInfo.agent | Agent role name configured for the subagent |
All three default to None and are only emitted into hook_event when
truthy. Payloads are byte-identical for callers that do not supply them,
preserving backward compatibility for existing hook scripts.
Caller sites:
subagent.py: passes all three at bothfire_tool_hooks(PreToolUse) andhook_store.fire(PostToolUse) call sitestask_executor.py: passessession_keyandagent(nosubagent_id)chat_runner.py/llm_helpers.py: unchanged (parent context, defaults to None)
Skill Integration
skills/subagent/SKILL.md (project-level) triggers on keywords: background, spawn, bg, subtask, parallel, separately, concurrently. Instructs the LLM to use kirocrew spawn "task" via bash to spawn subagents.
CLI: kirocrew spawn "task"
POSTs to http://localhost:5476/api/spawn (dashboard API). Returns immediately with subagent ID. Gateway runs the task async and posts result to Slack when done.
MCP Tool: spawn_run
Exposed via kirocrew-core MCP server. Always fire-and-forget — results
are delivered back to the calling session via completion event injection.
Single task:
spawn_run(task="search docs for X")
Batch parallel:
spawn_run(tasks=["search docs for X", "check pipeline status", "review CR-123"])
All agents spawn at once. The tool returns immediately with agent IDs.
Results arrive as [Subagent completion event] messages in the session,
processed by the LLM automatically.
Parameters:
task(str): single task descriptiontasks(list[str]): multiple tasks for parallel executioncwd(str, optional): absolute path to launch subagent in. Must be under a configuredsubagent_cwd_allowed_rootsentry (default:~/workspace,~/workspaces,~/workplace,~/workplaces). Validated via realpath + prefix match. Pool skipped when cwd is set. These roots are a least-privilege allowlist and are never widened automatically: a persisted list whose roots all fail to exist on the host rejects every cwd, and the operator must editagent.subagent_cwd_allowed_roots(or delete the key to take the shipped default). Neither the loader nor the guard stats the configured roots.max_turns(int, optional): override tool-call budget for this spawn (default: config or 100)agent(str, optional): agent name for the subagentreasoning_effort(str, optional): per-call reasoning-effort override (low/medium/high/xhigh/max), batch-wide likemodel. Precedence: per-call value →agent.role_efforts['subagent']pin → provider default;""/absent changes nothing. Like a model/effort role pin, a non-empty value forces the dedicated-process path (the parent's shared runtime cannot switch effort per session), so a wide fan-out pays a full process per subagent — and that cost is paid even when the resolved model turns out not to support effort (the level is then dropped at the provider factory). Carried through the stagger queue and the retry endpoint like the context-group flags. NOT inherited byspawn_continue— a continuation resolves effort fresh (role pin, else default), the same parity asmodel. When the requested effort cannot take effect, the gateway says so:/api/spawnresolves the model the factory's effort gate will see (per-call value, else the subagent role pin, else the session chain for the effective agent — a crew's own model pin, else a non-sentinel globalagent.model; a named kiro agent's own pin resolves downstream and cannot carry the overlay) and returns aneffort_droppedreason on the success response, which the tool renders as one attributed line per distinct verdict — subagents sharing an identical verdict (the usual case, since the value is batch-wide) are collapsed into a single line naming all of them, while differing verdicts keep their own attributed lines — including the default case where nothing is pinned and the model resolves to "auto". When the effort WILL apply, the response instead carries aneffort_appliednote naming the resolved model and the family-specific settings key (reasoningfor GPT,output_configfor Claude) it is delivered under, rendered the same way — so both outcomes of a requested effort are visible in the tool result. A role-pinned effort that will be dropped (no per-call effort involved) still surfaces in the gateway log at warning level, since the tool caller never asked for it — that warning is emitted by the provider factory's effort gate itself (config/loader.py), the single authority that drops the level, so one log line covers every surface that funnels through it (spawn, dashboard slot, cron) and cannot drift from the decision it reports on. The provider factory remains the single dropping authority; the report never rejects or alters a spawn. Per-TASK variation inside one call is deliberately not supported (see issue #2140).include_memory/include_lessons/include_project(bool, optional, defaulttrue): which switchable context groups the subagent inherits, applied to every task in a batch spawn. All-on is byte-identical to the injection a normal session gets, so a caller that omits them changes nothing.include_memory=falsedrops preferences, projects, daily history, semantic and episodic memory, and prior-session provenance — the normal choice for fan-out whose task text is self-contained.include_lessons=falseadditionally drops the user's learned corrections and profile, so keep it on for any subagent that writes code, edits files, or runs git.include_project=falsedrops the docs pointer and the project-directory line. It also drops the injected steering block, but ONLY on the Claude Code backend: on the ACP/kiro backendkiro-cli --agentloads the agent'sresources(including steering globs) itself, which Kiro Crew cannot suppress from here, so steering still reaches an ACP sub-agent regardless of this flag. The conduct group — critical output-format rules, date, agent identity, runtime, workspace identity, and the skills index — is never switchable, because a subagent without it cannot discover its own capabilities or format what it reports back. A subagent is told by name which groups were withheld ([CONTEXT SCOPE]) so it reports the gap rather than guessing. Resolved once at spawn, carried through the capacity-queue round-trip andPOST /api/spawn/{id}/retrylikeapproval_mode/silent/keep.spawn_continuedoes not take the flags but does inherit them from the run it continues: a continuation rebuilds session context (get_or_createreportsis_new=Trueeven when it restores the session viasession/load), so without inheritance a scoped-down run would regain a group on its follow-up turn. Seememory-skills-hooks.md§ Switchable context groups for the section-by-section mapping.
Response semantics:
- An ID means the submission was accepted. A running subagent returns its durable agent ID; capacity/stagger queueing returns a temporary
qNreceipt that is replaced by the durable ID when the queue drains. Usespawn_listor the completion event to discover the durable ID rather than treating the receipt as a result path. - An explicit HTTP error response means the submission was rejected and is reported as
failed to start; rejected work is never described as queued. - A transport failure has unknown acceptance status because the gateway may have accepted the work before the response failed. The response warns against automatic retries and directs callers to wait and recheck
spawn_listor completion events first. An empty immediatespawn_listresult is inconclusive because the stagger queue is not listed. If the request was truly lost, accepted siblings may remain held until the_WAVE_STUCK_SECSbackstop (1800s / 30 minutes) reconciles the wave. - If every submission is explicitly rejected (with no transport uncertainty), the response states that none of the requested subagents were started and does not promise completion events or suggest polling.
- For a partial batch, accepted IDs remain paired with their tasks, rejected tasks appear in a separate failure section, and completion guidance applies only to accepted submissions.
MCP Tool: spawn_sub_agents
Exposed via kirocrew-core MCP server. Unlike fire-and-forget spawn_run,
spawn_sub_agents is blocking: it spawns one or more sub-agents in
parallel, waits until all of them finish, then returns their collected
results inline to the calling tool invocation.
Each sub-agent runs as its own KiroCrew-owned ACP session (via
SubagentManager), so its text and tool calls stream live to the Activity
tab (subagent_spawn / subagent_chunk / subagent_tool / subagent_done
WS events) while the parent blocks.
Native kiro-cli subagent/use_subagent crews run inside the parent's
kiro-cli process rather than as KiroCrew-owned sessions. KiroCrew surfaces
those in the Activity tab too, by observing kiro-cli's sub-agent
notifications — one card per sub-agent, with each inner tool call and its
output attributed to the right card.
spawn_sub_agents(agents=[
{"agent_or_mode": "gpu-multiagent-explorer", "prompt": "list python modules"},
{"agent_or_mode": "gpu-multiagent-explorer", "prompt": "summarize last 5 commits"},
])
Parameters:
agents(list[dict], required): each item is{prompt: str, agent_or_mode?: str}.promptis truncated toMAX_MEDIUM_STRING;agent_or_modetoMAX_SHORT_STRING. Entries with an empty prompt are skipped.cwd(str, optional): absolute path to launch all sub-agents in. Must be under a configuredsubagent_cwd_allowed_rootsentry (default:~/workspace,~/workspaces,~/workplace,~/workplaces), same validation asspawn_run.
Blocking poll semantics:
- Each sub-agent is spawned via
POST /api/spawn(withparent_session), then the handler pollsGET /api/spawn/{id}every 2s until every sub-agent reportsdone(orerror). - An errored/crashed sub-agent is treated as settled so one bad agent cannot keep the loop spinning until the deadline.
- The loop pings
POST /api/session-keepaliveevery 60s so the gateway'sis_responsive()does not flag the (legitimately long-blocked) session as stale and SIGTERM the ACP subprocess mid-poll. Thewaittool pings the same endpoint for the same reason but on a 5s interval and with a body, because there the reply doubles as an early-end control channel (seemodules/learn-cron-dashboard.md§ Wait countdown and early end); this loop sends{}and ignores the reply, so 60s is sufficient. max_waitdefaults to 7200s (2 hours), clamped to[60, 7200], and is configurable via theKIROCREW_SPAWN_SUB_AGENTS_MAX_WAITenvironment variable. The deadline usestime.monotonic().- Returns a newline-separated list of per-agent JSON results (
status:completed/error/timed_out), all redacted for credentials and exfiltration URLs.
Difference from spawn_run: spawn_run returns immediately and delivers
results later via completion-event injection; spawn_sub_agents blocks and
returns the aggregated results directly, so the calling agent can reason over
them in the same turn.
Orphan Recovery & Tombstoning
Folder-per-agent persistence at ~/.kiro/crew/subagents/{id}/:
~/.kiro/crew/subagents/{id}/
state.json # {task, parent_session_key, started, pid}
result.txt # full result text (written on completion)
tombstone.json # {error, elapsed, timestamp} (written on failure/orphan)
Gateway Restart Reconciliation
On startup, SubagentManager scans ~/.kiro/crew/subagents/ and reconciles:
- PID alive → kill process group, deliver result if available, tombstone if not
- PID dead + result.txt exists → deliver result to parent session
- PID dead + no result → write tombstone with "orphaned" error
Orphan delivery is wired (not a stub): the gateway registers on_orphan_notify (session injection — rides the parent slot's batched pending-failures drain) and on_orphan_dm (fallback). The DM fallback collects every undelivered orphan across the reconciliation scan and sends ONE digest message ("N subagent(s)…") — never N pings; a lone orphan keeps the plain per-agent message.
Tombstone Lifecycle
- Created on: process death without result, delivery failure, timeout (
cause=error/timeout/cancelled/reaped/gateway_restart), and on successful delivery (cause="delivered", viamark_delivered) soresult.txtis retained for the grace window instead of deleted immediately. - Pruned by reaper:
deliveredtombstones afteragent.subagent_result_ttl_secs(default 1h); all other tombstones after 7 days.prune_stale_tombstonestakes a per-cause cutoff for this. spawn_statusfalls back to persistence layer for completed/tombstoned agents, reading the retainedresult.txt(and honoring offset/limit/grep).
MCP Tool: spawn_status
Retrieves a completed subagent's transcript by ID. The completion event now
carries a summary + the result_path whenever the completion copy was
truncated (result_truncated) or in orchestrator mode, so the parent reads the
full transcript on demand instead of re-running the subagent.
The full transcript stays in ~/.kiro/crew/subagents/<id>/result.txt for a
retention grace window after delivery — on success the folder is not
deleted immediately; mark_delivered writes a cause="delivered" tombstone and
the reaper prunes it after agent.subagent_result_ttl_secs (default 3600s / 1h).
This fixes the prior day-1 bug where delete_agent_folder ran immediately on
delivery, so a later spawn_status found no file and silently fell back to the
truncated in-memory info.result ("truncated at the same place").
Parameters:
agent_id(str, required): subagent ID from the completion event (alnum, max 64 chars)offset(int, optional): 0-based start line for a paged read (line-oriented, like reading code)limit(int, optional): max lines to return (1–2000). Omit for the full transcript.grep(str, optional): case-insensitive regex; return only matching transcript lines (offset/limit then apply to the matches)
When any of offset/limit/grep is set, the /api/spawn/{id} response
includes a result_meta block (total_lines, matched_lines, offset,
returned_lines, has_more) and the tool output is prefixed with a one-line
continuation header (showing lines X-Y of N | more available — call again with offset=Y). With no paging params the full-transcript contract is unchanged. The
line split + regex run via asyncio.to_thread so a pathological pattern never
stalls the event loop.
Completion Event Truncation Modes
The character cap and which end of the transcript to keep are both configurable. Defaults preserve original behavior — opt-in to the others when a particular agent style benefits from the change.
When truncation drops content (SubagentInfo.result_truncated), the completion
event is not a raw truncated blob: it carries a first+last-words preview + the
result_path (via context_management.summarize_result) so the parent reads
the full transcript on demand (read / grep / spawn_status) instead of
re-running the subagent. This is the same shape orchestrator-mode deliveries
have always used, now applied to chat mode too (gated on result_truncated so
small results still inline in full).
| Config key | Values | Default | Effect |
|---|---|---|---|
agent.completion_keep | head / tail / both | head | Which end of the transcript to keep when the cap is exceeded |
agent.completion_keep_chars | int (0 disables truncation) | 3000 | Character cap applied after completion_keep |
The helper apply_completion_keep(text, mode, max_chars) lives in
context_management.py. head is identical to the earlier
behavior. tail is appropriate for agents that summarize at the end
(developer/reviewer/on-call). both keeps roughly half the budget at
each end with a middle elision marker.
Unknown agent.completion_keep values cause kirocrew gateway to fail
at startup via _validated_completion_keep in config/loader.py. The
dashboard PATCH endpoint enforces the same enum via
_EDITABLE_CONFIG["agent.completion_keep"].
The values are threaded into SubagentManager.__init__ from
gateway.py (completion_keep=, completion_keep_chars= constructor
kwargs sourced from cfg.agent.*). User-facing docs:
src/kiro_crew/docs/configuration.md,
src/kiro_crew/docs/subagents.md,
src/kiro_crew/docs/troubleshooting.md.
Dashboard API: POST /api/spawn
Request: {"task": "..."}
Response: {"id": "abc123", "task": "...", "status": "spawned"}
Errors: 400 (missing task), 429 (capacity reached), 503 (subagents not available)
Typed rejections. A rejection raised INSIDE spawn() answers 400 with a
machine-readable code beside the advisory error prose (plus counted: true —
see Wave liveness above): agent_not_found for a named-but-unknown agent,
spawn_rejected for every other kind (empty task, low memory, cwd refusal,
governance). code is the contract and error is advisory (RFC 9457 3.1.3),
which is what lets the refusal sentence be reworded without breaking a client.
The identifier is minted AT the decision — subagent.AGENT_NOT_FOUND_CODE,
returned by _validate_agent — carried on SubagentInfo.error_code, and
forwarded by the handler without being respelled there, so the value has exactly
one spelling in the tree.
spawn_run switches on that code for the wave short-circuit (#4842): once the
gateway has refused an agent name, the remaining members of a wave sharing it are
not re-posted. Fail-soft in both version directions — an old client still
text-matches the unchanged prose, and a new client against a gateway that sends
no code loses only the short-circuit (every member is dispatched and refused
individually) and never refuses a name the gateway would have accepted. That
asymmetry is why a missing code is safe here, and why a code is never used to
REJECT a spawn.
The request-validation errors (bad JSON, missing task, bad approval_mode /
batch_id), the 429 capacity answer and the 503 are prose-only today; converting
them is Track B work tracked by error-code-baseline.json.
Not yet true of the sibling endpoints: POST /api/spawn/{id}/continue and
.../release DO answer with a code, but they derive it by prefix-matching the
manager's prose (info.error.startswith("conversation_busy")), because
continue_conversation mints those two decisions as sentences rather than
returning an identifier. SubagentInfo.error_code is the carrier that would let
them be minted at the decision the way the unknown-agent refusal now is; until
that migration, treat conversation_busy / conversation_gone as inferred, not
minted. Two more consumers reconstruct the same two decisions from prose
internally (crew_chat's queue hold, continuation's busy retry), so the
migration has to move them together.
Handler keywords (instant, no LLM)
User-typed spawn <task>, bg <task>, spawn list, spawn status are intercepted by the handler for instant execution.
Session sharing (shared AcpRuntime)
When agent.session_sharing is enabled (default on for the kiro backend) and
the parent session is kiro-backed, subagents no longer spawn a fresh kiro-cli
process each. Instead they open an additional ACP session on a shared
AcpRuntime — one process multiplexes the parent session plus all of its
subagents. Startup drops from ~3–5 s to ~200 ms and per-subagent memory from
~400 MB to near-zero.
Decision + lifecycle:
SubagentManager._should_use_session_sharing(info)gates the path: config flag on, parent session eligible (SessionManager.is_session_sharing_eligible), and no backend-specific overrides (model/allowed_tools/bare)._create_shared_session()resolves the parent'sAcpRuntimevia_get_parent_runtime()(falling back toSessionManager.get_subagent_runtime()— a companion runtime), callsruntime.create_session(), and wraps the handle inAcpSessionProvider.SubagentInfo._session_sharing/_shared_providerrecord the shared path.- On any failure the code falls back transparently to the legacy
per-process path (
get_or_create). - Cleanup (
_runfinally +_force_reap) calls_shared_provider.shutdown()to tear down only the session — it never kills the shared runtime, which other subagents may still use. The runtime is killed when the parent session ends (SessionManager.release_subagent_runtime, invoked fromreset()).
Non-kiro (alternate ACP backend) parents are never eligible and always use the
legacy AcpClient per-process path regardless of the flag.