The bounded AI tool loop

July 26, 2026 · View on GitHub

One cancellable, explicitly bounded state machine drives a multi-turn tool conversation. It consumes the StreamEvent union from the protocol layer (see ai-protocol.md), presents every model-requested mutation for review, executes only what the user authorized, returns structured results so the model can correct itself, and leaves the document in a consistent, single-undo state whether the turn completes, fails, is cancelled, or hits its ceiling.

The core security property:

No byte sequence originating in model output, in a tool result, or in document content can cause a tool to execute that the user did not authorize in this turn, or cause an authorization granted for one call to apply to a different call, a different input, or a different iteration.

Implementation: src/lib/ai/loop/. Adversarial proof: injection.test.ts.

The turn is a pure reducer

reduceTurn(state, input) (turn-machine.ts) is total, pure, and the only writer of call status and phase. It performs no I/O, holds no timers, and touches no store. The runner (turn-runner.ts) performs effects by reading the state the reducer produced; it can never execute a call the reducer refused to move to running. Every security property is therefore provable with synchronous array inputs and no mocks.

Authorization state — grants and denials — lives inside TurnState, not beside it. No code path outside a user command can produce a grant.

Phases

PhaseMeaningStop control
idleno turn in flightsend enabled
requestinga provider request is open, no message_start yet; the only phase in which a retry is legalStop
streamingbetween message_start and a terminal eventStop
awaiting_approvalthe stream closed with tool_use and at least one call is pendingStop
executingat least one call is approved or runningStop
settledterminal, carrying outcome: completed | cancelled | bounded | failedsend re-enabled

The reducer accepts protocol StreamEvents and a small set of commands: submit, approveCall, approveBatch, denyCall, startCall, callSettled, advance, retry, cancel, and undoTurn. Commands the runner issues (startCall, callSettled, advance, retry) carry the clock as nowMs, so the reducer never reads Date.now(). advance is the runner's per-iteration progression signal: when an iteration's calls are all terminal, it answers them with tool_result blocks and either starts the next iteration or settles bounded.

A settled turn is terminal. The one exception is undoTurn, which flips the turn's applied calls to undone; every other late input is dropped and recorded as a post_settlement_event violation, so a provider quirk or an injected late frame cannot revive it.

Call statuses

StatusScopeMeaning
pendingper-callawaiting review; shows Approve / Deny
approvedper-call, per-batch, or per-session (static read-only policy)queued for execution under a grant
runningper-callthe tool is executing
succeededper-callexecuted and committed; contributes a tool_result
failedper-callthe tool or the commit failed; the structured failure is returned so the model may retry
undoneper-turnevery succeeded call flips together, because the turn is one undo entry
deniedper-call (user_declined) or per-turn (turn_cancelled, limit_exceeded)never executed and can never execute later this turn

denied is a deliberate seventh status. A user refusal must not render as an execution error, and must not invite the model to retry, so it is not folded into failed.

Every call the assistant message opened — including denied, unknown-tool, and never-run calls — is answered with a synthesized tool_result before the turn's messages are written. Without this the next request is rejected by both providers.

Authorization

A grant (authorization.ts) is bound to one call id, one tool name, one canonical input digest, and one iteration, and is single-use. Grants come from exactly three constructors: grantForCall, grantForBatch (an explicit id list captured when the user clicked, never a predicate), and autoGrantReadOnly (the static read-only policy, which refuses any mutate or destructive tool).

authorizeStart(state, callId) is the only predicate that lets a call reach running. It re-checks the tool, denial stickiness, the grant's presence, its digest, its iteration, and its single-use consumption. Its refusals, and the reducer's own refusals, are one closed union:

ViolationMeaning
no_grantno grant for this call
digest_mismatchthe grant's input digest no longer matches the call
foreign_iterationthe grant was issued in a different iteration
grant_already_consumedthe grant was already spent
unknown_toolthe tool is not in the turn's frozen tool set
denied_replaythe user already declined an identical (tool, input) this turn
limit_exceededa tool-call cap was reached; the call was never prepared
duplicate_call_idthe model reused a call id
post_settlement_eventan input arrived after the turn settled

Denials are sticky by (toolName, inputDigest): a re-request with identical input is auto-denied without re-prompting, while a re-request with different input is a new call that prompts normally. Destructive tools are never covered by a batch approval, and read-only auto-approval is keyed exclusively to a static local registry field, proven unreachable for mutate and destructive tools.

Bounds

Every ceiling is one field of a frozen TurnLimits (limits.ts) and is enforced in exactly one function, budgetExhaustion, which returns the first exhausted bound in a fixed priority order (iterationstool_callsretriesdeadline).

BoundDefaultReason
maxIterations8worst-case provider requests one user message may cost
maxToolCallsPerTurn32total tool calls across the turn
maxToolCallsPerIteration12reviewable in one set of cards
maxRetriesPerTurn3turn-wide retry budget on top of the transport's per-request retry
turnDeadlineMs300000wall-clock ceiling so a stalled turn ends
reserveOutputTokens4096output tokens history budgeting reserves

When a bound is hit the turn settles bounded with an informational notice — never an error banner — and keeps whatever was committed.

Transaction and undo

commitToolOutcome (transaction.ts) is the only path from a tool result to the document. It refuses in four cases and commits in one:

  1. the tool failed — nothing committed;
  2. a read tool returned a document — refused read_tool_mutated;
  3. the live document changed under the call — refused document_changed;
  4. the proposed document fails validateThreatModel — refused invalid_document, returning the validator's message so the model can fix the reference it broke;
  5. otherwise the pre-turn snapshot is pushed once per turn and the document is swapped in with restoreSnapshot (not setModel, so selection survives).

A tool returns a whole next document, so atomicity is structural: there is no half-applied intermediate, cancelling mid-call discards a value rather than undoing writes, and a failing tool commits nothing because it returned nothing.

undoTurn reverts the whole turn in one step, and turnUndoAvailability reports undoable only while the turn's snapshot is still the top of the history stack (a deep-equality check defeats the 20-entry trim aliasing an old index to a newer entry). See ADR-011.

The #64 boundary

#64 supplies tools; the loop supplies the machine. A tool is declared with defineExecutableTool and exposed to the loop as an erased RegisteredTool (tool-runtime.ts):

  • effect: "read" | "mutate" and destructive: boolean are static, local, model-independent declarations — the only inputs to auto-approval policy.
  • prepare(raw) is the only way to obtain a runnable PreparedCall; it closes over the tool's typed input, so an unvalidated input is unrepresentable at the execution boundary. PreparedCall carries a plain-text summary (rendered as text, never Markdown), a canonical inputDigest, and run(ctx).
  • run(ctx) receives the current document and the turn's abort signal and returns a ToolOutcome: an ok result optionally carrying a whole next document, or an error result returned to the model verbatim.

createToolRegistry freezes the tool list and resolves names by exact string match — no trimming, case folding, or normalization. #64 extends this registry; it does not rewire the loop. The shipped tools split into two capabilities that createAiToolRegistry (src/lib/ai/tools/tool-registry.ts) composes, read tools first:

  • Twelve graph action tools (src/lib/ai/tools/graph-action-tools.ts) adapt the existing fenced actions: each delegates run to the pure applyAction, all are mutate, and the four delete_* tools are destructive.
  • Four read tools (#203, below) are all effect: "read" and destructive: false, so they are auto-approved and never pause the turn.

Read tools (#203)

The read tools let a model query the current document and the typed component catalog (#59) without proposing an edit. All four are pure functions of ctx.document (the catalog tool ignores it): none reaches a store, a setting, an adapter, or the environment, and ToolExecutionContext carries only a ThreatModel and an AbortSignal, so key/header/path exposure is structural, not defended. read-tool-invariants.test.ts enforces the import allowlist, the output-key allowlist, and zero mutation.

ToolInputReturns
get_document_summary{}version, metadata, per-kind counts, and threat breakdowns by severity, STRIDE category, and mitigation status
get_entity{ kind, id }the full bounded projection of one entity, by stable id
search_entities{ kind, …filters, offset?, limit? }a filtered, document-ordered, paginated page of one section
search_component_catalog{ query?, category?, provider?, include_deprecated?, offset?, limit? }a paginated page of catalog entries in registry declaration order

kind is exactly the eight array-valued .thf sections (elements, data_flows, trust_boundaries, threats, layers, groups, relationships, diagrams), compiler-enforced by a mapped type over ThreatModel. Results are in document order (catalog: declaration order) — never name-sorted, because a collator's order is not reproducible across engines.

Bounds (src/lib/ai/tools/read-result.ts, all frozen constants):

ConstantValueGuards
READ_RESULT_MAX_BYTES8192UTF-8 bytes of the serialized payload (markers excluded)
DEFAULT_PAGE_LIMIT20page size when limit is omitted
MAX_PAGE_LIMIT50a larger limit is a schema rejection, not a silent clamp
SCALAR_MAX_CODE_POINTS200ids, names, titles, and every short scalar
TEXT_MAX_CODE_POINTS400description / mitigation.description, only in get_entity
LIST_MAX_ITEMS10technologies, tags, derived id-lists, catalog aliases/keywords/variants
ECHO_MAX_CODE_POINTS80the longest untrusted value echoed in a failure message

Truncation is never silent: a cut scalar/text field ends in and its entity carries "truncated": true; a capped list reports "<field>_total".

Failures are a closed ReadToolFailure union rendered by describeReadFailure, returned as { status: "error", result } so the model can correct itself — never an empty success that reads as "there are none":

  • unknown_idget_entity found no entity with that id in that section (names the section's entry count and points at search_entities);
  • invalid_filter — a filter was set that does not apply to kind (names the applicable filters);
  • offset_out_of_rangeoffset >= total while total > 0.

A genuine zero-match query (total: 0, offset: 0) stays a success with stopped_by: "end"; more matches than one page holds is reported by total, next_offset, and stopped_by, not as a failure.

Envelope. Every successful result is canonicalJson (key-sorted, no whitespace, so byte-identical inputs give byte-identical results) wrapped in the #177 untrusted-data markers:

<<<UNTRUSTED_DOCUMENT_DATA>>>
{"page":{…},"results":[…],"tool":"search_entities"}
<<<END_UNTRUSTED_DOCUMENT_DATA>>>

Every document- and catalog-derived string is sanitized (control/bidi code points stripped, capped in code points) and escaped (src/lib/ai/untrusted-text.ts), so no field can reproduce, close, or forge a marker. Only authored markers, keys, and enum values are emitted literally, so counting raw markers in a result is exact. Failure messages are authored text and are not fenced; only the echoed value inside them is sanitized and escaped.