TinyAgents System Specification

July 23, 2026 ยท View on GitHub

TinyAgents is a Rust-native LLM application framework inspired by LangChain, LangGraph, and CodeAct-style recursive language-model runtimes. The system is organized around five modules:

  1. the harness
  2. the graph
  3. the registry
  4. the expressive language
  5. the REPL language

The goal is to make agent systems easy to define, inspect, run, test, and eventually serialize without hiding the Rust types that make production systems reliable.

Reference Positioning

TinyAgents should synthesize the reference systems rather than clone any one of them:

  • LangGraph contributes the durable execution model: explicit state graphs, virtual START and END, Pregel-style supersteps, reducers/channels, commands, Send fanout, checkpointing, interrupts, subgraphs, streaming, and time travel.
  • LangChain contributes the harness model: provider-neutral models, tools, middleware, runtime context, memory, retrieval, structured output, tracing, usage, cost, and conformance tests for integrations.
  • rust-langgraph shows the Rust-facing precedent for a stateful graph runtime with nodes, conditional edges, checkpoints, streaming, optional model adapters, and ReAct/tool helpers. TinyAgents should go deeper on typed state, harness composition, registries, and language-backed graph definitions.
  • OpenHuman PR #4261 contributes the closest product-shaped precedent: a harness-decoupled graph engine, persistent checkpoints, HITL, graph observability, blueprints, JSON-RPC run control, and a behavior-preserving cutover from an implicit turn loop to an explicit phase machine.
  • RLM contributes the REPL/code-act model: context and prompts as runtime values, recursive sub-model or sub-agent calls as functions, persistent session variables, trajectory logging, and sandbox choices.

The target architecture is therefore layered: the harness owns model/tool execution and policies, the graph owns deterministic state transition and durability, the registry owns named capabilities, .rag owns serializable graph blueprints, and .ragsh owns capability-bound interactive orchestration. No layer should bypass another layer's safety, policy, observability, or test contracts.

Detailed Module Docs

Docs should follow the module layout. Do not place standalone specification files directly in docs/ or docs/modules/; each high-level topic should have its own directory with a README.md entrypoint and any supporting files beside it.

Design Goals

  • Make simple agent workflows concise.
  • Make complex workflows explicit, inspectable, and testable.
  • Treat graph execution as a first-class runtime, not an incidental callback chain.
  • Keep model providers, tools, memory, and tracing behind stable traits.
  • Support both Rust builder APIs and a compact expressive language for workflow definitions.
  • Support a capability-bound REPL language for interactive graph and harness orchestration.
  • Allow agents to author, inspect, compile, and run graph blueprints through the same registry-bound compiler path used by human-authored .rag files.
  • Allow parent orchestrators and humans to steer orchestrator agents and sub-agents through typed, policy-checked, observable commands.
  • Prefer deterministic state transitions around inherently nondeterministic LLM calls.
  • Keep every generated or hand-authored graph explainable as topology, capabilities, policies, state channels, checkpoints, and events.

Module 1: Harness

The harness is the provider-neutral runtime for model calls, tools, middleware, structured output, streaming, usage/cost, retry/limits, cache, memory/embeddings, sub-agents, and steering. See harness-spec.md for the full specification (core types, model/tool/message abstractions, agent loop, middleware, memory, structured output, observability, and testability), and docs/modules/harness/README.md for the per-topic implementation docs.

Per-tool deadlines are opt-in at the harness boundary through AgentHarness::with_tool_timeout_settings. A tool's ToolTimeout policy is resolved from the final post-middleware call: Inherit uses the shared dynamic default, Millis is clamped and padded with configured grace, and Unbounded has no per-tool deadline. Expiry is a recoverable tool result returned to the model; only the enclosing run wall-clock deadline aborts the run.

Module 2: Graph

The graph is the durable, typed state-graph runtime: START/END, nodes, reducers/channels, routing, supersteps, checkpointing, interrupts, streaming, subgraphs, and execution guarantees. See graph-spec.md for the full specification, and docs/modules/graph/README.md for the per-topic implementation docs.

Module 3: Expressive Language

The .rag expressive language is a declarative, side-effect-free blueprint format that compiles through lexer -> parser -> compiler into the same graph/harness runtime types as hand-written Rust. See expressive-language-spec.md for the goals, grammar sketch, and compilation pipeline, and docs/modules/expressive-language/README.md for implementation status.

Package Layout

The crate is a single library at the repository root (Cargo.toml), with src/lib.rs re-exporting the public surface and src/error.rs holding the crate-wide error type. Each of the five surfaces lives in its own module directory:

src/
  error.rs
  lib.rs
  graph/       # durable typed state graphs (checkpoint, interrupt, streaming, ...)
  harness/     # provider-neutral model calls, tools, middleware, streaming, ...
  language/    # the declarative `.rag` blueprint format (lexer/parser/compiler)
  registry/    # the named capability catalog (models, tools, agents, stores, ...)
  repl/        # the imperative `.ragsh` session runtime

Provider implementations (OpenAI and the OpenAI-compatible endpoints for Anthropic, Ollama, DeepSeek, Groq, xAI, OpenRouter, Together, and Mistral) live inside src/harness/providers/ and are compiled in unconditionally. Two Cargo features gate optional dependencies: sqlite (embedded SQLite checkpointer) and repl (embedded Rhai engine for .ragsh sessions).

Milestones

All five milestones below have shipped as of v1.5.0.

Milestone 1: Core Runtime (shipped)

Chat message primitives, the model and tool traits, the state graph with direct and conditional edges, and the initial test/example suite.

Milestone 2: Harness (shipped)

The AgentHarness type, model and tool registries, run context, callback events, run status store, durable event journal, cache-backed observability projections, and mock model/tool testkit utilities.

Milestone 3: Expressive Language (shipped)

The .rag AST, lexer, parser, compiler into the graph runtime, parse/ validation diagnostics with source spans, and example .rag workflow files (see examples/rag_blueprint.rs, examples/openai_self_blueprint.rs).

Milestone 4: Provider Integrations (shipped)

OpenAI and OpenAI-compatible provider adapters (Anthropic, Ollama, DeepSeek, Groq, xAI, OpenRouter, Together, Mistral), plus the offline deterministic mock provider.

Milestone 5: Production Runtime Features (shipped)

Streaming events, checkpointing and resume support, the graph run status store, event journal with listener replay, graph export, and an embedded Langfuse tracing integration (LangfuseClient, GraphLangfuseExporter).

Open Questions

Historical decisions that have since been settled, kept for context:

  • The expressive language file extension is .rag (interactive/imperative orchestration uses the separate .ragsh extension).
  • State schemas remain Rust-owned; .rag binds to them by name through the registry rather than declaring schemas itself.
  • Provider crates live in this crate as always-compiled modules behind src/harness/providers/, not separate crates or feature flags.
  • Memory and embeddings are async, matching the rest of the harness surface.

Remaining open question:

  • Should graph nodes support typed route enums as a stronger alternative to string-keyed conditional routing before further serialization work lands?