amplifier-module-tool-rlm

January 15, 2026 · View on GitHub

Recursive Language Model (RLM) tool for Amplifier - enables processing of arbitrarily long contexts through recursive decomposition.

The Problem: Context Window Limits

Modern LLMs have context window limits:

ModelMax ContextPractical Limit
Claude Sonnet~200K tokens~150K usable
GPT-4~128K tokens~100K usable
Gemini~1M tokens~800K usable

What happens when your document exceeds these limits?

  • Truncation loses critical information
  • Summarization loses detail
  • RAG may miss relevant chunks

RLM solves this by letting the LLM recursively process chunks while maintaining the ability to synthesize information across the entire document.

How RLM Works

RLM treats the input as an external environment that the LLM can programmatically explore:

┌─────────────────────────────────────────────────────────────┐
│  Your 5M Token Document                                      │
│  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐            │
│  │ Chunk 1 │ │ Chunk 2 │ │ Chunk 3 │ │ Chunk N │ ...        │
│  └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘            │
│       │           │           │           │                  │
│       ▼           ▼           ▼           ▼                  │
│  ┌─────────────────────────────────────────────────────┐    │
│  │  Python REPL (sandboxed Docker)                     │    │
│  │  - Search with regex                                │    │
│  │  - Extract relevant sections                        │    │
│  │  - Make recursive LLM calls on chunks               │    │
│  │  - Combine facts to compute answers                 │    │
│  └─────────────────────────────────────────────────────┘    │
│                           │                                  │
│                           ▼                                  │
│                    Final Answer                              │
└─────────────────────────────────────────────────────────────┘

Key Mechanism

  1. Context as Variable: Content loaded into sandboxed Python REPL as context
  2. Programmatic Exploration: LLM writes Python code to search, filter, chunk
  3. Recursive Sub-calls: LLM can call itself on smaller pieces
  4. Fact Synthesis: Combine information from multiple chunks into final answer

When to Use RLM

ScenarioUse RLM?Why
Document > 200K tokens✅ YesExceeds model context window
Multi-hop reasoning✅ YesNeed to combine facts from different sections
Analytical queries✅ Yes"Calculate X from data scattered across doc"
Simple lookup in small file❌ NoRegular read is faster
Real-time streaming❌ NoRLM processes in batches
Vague exploratory queries❌ NoWorks best with specific questions

Ideal Use Cases

  • Financial Analysis: "What's the per-engineer investment?" (requires finding budget AND headcount)
  • Legal Document Review: "Find all clauses related to liability across this 500-page contract"
  • Codebase Understanding: "How does the authentication flow work?" (across multiple files)
  • Research Synthesis: "What are the key findings across these 50 papers?"

Validated Results

We tested RLM with multi-hop analytical queries requiring fact synthesis:

Test Setup

  • Query: "What is the per-engineer quarterly investment in Project Titan?"
  • Answer requires: Finding budget ($12.3M) AND team size (191) from different sections
  • Expected: $64,400 per engineer per quarter

Results

File SizeTokens (approx)RLM ResultStatus
256KB~64K$64,400 ✅Correct
1MB~250K$64,400 ✅Correct
5MB~1.25M$64,400 ✅Correct

Key Finding: RLM correctly performs multi-hop reasoning even when the answer requires combining information from different document sections.

Installation

Via Bundle Configuration

Add to your bundle's behaviors/ YAML:

tools:
  - module: tool-rlm
    source: git+https://github.com/michaeljabbour/amplifier-module-tool-rlm@main

For Development

git clone https://github.com/michaeljabbour/amplifier-module-tool-rlm
cd amplifier-module-tool-rlm
uv sync

Requirements

  • Docker: Required for sandboxed Python REPL execution
  • Amplifier: This is a tool module for the Amplifier AI agent framework
  • Python 3.11+
  • LLM Provider: Anthropic, OpenAI, or other configured provider

Usage

Basic Usage

In an Amplifier session:

Use the rlm tool with file_path="/path/to/large/document.txt" and query="What is the total revenue for Q3?"

With Inline Content

Use the rlm tool with content="<paste your content here>" and query="Summarize the key findings"

Parameters

ParameterRequiredDefaultDescription
queryYes-Your question or task (be specific!)
file_pathOne of-Path to file to process
contentthese-Inline content to process
content_typeNo"text"Hint: "code", "document", or "data"

Output

{
    "answer": "The per-engineer quarterly investment is \$64,400",
    "trajectory_steps": 15,    # REPL execution steps
    "llm_calls": 3,            # Total LLM invocations
    "tokens_in": 125000,       # Input tokens consumed
    "tokens_out": 2500,        # Output tokens generated
    "trajectory": [...]        # Full execution history for debugging
}

Best Practices

Write Specific Queries

# Good - specific and answerable
"What is the per-engineer quarterly investment in Project Titan?"
"Find all references to 'liability' in sections 4-7"
"What are the three main risk factors mentioned?"

# Bad - too vague
"Tell me about this document"
"What's interesting here?"

Provide Context Type Hints

# For code files
content_type="code"

# For business documents
content_type="document"

# For structured data (JSON, CSV)
content_type="data"

Configuration

tools:
  - module: tool-rlm
    config:
      max_recursion_depth: 5      # Max nesting of LLM sub-calls
      max_llm_calls: 100          # Budget for total LLM calls
      max_trajectory_steps: 50    # Max REPL execution steps
      exec_timeout: 60            # Timeout per execution (seconds)
      default_provider: anthropic # Provider for sub-calls

Testing

# Run unit tests (51 tests)
uv run pytest tests/ -v --ignore=tests/test_integration.py

# Run with coverage
uv run pytest tests/ --cov=amplifier_module_tool_rlm

# Type checking
uv run pyright

# Lint
uv run ruff check .

Architecture

amplifier-module-tool-rlm/
├── amplifier_module_tool_rlm/
│   └── __init__.py          # RLMTool, REPLManager, mount()
├── tests/
│   ├── test_tool.py         # Tool interface tests
│   ├── test_repl_manager.py # REPL execution tests
│   ├── test_config.py       # Configuration tests
│   └── test_models.py       # Data model tests
├── pyproject.toml           # Entry point: tool-rlm
├── README.md
└── LICENSE

References

License

MIT License - See LICENSE file for details.

Troubleshooting

Docker not running

Error: Docker not available or not running

Solution: Ensure Docker daemon is running:

docker ps  # Should return without error

Provider not configured

Error: Provider 'anthropic' not found

Solution: Ensure your bundle includes a provider module:

providers:
  - module: provider-anthropic
    source: git+https://github.com/microsoft/amplifier-module-provider-anthropic@main

Container timeout

Error: Code execution timed out

Solution: Increase the execution timeout in config:

tools:
  - module: tool-rlm
    config:
      exec_timeout: 120  # Increase from default 60s

Max LLM calls exceeded

Warning: Max LLM calls (100) reached

Solution: For very large documents, increase the limit:

tools:
  - module: tool-rlm
    config:
      max_llm_calls: 200  # Increase budget

Query returns no results

If RLM returns empty or incorrect results:

  1. Be more specific - Vague queries work poorly
  2. Check document format - Ensure content is readable text
  3. Try smaller chunks - Very dense documents may need multiple passes

Debug mode

Enable debug logging for troubleshooting:

export RLM_DEBUG_LOG=/tmp/rlm_debug.log
amplifier
# After session, check: cat /tmp/rlm_debug.log