Chroma Context-1 Data Generation

April 1, 2026 · View on GitHub

Synthetic data generation pipeline from our technical report.

Generates synthetic multi-hop search tasks across multiple domains. Each domain follows an explore → verify → extend pattern to produce multi-step retrieval tasks.

Context-1 model weights are available here.

Setup

# Install dependencies
uv sync

# For all optional dependencies (reranking, patents, indexing)
uv sync --all-extras

# Configure environment
cp .env.example .env  # then fill in API keys

Required environment variables

Variable	Used by
`ANTHROPIC_API_KEY`	All domains
`SERPER_API_KEY`	Web (search + scrape)
`JINA_API_KEY`	Web (backup page fetcher)
`OPENAI_API_KEY`	Web, SEC, Email, Patents (embeddings)
`CHROMA_API_KEY`	Web, SEC, Email, Patents (indexing)
`CHROMA_DATABASE`	Web, SEC, Email, Patents (indexing)
`BASETEN_API_KEY`	SEC (reranking)

Domains

Each domain has its own pipeline command and README with full documentation:

Web — multi-hop search tasks from the open web
SEC — SEC filing tasks
Patents — patent prior-art tasks
Email (Epstein) — email search tasks

Project structure

agentic_search_data_gen/
├── core/                    # Shared base classes and utilities
│   ├── explore.py           # BaseExplorerAgent
│   ├── extend.py            # BaseExtenderAgent
│   ├── verify.py            # BaseVerifier
│   ├── distract.py          # BaseDistractorAgent
│   ├── rerank.py            # Baseten reranker client
│   ├── indexing.py          # ChromaDB indexing utilities
│   └── utils.py             # Anthropic client, token counting, quote matching
├── domains/
│   ├── web/                 # Web search tasks
│   ├── sec/                 # SEC filing tasks
│   ├── patents/             # Patent prior-art tasks
│   └── epstein/             # Email search tasks

If you find our work useful please consider citing us

@techreport{bashir2026context1,
  title        = {Chroma Context-1: Training a Self-Editing Search Agent},
  author       = {Hammad Bashir and Kelly Hong and Patrick Jiang and Zhiyi Shi},
  year         = {2026},
  month        = mar,
  institution  = {Chroma},
  url          = {https://trychroma.com/research/context-1}
}