Meta-Harness

Meta-Harness is a framework for automated search over task-specific model harnesses: the code around a fixed base model that decides what to store, retrieve, and show while the model works. This repo contains the framework and two reference experiments from the paper. The paper is Meta-Harness: End-to-End Optimization of Model Harnesses.

If you build with Meta-Harness, please send us a link. We may add your repository, artifact, post, or paper to this README.

Contents

• The reusable framework and onboarding flow for applying Meta-Harness to a new domain.
• Two paper reference experiments under reference_examples/:
  • reference_examples/text_classification/: memory-system search for text classification.
  • reference_examples/terminal_bench_2/: scaffold evolution for Terminal-Bench 2.0.
• The optimized Terminal-Bench 2 harness from the paper lives in the separate artifact repo: stanford-iris-lab/meta-harness-tbench2-artifact.

Community Projects

• Don't Train the Model, Evolve the Harness: Applies Meta-Harness to Harvey's Legal Agent Benchmark.
• dkhanal/meta-harness: Independent implementation with a Symptom2Disease demo.
• SuperagenticAI/metaharness: Python library and CLI for harness optimization with Codex.
• Harness Forge: Reimplements Meta-Harness as a native Claude Code skill.
• meta-harness-on-islo: Applies the optimization loop to Islo sandboxes.

Quick Start

Text classification:

cd reference_examples/text_classification
uv sync
uv run python meta_harness.py --iterations 1

Terminal-Bench 2 smoke task:

cd reference_examples/terminal_bench_2
uv sync
uv run bash scripts/run_eval.sh agents.baseline_kira:AgentHarness full 1 1 -i extract-elf

Use the subdir READMEs for setup details, expected runtime, and additional commands.

Applying Meta-Harness To A New Domain

Start by pointing your coding assistant to ONBOARDING.md and having a conversation with it. This should produce a domain_spec.md file with concrete details on how to proceed with implementing Meta-Harness for your domain.

The shipped examples currently assume Claude Code as the proposer agent. To use a different proposer agent, adapt the example claude_wrapper.py scripts in reference_examples/text_classification/claude_wrapper.py or reference_examples/terminal_bench_2/claude_wrapper.py. The main requirement is a wrapper that cleanly logs proposer interactions.

Release Note

This is a cleaned up version of the code we used for the paper. It has not been tested beyond verifying that it runs. Please let us know if anything goes wrong.

Citation

If this repository is useful for your research, please cite the paper:

@misc{lee2026metaharnessendtoendoptimizationmodel,
      title={Meta-Harness: End-to-End Optimization of Model Harnesses},
      author={Yoonho Lee and Roshen Nair and Qizheng Zhang and Kangwook Lee and Omar Khattab and Chelsea Finn},
      year={2026},
      eprint={2603.28052},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2603.28052},
}