Benchmark research workspace
August 16, 2026 ยท View on GitHub
This is LoopX's active workspace for benchmark practice and narrow runner examples. Its authority is the Long-Horizon Harness Benchmark and Research Program v0.
The workspace follows three rules:
- benchmark-native task, runner, verifier, and score semantics remain authoritative;
- current real-run practice outranks legacy LoopX benchmark abstractions;
- only generalized code and public-safe conclusions enter this directory.
Reusable product policy stays in
benchmark-toolkit. The
toolkit owns provider-neutral permission, artifact, and integrity boundaries.
This directory may contain thin examples and practice notes, but it is not
installed as a second LoopX Python package and does not grant execution or
publication authority.
Current work
deepswe/README.mdrecords the current public-safe DeepSWE method: frozen selection, matched-arm authority, native Goal proof, independent verification, invalid-run replacement, and compact evidence.native_codex_goal.pyis a compatibility import for the benchmark toolkit's installed native Goal runtime. The runnabledeepswe/run_native_codex_goal.pyexample connects that runtime to a realcodex app-server. Benchmark-family adapters should import the installed runtime and its formal isolated profile helper, then retain only their isolation, environment bridge, verifier, and scoring concerns. The profile helper renders the real Goal prompt with its installed CLI and proves that the prompt, discovered skills, and release-snapshot CLI belong to one pinned product path.
Legacy runners and dated packets are archived under
deprecate/benchmark-legacy/. They are
candidate evidence only, not the architecture for new work.