Model invocation contract

August 17, 2026 · View on GitHub

中文

Model invocation contract

All rollout backbones use temperature 0, planner prompt v4, verifier v3, instruction v1, a six-step planning horizon, and a 100-environment-step cap. max_tokens remains model-specific.

ModelHistorical backendmax_tokensReasoning budgetJSON request
GPT-5.5Azure OpenAI direct4096512no explicit argument
Gemini-3.1filesystem queue1200512yes
Gemini-2.5filesystem queue1200512yes
Claude-4.8filesystem queue12000yes
Gemma-4-31BvLLM1200unsetyes
Qwen3.6-27BvLLM4096unsetyes
Qwen3.6-35B-A3BvLLM4096unsetyes
Qwen3.5-27BvLLM4096unsetyes
InternVL3.5-38BvLLM2048unsetyes

The machine-readable provenance is configs/paper_table_model_invocations.json. It intentionally omits private endpoints and credentials.

OpenAI-compatible adapter

{
  "backend": "openai_compatible",
  "model": "served-model-name",
  "base_url": "http://127.0.0.1:8100/v1",
  "api_key_env": "OPENAI_API_KEY",
  "max_tokens": 4096,
  "temperature": 0.0,
  "response_format": {"type": "json_object"},
  "extra_body": {}
}

Local OpenAI-compatible endpoints need no credential. For a remote endpoint, export the named environment variable; do not store secrets in this file.

Filesystem queue adapter

The rollout atomically moves each request directory into <queue_dir>/pending/. A worker writes its response to <queue_dir>/done/<call_id>/response.json:

{"content": "{...model JSON response...}"}

The request and response directories are removed after the response is read. They are transport state and are not part of the saved episode.

For exact cost metrics, the response may additionally contain a structured usage object or the worker's Usage(...) string. The adapter accepts OpenAI-style totals as well as Gemini's separate prompt, visible candidate, and reasoning-token counts. Hidden reasoning is never folded into the paper's visible-output column. If no usage is returned, metrics.json labels the character-count fallback as an estimate.