config/

Everything that defines what the experiment asks and how it is executed lives here, version-controlled, so that any collected data point can be traced back to the exact prompt wording and settings that produced it.

Files

File	Role
prompts.json	The full prompt battery: system prompts + 15 tasks × 4 languages, with per-task metadata (paper, condition, answer space, normalization rule). The scientific core of the project.
run.config.json	Execution parameters: temperatures, repetitions, concurrency, retry policy, pilot model list, main-run scoping rules.
models.selected.json	Generated by npm run assess-models from the OpenRouter catalog, then manually reviewed. Not committed until reviewed.

Rules (read before changing anything)

1. Never edit a prompt after data collection has started for a given run. If a prompt must change, bump version in prompts.json (minor for additions, major for wording changes) and start a new run_id. The runner stores the SHA-256 of prompts.json in each run's manifest; stats scripts refuse to merge runs with different major versions.
2. Translations must be human-reviewed before the main run. Record reviewer initials and date in translation_review in prompts.json. The pilot may run with unreviewed translations (that is part of what the pilot is for).
3. models.selected.json is machine-generated but human-curated: after running npm run assess-models, open it, check family_guess assignments, add a manual "family" field where the guess is wrong (stats prefer family over family_guess), and set "included": false for models you want to exclude (with a short "exclusion_reason"). This curation step is part of the method (see paper1 §Method) and must be documented — do not silently delete entries.
4. JSON has no comments; documentation lives in description/notes fields. Keep them truthful — they are the primary documentation for anyone (human or LLM) continuing this work.

Design decisions already made (do not re-litigate without the PI)

• One-word answer constraint goes in the system prompt, not the user prompt.
• Counterpart identity is never disclosed in game conditions (no "you play a copy of yourself" variants).
• Languages: en, ru, zh, ar (high-resource, three scripts, distinct cultural spheres).
• Colors are probed in all languages and additionally mapped to canonical color codes for the cross-language analysis.
• Answers are compared within language; refusals/invalid answers are coded, never dropped.
• Study A (paper 1) uses paper: 1 tasks; Study B (paper 2) uses paper: 2 tasks. Data are collected in the same runs but analysed and published separately.