Research and experiments

Jev Atari Lab has two connected goals: work toward mastering every game in the pinned Atari scope, and discover effective teacher-driven optimizers for structured question policies. Atari provides environments in which proposed optimization patterns can be measured, challenged and reproduced.

As directed by the project owner on 2026-09-18, this directory is the canonical home for this project's research ideas, hypotheses, methods, decisions and paper planning. Readers do not need access to a private research repository. Historical notes elsewhere remain provenance; new project research belongs here. Original machine-readable evidence and immutable LFS archives remain in experiments/; code and tests retain their own directories.

Research map

Document	Purpose	Status
Research direction	Two goals, hypotheses, current evidence and limits	Adopted project direction
Teacher optimizer	What is optimized, optimizer components, round records and pattern discovery	Design; first implemented teacher study incomplete
Evaluation framework	Edit, behavior, outcome and cost measures; controls and uncertainty	Proposed framework; some underlying logging exists
Research roadmap and paper plan	Staged studies and evidence required for paper claims	Planned studies; no new live run authorized by this document
Related work	Primary sources and boundaries of novelty claims	Targeted review; methods not reproduced
Research provenance	Origins, ownership transition and implementation decisions	Historical record

Experimental evidence

Game-specific research starts at Pong, with its own evaluation profile and teacher log. The approved longitudinal teacher study has a frozen protocol and an explicit parser continuation record. Its incomplete results lead to a proposed strategy-versus-execution study. For new games, use the shared evaluation-profile and teacher-round templates. Outcomes and mastery criteria are game-specific; provenance, cost and evidence requirements are shared.

Experiment	What it establishes	What remains open
First teacher study	Audited A/B proposals, behavioral probes and rejection; technical stop in round two	Multi-round learning and final generalization
Value pilot	A question revision improved Brier score but regressed MAE and was rejected	Improved control or TD learning
Score/Choice comparison	Offline representation comparison	General representation superiority
Direct-policy pilot	Real Jev-controlled Pong trajectories	Complete-match performance
Fixed-frame controls	Existing Jev and Python rules differ in actions and short-run return	Experience-driven learning
No-FIRE revision	Higher confidence/adherence did not improve aggregate return	Reliable causal effect of the wording
Match calibration	Nine local training episodes and duration/censoring evidence	A reliably stronger interception controller
Long Jev trial	20,000-frame fixed-policy run ending at 7:18, unfinished	Native completion, wins and population performance

Experiment protocols remain frozen with their reports. Later research directions must not retroactively change budgets, gates, data splits or interpretations of those runs. A video is evidence of one execution, not a learning curve.

Reproduction and environment contracts

• Pong benchmark interpretation and game coverage.
• Observation contract and Pong protocol.
• Replay guide, validation history, and artifact journal.
• Frozen protocols: fixed-frame controls, no-FIRE follow-up, and native-match feasibility.

Use explicit evidence labels in new research: observed for our executed runs, source-reported for external performance claims, proposed for unrun methods, and unconfirmed for unresolved conclusions. Link claims to a report, source, program version and experiment revision as appropriate. Preserve negative results.

The question execution diagnostics protocol freezes the next offline adherence analysis and separate literal Python strategy trials. No new model calls.

Question diagnostics results: low interception-rule adherence, 16 separate literal-control episodes and no new model calls.