Effect read-back oracle
July 20, 2026 · View on GitHub
Generated by python -m benchmark.effect_readback.run. Judged by the MockMed
fault_server record store (independent ground truth), NOT the screen. Zero
model calls.
The gate (AGENTS.md safety asymmetry): the only dangerous error is a FALSE CONFIRM — the oracle saying the record is correct when it is not. It must be ~0. A false halt (INDETERMINATE/REFUTED when actually fine) is safe: it just halts for a human.
Per-fault verdicts
| fault | value present | same-surface | different-path |
|---|---|---|---|
ok (Clean write (control)) | yes | confirmed | confirmed |
partial (Partial save (note dropped)) | NO | confirmed | indeterminate |
optimistic (Optimistic-UI success, server rejects) | NO | confirmed | indeterminate |
session (Session expired (401, nothing persisted)) | NO | refuted | indeterminate |
timeout (Commit then client timeout) | yes | confirmed | confirmed |
duplicate (Duplicate submission (value present)) | yes | confirmed | confirmed |
stale (Lost update (value present)) | yes | confirmed | confirmed |
Aggregate
| path | false-CONFIRM | false-halt | true-CONFIRM | blind spots |
|---|---|---|---|---|
| same-surface | 0.667 (['partial', 'optimistic']) | 0.0 (none) | 1.0 | ['duplicate', 'stale'] |
| different-path | 0.0 (none) | 0.0 (none) | 1.0 | ['duplicate', 'stale'] |
Decision
- Different-path read-back false-CONFIRM rate = 0.0 → at/below the ~0 bar, so different-path read-back is enabled as the out-of-the-box default oracle (auto-wired, no connector) for GUI-only recordings.
- Same-surface read-back false-CONFIRM rate = 0.667 → ABOVE the bar (a phantom/optimistic/partial save still paints the note on the write's own surface), so same-surface read-back is wired but kept non-default: a halt-inducing consistency signal an operator opts into via
effects.kind: onscreen, never an automatic pass. - Blind spots (
duplicate,stale): the note value IS present, so the value read-back CONFIRMS; the duplicate row / lost concurrent row are STRUCTURAL faults a field read-back cannot see. The structured system-of-record oracle (record_writtencount /forbid_collateral_loss) remains the transactional guarantee where a read API exists (docs/LIMITS.md); the ultimate safety net is the identity gate + halt.