Rendering snapshot tests
August 6, 2026 · View on GitHub
This directory holds the byte-exact PNG baselines that gate the rendering
pipeline. The harness runs the existing roller binary in a headless snapshot
mode across the seven intro replays in fatdata/ and a small set of
deterministic named frontend scenes, captures the indexed framebuffer to PNGs,
writes them straight here, and uses git diff to detect any drift.
Why this matters: any change that alters a rasterizer, a palette path, or replay-driven game logic shows up as a tracked-file diff in your PR. The reviewer sees the side-by-side image diff in GitHub. No custom diff viewer required.
For the why behind the design (byte-exact comparison, indexed PNG storage,
single-host pinning, Git LFS scope, etc.) see
docs/adr/0001-rendering-snapshot-tests.md.
Prerequisites
# One-time, per checkout:
git lfs install
git lfs pull # if you cloned without LFS first
The baselines under baselines/ are stored via Git LFS (scoped via
.gitattributes). Without git lfs your working tree contains ~134-byte
pointer files instead of real PNGs, and the harness will fail loudly.
You also need a populated fatdata/ directory at the project root (or pass
-Dassets-path=...); the snapshot binary loads the intro replay files from
there, same as a normal zig build run.
Running the harness
zig build test-snapshots
On the canonical host (Apple Silicon macOS at the time of writing) this:
- Builds the
rollerbinary if needed. - Runs
roller --snapshot introN.gss --frames ... --out tests/snapshots/baselines/once per intro replay, serially, then runsroller --snapshot-scene NAME --frames ... --out tests/snapshots/baselines/once per named scene. - Runs
git diff --exit-code --stat -- tests/snapshots/baselines/.
If the captures match HEAD, exit 0. If anything diverged, the build fails and
git status shows you what changed.
I changed rendering code and want to bless the new pixels
- Run
zig build test-snapshots. Expect it to fail. - Inspect the working-tree diff:
For visual review, open the PNGs in your image viewer of choice or push the changes to a draft PR — GitHub renders LFS-backed PNG diffs as a side-by-side image viewer.git diff --stat tests/snapshots/baselines/ - If the changes look intentional and correct:
The commit is the explicit blessing — there's no separate flag.git add tests/snapshots/baselines/ git commit -m "..." - If the changes look wrong, revert:
then debug the rendering change that caused the divergence.git checkout -- tests/snapshots/baselines/
I'm on Linux/Windows and want to sanity-check the capture path
The baselines are pinned to one host architecture; running on a divergent host would write divergent pixels into the LFS-tracked files. To sanity-check captures without mutating the baselines:
zig build test-snapshots -Dscratch
# Captures land in zig-out/snapshot-scratch/, the working tree stays clean.
To eyeball your platform's pixels against the canonical baselines:
diff -rq tests/snapshots/baselines zig-out/snapshot-scratch
# Or open the corresponding PNGs side by side in your viewer.
If you get pixel-exact equality on a non-canonical host, your platform is rendering identically to the pinned host (rare and worth noting). Otherwise differences are expected; treat the canonical baselines as authoritative for now. Cross-platform pixel parity is an open question tracked separately in a future ADR.
What the build options do
| Flag | Effect |
|---|---|
| (none) | Run captures into tests/snapshots/baselines/, then git diff --exit-code against HEAD. Fail on divergence. |
-Dscratch | Run captures into zig-out/snapshot-scratch/ and skip the diff check. Working tree stays clean. |
-Dassets-path=PATH | Use PATH instead of ./fatdata as the data root. |
The list of replays and which frames are captured per replay live in
build.zig's snapshot_replays table.
Named scene snapshots live in build.zig's snapshot_scenes table. Each entry
declares the scene name and capture frame(s). Animated frontend and winner
scenes usually capture a later settled present such as frame 30, while
single-present static result screens can capture frame 1. Scene PNG names use
<scene-name>_<present-index>.png, for example menu-main_30.png or
race-result_1.png.
File layout
tests/snapshots/
├── README.md
└── baselines/ ← indexed PNG baselines, LFS-tracked
Replay baseline filenames use <replay-stem>_<frame-index>.png; named scene
baseline filenames use <scene-name>_<present-index>.png. Treat build.zig's
snapshot_replays and snapshot_scenes tables as the source of truth for the
current generated file set.
Each PNG is a 640x400 8-bit indexed image with the active 256-entry palette in the PLTE chunk. Any image viewer can open them.
Troubleshooting
zig build test-snapshots fails immediately with "git diff" output showing
every PNG modified. Your captures match what the snapshot binary produces
today, but those bytes don't match what's in HEAD. Either:
- You're running on a non-canonical host (use
-Dscratchinstead). - You changed something on the rendering path that genuinely shifted pixels (intended or not — review the diff).
- Your LFS smudge filter didn't run; the working-tree files are LFS pointers,
not real PNGs. Try
git lfs pulland retry.
zig build test-snapshots complains it can't find fatdata/. Either
symlink your fatdata into the project root, or pass
-Dassets-path=/path/to/fatdata.
Captures hang or never finish. A given replay shouldn't take more than ~30
seconds on a modern machine. If a run sits indefinitely, suspect determinism:
check that no recent change touched the snapshot init path (especially anything
that consumes time(NULL) or SDL_GetTicks for branching).
intro7 only captures three frames where the others capture four.
Intentional. Frame 900 of intro7 produces non-deterministic pixels across runs
even with srand(0) (suspected: an unseeded RNG consumer or wall-clock read in
the deep replay path). Excluded from the baseline list pending investigation.
You see *.png filter=lfs diff=lfs merge=lfs -text show up in
.gitattributes for the wrong directory. Some editor / pre-commit hooks
auto-track binaries via LFS the first time you add one. Revert that — the
baselines are the only PNGs in the repo that should be LFS-backed (the
project's icons and screenshots in images/ stay in plain git).
Where the policy is documented
docs/adr/0001-rendering-snapshot-tests.md— comparison policy, format choice, single-host pinning, LFS scope, scratch flow.docs/plans/2026-05-07-rendering-snapshot-tests-design.md— original design document.
Where the implementation is
PROJECTS/ROLLER/png_writer.{c,h}— indexed-PNG writer (usesIMG_SavePNGfrom SDL3_image, which delegates to libpng)PROJECTS/ROLLER/snapshot.{c,h}— snapshot-mode state, CLI parsing, per-tick zero-screen hook, present hook, manual tick driver, fixed settings applicationbuild.zig—test-snapshotsstep,snapshot_replaysandsnapshot_scenestables,-Dscratchflag