Testing

Tests should prove product behavior, not implementation trivia. For analyzer and output work, use BDD-shaped names and fixtures that make the behavior question plain:

given_changed_boundary_when_equal_value_is_missing_then_reports_weak_exposure

Behavior changes should have a three-way match:

spec -> test -> code

See Spec-test-code traceability for the expected mapping. See Test taxonomy for required proof levels by change type.

Run everything:

cargo xtask shape
cargo xtask fix-pr
cargo xtask pr-summary
cargo xtask precommit
cargo xtask check-pr
cargo xtask fixtures
cargo xtask goldens check
cargo xtask golden-drift
cargo fmt --check
cargo check --workspace --all-targets
cargo test --workspace
cargo clippy --workspace --all-targets -- -D warnings
cargo doc --workspace --no-deps
cargo xtask ci-fast
cargo xtask ci-full
cargo xtask check-static-language
cargo xtask check-no-panic-family
cargo xtask check-file-policy
cargo xtask check-covered-by
cargo xtask check-executable-files
cargo xtask check-workflows
cargo xtask check-spec-format
cargo xtask check-spec-numbering
cargo xtask check-fixture-contracts
cargo xtask check-traceability
cargo xtask check-capabilities
cargo xtask check-workspace-shape
cargo xtask check-architecture
cargo xtask check-public-api
cargo xtask check-output-contracts
cargo xtask check-doc-index
cargo xtask check-pr-shape
cargo xtask check-generated
cargo xtask check-badge-diff-policy
cargo xtask check-generated-clean
cargo xtask check-dependencies
cargo xtask check-process-policy
cargo xtask check-network-policy

Package check:

cargo package -p ripr --list
cargo publish -p ripr --dry-run

The current test suite covers:

• unified diff parsing
• Rust test/assertion extraction
• harness trial-subject evidence parity (helper callbacks, method oracles, dormant templates, shadow fail-closed boundaries)
• JSON escaping
• simple end-to-end diff analysis
• CLI smoke behavior
• property-based parser invariants (diff parser)

Property-Based Tests

The diff parser (crates/ripr/src/analysis/diff/parse.rs) has property-based tests using proptest (a dev-dependency). These generalize the hand-rolled LCG fuzz tests with automatic shrinking and broader input coverage.

Properties checked:

1. Totality: parse_unified_diff never panics on arbitrary string input.
2. Structural invariants: no empty paths, no newlines in line text, for generated diff-like text.
3. Line number validity: every added line's new_side_line is >= 1.

Convention: property tests use Result<(), String> bodies, no unwrap/expect, per the workspace lint posture. See docs/TEST_TAXONOMY.md for the Property and Fuzz test type definitions.

Harness Trial-Subject Evidence Parity Scan

The repository-governed test-harness registry (crates/ripr/src/analysis/facts/harness_registry.rs, RIPR-SPEC-0173) derives evidence for registered harness trial subjects (for example libtest_mimic Trial::test registrations). Its parity scan keeps a trial subject's observation evidence aligned with what an ordinary #[test] carrying the same code would show, and every widening is fail-closed. The discriminating unit tests live in crates/ripr/src/analysis/facts/harness_registry/tests.rs:

• helper-callback bodies contribute one level of parsed evidence (calls, oracles, literals) with real line attribution, only when the callback name provably binds the file-level fn (shadowed_callback_... and ..._const_or_static_shadow_... pins block let/parameter/const/ static/import/nested-module shadows);
• method-position .unwrap()/.expect() calls register smoke oracles with receiver-ful text across keyword, indexed, cast, operator, and negation receiver forms (trial_method_oracle_receivers_...);
• assertion macros in every delimiter Rust permits ((...), [...], {...}) keep their complete invocation text, classification, and observed tokens (trial_alternative_delimiters_...);
• dormant macro_rules! templates — in any delimiter, in a closure or a helper body — contribute no oracle, call, or literal evidence while live surrounding evidence still admits (trial_dormant_... and given_dormant_template_in_helper_... pins).

When you touch the trial evidence scan, work fixture-first: verify the new discriminating test fails on the pre-fix head, then check the ordinary-parser parity pins and the fail-closed negative controls together, so a widened admit cannot land without its boundary.

Error-Handling Bar

The target rule is:

No panic, unwrap, expect, todo, or unimplemented in production or tests.

New tests should return Result when setup can fail and should use explicit assertions. Existing panic-family usage is tracked engineering debt and should be paid down in scoped PRs rather than copied into new tests.

VS Code Extension Tests

The VS Code extension smoke tests run inside a real VS Code instance through @vscode/test-electron:

cd editors/vscode
npm ci
npm run test:e2e

The test suite:

• opens a fixture Rust workspace (test-fixtures/workspace/Cargo.toml)
• activates the extension
• asserts commands are registered (ripr.restartServer, ripr.selectWorkspaceRoot, ripr.showOutput, ripr.copyContext, ripr.copySuggestedAssertion, ripr.copyTargetedTestBrief, ripr.copyAgentPacketCommand, ripr.copyAgentBriefCommand, ripr.copyAfterSnapshotCommand, ripr.copyAgentVerifyCommand, ripr.copyAgentReceiptCommand, ripr.openRelatedTest, ripr.openSettings)
• verifies the defaults-first editor check mode is draft
• pins the contributes.menus editor/context repair-loop entries (ripr groups, resourceLangId gating) and the two default contributes.keybindings, keeps the payload-bound targeted-test and agent-loop commands code-action-only, and fails closed when a menu or keybinding references an unregistered command
• verifies copyContext completes without crash when no editor is active
• verifies copyContext accepts a structured target with finding_id and probe_id without crashing
• verifies copyContext asks LSP ripr.collectContext first for seam_id targets and falls back to the CLI when LSP returns no packet or errors
• verifies suggested-assertion and targeted-test-brief commands copy valid payloads and ignore malformed arguments without throwing
• verifies agent-loop command copying writes command text and the contributed command handlers ignore malformed arguments without throwing
• verifies LSP agent-loop command payloads stay workspace-relative across platform-shaped roots and fail closed for stale seam diagnostics
• verifies the live real-server boundary-gap path publishes a seam diagnostic, renders hover evidence, exposes seam actions, copies seam packet and verify command payloads, and opens the best related test
• verifies explicit editor status states for disabled config, missing workspace, unavailable server, queued/running/complete/no-seam/failed refreshes, and stale dirty Rust buffers that stay stale until save or close
• verifies an existing workspace-matched first-useful-action.json is projected through status bar and ripr: Show Status without overriding stale evidence or accepting reports for another workspace
• verifies openRelatedTest opens URI/line targets and ignores malformed arguments without throwing
• verifies restartServer is callable even when server resolution fails

CI runs the suite headless with xvfb-run -a npm run test:e2e. The test runner stores downloaded VS Code archives under target/ripr/vscode-test-cache so generated editor-host state stays with the rest of the repo-local build output.

Golden Output

When changing user-visible output, update or add golden coverage for:

• human output
• JSON output
• context packets
• LSP diagnostic shape, when applicable

Golden updates must preserve the static language boundary: draft static output does not use mutation-runtime terms such as killed or survived.

CI Test Results

CI uploads Rust test results to Codecov Test Analytics from JUnit XML generated by cargo nextest:

cargo nextest run --workspace --all-features --profile ci

Doc tests remain a separate Cargo test step:

cargo test --workspace --doc

The JUnit output path is configured in .config/nextest.toml at target/nextest/ci/junit.xml.