auditlite

Chain-of-custody for AI-assisted analyses.

Traditional reproducibility asks: can the analysis be rerun? AI-assisted analysis adds a second question: can it be reconstructed? The final result can look coherent while the reasoning path is scattered across prompts, generated code, manual edits, reruns, copied outputs, and decisions nobody wrote down. Three months later — or one reviewer letter later — nobody can say which code the model wrote, why an exclusion happened, or whether Table 1 still comes from the current script.

auditlite keeps one append-only, hash-chained trail per project (audit/trail.jsonl) and records what changed, why it changed, who or what changed it, and whether the reported result still follows from the current analysis.

What it is not. auditlite is not an AI detector — it records declared provenance and flags where declarations are missing. An intact hash chain shows the trail has not been rewritten after the fact; it does not show the recorded events are true. Checks report support for reconstruction, never correctness of the science.

Install

pip install auditlite

Python ≥ 3.10, no dependencies. Or the protocol-identical R package:

remotes::install_github("heidihelena/auditlite", subdir = "r")

Trails are cross-compatible: a trail written in Python checks cleanly in R and vice versa, hash for hash — mixed Python/R projects keep one trail.

What gets recorded

Kind	Captures
provenance	which code was human-written / AI-generated / AI-modified; model, version, prompt, context files, external tools (declared)
decision	why a variable, model, exclusion, transformation, or sensitivity analysis was chosen; whether an AI suggestion was accepted, edited, or rejected; whether the AI introduced a new analytic decision
run	exact script hash, command, Python + platform + package versions, declared seeds, input and output file hashes, exit code
claim	a manuscript table or figure tied to the run and file it came from
deviation	departures from the analysis plan, with the reason

The checks

auditlite check runs eight integrity checks. Each answers three ways — ok, flag, or not_checkable — because a check that lacks the events it needs should say so rather than guess.

• chain — has the trail itself been edited, reordered, or truncated?
• script_drift — were scripts edited after their last recorded run?
• input_drift — does the data on disk still match what the runs consumed?
• stale_outputs — were outputs modified outside a recorded run?
• claim_freshness — do manuscript tables/figures still match the files they were recorded from?
• downstream_review — when a source input or script has changed, what is derived from it downstream (outputs, and the manuscript claims that rest on them) and should be re-checked? Follows the whole derivation chain, not one hop — so it catches an input edited but not yet re-run, where the old output still matches its fingerprint while no longer following from the current data.
• provenance_coverage — did any script execute without a declared origin?
• decision_rationale — were decisions recorded without a why (worst when the AI introduced them)?

downstream_review reports review obligations, never invalidation: it says what to re-check, not that anything is wrong. It uses only the dependency graph the runs already record (inputs → outputs, chained across runs; claims → their artifact), so no new declarations are required.

Exit code 1 if anything is flagged, so it drops into CI or a pre-submission checklist as-is.

Workflow

auditlite init --project lung-registry-survival

# declare provenance when code enters the project
auditlite provenance analysis.py --origin ai_generated \
    --model claude-sonnet-5 --prompt "primary survival analysis per SAP v2" \
    --context sap_v2.md --context codebook.csv

# trace analytic decisions as they happen
auditlite decision --what exclusion --choice "drop ECOG 4" \
    --why "n=3 in stratum, prespecified minimum is 10" \
    --source ai_suggested --disposition edited

# run through auditlite so the execution is fingerprinted
auditlite run --input data.csv --output table1.csv --seed numpy=42 \
    -- python analysis.py

# tie manuscript numbers to the run that produced them
auditlite claim table1 table1.csv

# before submission (or in CI)
auditlite check
auditlite report        # writes audit/AUDIT.md

Same API from Python:

import auditlite

trail = auditlite.init(project="lung-registry-survival")
trail.provenance("analysis.py", origin="ai_modified", model="claude-sonnet-5")
trail.decision("model", "logistic regression", "binary outcome, prespecified")
trail.run(["python", "analysis.py"], inputs=["data.csv"], outputs=["table1.csv"])
trail.claim("table1", "table1.csv")

for result in auditlite.run_checks(trail):
    print(result.check, result.status, result.detail)

And from R (same trail file, same hashes):

library(auditlite)

trail <- audit_open("audit/trail.jsonl")   # or audit_init(project = "...")
audit_provenance(trail, "models.R", origin = "ai_generated",
                 model = "claude-sonnet-5")
audit_decision(trail, "sensitivity", "E-value for unmeasured confounding",
               "reviewer 2 concern about smoking status")
audit_run(trail, c("Rscript", "models.R"),
          inputs = "data.csv", outputs = "table2.csv")
audit_check(trail)     # the original seven checks, same three-way statuses
audit_report(trail)    # writes audit/AUDIT.md

Version note (0.2.0): downstream_review ships in the Python package first. The trail format is unchanged, so trails stay cross-compatible hash-for-hash; R gains the eighth check in a follow-up. The R package's existing seven checks are unaffected.

The trail format

One JSON object per line. Every record carries the SHA-256 of the previous record, so any later edit breaks the chain from that point forward:

{"seq": 4, "ts": "2026-07-11T09:12:03+00:00", "kind": "run", "actor": "human",
 "body": {"command": ["python", "analysis.py"], "scripts": [...], "inputs": [...],
          "outputs": [...], "seeds": {"numpy": 42}, "exit_code": 0,
          "environment": {"python": "3.12.4", "platform": "...", "packages": {...}}},
 "prev": "9f2c…", "hash": "a41b…"}

Plain text, local-first, no server, no account. Commit audit/ next to the analysis and the trail travels with the project.

Siblings

• recoverlite checks whether a planned method can recover its target.
• assesslite checks whether a conclusion survives plausible assumptions.
• auditlite checks whether the entire analytic chain can be reconstructed and defended.

License

Apache-2.0. © Heidi Helena Andersen.