Ablation-backed context attribution

August 20, 2026 ยท View on GitHub

Context attribution separates three evidence levels:

  • causal_ablation: matched with/without-component trials from a trusted producer claim to isolate one immutable component.
  • paired_shadow: matched outcomes compare bundles, but do not isolate the component from other edits.
  • component_correlated: edit-size attribution identifies correlation only and is never reported as causal.

Every controlled trial joins to a component and distinct tested/comparison bundles by digest and records the evaluator epoch, cohort, fixture, seed, score pair, token cost, test time, and known interaction components. Aggregation rejects duplicate fixture/seed pairs and mixed comparison/cohort groups. A controlled record stores deterministic matched-pair keys and exact source-trial digests, so reconstruct_causal_credit / reconstructCausalCredit can verify its provenance before recomputing the mean effect from durable trials.

attribute_manifest_verified_trials / attributeManifestVerifiedTrials resolve both immutable manifests, recompute their digests and canonical manifest diff, require the same scenario/evaluator epoch, and bind the diff to the source trial. causal_ablation is accepted only when the tested manifest contains the declared (kind, key, digest), the comparison omits it, and that is the sole component change. Replacements, unrelated bundles, multi-component edits, missing/tampered manifests, and post-hoc relabeling fail closed. reconstruct_manifest_verified_causal_credit redoes the same checks before replaying credit.

The live context promotion path persists immutable manifest_diff.json on proposal and verifies it again before cutover. An exact component-addition experiment also writes held-out source trials and verified records to causal_attribution.json before promotion. Comparisons that are only replacement or multi-component evidence remain noncausal; they cannot reach that artifact by asserting an evidence label.

Re-ablation

plan_reablation / planReablation runs on a configured cadence, a score plateau, or a bundle-composition change. It uses a hard budget and prioritizes components that are expensive in prompt tokens, low confidence, old, interaction-prone, or last tested in a different bundle.

Later trials append to ContextAttributionLedger and link to the prior attribution rather than replacing it. This allows a component once marked retained to become uncertain, a demotion_candidate, or harmful after interactions change.

The ledger schema is version 2. Python model loading and the TypeScript parseContextAttributionLedger helper migrate schema-1 history. Because those records did not persist the new comparison, classification-policy, matched-pair, or source-trial bindings, migration marks them legacy_unverified. They remain available as history and may be superseded by fresh evidence, but cannot pass controlled replay or drive prompt demotion until re-ablation creates a fully bound record. Schema-1 records that already contain every current binding are preserved as verified. Digest-bound string arrays use ECMAScript UTF-16 ordering in both runtimes.

Prompt selection

select_prompt_components / selectPromptComponents can omit current-bundle components classified as harmful or neutral-and-expensive while retaining their full attribution history. Evidence from an older bundle composition is treated as uncertain and kept until re-ablation; evaluator-epoch mismatches are ignored.

Shared Python/TypeScript cases for isolated causal credit, interaction re-ablation, and insufficient budget live in context-attribution-parity-fixture.json. Exact valid removal and invalid replacement behavior is pinned by fixtures/context-bundles/causal-attribution-manifest-parity.json.