MCP Evidence Validator

September 22, 2026 · View on GitHub

Validate what an MCP server declares against what it actually does, and produce a tamper-evident evidence record you can hand to an auditor.

Status: v0.5.0 — published on PyPI with build attestations · License: Apache-2.0 · Language: Python 3.10+ (stdlib only, zero dependencies)

Built by Empire Labs Pty Ltd and published as part of the Empire Stack — the evidence layer for agent actions. Related public work:

witnessos — credential-brokered enforcement and cryptographic evidence for agent actions; the evidence-grade ladder these checks descend from (E0 Declared → E1 Observed → E2 Enforced → E3 Corroborated → E4 Anchored).
evidence-record-spec — portable, evidence-grade record format for agent actions.

Why

Model Context Protocol (MCP) is the industry standard for connecting AI agents to tools and data. It is now governed by the Agentic AI Foundation (AAIF) under the Linux Foundation, with 97M+ monthly SDK downloads and 10,000+ active public servers.

But the protocol leaves the evidence problem open:

A server declares tool schemas and permissions, but nothing verifies those declarations against observed runtime behaviour.
An annotation can be accurate at declaration time and silently stale minutes later, when the contract it was bound to mutates underneath it.
MCP has already seen the first malicious server (Sept 2025), CVE-2025-6514 (CVSS 9.6, RCE), and a hosting-platform breach affecting 3,000+ downstream applications.

The declared-vs-observed gap is the measurement finding that matters. This validator is a reference implementation for closing it.

What it does

Captures declarations - the tool schemas, permissions, and annotations an MCP server publishes.
Observes reality - the tool invocations, argument shapes, and contract hashes seen at runtime.
Checks the gap - declared annotation still bound? Contract mutated since declaration? Privilege use inside declared scope?
Produces an anchored evidence ledger - every check result is committed to a SHA-256 hash chain, and validate prints the head digest of that chain. Record the head somewhere the ledger file's holder cannot edit, and verify refuses any ledger that does not reach it. A chain read on its own proves ordering to whoever holds the file and nothing to anyone else: an editor who can change a record can replay the chain over the change, drop the last block, or publish any prev_hash and index they like.

Install

pip install mcp-evidence-validator

Python 3.10+, standard library only — no dependencies. Or from source:

git clone https://github.com/narko4u/mcp-evidence-validator.git
cd mcp-evidence-validator
pip install .

Releases carry a build attestation you can check before running the code you installed — see Verifying releases below.

Quick start

After installing from source above, run these commands from the repository root so the bundled example files are available.

# Compare a declared manifest against observed runtime records, and write the
# head digest of the resulting ledger to a separate file
mcp-ev-validate validate --declared examples/fictional-server-declared.json --observed examples/fictional-server-observed.json --out evidence.json --head-out evidence.head

# Audit a ledger later, against the head you kept elsewhere
mcp-ev-validate verify --ledger evidence.json --expected-head "$(cat evidence.head)"

--expected-head is required. Keep the head where the ledger's holder cannot reach it: a signature, a commit in another repository, a transparency log entry, or a line in the auditor's own notes. Both files land in the current directory, so the commands run unchanged on Windows.

The validate command prints a JSON report. Its summary contains:

{
  "declared_tools": 2,
  "observations": 3,
  "findings": 2,
  "contract_recipes": ["1"],
  "checks": [
    "bound_unmutated",
    "contract_mutated",
    "scope_violation",
    "recipe_mismatch"
  ]
}

contract_recipes names the recipe each contract hash was computed under — ["1"] here because the constructed example predates the current recipe and does not state one, so validate also notes that on stderr. See Contract recipes.

The report's findings array describes the deliberately undeclared admin_token argument in observation 2 and the changed contract in observation 3. Validation exits 0 when the report is produced, even when it contains findings; inspect the report to assess the observed behaviour.

Record the head somewhere the ledger file's holder cannot edit. verify checks the chain against that head and, for this example, exits 0 with:

ledger intact: 3 blocks, chain verified against the expected head
block types: {"declaration": 1, "observation_batch": 1, "report": 1}
contract recipes: 1

The contract recipes: line is the verification being explicit about how the evidence it just checked was produced: a contract hash means nothing without the recipe that produced it.

verify exits 1 when the ledger does not reach the expected head, and 2 when it is invoked without one. It checks the ledger against the head you pass in, so it does not vouch for the observations being free of findings: a chain read on its own proves ordering to whoever holds the file, and nothing to anyone else.

Or run as a module: python -m mcp_evidence_validator validate --declared ...

Output is a machine-readable evidence record with a chain of hash-linked entries plus a human-readable findings summary. The verify subcommand replays the chain, checks the published prev_hash and index of every block against that walk, and compares the head it reaches to the one you pass in. Try editing evidence.json and re-verifying against the original head: the edit is refused.

Covered server types

Example	Server	Declared	Observed	Recipe	Shows
`examples/fictional-server-*.json`	fictional weather server	—	—	1 (legacy)	all three checks against constructed data
`examples/filesystem-server-*.json`	`@modelcontextprotocol/server-filesystem`	2026.1.14	2026.8.31	3	three real contract mutations between two releases, two of which recipe 1 could not see

The second pair is a real capture, not a constructed one:

mcp-ev-validate validate --declared examples/filesystem-server-declared.json --observed examples/filesystem-server-observed.json --out fs-evidence.json --head-out fs-evidence.head

It reports three findings, one per observed call. read_media_file's contract moved between the two versions in a way that matters to a caller: the description went from "Read an image or audio file" to reading any file, "returned as an embedded resource" when it is neither image nor audio, and the output schema gained an anyOf branch whose second case is a resource carrying a blob, while the first case lost blob from its type enum. A client that bound a handler to the earlier output shape at declaration time is now holding a stale annotation; the check says so, and the two hashes it prints are the two real contracts.

The other two, read_text_file and get_file_info, are the more interesting findings: the 2026.8.31 release added an openWorldHint annotation to every tool it serves, and those two changed nothing else. Under recipe 1 they were byte-identical and reported healthy. Under recipe 2 their contracts moved, correctly, because what a tool declares about its own side effects is part of what it declares.

Both sides of this pair are now hashed under recipe 3, so the contract hashes differ from the v0.4.x ones for the same declarations — the recipe widened, the servers did not change. This capture happens to show title and execution identical between the two versions, so recipe 3 adds no finding here: it closes a gap the capture demonstrates the fields are exposed to, not one it exhibits. The mutation is pinned in the test suite instead, where it can be stated as a construction rather than implied as evidence.

The three calls in the observed file were made against the running server and every contract_hash is the validator's own fingerprint over what that server served. The raw replies are committed in examples/captures/ and tests/test_filesystem_example.py recomputes each hash from them, so the pair cannot silently drift from its evidence. To recapture:

python3 examples/capture_mcp_server.py \
    --package @modelcontextprotocol/server-filesystem \
    --declared-version 2026.1.14 --observed-version 2026.8.31 \
    --root /tmp/mcp-capture-root --label filesystem-server

That needs Node.js (npx) and registry access. The committed files are static, so neither the tests nor the validator need either.

To rebuild the pair from the committed captures — which is how a pair is migrated to a new contract recipe, with no Node.js and no network:

python3 examples/rebuild_pair.py --recipe 3     # rewrite the pair and its derived hashes
python3 examples/rebuild_pair.py --check        # report differences, write nothing

Concepts

Term	Meaning
Declaration	What a server publishes: tool name, description, input schema, permission scope
Contract	A canonical, hashable form of a declaration (canonical JSON → SHA-256)
Contract recipe	Which declaration fields a contract hash covers. Carried in the evidence next to the hash, never assumed — see Contract recipes
Annotation	A statement binding a declaration to a contract (e.g. "this tool is scoped to read-only"). Distinct from a tool's MCP `annotations` hints (`readOnlyHint`, `destructiveHint`, …), which recipe 2 folds into the contract itself
Observation	A runtime fact: invocation record, argument values, contract hash at call time
Finding	A measurable gap between declaration and observation
Ledger	An append-only SHA-256 hash chain over every captured record

Contract recipes

A contract hash answers "has this declaration changed?" — which is only useful if you also know which parts of the declaration it covers. That field set is the recipe, and it travels with the evidence instead of being assumed:

Recipe	Covers	Applies to
`1`	name, description, input schema, permissions	every ledger written before v0.4.0, and any declaration that does not state a recipe
`2`	recipe 1, plus the declared output schema and the tool's MCP annotation hints	declarations that opted up before v0.5.0
`3`	recipe 2, plus the tool's declared title and its execution parameters (`taskSupport`)	the current default for new declarations

Each recipe covers what the raw captures actually hold. _meta and icons sit outside all of them deliberately: no capture of a served tool has ever carried either, and a recipe that folded in a field the declaration does not carry would hash the fallback default and report a contract no server ever served. That exclusion is measured against the captures in the test suite rather than asserted in a comment.

Where it lives:

a declaration states contract_recipe at the manifest level, and an individual tool can override it;
an observation may state the contract_recipe its contract_hash was computed under; one that carries a hash but states none is read as recipe 1 — the same default a silent declaration gets — and one that carries no contract_hash at all (a scope-only observation) states no recipe either way;
the report's summary lists every recipe in play, including one stated on an observation for a tool the declaration does not carry, and verify prints them for a ledger.

Four rules make that safe rather than decorative:

No silent reinterpretation. A declaration that states no recipe is hashed under recipe 1, exactly as before — so ledgers issued before the recipe existed still verify, and nothing already published changes meaning. New declarations opt up by stating the recipe.
No guessing across recipes. If an observation states a different recipe from the declaration's, the comparison is refused with a recipe_mismatch finding rather than reported as drift. Two hashes computed different ways are not evidence of change in either direction.
Silence means legacy, not unknown — where there is a hash to interpret. An observation that carries a contract_hash but states no recipe was produced before recipes existed, so it is read as recipe 1 — and a pre-v0.4.0 ledger compared against a recipe-2 declaration is therefore refused, not reported as drift. The finding says so, and states the one-line change (contract_recipe: "1" on the declaration) that judges that ledger under the recipe that produced it. The inference stops at the hash: an observation that carries no contract_hash states no recipe either way, so none is inferred for it and no finding claims a hash the evidence does not hold. Arguments on such an observation are still judged against the declared input schema.
A recipe is read wherever it appears. An observation for a tool the declaration does not carry is still parsed and counted, so the summary cannot describe a ledger as holding one recipe while the evidence also holds a hash made under another. An unknown recipe is rejected there too, rather than escaping validation by virtue of the tool being undeclared.

Migrating a pair between recipes re-derives every contract hash from the raw captures, never from a stored hash — which is why the example pair can be rebuilt offline:

python3 examples/rebuild_pair.py --recipe 3

The example in this repository shows what the wider recipe buys: two of its three findings are contract changes that recipe 1 reported as healthy.

Check types

Bound, contract unmutated - healthy baseline: annotation still matches the contract it was declared against.
Bound, contract since mutated - finding: the annotation was accurate at declaration time but is stale because the contract moved underneath it. Pairs with a review-scheduling gate.
Observed outside declared scope - finding: runtime behaviour exceeds what the declaration permits (arguments, tools, or permissions not present in the declaration).
Recipe mismatch - finding: the observed contract hash was computed under a different recipe from the declaration's (including an observation that states none, and is read as recipe 1), so the comparison is refused. High severity because it means the check could not run — not because the server did anything wrong.

Roadmap

Project status

A reference implementation of declared-vs-observed evidence for MCP servers, built and maintained by Empire Labs Pty Ltd as the open layer of its evidence-engineering work. The enforcement and cryptographic-evidence side of that work is witnessos; the record format is evidence-record-spec.

It is also the implementation this project puts behind the argument when the declared-vs-observed gap comes up in standards venues — including the Agentic AI Foundation's Security & Privacy work on tamper-evident evidence for agent actions — because a gap is easier to settle with a working validator than with a slide.

The core ideas are exercised in production-grade systems elsewhere in the organisation. This repository contains no proprietary code and no client data: the fictional example is constructed to exercise the checks, and the filesystem example is a capture of a public open-source MCP reference server, committed together with the raw replies it was derived from.

Security

See SECURITY.md for the vulnerability reporting policy. Security issues are handled privately — do not open a public issue.

Verifying releases

Two independent things are worth checking, and they answer different questions.

That the file you installed came from this repository

The wheel and sdist on PyPI carry PEP 740 build attestations, published by this repository's release.yml workflow under GitHub Actions OIDC. No signing key exists to leak. With uv this needs no installation:

uvx pypi-attestations verify pypi \
  --repository https://github.com/narko4u/mcp-evidence-validator \
  pypi:mcp_evidence_validator-0.5.0-py3-none-any.whl

which prints OK: mcp_evidence_validator-0.5.0-py3-none-any.whl. Name the file you installed — pypi:<filename> has PyPI serve it, or pass a local path or a files.pythonhosted.org URL to check a file you already hold. The check is not decorative: pointing it at a different repository fails with provenance was signed by repository "narko4u/mcp-evidence-validator", expected .... With pip, pip install pypi-attestations and drop the uvx. PyPI's own project page shows the same attestations under each file's Provenance link.

That the release bundle is signed and internally consistent

Releases are signed with sigstore keyless signing (GitHub Actions workload identity). Each release contains:

SHA256SUMS — hashes of every release asset
SHA256SUMS.sig — the cosign signature over SHA256SUMS
SHA256SUMS.pem — the ephemeral signing certificate

Set TAG to the release you are verifying. The expected signer identity is the repository's release.yml workflow running under that tag (GitHub Actions OIDC, issuer https://token.actions.githubusercontent.com), so a command pinned to some other release fails with none of the expected identities matched what was in the certificate: the certificate for v0.5.0 names v0.5.0 and nothing else. This needs the cosign CLI — which is why the attestation check above is the quicker one.

TAG=v0.5.0   # the release you are verifying
gh release download "$TAG" --repo narko4u/mcp-evidence-validator --pattern 'SHA256SUMS*'

cosign verify-blob \
  --cert SHA256SUMS.pem \
  --signature SHA256SUMS.sig \
  --certificate-identity "https://github.com/narko4u/mcp-evidence-validator/.github/workflows/release.yml@refs/tags/${TAG}" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
  SHA256SUMS

That prints Verified OK. The certificate asset is stored base64-encoded, which cosign reads as it comes out of the release; base64 -d SHA256SUMS.pem > cert.pem gives the plain PEM for a tool that wants one. Then check the assets against the signed list:

sha256sum -c SHA256SUMS

Acknowledgements

This validator checks implementations of the Model Context Protocol (MCP). MCP is an independent project with its own maintainers and specification process. It is credited here as the protocol being validated; that does not imply that MCP's maintainers or publisher have reviewed, endorsed or certified this tool.

The validator itself has no third-party runtime dependencies - it runs on the Python standard library alone.

Contributing

See CONTRIBUTING.md. Contributions welcome under the Apache-2.0 licence; please follow the Code of Conduct.