Compliance

August 12, 2026 · View on GitHub

The one-command acceptance gate is:

make compliance

It runs protocol, dual-era, golden tool, security, E2E, runtime-semantics, dogfood, compliance-report, required docs/evidence/workflow, and schema-drift checks. Report files:

Always inspect suite, passed, tests_run, security, e2e, dogfood, and required_tools. If suite is not all, required tool coverage is not_measured by design.

The CI-shaped local gate is:

make ci

It adds lint, typecheck, unittest discovery, required docs checks, schema-drift checks, full deterministic dogfood runner, and SWE-bench preflight.

Coverage

  • Protocol versions: full support for MCP 2026-07-28 — every implemented method, its params._meta validation, the SEP-2243 mirror headers, server/discover, result shaping, and the HTTP status each error maps to — alongside the handshake era 2025-11-25 and 2025-06-18. tools with listChanged: false is the only advertised capability in either era.
  • MCP initialize, tools/list, tools/call, schemas, annotations, structured success/failure output, unknown tool behavior, protocol errors, trace redaction, and stdout cleanliness.
  • Interoperability with a client we did not write: the official MCP python SDK drives the server over stdio and HTTP, and CI fails rather than skips if it is not installed.
  • Tool golden cases for read/list/search/patch/exec/stdin/kill/git status/git diff/image.
  • Security cases for traversal, absolute paths, symlink escape, command workdir escape, direct and interpreter-mediated outside reads, direct syscall outside reads and writes where Landlock is available, destructive command policy, shell-expansion gating, obfuscated network access, risky env rejection, Linux Landlock fallback warnings, command timeout enforcement, watchdog cleanup, bounded output buffers, request-permission non-grants, and concurrent read-only calls.
  • Deterministic E2E loops for JavaScript bugfix, Python function add, long-running stdin, command termination, workspace escape denial, and image viewing.
  • MCP-only dogfood without direct filesystem or shell bypass during task execution.
  • Compliance report generation semantics, including non-overclaiming partial-suite tool coverage.

Individual Gates

make test-mcp-contract
make test-dual-era
make test-tool-golden
make test-security
make test-e2e
make test-runtime-semantics
make test-docs-required
make test-schema-drift
make dogfood-mcp
make dogfood-runner
make dogfood-smoke
make benchmark-smoke

GitHub Actions workflows: