AIR Blackbox

June 4, 2026 · View on GitHub

PyPI Downloads License EU AI Act Audit chain Status

The flight recorder for autonomous AI agents. Record, replay, enforce, and audit, with a tamper-evident chain you can hand to a regulator.

One proxy swap. Complete coverage. Runs locally.

# Before
client = OpenAI(base_url="https://api.openai.com/v1")

# After, everything else in your code stays identical
client = OpenAI(
    base_url="http://localhost:8080/v1",
    default_headers={"X-Gateway-Key": "your-key"}
)

With the trust layer enabled, every LLM call generates a tamper-evident, replayable audit record. No SDK changes. No refactoring. Records are written asynchronously, off your request path.

Signed, verifiable, and yours to keep

A log file is not evidence. Anyone with write access can edit a plain log and no one can tell. AIR Blackbox makes your AI decision records provable instead.

Tamper-evident by default. Every record is chained with HMAC-SHA256: alter one field and that record's hash no longer matches, and every record after it breaks with it. This runs on every install.

Independently verifiable. Receipts are signed so a third party can verify them with only a public key, no shared secret and no trust in you. Signing uses Ed25519 by default (install the gate extra, see below). The gateway can additionally sign with ML-DSA-65 (FIPS 204 / Dilithium3), NIST's standardized post-quantum scheme, via Cloudflare's circl, for evidence that stays verifiable after large quantum computers arrive. Keys are generated locally and never leave your machine.

Built for the audit. EU AI Act Article 12 requires high-risk systems to automatically record events over the system's lifetime with traceability appropriate to their purpose, and Article 19 requires keeping those logs for at least six months. A tamper-evident, independently verifiable chain is how you prove that traceability and exceed the regulatory floor. For Annex III high-risk systems like hiring AI, the logs must also record who verified each result, which is exactly the human-oversight record AIR captures.

What you get

Tamper-evident audit chain, every call produces an HMAC-SHA256 chained .air.json record, written asynchronously. Tamper with one record and every record after it breaks.

Self-verifying evidence bundle, one command packages the audit chain, scan results, a manifest, and a standalone verify.py. An auditor runs python verify.py and gets PASS or FAIL on the chain in seconds, using only the Python standard library. No pip install on their end. No internet connection. No hosted service.

EU AI Act gap analysis, 51+ checks across Articles 9, 10, 11, 12, 13, 14, and 15. Maps to ISO 42001, NIST AI RMF, and Colorado SB 24-205. One scan, four frameworks, one report.

PII and injection scanning, 20 weighted patterns across 5 attack categories detected before the prompt reaches the model. Configurable sensitivity. Auto-blocking.

Replay, load any past episode from the audit chain, verify the chain, and replay every step with timestamps. Incident reconstruction without guesswork.

Framework trust layers, drop-in wrappers for LangChain, CrewAI, OpenAI Agents SDK, Anthropic, AutoGen, Google ADK, and Haystack. Same audit chain, native integration.

Quickstart

pip install air-blackbox

# Run your first gap analysis, works on any Python AI project
air-blackbox comply --scan . -v

# Find undeclared model calls hiding in helpers and utilities
air-blackbox discover

# Replay any recorded episode
air-blackbox replay

# Generate a signed evidence package for audit or regulator review
air-blackbox export

The base install scans, discovers, replays, and exports, with an HMAC-SHA256 audit chain. To sign receipts with Ed25519 for third-party verification, install the signing extra:

pip install "air-blackbox[gate]"   # adds Ed25519 signing via cryptography

ML-DSA-65 post-quantum signing is provided by the gateway (see the full stack below).

Claude Code plugin, fastest path for developers who live in their editor:

/plugin marketplace add airblackbox/air-blackbox-plugin
/plugin install air-blackbox@air-blackbox

Full stack (Gateway + Episode Store + Policy Engine + observability):

git clone https://github.com/airblackbox/air-platform.git
cd air-platform
cp .env.example .env      # add OPENAI_API_KEY
make up                   # running in ~8 seconds
  • Traces: localhost:16686 (Jaeger)
  • Metrics: localhost:9091 (Prometheus)
  • Episodes: localhost:8081 (Episode Store API)

How it fits your stack

Your Agent
    │
    ▼
AIR Gateway                    ← swap base_url here
    │
    ├── PII + injection scan   (before prompt reaches model)
    ├── HMAC audit record      (async, tamper-evident chain)
    └── Signature              (Ed25519 default, ML-DSA-65 option, keys stay local)
    │
    ▼
LLM Provider                   ← OpenAI / Anthropic / Azure / local
    │
    ▼
AIR Record                     ← tamper-evident .air.json
    │
    ▼
Evidence Bundle                ← self-verifying .air-evidence ZIP

Works with any OpenAI-compatible API. Same format, same integration, regardless of provider.

Why not just log everything?

You probably already have logging. The problems logging doesn't solve:

Tamper-evidence, anyone with write access to your log store can alter a record. HMAC chains make alteration detectable. Signatures (Ed25519 by default, ML-DSA-65 in the gateway) prove who signed and when, verifiable by a third party with only the public key.

Prompt reconstruction, most logging captures responses but not the full prompt context, tool calls, and intermediate reasoning. AIR records the complete episode.

Compliance structure, EU AI Act Article 12 requires high-risk systems to automatically record events over their lifetime with traceability, and Article 19 requires retaining those logs for at least six months. Raw, editable logs don't prove integrity. A tamper-evident, verifiable evidence bundle does.

Secrets leaking into traces, every team that builds their own logging eventually discovers credentials in their observability backend. AIR strips and vault-encrypts API keys before writing any record.

Runtime control, air-gate and air-controls

air-blackbox scans your code before you ship. Two sibling packages cover what your agents do after they're live.

air-gate, human-in-the-loop gating

Before an agent sends that email, deletes that file, or executes that SQL, air-gate pauses, checks a policy, optionally asks a human via Slack, and signs the decision to a tamper-evident audit chain. A building block for EU AI Act Article 14 (Human Oversight), in about a dozen lines of Python.

pip install air-gate

from air_gate import GateClient

gate = GateClient()  # local mode, zero config

result = gate.check(
    agent_id="support-bot",
    action_type="email",
    action="send_email",
    payload={"to": "customer@example.com", "body": "..."},
)

if result["decision"] == "auto_allowed":
    send_the_email()
elif result["decision"] == "blocked":
    log.warning("Blocked by policy:", result["reason"])
# MEDIUM/HIGH-risk actions pause until a human approves in Slack

# Verify the full audit chain anytime
assert gate.verify()

Highlights:

  • Risk-tiered YAML policy, auto_allow, require_approval, block, per-action-type
  • Slack approval flow, human approves from their phone, callback URL fires back to the agent
  • PII auto-redaction, 25+ categories across five verticals (universal, finance/PCI-DSS, healthcare/HIPAA, legal, recruiting/EEOC)
  • LangChain and OpenAI function-tool wrappers, one-line integration
  • Library or server mode, GateClient() for zero config, FastAPI + Slack bot for team workflows

Full repo: airblackbox/air-gate

air-controls, runtime visibility

Your agents are making thousands of decisions per day. air-controls is the dashboard that makes them legible. Action timeline, cost per call, risk scoring, kill switch. Same HMAC audit chain as air-blackbox.

pip install air-controls

# LangChain
from air_controls import ControlsCallback
cb = ControlsCallback(agent_name="sales-bot")
chain.invoke({"input": "..."}, config={"callbacks": [cb]})

# CrewAI
from air_controls import CrewMonitor
mon = CrewMonitor(agent_name="research-crew")
mon.run(crew)

# Any OpenAI / Anthropic agent
from air_controls import monitor

@monitor(agent_name="my-bot")
def process_customer(query):
    return openai.chat.completions.create(...)

air-controls status                 # live dashboard of all agents
air-controls events sales-bot       # event timeline for one agent
air-controls pause sales-bot        # kill switch
air-controls verify                 # verify audit chain integrity

Local-first. SQLite backing store. No cloud. No phone-home.

Full repo: airblackbox/air-controls, with an MCP server at air-controls-mcp for Cursor, Claude Code, and Windsurf.

How the pieces compose

   Build time                                 Runtime
───────────────────                   ────────────────────────

air-blackbox  ──────────┐       ┌──── air-controls
(scan code,             │       │     (monitor what agents do,
 find gaps,             │       │      action timeline, cost,
 export evidence)       │       │      kill switch)
                        │       │            │
                        │       │            │ escalates to
                        │       │            ▼
                        │       │      air-gate
                        │       │      (pause dangerous actions,
                        │       │       human approval, Slack)
                        │       │
                        ▼       ▼
                  Shared HMAC-SHA256 audit chain
                  Shared signatures (Ed25519 default, ML-DSA-65 option)
                  Shared .air-evidence bundle format

All four deployable as one Docker Compose stack:
    air-platform  (make up, 8 seconds to full stack)

The full ecosystem

air-blackbox is the scanner and the entry point. Everything else extends it.

PackageStageWhat it does
air-blackboxBuild-timeEU AI Act scanner, 51+ checks, signed evidence bundles (this repo)
air-trustBuild + runtimeCryptographic primitives and trust layer wrappers (see repo)
air-controlsRuntimeLive agent visibility, timelines, costs, kill switch
air-gateRuntimePre-execution human-in-the-loop gating with Slack approvals
air-platformDeploymentDocker Compose full stack in one command
air-blackbox-mcpIDEMCP server for Claude Desktop, Cursor, Claude Code
air-controls-mcpIDEMCP server for runtime agent visibility
air-blackbox-pluginIDEClaude Code plugin (slash commands for the scanner)
compliance-actionCIGitHub Action, run compliance checks on every PR
otel-prompt-vaultInfraOTel processor: offloads sensitive content to external storage
otel-collector-genaiInfraOTel processor: redaction, cost/token metrics, loop detection
otel-semantic-normalizerInfraOTel processor: normalizes LLM attributes to gen_ai.* schema

Install any one. They compose when you want them to.

Validated by

  • Julian Risch (deepset, Haystack maintainer), public validation on LinkedIn and GitHub issue #10810; response in under 38 minutes
  • Piero Molino (Ludwig maintainer), merged EU AI Act compliance changes driven by AIR scan results within hours of the issue being opened
  • arXiv AEGIS (March 2026), independent researchers published the identical interception-layer architecture for AI agent governance
  • McKinsey State of AI Trust 2026, trust infrastructure named as the critical agentic AI category
  • Listed in EthicalML/awesome-artificial-intelligence-regulation and GenAI-Gurus/awesome-eu-ai-act

How we compare

AIR BlackboxDocument generators (ArcKit, etc.)Hosted scanners (ark-forge, etc.)
Scans actual code✅❌ (generates docs from prompts)✅
Pre-execution gating with receipts✅ air-gate❌❌
Independently verifiable signatures✅ Ed25519, ML-DSA-65 option❌❌
HMAC audit chains✅ Local, self-verifiable❌Partial (usually hosted)
Everything runs locally✅✅❌
MCP server + Claude Code plugin✅Partial❌
PricingFree, Apache 2.0Free (docs only)Free tier + paid signing

Use a document generator for RFPs, business cases, and governance board paperwork. Use AIR Blackbox to prove to an auditor what your AI system actually did, with an audit chain anyone can verify independently.

Philosophy

AIR is a witness, not a gatekeeper, until you tell it to be.

  • Non-blocking, recording or gating failures never break production flow
  • Lossy-safe, dropped audit records are acceptable; dropped user requests are not
  • Self-degrading, if the collector is down, spans drop silently; warnings logged, never errors returned

You cannot detect what you cannot see. You cannot prevent what you cannot detect. You cannot trust what you cannot prove. AIR Blackbox is the layer that makes proof possible.

Contributing

See CONTRIBUTING.md.

False positive on a compliance check? Correct it, your correction flows into training data for the fine-tuned scanner model. The scanner gets smarter with every fix your team submits.

Good first issues: labeled good first issue, mostly new compliance checks and framework integrations.

License

Apache-2.0, airblackbox.ai

This is not a certified compliance test. It is a starting point to identify potential gaps.

If this helps you prepare for EU AI Act enforcement, star the repo, it helps other teams find it.