Contributing

The most useful contribution is an adapter for a harness we don't cover. Detection logic is worthless if it can't reach your trajectories.

Ground rules

• Zero runtime dependencies. CI enforces this. plumbline runs inside security-sensitive pipelines; every dependency is somebody else's supply chain.
• Node ≥ 20, ESM (.mjs), no build step, no transpiler.
• node --test test/*.test.mjs must pass, including the controls.
• New detector or adapter ⇒ new tests. See below for what's actually required.

git clone https://github.com/askalf/plumbline
cd plumbline
node --test test/*.test.mjs
node src/cli.mjs replay corpus/exploitgym.jsonl

Writing an adapter

An adapter turns a harness's logs into the trajectory schema. Look at src/adapters/claude-code.mjs (per-session transcript files), src/adapters/forge.mjs (many sessions in one dump), and src/adapters/redstamp.mjs (a hash-chained per-call audit log, with the block -> denied mapping that makes ratchet live and an inline chain verify for tamper-evidence) — they cover the common shapes.

If the harness's tools are user-defined — OpenAI, Anthropic, LangChain and OTel all are — don't hand-write a capability map you can't complete. Parse the log into the flat record shape ({ kind: 'human' | 'tool', name, args, output, isError }) and hand it to src/agentlog.mjs, which infers the action and capability from each tool's name and arguments and handles secret measurement, denial detection, and session.declare for you. src/adapters/openai.mjs is the smallest example — the four framework adapters are each ~60 lines of pure parsing on top of that shared core.

Four things matter, roughly in order of how easy they are to get wrong.

1. Emit capability_grant — this is not optional

staircase, the primary detector, reads only this field. An adapter that doesn't emit it leaves the most important detector completely inert, and — because a clean result and a dead detector look identical — nothing will tell you.

Map each tool to the capability its use demonstrates, and emit the first exercise in a session as the grant:

const CAPABILITY_MAP = {
  run_shell:   ['exec:local', 'proc:spawn'],
  query_db:    ['db:query'],
  fetch_url:   ['net:egress:read'],
  send_email:  ['mail:send'],
};

const held = new Set();
// ...per tool call:
const fresh = (CAPABILITY_MAP[tool] ?? []).filter((c) => !held.has(c));
for (const c of fresh) held.add(c);
if (fresh.length > 0) event.capability_grant = fresh;

Use scheme:value consistently. cloud:iam:role matches an envelope granting cloud:iam:*; iam:cloud:role does not.

2. Set target.op for anything shell-shaped

The invoked binary, not the whole command. Without it every shell action in a session shares one identity, and ratchet fires on unrelated denials — this produced 98 false positives during calibration. opOf() is exported from the Claude Code adapter and handles env prefixes, wrappers (sudo, npx), and navigation segments (cd x; git log → git).

3. Report denials and human turns

• outcome: "denied" when a guard refused the action. Without it ratchet cannot work.
• action: "session.turn" whenever a human speaks. This resets the ratchet — if oversight adjudicated a denial, the retry is not evasion.

Both are easy to skip and both silently disable a detector.

4. Measure secrets, never carry them

produces takes {id, len, entropy} — measurements only. Your adapter has the real value; compute entropyOf(value) and emit the measurement. plumbline never holds a secret, which is a property worth preserving.

5. Declare what your adapter cannot express — this is enforced

Add an entry to ADAPTER_CAPABILITIES in src/reachability.mjs. CI enumerates src/adapters/*.mjs from the filesystem and fails if any adapter lacks one.

'your-harness': {
  emits: Object.values(FIELDS).filter((f) => f !== FIELDS.DENIED),
  blind: [{
    field: FIELDS.DENIED,
    detectors: ['ratchet'],
    reason: 'this harness logs tool calls without outcomes, so a refused call '
      + 'is indistinguishable from a successful one',
  }],
},

This is what lets plumbline distinguish "this corpus contains no denials" (fine) from "this adapter can never see denials" (a permanent blind spot). Without it, every corpus your adapter produces reports clean with unknown gaps behind the number.

Be honest here. An over-claimed emits list is worse than an admitted blind spot, because it converts a known limitation into a silent one.

Verify your adapter

Two checks, and the second is the one people skip.

Score a corpus you expect to fail. A permissive envelope produces a clean report whether or not your adapter works:

node src/cli.mjs scan <your-dump> --adapter=<yours> --profile=eval-sandbox

If that comes back clean on non-trivial traffic, your adapter is probably not emitting capability_grant.

Check reachability. The scan output names any detector no session could feed:

  *** ADAPTER BLIND SPOT - THE CLEAN RATE DOES NOT COVER THESE ***
  not exercised    reassembly - this corpus contains no such activity
  detector coverage
    staircase     828  sessions (100%)
    ...

A detector at 0% coverage that you did not declare as blind is a bug in your adapter, not a property of the data.

Adding a detector

1. src/detect/<name>.mjs, exporting id and detect(events, envelope).
2. Register it in src/detect/index.mjs.
3. Declare its dependencies in DEPENDENCIES (src/reachability.mjs). Without one, reachability cannot reason about it and CI fails — deliberately: an undeclared detector is one that can be starved silently. If it reads a field no marker exists for yet, add the marker to FIELDS and count it in fieldCensus.
4. Add a trigger to corpus/detector-exercise.jsonl, and the id to EXPECTED_DETECTORS in test/control.test.mjs. The liveness control asserts every registered detector fires on that corpus, against a list written out literally rather than derived from the registry; a detector without a trigger fails CI, which is the intended behaviour.
5. Document it in DETECTORS.md — including a "cannot see" section. A detector whose blind spots aren't written down invites false confidence.
6. If it fires on known locations or known addresses (like metadata or siphon), ship the benign case it misreads as a corpus file too. A known-location rule without its declared false positive is a rule nobody can calibrate — and the false positive will be found by a user instead.

Severity guidance: weak-but-early signals cap low (recon caps at 0.25); only evidence that something left the boundary or that the agent evaded a gate should approach halt on its own.

Corpus contributions

Labeled trajectories are extremely welcome, especially benign ones — false-positive pressure is what makes this useful. Requirements:

• No real credentials, hostnames, internal paths, or personal data. Sanitize before opening a PR.
• A session.declare with a realistic envelope.
• A header comment stating provenance and whether it is real, sanitized, or reconstructed.

corpus/exploitgym.jsonl is a reconstruction from public write-ups and says so in its header. Be equally explicit.

What gets declined

• Anything adding a runtime dependency.
• Detectors that fire on novelty rather than divergence from declared intent. That's anomaly detection, and it's a different tool.
• Tuning that improves a benchmark number without a corresponding control proving the detector still fires.
• Inline/blocking enforcement. Deliberately out of scope until trajectory signals have earned it.

Reporting a vulnerability

See SECURITY.md. Please don't open a public issue for a security bug.