Real User Proof v2 and the legacy activation cohort

July 31, 2026 ยท View on GitHub

The version-2 instrument separates:

  1. a counterbalanced bare-versus-Citadel controlled utility trial;
  2. meaningful D7/D30 retention based on another canonically verified task; and
  3. receipt-owned exit and restore evidence.

Run node scripts/product-proof-trial.js help for the local-only v2 workflow. Every v2 report remains claim_status: instrument_only and utility_claim: false until an independently run, preregistered cohort supplies the required human evidence. Assignment failures, timeouts, abandonments, and missing records remain in intention-to-treat denominators. Public previews suppress cells smaller than five and make no network request.

The schema-1 activation cohort below remains readable for compatibility. Its return_session field is a legacy session-reopen diagnostic and can no longer satisfy meaningful D7 retention.

Legacy schema-1 activation cohort

Citadel has public attention. This legacy cohort asks whether people reach a handoff, resume work, and reopen a session. It does not establish verified comparative utility or meaningful task retention.

The cohort is voluntary, public, and privacy-minimal. Failures count. Missing evidence stays unknown. A star, clone, or successful fixture is not counted as human activation.

Public submissions live in GitHub Discussion #182.

Share your activation journey

From a repository where Citadel is installed, run:

node .citadel/scripts/activation-telemetry.js share

The command writes .planning/product-proof/activation-share.json and prints the same payload. It does not open a network connection or post anything.

Review the file, then post that JSON object inside a json fenced code block in Discussion #182. Run the command again after day seven and reply with the updated object. The stable opaque submission ID lets the maintainer replace the earlier observation instead of counting one installation twice.

Only a block shaped like this qualifies for collector ingestion:

```json
{ "schema": 1, "kind": "activation_cohort_submission", "...": "the remaining exact share fields" }
```

JSON mentioned in prose, untagged fences, inferred claims, malformed objects, and objects with extra fields do not count.

If your install keeps Citadel in a separate source clone, run the source script and point it at the target project:

node /path/to/Citadel/scripts/activation-telemetry.js share --root /path/to/your/project

What the bundle contains

  • An opaque random submission ID that is separate from the local installation ID.
  • The Citadel version and whole-day observation age.
  • Bounded booleans for install, setup, route, verified handoff, resume, return, and install or route failure.
  • A local event count used as a basic integrity check.
  • Explicit consent for aggregate use.

The schema rejects extra fields. It cannot contain prompts, repository names, paths, commands, source code, user identity, tokens, or secrets.

Posting is not anonymous. GitHub shows the account that wrote the comment. The bundle itself contains no GitHub username or other personal identity.

Legacy instrumentation thresholds

These six thresholds can show that the legacy activation funnel has enough observations to inspect. They cannot authorize a product-utility or meaningful retention claim:

GateTargetDenominator
Shared installations25Unique opaque submission IDs
Setup completion60%Successful installs
Verified handoff40%Successful installs
Durable resume25%Successful installs
Seven-day return15%Successful installs observed for at least seven days
Install or route failure10% maximumShared install attempts

Before 25 submissions, the diagnostic status is collecting. After 25 submissions but before 25 are seven-day eligible, it is observing. The legacy labels ready and needs_attention mean only that this diagnostic funnel has or has not met its historical thresholds. They are never a Real User Proof v2 gate.

Likewise, scripts/product-proof-cohort.js is retained for schema-1 compatibility. It emits claim_status: superseded_instrument_only, utility_claim: false, and can no longer set milestone_ready.

This is an opt-in cohort, not a census. Install failures that cannot run the share command are underrepresented, so the failure rate must not be described as the failure rate of every clone or installation.

Maintainer ingestion

Collect the current complete Discussion snapshot with the authenticated gh session:

node scripts/activation-cohort-collect.js --dry-run --json
node scripts/activation-cohort-collect.js --json

The collector calls only the read-only Discussion comments endpoint through gh api --paginate --slurp. It does not post, edit, react, infer a journey from prose, copy credentials, or persist tokens. It parses only json fenced blocks and validates every candidate through the same exact submission schema used by manual ingestion.

The complete snapshot is reconciled by opaque submission ID. A later observation replaces an earlier one; a current edit is revalidated; duplicate or older observations do not add another installation; and a deleted or no-longer-qualified comment disappears from the collector snapshot. The ignored local evidence envelope retains the source comment URL. The aggregate report never contains those URLs.

Use a fixture for deterministic offline verification. Fixture mode never invokes gh or another network client:

node scripts/activation-cohort-collect.js \
  --fixture scripts/fixtures/activation-discussion/initial-pages.json \
  --root /tmp/citadel-cohort \
  --dry-run \
  --json

Rate limits and invalid API responses fail closed before local cohort files are changed.

Manual ingestion remains available for a single reviewed comment:

Save one posted JSON object as a temporary file, then bind it to the final public comment URL:

node scripts/activation-cohort.js ingest \
  --bundle /path/to/activation-share.json \
  --evidence-url https://github.com/SethGammon/Citadel/discussions/182#discussioncomment-123456

The command updates the ignored local cohort store and writes .planning/product-proof/activation-cohort-report.json. The dashboard Activation panel reads that report and shows the current status, denominators, and gate results.

Rebuild the report at any time:

node scripts/activation-cohort.js report

The local evidence store preserves the public comment URL. Updated observations use the latest observation_day for each opaque submission ID. Do not combine a collector-managed snapshot with unrelated manual evidence in the same file; the collector intentionally reconciles that file to the current Discussion.

Stopping condition

Do not claim retained human use until the report says ready. If the cohort reaches needs_attention, inspect the failed gate, fix the product seam, and begin a new versioned observation window without deleting the negative result.