Key files

September 11, 2026 · View on GitHub

On-demand reference. CLAUDE.md (the always-loaded orientation file) routes here via its Reference map. Read a file's entry before editing that file.

Entries describe CURRENT behavior + load-bearing invariants only. Release history lives in CHANGELOG.md + git log / git blame, NOT here. Do not append per-release **vX.Y.Z:** narration — CI enforces this (scripts/check-key-files-current-state.sh).

  • src/core/runtime-version.ts — Bun 1.3.11 minimum enforced by the CLI entrypoint and guarded HTTP transport, with upgrade/restart guidance. Package engines and runtime checks share the same supported floor.

  • src/core/data-frontmatter.ts — shared data-only YAML/JSON parsing and serialization. Rejects unsupported language selectors before parsing, uses js-yaml safe schema with existing scalar types, sanitizes parser errors, and keeps body text opaque. Used by markdown, capture, recipes, plugins, and greenfield ingestion. Independent fixtures: test/data-frontmatter.test.ts, test/frontmatter-security.test.ts.

  • src/core/oauth-grants.ts — client-row-locked grant transactions shared by OAuth issuance, code/refresh consumption, and revocation. Preserves scope/resource ceilings and rolls back consumption with replacement writes. Bounded pending consent requests snapshot current operation and delegation policies. src/commands/serve-http-oauth.ts authenticates request details and enforces session-bound CSRF decisions; admin/src/pages/OAuthConsent.tsx renders the existing admin login/approval flow without exposing protected details in the page shell.

  • src/core/minions/submission-authority.ts — versioned authority kept outside caller job data. Submission normalization and worker revalidation bind remote jobs to immutable principal/grant/source/payload ceilings; application authority preserves trusted internal work. src/core/minions/authorize-legacy.ts provides explicit local preview and transactional snapshot comparison for selected historical rows. src/core/minions/source-filesystem.ts provides reentrant canonical root/worktree locking and descriptor-based confined reads/writes. Both worker modes enforce the same authority gate. Upgrade protocol: docs/guides/authorization-upgrade.md.

  • src/core/guarded-http.ts + src/core/ssrf-validate.ts — DNS answer classification, validated-address connections with original TLS/HTTP hostname and port, shared deadline/redirect policy, header-only probes, and bounded gzip/deflate/Brotli decoding. Ambient proxies fail closed with restart guidance. Consumers: URL reachability, remote images, and HTTP integration checks. Real TLS fixtures run on Bun 1.3.11 and 1.3.13.

  • docs/operations/conversation-parser-llm-fallback.md — operator and maintainer contract for the default-off LLM parse fallback: exact config key, deterministic-first dispatch boundary, sampled data surface, untrusted-content prompt handling, page-date/cache-key coupling, timestamp validation, cache/checkpoint behavior, observability, limitations, and focused test commands.

  • src/core/operations.ts — Contract-first operation contract, served through a ~300-line assembly façade: the op definitions live in the domain modules under src/core/ops/ (next entry) and are spread into the single exported operations array here; the shared contract types (src/core/ops/contract.ts) and the security/scope layer (src/core/ops/context.ts) are re-exported through this file, so every import path below resolves through the façade unchanged. The rest of this entry describes the surface as consumers see it. Same-source graph mutations (add_link, add_timeline_entry) preflight each endpoint against the scalar write source via requireWritablePage (ops/context.ts): a page readable only from another granted source returns endpoint-specific permission_denied naming the boundary, pages outside the caller's get_page visibility scope stay page_not_found (a soft-deleted foreign page is indistinguishable from absence), and a mutation-time engine miss — the typed PageMissingError from src/core/engine-errors.ts, thrown by both engines' single-statement endpoint resolution — is reclassified into the same envelope by instanceof, never message matching. Exports upload validators validateUploadPath, validatePageSlug, validateFilename, plus matchesSlugAllowList(slug, prefixes) (glob matcher: <prefix>/* matches recursive children; bare <prefix> matches exact only). OperationContext.remote is a REQUIRED field flagging untrusted callers; OperationContext.allowedSlugPrefixes is the trusted-workspace allow-list set by the dream cycle; OperationContext.auth?: AuthInfo is threaded through HTTP dispatch for scope enforcement in serve-http.ts before the op runs. OAuth whoami exposes the authenticated AuthInfo.sourceId and AuthInfo.allowedSources grants as source_id and federated_read; absent grants serialize fail-closed as null and [], while local, legacy, and stdio response shapes stay unchanged. enforceSubagentSlugFence(ctx, slug, opName) is the shared fail-closed subagent write fence: when viaSubagent and allowedSlugPrefixes is set, the slug must match the allow-list; else the legacy wiki/agents/<id>/... namespace check applies. Both put_page and add_timeline_entry (subagent-allowlisted) route through it. Auto-link/timeline hooks are skipped for remote-owned subagents even with explicit prefixes; only local subagents qualify as trusted workspaces. enforceClientSlugFence(ctx, slug, opName) is the OAuth-client write fence: when ctx.auth.boundSlugPrefixes is present (threaded from oauth_clients.bound_slug_prefixes at token-verification time), every direct slug-mutating write op — put_page, delete_page, restore_page, add_tag, remove_tag, add_link/remove_link (from endpoint only; linking TO a readable page is a reference), add_timeline_entry, revert_version, put_raw_data — rejects out-of-prefix slugs with permission_denied, BEFORE each op's dry-run short-circuit. Direct writes use the shared boundary-aware prefix rules; empty-array binding is deny-all (fail-closed); no auth / no binding = no fence. The match rule itself lives in the exported slugUnderBoundPrefixes(prefixes, slug) so non-op write surfaces reuse it verbatim. It is BOUNDARY-AWARE (a prefix matches whole segments, so emp-alice does not admit emp-alice-2/…), lowercases both sides (stored slugs are lowercased by validateSlug, so comparing the caller's raw string would let a mixed-case slug commit and only then trip the resolved-slug re-check), accepts BOTH the trailing-slash and the <prefix>/* glob spelling via normalizeSlugPrefix (direct and delegated fences share normalization while storing independently reviewed spans), and ignores empty-string prefixes. assertValidSlugPrefixes (oauth-provider.ts) rejects empty, whitespace-bearing, non-lowercase, and boundary-less entries at registration and rescope. submit_agent validates tools and namespaces against the independently stored bound_tools, delegated_slug_prefixes, and delegated_namespace grants. It normalizes accepted prefixes for matchesSlugAllowList and rejects explicit empty remote tool or prefix overrides. Workers interpret explicit empty tool lists as no tools, with defaults reserved for absent trusted-local bindings. put_page additionally fences the RESOLVED slug when importFromContent's dedup pre-check redirects the write to a different page (same content_hash / frontmatter.id), since the disk write-through runs against that slug. That re-check applies whichever confinement the CALLER is under — OAuth binding and/or subagent allow-list/legacy namespace — via slugOutsideCallerFence(ctx, slug), which composes slugUnderBoundPrefixes with the subagent fence's own match rule: the delegated submit_agent → subagent context carries viaSubagent + allowedSlugPrefixes while its auth carries current read scope without the parent's direct-write fence, so an auth-only check would let a slug-bound client holding agent scope reach an out-of-fence page simply by delegating the write. Denials never name the resolved slug (it would be a slug-enumeration oracle). Pinned by test/put-page-dedup-fence.test.ts. CLIENT_FENCED_WRITE_OPS + enforceBoundClientOpAllowList(auth, op) are the fail-closed companion, applied once in src/mcp/dispatch.ts (the choke point both MCP transports share): a slug-bound client calling ANY write/admin op not on the allow-list gets permission_denied. This covers the ops that write by a key other than a slug and therefore cannot be fenced — extract_entities/extract_facts (mutate people/*, companies/*), forget_fact (numeric fact id, crosses sources), ontology_propose — and makes a write op added later denied-by-default instead of silently unfenced. think is on the allow-list because remote callers cannot persist from it. Pinned by test/client-slug-fence.test.ts and over-the-wire by test/e2e/qm-provisioning.test.ts. Every Operation carries scope?: 'read' | 'write' | 'admin' + localOnly?: boolean; think is read-scoped for OAuth/MCP because remote callers have save/take forced off before persistence, while local CLI can still persist via remote:false; sync_brain, file_upload, file_list, file_url are admin + localOnly (rejected over HTTP). Four trust-boundary call sites (put_page allowlist, file_upload trust-narrowing, submit_job protected-name guard, auto-link skip) use FAIL-CLOSED semantics: ctx.remote === false for trusted-only sites, ctx.remote !== false for "untrust unless explicit-false" — anything not strictly false is treated as remote (so a read+write OAuth token over HTTP MCP cannot submit shell jobs). sourceScopeOpts(ctx) encodes the source-scoped read precedence ladder — federated array (ctx.auth.allowedSources) wins over scalar (ctx.sourceId/ctx.auth.sourceId) over nothing; every read-side op handler routes through it so a source-bound OAuth client can't see neighboring sources via search/query/list_pages/get_page/find_experts/query's image path, plus the by-slug reads get_tags/get_links/get_backlinks/get_timeline/get_chunks (chunks follow the same ladder as get_page, so a federated grant that can open a page can read its chunks — and the chunk payload never carries embedding vectors) (and get_page's tag fetch, which resolves against the concrete page's own source_id). assertExplicitSourceLive(ctx, sourceIdParam) is the async companion for the ops that accept a per-call source_id (get_page, list_pages, search, query): called right after federatedSearchScope (so the grant check has already run and it can only name a granted source), an explicit id with no live unarchived sources row throws unknown_source instead of silently scoping the read to an empty source — the CLI's --source existence rule applied to the op path; __all__ and an omitted param skip it. linkReadScopeOpts(ctx) is the link-read sibling for get_links/get_backlinks: a link row references three pages (from/to/origin), and the engine's federated (sourceIds[]) branch scopes ALL THREE while its scalar (sourceId) branch scopes only the near endpoint (by design — trusted internal callers like reconcileLinks and back-link validators call the engine with a scalar scope and need the cross-source view). For an UNTRUSTED remote caller (ctx.remote !== false) carrying only a scalar scope it promotes that scope to a single-element sourceIds:[id], routing them through the all-endpoint branch so a far/origin slug from outside the grant isn't disclosed; a federated array passes through unchanged and trusted local CLI (ctx.remote === false) keeps the scalar cross-source view. thinkSourceScopeOpts(ctx) maps the same precedence ladder onto runThink's public options (allowedSources/sourceId) so the think op's gather and trajectory stages inherit the caller's source grant. put_page's inline disk write-through is the shared writePageThrough helper (src/core/write-through.ts), ATOMIC via temp-sibling + rename so a crash or concurrent gbrain sync can't read a half-written .md; same helper backs gbrain brainstorm/lsd --save. Link provenance surface: add_link (gbrain link/link-add) + remove_link (gbrain unlink/link-rm) expose link_source/link_type; add_link rejects the reconciliation-managed built-ins via MANAGED_LINK_SOURCES (markdown/frontmatter/mentions/wikilink-resolved) and defaults omitted provenance to 'manual' (the engine's own default stays 'markdown' for internal callers); list_link_sources (gbrain link-sources, read) lists provenances via sourceScopeOpts. CLI aliases register through cliHints.aliases (collision-guarded in src/cli.ts).

  • src/core/ops/ — the operations contract's module directory (the meat behind the operations.ts façade). contract.ts is the foundation contract: the error envelope (ErrorCode/OperationError/verbError), the shared param/logger/auth/context types, and the Operation interface — re-exported wholesale by the façade. context.ts carries the context validators + scope resolvers: the upload/slug/filename validators, the subagent and bound-client slug fences, and the source-scope resolution ladder (some internal helpers are exported only here and deliberately NOT re-exported from the façade — import those from context.ts directly). The remaining modules are one-per-op-domain (pages.ts, search.ts, takes.ts, tags.ts, links.ts, timeline.ts, admin.ts, skills-catalog.ts, sync-status.ts, raw-data.ts, chunks.ts, ingest-log.ts, files.ts, jobs.ts, orphans.ts, calibration.ts, salience.ts, facts.ts, sources.ts, transcripts.ts, insights.ts, image.ts, extraction.ts, code-intel.ts, chronicle.ts, embedding-migration.ts, request-tools.ts, schema-packs.ts, skillopt.ts), each exporting a <domain>Operations map that the façade spreads into the single operations export. Add a new op in the matching domain module — an existing domain needs no façade change; a brand-new domain module gets one spread line in operations.ts. facadeExpansion in scripts/generate-flag-registry.ts maps the façade to this whole directory so every module's --flag text stays on the command flag-scan surface.

  • src/core/ops/links.ts — the link-domain op module (add_link, remove_link, get_links, get_backlinks, list_link_sources, traverse_graph). Two load-bearing behaviors. (1) Pack vocabulary at the write surface: an EXPLICIT link_type on add_link must be declared by the active schema pack when one resolves (schema-pack/write-vocabulary.ts; no resolvable pack = no enforcement; omitted/empty link_type is the untyped-edge default and stays unchecked; the check runs before the dry-run return so dry runs preview the rejection). (2) traverse_graph output shape: trusted local no-filter callers (ctx.remote === false, no link_type/direction) keep the legacy GraphNode[] shape that gbrain graph renders; remote callers default to direction: 'both' and always receive explicit GraphPath[] edges, so a page whose typed edges are all inbound does not read as edge absence. An explicit direction param wins for every caller. Depth: default 5 (DEFAULT_TRAVERSE_DEPTH), except a remote call that let direction default to both ALSO defaults depth to REMOTE_BIDIRECTIONAL_DEFAULT_DEPTH (2) — bidirectional path enumeration is combinatorial on entity hubs and this is the per-agent-turn path; a link_type-only remote call still takes both/2; an explicit depth is honored up to TRAVERSE_DEPTH_CAP (10, clamped with a warn). The edge walk runs through engine.traversePathsDetailed, and a hit on TRAVERSE_PATH_ROW_CAP (engine-constants.ts) surfaces as a stderr warn naming the cap (shallowest edges kept) — the GraphPath[] wire shape stays unchanged. Pinned by test/traverse-graph-op-default.test.ts + test/traverse-paths-row-cap.test.ts.

  • src/core/ops/pages.ts — the page-domain op module (get_page, put_page, delete_page/restore_page, capture, …; the source-scoping contract of the delete/restore trio is described under src/core/destructive-guard.ts). get_page resolution ladder: exact read in the caller's scope → alias hop → fuzzy. The alias hop scopes its lookup to the federated grant (sourceIds[]) > the scalar sourceId > (trusted unscoped only) every LIVE source — archived sources' alias rows count only when include_deleted asks for archived material — through engine.resolveSlugWithAliasDetailed, then reads the canonical page IN THE SOURCE THAT OWNS THE ALIAS ROW (getPage(canonical, { sourceId: hit.source_id })): a federated getPage prefers the anchor source, so an unrelated live page at the canonical slug in another granted source would otherwise shadow the alias owner's page. The redirect target composes with the same private-page gate as the exact read and reports resolved_slug. capture: an EXPLICIT type (the type param, else a frontmatter type: in the content — explicitCaptureType in src/core/capture-content.ts, BOM/CRLF tolerant, malformed YAML → none) is validated against the active pack when one resolves (schema-pack/write-vocabulary.ts) and IS the effective type for both the default slug and the merged frontmatter — approved as X, stored as X; no explicit type stamps note (never checked). Sits at its module-size ratchet ceiling (scripts/module-size-limits.tsv); the next growth peels a submodule. Pinned by test/get-page-federated-scope.test.ts, test/capture-explicit-type.test.ts, and the engine-parity suite (both engines pick the same owning row). The alias hop has no catch of its own: both engines return null for the pre-v104 missing-table case, so any other rejection (connection reset, timeout) propagates instead of degrading to page_not_found.

  • src/core/ops/admin.ts — the admin/diagnostics domain module (get_stats, get_health, get_brain_identity, …). All three aggregate ops confine remote callers via the same ladder reads use: ctx.remote === false keeps the trusted brain-wide view; every other caller gets sourceScopeOpts(ctx) (federated array > scalar > the unmatchable __all__ sentinel, which fail-closes to zeros) threaded into engine.getStats(scope)/getHealth(scope) — aggregates leak by subtraction, so they scope exactly like reads, and read-scope get_brain_identity is confined the same way. get_health's migrations {pending, partial, wedged, skipped_future} block stays GLOBAL for scoped callers by decision: it is a host filesystem ledger with no per-source semantics, a wedged host migration is exactly what a remote agent needs to see to explain degraded behavior, and it is composed at the op layer (not in BrainEngine.getHealth — growing the engine interface would force both engines to duplicate a file read).

  • src/core/ops/search.ts — the search-domain op module (search, query, search_stats, search_modes, search_tune, cache_stats). query is the hybrid entry (hybridSearchCached) plus two non-hybrid legs that share ONE effective-row-contract helper, resolveEffectiveLimit(ctx, p): an explicit limit wins, otherwise the mode-derived searchLimit resolved through the same trust-gated chain hybridSearch uses (resolvePerCallMode ignores a remote caller's mode, so a remote client cannot select the tokenmax row count); the config reads run lazily on those paths only. The legs: the image-similarity branch (embedding_image vector search, test/query-image-mode-limit.serial.test.ts) and the CRAG escalation slice — when search.crag_escalation is on and shouldEscalateRetrieval (src/core/search/crag.ts) says the first pass graded weak, was not already escalated, AND did not already run with the caller's expansion on (callerExpanded), the op re-runs once at limit: max(effectiveLimit, 50) with expansion + relational on and autocut off, keeping the better-graded run.

  • src/core/engine.ts — Pluggable engine interface (BrainEngine). clampSearchLimit(limit, default, cap) takes an explicit cap so per-operation caps can be tighter than MAX_SEARCH_LIMIT. Exports LinkBatchInput/TimelineBatchInput for the bulk-insert API (addLinksBatch/addTimelineEntriesBatch). readonly kind: 'postgres' | 'pglite' discriminator lets src/core/migrate.ts and others branch without instanceof + dynamic imports. Methods: batchLoadEmotionalInputs(slugs?) (CTE-shaped read with per-table aggregates so page × N tags × M takes never produces N×M rows), setEmotionalWeightBatch(rows) (UPDATE FROM unnest(\$1::text[],\$2::text[],\$3::real[]) composite-keyed on (slug, source_id)), getRecentSalience(opts), findAnomalies(opts). PageFilters has sort?: 'updated_desc'|'updated_asc'|'created_desc'|'slug' + PAGE_SORT_SQL whitelist consumed by both engines. listAllPageRefs(): Promise<Array<{slug, source_id}>> ordered by (source_id, slug) — cheap cross-source enumeration instead of a getAllSlugs()→getPage(slug) N+1 (which would silently default to source_id='default'); parity across postgres-engine.ts + pglite-engine.ts; Pinned by test/e2e/multi-source-bug-class.test.ts. SearchOpts+PageFilters add sourceIds?: string[] (federated read axis; both engines apply WHERE source_id = ANY($N::text[]) when set, preserve scalar sourceId fast path when unset); traverseGraph(slug, depth, opts?) and traversePaths(slug, opts?) accept opts.sourceId/opts.sourceIds. traversePathsDetailed(slug, opts?) returns { paths: GraphPath[], truncated } — the final SELECT of the path-enumerating recursive CTE is bounded at TRAVERSE_PATH_ROW_CAP + 1 rows on both engines (the probe row signals overflow; ORDER BY depth means the deepest edges are dropped, so a truncated result is still the complete shallow neighbourhood; the cap counts raw rows before the in-memory edge dedup, so paths.length alone cannot reveal truncation) and traversePaths is its .paths projection. resolveSlugWithAliasDetailed(slug, sourceOrSources) returns { canonical_slug, source_id } | null — the winning alias row AND its owning source under the same scope/precedence rules (ORDER BY array_position(scope, source_id), id; null when nothing matches or the slug_aliases table does not exist yet); resolveSlugWithAlias is its canonical-slug projection (falls back to the input slug). A consumer that goes on to read the canonical page must scope that read to source_id. The by-slug read methods carry the same federated axis: getTags/getLinks/getBacklinks/getChunks opts and TimelineOpts (consumed by getTimeline) accept sourceIds?: string[] taking precedence over the scalar sourceId (source_id = ANY($::text[]) scoping the slug→page-id lookup); getChunks falls back to the 'default' source when neither is set (importCodeFile's incremental-embedding reuse relies on it) and SELECTs an explicit non-vector column list — embedding vectors never ride the payload since rowToChunk discards them (getChunksWithEmbeddings stays scalar-only by design: engine-internal, zero remote-reachable callers); the link reads (getLinks/getBacklinks) scope ALL THREE endpoints (from/to/origin) on the federated branch while the scalar branch scopes only the near endpoint for trusted internal cross-source callers. traverseGraph opts has frontierCap?: number (per-iteration recursive-CTE cap, approx per-BFS-layer); return type Promise<GraphNode[]> for MCP wire stability; export TraverseGraphOpts; Postgres uses parenthesized LIMIT N ORDER BY (slug, id) inside the recursive term, PGLite mirrors with positional params; Pinned by test/regressions/v0_36_frontier_cap.test.ts. Phantom-redirect methods: refreshPageBody(slug, sourceId, compiled_truth, timeline, content_hash) narrow-UPDATEs three columns + updated_at, skipping soft-deleted rows (content_hash refresh required so gbrain sync sees the canonical as unchanged after fence merge); migrateFactsToCanonical(phantomSlug, canonicalSlug, sourceId) UPDATEs entity_slug+source_markdown_slug on every active fact row keyed on the phantom, preserving embedding/validUntil/kind/status/source_session/confidence; parity at test/phantom-redirect-engine-parity.test.ts. getAdjacencyBoosts(pageIds): Promise<Map<number, AdjacencyRow>> powers the per-query graph-signals stage — one SQL query returning inbound-link counts among top-K plus a cross-source count (links from differing source_id); COALESCE(p.source_id,'default') null safety, HAVING >= 1, cross-source CASE-WHEN excludes the target's own source; parity SQL across both engines; SearchResult gains optional base_score, backlink_boost, salience_boost, recency_boost, exact_match_boost, graph_adjacency_boost, graph_cross_source_boost, session_demote_factor, reranker_delta + internal staging fields; Pinned by test/e2e/graph-signals-engine.test.ts. Two REQUIRED methods: deletePages(slugs, {sourceId}): Promise<string[]> (single-batch primitive returning slugs actually deleted) and resolveSlugsByPaths(paths, {sourceId}): Promise<Map<path,slug>> (batch path→slug lookup); sourceId REQUIRED on both at the type level (asymmetric with single-row deletePage which keeps optional/'default'); both short-circuit on empty input and throw when > DELETE_BATCH_SIZE. softDeletePages(slugs, {sourceId}): Promise<string[]> is the batch soft-delete twin, mirroring deletePages' contract exactly (required sourceId, > DELETE_BATCH_SIZE throws, empty input short-circuits, caller owns chunking + decompose-to-one-element-batches on failure): one UPDATE … SET deleted_at = now() … AND deleted_at IS NULL … RETURNING slug round-trip returning only the slugs that actually flipped active→soft-deleted — the deleted_at IS NULL predicate is load-bearing so a re-run never refreshes an already-soft-deleted row's 72h purge clock — and NOTHING cascades (chunks/links/timeline stay behind the read-side deleted_at filters; the autopilot purge phase hard-deletes after 72h; a re-import within the window revives via putPage's upsert; deletePage/deletePages remain the purge/teardown primitives). getStats(opts?) / getHealth(opts?) take an optional {sourceId?, sourceIds?} scope (same shape as sourceScopeOpts output): omitted = brain-wide; scoped = EVERY counter/coverage/degree confines to the grant, including derived-table counts via their page joins, and link-derived numbers count only edges with BOTH endpoints in scope — so an excluded source's numbers can't be recovered by subtraction. Consumer is the remote-caller ladder in src/core/ops/admin.ts. Embedding-signature stale-detection quartet: countStaleChunks(opts?) gains optional signature?: string widening the stale predicate from embedding IS NULL to ALSO include chunks whose JOINed page embedding_signature IS NOT NULL AND <> $signature (NULL signature is GRANDFATHERED, never counted; omit signature for the legacy NULL-only count); sumStaleChunkChars(opts?: {sourceId?, signature?}): Promise<number> = SUM(LENGTH(chunk_text)) over stale chunks (same predicate + embed_skip filter + optional sourceId scope), used by gbrain sync --all cost preview via estimateCostFromChars; setPageEmbeddingSignature(slug, {sourceId?, signature}) stamps pages.embedding_signature after a page's chunks (re)embed, idempotent no-op when page absent; invalidateStaleSignatureEmbeddings({signature, sourceId?}): Promise<number> NULLs embedding+embedded_at on every chunk whose page signature is set AND differs, returning the count, called BEFORE listStaleChunks so signature-drift pages flow through the NULL-embedding keyset cursor unchanged (NULL never invalidated). Widens findOrphanPages(opts?: {sourceId?, sourceIds?}) (candidate-side scoping only; inbound links counted from any source). Pinned by test/sum-stale-chunk-chars.test.ts, test/embedding-signature-stale.test.ts, test/e2e/engine-parity.test.ts. Free-text alias layer: resolveAliases(aliasNorms, opts?): Promise<Map<string, Array<{slug, source_id}>>> (READ; maps each normalized alias to declaring (slug, source_id) pairs, source-scoped) and setPageAliases(slug, sourceId, aliasNorms) (WRITE; replaces the full alias set, delete-then-insert, empty clears, idempotent on the unique triple), called by the importFromContent ingest projection and the reindex --aliases backfill; parity across both engines, Pinned by test/search/page-aliases-engine.test.ts. searchVector in both engines injects the shared buildBestPerPagePoolCte per-page max-pool so a page surfaces on its strongest chunk. executeRawDirect(sql, params?, opts?) is the lock-hot-path sibling of executeRaw: same single-statement contract, but routes to the direct session-mode pool when dual-pool is active (Postgres/Supabase port 5432) so a long-held lock heartbeat survives the transaction pooler's per-transaction connection recycling; PGLite delegates straight to executeRaw (no pooler). Both engines implement it; the Minion lock path (claim/renewLock) is the consumer. reconnect(ctx?: {error?}) is a REQUIRED lifecycle method on BOTH engines: it recovers a dropped connection using the config captured at the last connect(), so callers (autopilot health probe, batchRetry) never disconnect() + bare connect() (which loses the config and throws database_url undefined forever, and opens a null-connection window). PostgresEngine rebuilds its pool with a _reconnecting reentrancy guard and emits pool-recovery audit; PGLite is single-writer in-process so it just restores the saved data dir for parity. Plus two interface members: (1) optional findDuplicatePage?(sourceId, {hash, frontmatterId?}): Promise<{slug, id} | null> (identity precedence is content_hash OR frontmatter->>'id', both with deleted_at IS NULL); (2) resolveSlugs(partial, opts?) extended with {sourceId?, sourceIds?} so the MCP fuzzy get_page path scopes by source (field names match sourceScopeOpts(ctx) output so handlers spread directly; no opts gives the unscoped behavior). Plus a stable tiebreaker ORDER BY score DESC, page_id ASC, chunk_id ASC in searchVector in both engines: on a score tie (basis-vector eval fixtures) older page_id wins, so a new index on pages cannot flip ranking on tied scores.

  • src/core/engine-constants.ts — single source of truth for engine batch-sizing constants. Exports DELETE_BATCH_SIZE = 500 consumed by both engines' deletePages + resolveSlugsByPaths and by the sync delete + rename loops. Lives outside engine.ts (the interface module) to avoid circular-import worry — bounded per-statement work for predictable lock hold time + write amplification. Also exports TRAVERSE_PATH_ROW_CAP = 5000, the raw-row bound on traversePaths/traversePathsDetailed (both engines LIMIT CAP + 1; the extra row is the truncation probe; counted before edge dedup).

  • src/core/background-work.ts — process background-work registry: the single owner of "drain every fire-and-forget DB-write sink before the engine goes away," with TWO exit points. registerBackgroundWorkDrainer({name, order, drain(timeoutMs, mode), abort?}) over a Map<name, BackgroundWorkDrainer> (idempotent registration by name; __registerDrainerForTest returns an unregister handle); mode is 'exit' | 'disconnect' ('exit' = CLI teardown, engine still live, residual buffers may flush; 'disconnect' = an engine is mid-disconnect, sinks await only IN-FLIGHT work and never start new writes). drainAllBackgroundWorkForCliExit({timeoutMs}) runs mode 'exit' with abort allowed; drainBackgroundWorkBeforeDisconnect({timeoutMs}) is called by BOTH engines' disconnect() so an in-flight statement settles before the underlying handle closes — PGLite's close() deadlocks PERMANENTLY with a statement in flight — and it NEVER calls abort() (permanent process state, wrong for a long-lived gbrain serve disconnecting one engine); a partial disconnect-drain warns once per sink to stderr. The module stays a zero-import leaf on purpose (both engines import it statically). Drains in explicit (order, name) order — facts FIRST (order 0) so its abort-path DB logIngest runs against the freshest live engine — and AWAITS abort() only when drain() reports unfinished>0 in 'exit' mode. Best-effort per drainer: one sink's failure never blocks the others or the disconnect. SIX sinks register at module import: facts/queue.ts (order 0; abort=shutdown() cancels a hung facts:absorb Haiku via internalAbort), last-retrieved.ts (order 1), search/hybrid.ts (order 2; awaitPendingSearchCacheWrites bounded via Promise.race), eval-capture.ts (order 3; captureEvalCandidate self-tracks its promise via awaitPendingEvalCaptures), context/volunteer-events.ts (order 4 — batched volunteer-event INSERTs, drained like the rest), search/telemetry.ts (order 5 — awaitPendingTelemetryFlush: awaits the in-flight flush in both modes, flushes residual buckets only in 'exit' mode so short-lived CLI calls land in search_telemetry on clean exit). Every cli.ts teardown site reaches it through finishCliTeardown (src/core/cli-force-exit.ts), which drains the registry before engine.disconnect(), so db.close() cannot race an in-flight job and pin the PGLite single-writer lock. Exports backgroundWorkSinkCount() so the teardown helper computes its backstop deadline from the registered sink count, plus the shared teardown budgets: MAX_TIMER_DELAY_MS (the 2312^{31}−1 setTimeout ceiling; process-watchdog.ts aliases it as MAX_WATCHDOG_TIMER_MS), SINK_DRAIN_TIMEOUT_MS (the per-sink drain bound, used as the runDrainers default), and pgliteCloseTimeoutMs() (the env-tunable in-loop close bound, defined here so cli-force-exit's computed deadline budgets the SAME bound the engine honors). CLI-EXIT-ONLY: the facts shutdown() abort is permanent process state, never call in a long-lived gbrain serve. Companion changes: src/core/ai/gateway.ts withDefaultTimeout(caller, ms) bounds every outbound AI call (chat 300s, embed+multimodal 60s; env GBRAIN_AI_{CHAT,EMBED,MULTIMODAL}_TIMEOUT_MS; composed with caller signals via AbortSignal.any) and the teardown backstop honors an errored op's exit code so a hung disconnect can't mask failure as success (see cli-force-exit.ts); src/core/postgres-engine.ts reconnect() module-mode branch re-establishes via idempotent db.connect() + connectionManager.setReadPool refresh instead of db.disconnect() (no null window for concurrent ops; fail-loud on real connect failure); src/core/search/hybrid.ts embedQueryBounded + a shared QueryEmbedDeadline (6s, floored 2s per embed via MIN_QUERY_EMBED_BUDGET_MS; env GBRAIN_QUERY_EMBED_TIMEOUT_MS) bounds the cache-lookup AND inner query embeds so a stalled provider falls back to keyword instead of stalling the whole op. Pinned by test/core/background-work.test.ts, test/search/query-embed-deadline.test.ts, test/eval-capture-drain.test.ts, test/e2e/postgres-reconnect-singleton.test.ts, test/e2e/pglite-cli-exit.serial.test.ts, test/fix-wave-structural.test.ts.

  • src/core/search/read-policy-sql.ts — shared internal concrete-page SQL policy (PageReadScope in types.ts): nonempty federated grants precede scalar source; excludePrivate is resolved by ops/context.ts:readPolicyOpts, never an MCP override. Optional live-page filtering excludes deleted/quarantined pages and archived sources. Both engines apply policy inside content, version, timeline, graph and identity reads before limits. While page privacy is enforced, history requires both current-page and snapshot visibility. Links/timeline projections independently authorize originating pages.

  • src/core/search/read-enrichment.ts — narrow query helpers shared by PGLite and Postgres. Recursive relational reads authorize seeds, hops and link origins before limits; seed identity constraints remain separate from caller grants. Batched IDs or (source_id, slug) pairs authorize backlinks, adjacency, dates, flags, extraction state and take-derived salience. Restricted holders use permitted active takes and zero stored emotional-weight contribution; required expert admission runs after optional enrichment, uses an authorized effective-date map and propagates query failures. Callers omit failed optional signals or apply their existing neutral defaults.

  • src/core/search/safe-chunks.ts — internal remote chunk-read policy and current safe-index version. Remote reads require a completed index built from sanitized inputs; existing chunks remain available to trusted local callers while awaiting rebuild. Both engines apply the predicate before chunk ranking and limits, independently of page-visibility opt-outs. Import seals the index only after replacing chunks in its transaction; body/chunk mutations invalidate it. Uses the existing pages.chunker_version column, with no schema migration.

  • src/core/remote-body.ts — linear protected-body sanitizer over the existing Facts parser/renderer. Processes every protected fence, retains only world Facts, removes all Takes and drops malformed protected sections. Chunk creation sanitizes full bodies before splitting. Page/history responses apply it whenever trust is not explicitly local, independently of page-visibility opt-outs and holder grants.

  • src/core/search/query-cache.ts — stored semantic-cache data and maintenance remain, but semanticResultCacheAvailable() is false. hybridSearchCached bypasses both result lookup and writes regardless of configuration or useCache; request-local query embeddings still work. Search metadata, cache_stats and the mode dashboard report effective caching as disabled. Historical hit counts and search_telemetry remain intact. Restoration requires complete response-dependency provenance.

  • src/core/ops/insights.ts — stored contradiction reports load only for explicitly trusted, unscoped local callers. Other callers receive the fixed availability note and an empty contradictions list before any report storage access. Expert search threads page and holder policy and performs final authorized-page admission after enrichment.

  • src/core/search/graph-signals.ts — per-query graph-signals helper. applyGraphSignals(results, engine, opts) runs as the 4th post-fusion stage (after backlink/salience/recency). Three boosts: ADJACENCY_BOOST=1.05 (page linked from 2+ OTHER top-K results — local hub for THIS query), CROSS_SOURCE_BOOST=1.10 (page linked from 2+ DIFFERENT sources — corroborated across team brains, dormant in single-source brains), SESSION_DEMOTE=0.95 (3+ results from same chat session — keep the highest-scoring at full score, demote the rest). All three inherit the floor-ratio gate preventing weak pages from being boosted past strong ones via popularity. computeScoreDistribution(results) emits min/p25/p50/p75/p95/max + reorder_band_width. sessionPrefix(slug) extracts the chat-session anchor (chat/2026-05-15-...). Pure pairedBootstrapPValue(deltas, resamples, rng) exported for eval gates. Test seam via adjacencyFn DI. Fail-open: any error logs via logGraphSignalsFailure (JSONL audit via audit-writer) and returns the input array unchanged. Pinned by test/search/graph-signals.test.ts (incl. the IRON-RULE floor-gate guard).

  • src/core/search/explain-formatter.ts — renders SearchResult[] as a multi-line per-result breakdown for gbrain search --explain. Reads every boost-stamping field; also prints the raw query↔chunk cosine (SearchResult.cosine, the calibrated signal evidence keys off) next to the blended score when present — absent on keyword-only / no-embedding paths. Handles the "no boosts applied" empty path. 4-decimal precision with trailing-zero strip. formatResultsExplain(results, meta?) prepends header lines from the captured retrieval meta when present — formatAutocutSummary (the autocut decision) and formatDegradedSummary(meta.degraded), which renders the closed degraded[] vocabulary as degraded: reranker_skipped (no_key) (null when the run was clean); cli.ts:formatResult threads lastRetrievalMeta into it, so a silently skipped reranker is visible from the CLI. Pinned by test/search/explain-formatter.test.ts + test/cli-explain-degraded-render.test.ts.

  • src/core/search/mode.ts — Named search-mode bundles and retained semantic-cache key machinery; persisted result reuse remains disabled. MODE_BUNDLES (conservative/balanced/tokenmax) and the resolution chain (per-call SearchOpts → per-key search.* config → bundle → balanced fallback) resolve every search knob; knobsHash folds registered search knobs into the query_cache key, and KNOBS_HASH_VERSION (exported from this file — the single source of truth for the current cache-key version) is bumped whenever a new knob shapes results so stale cache rows become unreachable. graph_signals: boolean knob in ModeBundle (defaults: conservative=false, balanced=true, tokenmax=true). KNOBS_HASH_VERSION appends a gs= parts entry per the cache-key contamination convention so a graph-on cache write can't be served to a graph-off lookup. SearchKeyOverrides + SearchPerCallOpts + loadOverridesFromConfig + SEARCH_MODE_CONFIG_KEYS + resolveSearchMode + attributeKnob all carry the field. Opt-out: gbrain config set search.graph_signals false. query_cache rows written under an older hash version hash differently — natural row segregation, cleared within cache.ttl_seconds (3600s default). title_boost: number | undefined knob in ModeBundle (default 1.25 for all three modes; multiplier for the post-fusion title-phrase boost). Override chain: per-call SearchOptssearch.title_boost config (clamped [1.0, 5.0]) → bundle. KNOBS_HASH_VERSION appends a tib= parts entry so a title-boost-on cache write can't be served to a title-boost-off lookup. SEARCH_MODE_CONFIG_KEYS gains search.title_boost. Cross-modal knobs in ModeBundle: cross_modal_both_text_weight/cross_modal_both_image_weight (weighted RRF for 'both' modality, defaults 0.6/0.4), image_query_text_refinement_weight/image_query_image_refinement_weight (hybrid intersect for searchByImage query refinement, defaults 0.4/0.6), unified_multimodal + unified_multimodal_only (unified-column routing flags), cross_modal_llm_intent (opt-in LLM escalation). SEARCH_MODE_CONFIG_KEYS carries the corresponding config keys, and the modality knobs participate in knobsHash so a cached text-mode result can't be served to an image-mode caller. Retrieval-quality knobs autocut_min_top (default 0.35 in all three bundles; config search.autocut_min_top; folds into knobsHash as an acm= part) and evidence_cosine_floor (default 0.8 in all three bundles; config search.evidence_cosine_floor; labels evidence — result-set-shape-neutral, so not hashed) ride the same bundle → config → per-call chain. keywordOrFallback: boolean knob in ModeBundle (default true in all three bundles; config search.keywordOrFallback): the keyword arm's AND→OR zero-recall retry — corpora the FTS config can't stem (CJK/agglutinative text under 'english') can turn it off because the OR retry has no IDF demotion and its common-token hits read as noise in the RRF blend; folds into knobsHash as a kof= part. KnobsHashContext additionally carries the EFFECTIVE per-call salience/recency boost modes (sal=/rec= — explicit SearchOpts ?? the classifier's auto-suggestion, resolved by the same chain bare hybridSearch uses) and the per-engine search.intent_patterns config fingerprint (ipat=, from query-intent.ts's intentPatternFingerprint; threaded through ctx, never read process-globally, so a multi-engine process can't key one brain's rows under another brain's patterns) — a reordered or reclassified write can never serve a differently-configured lookup. Cache HITS slice the stored page to the same resolved result count as misses (per-call limit → the mode's searchLimit) in hybridSearchCached, so a mode's result-count knob holds on both paths; nonzero offsets (positive OR negative) skip the cache entirely. expansion_variant_budget: number | null knob in ModeBundle (null in all three bundles; config search.expansion_variant_budget accepts legacy/null or a number in (0, 4] through the ONE range contract normalizeExpansionVariantBudget in fusion-lists.ts — out-of-range falls through to the bundle; per-call HybridSearchOpts.expansionVariantBudget, normalized by the same function in both the inner search and the cache resolver): the total RRF weight the NON-EMPTY expansion variant/clause lists share at fusion (weight_i = b / n_voting_arms; the original list always keeps weight 1), null = legacy weight 1 on every list (byte-identical fusion), a no-op when expansion is off. It folds into knobsHash as the last, append-only evb= part (legacy or the budget to 3 decimals) — KNOBS_HASH_VERSION is 29 with that part in — so a budget-weighted write can never serve a legacy lookup. KNOB_NULL_LABELS + formatKnobValue in modes-report.ts render its legitimate null as legacy (null) in gbrain search modes (plain (undefined) stays reserved for genuinely unset knobs). relational_rerank_pin: number knob in ModeBundle (3 in all three bundles — a no-op under conservative, which has no reranker; config search.relational_rerank_pin accepts off/0 or an integer in [0, 10] through the ONE range contract normalizeRelationalRerankPin in relational-rerank-pin.ts, out-of-range falls through to the bundle; per-call SearchOpts.relationalRerankPin, normalized by the same function in both the inner search and the cache resolver): how many relational-arm rows pinRelationalRows re-pins above the reranked text rows after applyReranker. It folds into knobsHash as the append-only rrp= part, riding KNOBS_HASH_VERSION 29 together with evb= (one bump per wave — both parts landed before v29 shipped; a partial-knobs literal without the field hashes as the bundle default), so a pin-3 write can never serve a pin-0 lookup. Pinned by test/search-mode.test.ts (bundle snapshots + hash pins), test/search/knobs-hash-reranker.test.ts (the version comment chain), test/config-adaptive-return-keys.test.ts, test/search/modes-report-coverage.test.ts.

  • src/core/search/modes-report.tsbuildModesReport(engine)SearchModesReport (schema_version: 2), the read-only dashboard behind gbrain search modes and the search_modes MCP op: active_mode + validity, the per-knob resolved attribution (attributeKnob over KNOB_DESCRIPTIONS, including the five reranker_* knobs), the three frozen bundles, config_keys, and reranker_readiness{model, enabled, ready, required_key, key_present, sunset_passed, self_hosted, fix} from rerankerReadinessForEngine + describeRerankerFix (a thrown readiness check still yields a ready: false verdict whose fix says to run gbrain doctor; the block never vanishes silently). redactReadinessForRemote(report) is what the search_modes op returns when ctx.remote !== false: required_key, key_present and fix are dropped and self_hosted is forced false — which env vars exist on the host is fingerprinting data and the fix names them; ready stays because it is observable anyway (reranked results carry rerank_score). src/commands/search.ts:formatModesText renders the runtime Reranker: line (off (resolved) — … / <model> (enabled) — <KEY> present / <model> (enabled but NOT running) — <fix>) and a per-bundle reranker=… topNIn=… autocut=… line. Pinned by test/modes-report-reranker.test.ts.

  • src/core/context-engine.ts + src/openclaw-context-engine.ts — the deterministic context engine OpenClaw loads on every turn (assemble() injects the Live Context block, zero-LLM). createGBrainContextEngine({workspaceDir, resolveEntities?}) accepts an OPTIONAL host-injected resolver (ENGINE_API_VERSION 0.3.0, additive — older hosts work unchanged; the plugin entry maps ctx.resolveEntities/ctx.brainQuery onto it). Checkpoint compaction: compact() runs a time-bounded (8s), fail-open, lazily-imported checkpoint step BEFORE delegating — spools the since-last-boundary window (openclaw tail reader over the exported adapter mapper, 40-turn cap = the no-prior-boundary fallback) as a content-addressed corpus segment + ledger entry, then rung 2 (PGLite: one bankOnly+flushCorpusFile IPC round trip to serve) or rung 3 (Postgres: inline harvest over the reflex ladder's exported getDirectPostgresEngine singleton, under sweep claim fencing + capability/kill-switch gates, abort post-check) — and rides an additive result.gbrain_checkpoint bag on the delegate's return (ownsCompaction stays false). After the segment is spooled, the Memorable receipt lane runs (gate + stamp via memorableGateAllowed, span-filtered redacted tool calls, recordAndRelayReceipt with harness: 'openclaw' — per-compaction capture, content-hash dedup on retries, fail-open for the checkpoint; the whole block is skipped once the host's compact() deadline has fired; the rendered segment text is re-scanned with highEntropy at receipt time and the relay REFUSED fail-closed on any hit — the segment rendering feeds the ledger hash and cannot change, so the next compaction window re-evaluates; this lane NEVER compacts the receipts file — the hook lane is the ONE compactor, single-rewriter rule — but DOES trim the relay file, since an openclaw-only host has no hook lane and converging newest-keeping trims make concurrent trims safe). assemble() consumes sessionId ?? sessionKey and splices a deterministic envelope-bearing Compaction-checkpoint block at parts[1] (after Live Context) from the banked manifest via an in-process memo + hash-keyed polls (≤5; stale manifests can't satisfy a poll for the new segment; no manifest ⇒ byte-identical output). assemble() runs the Retrieval Reflex after the Live Context block: extracts the current turn's user text, builds prior-context text (every message EXCEPT the current turn — suppression must not see the triggering mention), passes the rolling window (getWindowTurns, last 12 user/assistant turns; the reflex slices to its configured retrieval_reflex_window_turns), and appends the pointer block. warmReflex() fires at construction.

  • src/core/context/ — Retrieval Reflex (Layer 1). entity-salience.ts: pure, zero-LLM, precision-biased extractCandidates(text) (capitalized runs + @handles, STOPWORDS + soft COMMON_WORDS + sentence-start guard, deterministic, capped; a lowercase weak pass additionally emits weak: true candidates — lowercase words ≥3 chars on a separate MAX_WEAK_CANDIDATES=32 budget that never evicts strong candidates — which downstream may resolve through the alias arm ONLY) + extractCandidatesFromWindow(turns) (merges per-turn extraction across the last N turns by normalizeAlias form with occurrence/newest-turn/user-mention metadata; salience-ordered — recency > frequency > user-role — so the cap drops stale assistant chatter first). retrieval-reflex.ts: resolveEntitiesToPointers(engine, sourceId, candidates, opts) — alias arm (resolveAliases, caught per-arm for pre-v110 brains) + exact title/slug-suffix arm (real slugs are namespaced people/x but slugify drops the prefix) + two lexical identity arms behind opts.lexicalArms (kill switch: config retrieval_reflex_lexical_arms / env GBRAIN_RETRIEVAL_REFLEX_LEXICAL_ARMS, default on): weak lowercase candidates probe the ALIAS arm ONLY (never title/slug-suffix, where ordinary lowercase words would fabricate pointers) and require GLOBAL uniqueness across all sources in play; the surname arm resolves a strong single capitalized token ≥3 chars via a lower(title) LIKE '% <token>' predicate on the same query, with exact-arm precedence and ambiguity counted over ALL person rows carrying the surname — ambiguity in either arm injects nothing; pointers carry source_id/arm/confidence/matchedNorm (ARM_CONFIDENCE alias 0.9 / title 0.8 / title-surname 0.72 (deliberately above the volunteer layer's 0.70 gate, below title) / slug-suffix 0.6 lives next to the arm definitions; arm-2 provenance classified in JS since the combined OR can't report which predicate matched); opts: sourceIds? federated scope (alias arm loops per source, arm 2 uses source_id = ANY), suppression? ('slug-and-title' default; 'slug-only' REQUIRED under windowing — the title rule would suppress every entity merely mentioned in a prior window turn), ambient-channel event logging is DELIVERY-side, not in-resolver — logDeliveredReflexPointers(engine, pointers) fires only once a block is actually handed to the consumer (serve's resolve-IPC onDelivered hook post-write; buildReflexAddition post-timeout on the direct rung), so abandoned/timed-out blocks never pollute the volunteered-vs-used stats; its event write is registered synchronously before return so the CLI background-work drain cannot miss it; synopsis runs through stripTakesFence/stripFactsFence (the same privacy boundary get_page applies) so private facts never reach the prompt; capped at MAX_POINTERS. reflex.ts: the orchestrator + engine-aware resolver ladder (host resolveEntities → PGLite serve IPC → Postgres cached process-singleton → disabled), zero-candidate fast path, fail-open + timeout, heartbeat write for the doctor check, reflexEnabled(cfg) (file/env gate, default ON; DB-plane does NOT gate — assemble() is sync); windowed extraction when windowTurns present and retrieval_reflex_window_turns (default 4; 1 = single-turn behavior) > 1 — switches suppression to slug-only; accept-side reflex-channel logging fires after the per-turn timeout admits the block (direct-Postgres rung only — IPC logs server-side at delivery; host-injected resolvers are a documented gap). resolve-ipc.ts: local unix-socket resolve protocol (client + server) so PGLite resolves through the single connection gbrain serve holds (a second opener would hit the exclusive lock; a subprocess would force-steal it past the 5-min staleness window and crash). Wired into both serve transports via src/mcp/resolve-ipc-binding.ts (engine-uniform; socket + secret keyed off hash12(database_url) under ~/.gbrain/run, cleaned up on shutdown). Doctor surface: retrieval_reflex_health in src/commands/doctor.ts (reads the heartbeat for truthful runtime status; categorized in doctor-categories.ts) + volunteer_channels (engine-aware sibling: groups context_volunteer_events by channel over 7 days so operators see which push channels — reflex/op/watch/claude-code/codex — actually fire; info-only; the LOCAL doctor runs it brain-wide while the remote report path threads the caller's source scope, so a source-bound token never sees other sources' activity counts/timestamps; counts are reconciled against the hook heartbeat over the same 7-day window — a mostly-degraded week gets a CAUTION note, since a server-side delivery count isn't proof of injection; quiet-channel guidance is engine-aware — Postgres brains are told the hook lane is quiet by design rather than to chase registration — and walks both quiet classes (installed-but-unregistered vs registered-but-quiet; the check can't inspect registration itself); pre-v117 tolerant, and transient DB errors are reported as such, never as an old schema; pinned by test/doctor-volunteer-channels.test.ts). Config: retrieval_reflex + retrieval_reflex_max_pointers + retrieval_reflex_window_turns + retrieval_reflex_lexical_arms in src/core/config.ts (env GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS, GBRAIN_RETRIEVAL_REFLEX_LEXICAL_ARMS). volunteer.ts: parseWindow (lenient user:/assistant: prefixes, unprefixed → one user turn), volunteerContext (extract → resolve → +0.05 multi-turn/newest-turn boost → min_confidence 0.7 gate → cap 3/5; deterministic rationale strings, never raw conversation text; slug-only suppression), volunteerUsageStats (per-arm/channel precision from the pages.last_retrieved_at > volunteered_at join — APPROXIMATE: the 5-min last-retrieved throttle causes false negatives, unrelated reads false positives). volunteer-events.ts: insertVolunteerEvents (ONE multi-row parameterized INSERT), logVolunteerEventsFireAndForget + bounded drain registered as the volunteer-events background-work sink (order 4), purgeStaleVolunteerEvents (90-day GC, called from the dream cycle's purge phase). Policy layer ships as the retrieval-reflex recipe (recipes/retrieval-reflex/). Pinned by test/context/entity-salience.test.ts, test/retrieval-reflex.test.ts, test/retrieval-reflex-pre-v110.test.ts, test/context/resolve-ipc.test.ts, test/doctor-retrieval-reflex.test.ts, test/volunteer-context.test.ts, test/e2e/volunteer-context-postgres.test.ts. Checkpoint-compaction modules in this dir: corpus-segments.ts — engine-free content-addressed compaction segments (<session>.seg-<hash24>.txt, idempotent by name; parsers accept 12–64 hex), the hash-keyed per-session ledger (atomic tmp+rename, fail-open, entry order = the only ordinal), sliceBoundaryWindow/splitByBoundaries, the redacted renderSegmentText (a segment is NEVER written unscanned — unlike the session-end full write, which degrades-and-writes), exact-set coverageComplete/decideCorpusMode (count equality can be fooled by a duplicated boundary), the compact-time bankCompactSegment step (per-step deadline degrades, segment-then-ledger crash order), the openclaw tail boundary reader (readOpenclawBoundaryTail, delegates mapping to the adapter's exported mapOpenclawLine; also returns turn-stamped toolCalls/toolCallTurnIndexes for the Memorable receipt — name-only v1, input: null by design until an observation run characterizes OpenClaw's args field), and orphan-sidecar/aged-ledger GC — pinned by test/corpus-segments.test.ts. checkpoint-harvest.ts — the serve-side prompt harvest of a segment: bounded FIFO (cap 8, 60s abort), sweep-shared claim fencing, capability-then-kill-switch gates, signal.aborted POST-check (the pipeline returns partials on abort; an aborted run writes nothing and stays retryable), receipt sidecar before idempotent manifest publish (source-scoped getPage verification — a link that resolves to nothing is never banked), .ingested last, explicit shutdownCheckpointHarvest() called by serve BEFORE engine.disconnect() (the background-work drain is CLI-exit-only by contract) — pinned by test/checkpoint-harvest.serial.test.ts. hook-heartbeat.ts — the hooks telemetry JSONL extracted from hook.ts (hook.ts re-exports; serve-side writers never import the command module); allowlist carries the checkpoint segment/inserted/duplicate/links count-only fields. session-state.ts also carries the v132 checkpoint_manifest helpers (getCheckpointManifest/appendCheckpointManifest: newest-first, dedup-by-slug, cap 20, seg-hash completion key, fail-open on pre-v132 schema) — pinned by test/checkpoint-manifest.test.ts. sensitivity-scan.ts + compile-view.ts — the compile-context stack: composed detector (secret-scan + ordered PII_PATTERNS + path/blocklist families + operator pattern file; uniform .gbrain-scan-allow fingerprint escape hatch; CONTENT hits drop an entry, CONFIG/loader failures THROW so the caller aborts without writing) and the deterministic compiled-view builder (recency decay anchored to the newest candidate updated_at, never wall-clock; total-order (score desc, slug asc); source-scoped listPages/getPage reads that skip op-layer write-backs; whole-file packToBudget math that never passes a <=0 budget) — pinned by test/sensitivity-scan.test.ts + test/compile-view.test.ts + test/e2e/compile-context-pglite.test.ts; the CLI shell is src/commands/compile-context.ts (targets claude-code|codex|openclaw, AGENTS.md managed-marker splice that throws on damaged markers, atomic writes, --check recompile-and-compare exit codes; the guide is docs/guides/checkpoint-compaction.md). It also owns the whole Memorable seam's engine-free plumbing: the session-receipt JSONL (session-receipts.jsonl, appendSessionReceipt — dedup by post-redaction content_hash so a resumed session neither duplicates its receipt nor re-fires the relay; tail-rewrite compaction triggered by the 32 MB byte ceiling alone — the read fires exactly when the pass will act, with the 2000-line budget enforced inside the pass — and a skipCompaction seam the openclaw compact() lane sets unconditionally: the hook lane is the ONE receipts compactor, so two processes can never race the read-filter-rename rewrite); ONE bounded adaptive readJsonlTail reader shared by lastReceiptMatches, lastRelayResult and readSessionReceiptsTail (a receipt line can exceed the 1 MB window — the window doubles rather than reading "empty"); maybeTrimRelayResults (BOTH capture lanes trim the child-appended memorable-relay.jsonl — an openclaw-only host has no hook lane, and converging newest-keeping trims make concurrent trims safe: mtime-stale preferred, hard 8 MB force backstop, line-boundary cut keeping the newest lines — the child appends open-by-path, so the residual loss window is one in-flight write); the CONSENT STAMP (memorable-consent.json, 0600, written ONLY by config-set's disclosure flow — deliberately outside config.json, which the external CLI full-file-rewrites; scope-bound to the disclosure sha256 + MEMORABLE_CAPTURE_HARNESSES, so widening the capture surface invalidates old stamps); memorableGateAllowed(cfg) ({allowed, reason: kill_switch|disabled|disclosure_missing} — one reason vocabulary for hook, context-engine and doctor); memorableConsentEvidence() (the CLI-side opt-in read fail-closed from ~/.memorable/config.json, dated provisional spec note, GBRAIN_MEMORABLE_CONFIG test seam); redactedToolCallsJson (the security-load-bearing span-filter + highEntropy redaction, ONE implementation for both capture lanes; string LEAVES are redacted raw before serialization — scanning the serialized JSON would let quoted secrets through via " escaping — with a post-serialization re-scan as belt-and-braces); recordAndRelayReceipt (per-receipt harness scope-binding first — a harness outside MEMORABLE_CAPTURE_HARNESSES is refused with memorable_harness_undisclosed → receipt → prior-run outcome surfacing with the child's reason clamped via the shared clampRelayCause (32 chars, so the 16-char prefix keeps composites inside the 48-char reason bound) and appended LAST → consent evidence → resolveMemorableBin → detached spawn; never throws; spawnFn test seam); and priorRelayFailure, which surfaces the PREVIOUS relay run's self-reported exit status so a relay that records nothing is never reported healthy. resolveMemorableBin does the pre-spawn PATH/MEMORABLE_BIN resolution so a missing CLI is a named heartbeat reason rather than a silent async ENOENT.

  • src/commands/watch.tsgbrain watch: the push transport. Reads turns from stdin as they arrive (user:/assistant: prefixes; unprefixed = user turn), keeps a rolling in-process window (--window-turns, default 4), calls volunteerContext per turn, streams pointers to stdout (--json for JSONL with turn attribution), logs channel: 'watch' events with a per-session id. Session dedupe feeds already-pushed slugs back as priorContext so the core's slug-only suppression dedupes. Blocks in the stdin iteration (interactive alive until Ctrl-C/Ctrl-D; piped ends at EOF) — deliberately NOT in DAEMON_COMMANDS; SIGINT closes the stream so teardown flows through finishCliTeardown. Per-turn resolution failures are fail-open. Registered in CLI_ONLY + CLI_ONLY_SELF_HELP + THIN_CLIENT_REFUSED_COMMANDS (thin clients use the volunteer_context MCP op). Pinned by test/watch-command.test.ts.

  • src/commands/integrations.ts — recipe install. The resolver-row install fence is keyed by manifest.recipe (gbrain:<recipe>:resolver-rows), so a second copy-into-host-repo recipe never writes a block mislabeled with the first recipe's name. Pinned by test/integrations-install.test.ts. Health-check DSL includes the staleness-aware heartbeat_max_age type: declares the sense's expected cadence (max_age: 48h), and integrations doctor FAILS when the newest heartbeat event is older — the only check type that catches a green-but-dead sense (all others are point-in-time). Not embedded-gated (reads only the local heartbeat file). Recipe frontmatter carries output_paths (repo-relative dirs the collector writes, e.g. calendar-to-brain → daily/calendar/); getConfiguredCollectorOutputs() surfaces them for the db_only-collision check/warning. Pinned by test/integrations-heartbeat-max-age.test.ts. Standalone integration recipe management (no DB needed). Exports getRecipeDirs() (trust-tagged recipe sources), SSRF helpers (isInternalUrl, parseOctet, hostnameToOctets, isPrivateIpv4). Only package-bundled recipes are embedded=true; $GBRAIN_RECIPES_DIR and cwd ./recipes/ are untrusted and cannot run command/http/string health checks.

  • src/core/audit/audit-writer.ts — shared JSONL audit primitive behind the audit modules. Exports createAuditWriter({kind, recordSchema}) returning {log, readRecent} plus shared helpers computeIsoWeekFilename(kind, now?) and resolveAuditDir() (honors GBRAIN_AUDIT_DIR). ISO-week file rotation; best-effort writes (stderr warn on failure, never throws); read-path scans current-week + previous-week files for boundary spans. Built on it: src/core/rerank-audit.ts, src/core/audit-slug-fallback.ts, src/core/minions/handlers/shell-audit.ts, src/core/minions/handlers/supervisor-audit.ts, src/core/facts/phantom-audit.ts. The graph-signals-failures audit (logGraphSignalsFailure) uses the same primitive. src/core/skillpack/audit.ts is the one audit that does not use it. Pinned by test/audit/audit-writer.test.ts.

  • src/core/cli-force-exit.ts — single owner of one-shot CLI exit + teardown, designed as a PAIR with the import.meta.main seam at the bottom of src/cli.ts. finishCliTeardown({engine, drainTimeoutMs?}) is teardown-ONLY (never exits on the clean path): arms a REF'D backstop (unref'd would let a hung teardown exit naturally, skipping the flush and exiting with whatever PGLite scribbled into process.exitCode) whose deadline is COMPUTED from the bounds it guards (computeTeardownDeadlineMs = sinks × drainTimeoutMs + sinks × SINK_DRAIN_TIMEOUT_MS disconnect-drain bound + the RESOLVED PGLite close bound (pgliteCloseTimeoutMs() from background-work.ts — an operator-raised GBRAIN_PGLITE_CLOSE_TIMEOUT_MS widens this backstop too, never a hardcoded copy of the default) + facts-abort grace + 2 × pool-end bound + slack, floor 10s — the disconnect-drain and close terms budget engine.disconnect()'s own drain pass and bounded close so the backstop can't fire while every component honored its own bound; GBRAIN_TEARDOWN_DEADLINE_MS env override is the incident escape hatch), drains every background-work sink, disconnects the engine (a throw is warned + swallowed — the exit code reports the OPERATION, not the cleanup), then returns. The exit VERDICT lives in a gbrain-owned channel (setCliExitVerdict/currentExitCode; mirror-writes process.exitCode but NEVER reads it back) because PGLite's Emscripten runtime scribbles its own status into process.exitCode at arbitrary points mid-run — every writer that means to set the CLI exit code (op-dispatch catch, reindex, frontmatter, transcripts, brainstorm, autopilot, doctor's FAIL verdict, extract, and cli.ts's swept inner exits — friction, claw-test, smoke-test, the no-DB eval runners, status/status-thin, whoknows-thin) calls setCliExitVerdict; test/cli-exit-verdict-pin.test.ts greps src/ so the next raw process.exitCode = write fails CI instead of silently reporting success on failure. The deadline arms at TEARDOWN start, never before the op handler (arming before the handler would measure handler + teardown combined, taxing PgBouncer deployments with a flat force-exit deadline on every query and killing any long op mid-run with exit 0). All nine cli.ts disconnect sites route through it; the ONE process exit happens in cli.ts's main().then/catch via flushThenExit(currentExitCode()), gated by shouldForceExitAfterMain() (daemon list: serve) — the CLI never waits for Bun's event loop to drain, because endPoolBounded deliberately races past stuck PgBouncer sockets that would keep it alive. flushThenExit(code) fences stdout+stderr (write('', cb) raced with an unref'd guard, EPIPE-safe both sync and async) then holds a REF'D aliveness grace for non-TTY stdio before process.exit — Bun delivers queued pipe writes only while the process is alive (no flush API reaches process.stdout's native queue; write callbacks fire on accept, not delivery), so the grace IS the flush. Scope claim is deliberately cli.ts-only: command modules' mid-run engine lifecycles stay local (process-exit semantics inside them would be wrong) and are absorbed by the final explicit exit. Pinned by test/cli-finish-teardown.test.ts, test/flush-then-exit-harness.test.ts (real spawned-Bun pipe semantics), test/cli-should-force-exit.test.ts, test/cli-pipe-truncation.test.ts (real-CLI piped --tools-json byte-stable), test/cli-exit-verdict-pin.test.ts, the teardown describes in test/fix-wave-structural.test.ts + test/e2e/pglite-cli-exit.serial.test.ts, and test/e2e/pgbouncer-teardown.test.ts (CI transaction-mode pooler).

  • src/commands/search.ts:gbrain search statsgraph_signals section (enabled/source/failures_count/failures_by_reason). JSON envelope adds a graph_signals sibling property; _meta.metric_glossary adds graph_signals.enabled + graph_signals.failures_by_reason. Human output prints the section after the existing block. Reads search.graph_signals config first, falls back to the mode default. Pinned by test/search/search-stats-graph-signals.test.ts. Both gbrain search stats and gbrain search tune also surface a coverage disclosure (JSON: {cli_invocations: 'recorded_on_clean_exit', reason}; human: a one-line caveat) sourced from telemetryCoverage()/TELEMETRY_COVERAGE_CAVEAT in src/core/search/telemetry.ts: a short-lived CLI call's buffer flushes during the bounded CLI teardown drain, so a CLEAN exit is recorded; hard kills, drains that exceed their bound, and anything buffered when an engine disconnects outside the CLI teardown path still drop — the disclosure is display-only and reads its truth from the telemetry module's single-source constants. Pinned by the coverage-disclosure tests in test/commands-search.test.ts.

  • src/core/engine-factory.ts — Engine factory with dynamic imports ('pglite' | 'postgres').

  • src/core/pglite-engine.ts — PGLite (embedded Postgres 17.5 via WASM) implementation, all BrainEngine methods. The facts/takes/code-edges/salience method clusters are implemented in narrow-deps modules under src/core/pglite-engine/ (see the engine-module-dirs entry below); the class methods delegate to them and the façade keeps its full public surface. listLinkSources({sourceId?, sourceIds?}) returns distinct link_source provenances + counts (ORDER BY count DESC, link_source ASC NULLS LAST; scalar + federated scoped; parity with postgres-engine.ts) powering gbrain link-sources. addLinksBatch/addTimelineEntriesBatch/addTakesBatch pass the whole batch as one JSONB document via jsonb_to_recordset((\$1::jsonb)->'rows') (bound through executeRawJsonb with a { rows } wrapper; rows built by the shared src/core/batch-rows.ts helpers, NUL-stripped), and are batchRetry-wrapped. connect() wraps PGlite.create() in a try/catch that classifies the failure and, for the wasm-abort verdict on a persistent data dir (torn WAL/checkpoint state after an unclean shutdown; it presents like a macOS WASM bug but is not one), runs in-place auto-repair via attemptWalRepairAndRetry (static import from pglite-repair.ts, per the engine-live rule; the retry create is preservingProcessExitCode-wrapped; success sets the public walRepairReceipt field + prints buildWalRepairNotice to stderr and returns with the lock held). The seam never throws, so every non-repaired path funnels through the single lock-release-then-throw site; repair refuses when the lock was acquired by reaping (LockHandle.reaped'possibly-live-writer'), when disabled (GBRAIN_PGLITE_WAL_REPAIR=off), on layout-validation failure, or inside the post-failure cooldown. disconnect() early-nulls the handle (so no NEW statement can reach it), then runs drainBackgroundWorkBeforeDisconnect() so statements ALREADY in flight settle against the still-open handle — PGLite's close() deadlocks permanently (close's promise AND the in-flight query's promise never settle) with a statement in flight; the ordering is load-bearing (drain above the null would let a new statement race the close). Teardown defense is layered and honestly scoped: the drain PREVENTS the known wedge; the in-loop close bound (GBRAIN_PGLITE_CLOSE_TIMEOUT_MS, default 5000ms, floor 1s, ceiling 2312^{31}−1, read per call) covers ONLY a close that still yields to the event loop — armed BEFORE close() is called and deliberately ref'd, a timed-out close degrades to a once-per-process stderr warning naming both env knobs and teardown proceeds (an abandoned close's later rejection is swallowed; the WAL-repair path covers a zombie instance on next open); a close that WEDGES the loop (blocked or microtask-starved) can never be caught by any same-loop timer and is observable/killable only by the opt-in out-of-band watchdog — a diagnostic/incident instrument, not ambient production protection: GBRAIN_PGLITE_CLOSE_WATCHDOG_MS (unset/0 = off; a positive value clamps UP to max(5000,registeredsinkcount×2000+closetimeout+2000)\text{max}(5000, \text{registered}-\text{sink}-\text{count} \times 2000 + \text{close} \text{timeout} + 2000) with a warn so a units typo never SIGKILLs a healthy slow teardown) + GBRAIN_PGLITE_CLOSE_WATCHDOG_GRACE_MS (default 30000) arm the shared process-watchdog.ts worker around a PGLite disconnect with a live handle (armed after the early-return, before the drain; disposed in a nested finally so a releaseLock throw can't leak it; SIGTERM at deadline, SIGKILL at deadline+grace; lock-only teardown and postgres pool teardown are out of scope). Pinned by test/pglite-engine-disconnect.serial.test.ts + test/search-telemetry-disconnect-hang.serial.test.ts + test/pglite-disconnect-watchdog.serial.test.ts (spawned fixture that genuinely starves the loop) + the structural pins in test/fix-wave-structural.test.ts. searchKeyword/searchKeywordChunks multiply ts_rank by the source-factor CASE at chunk grain; searchVector is a two-stage CTE — inner CTE keeps ORDER BY cc.embedding <=> vec so HNSW stays usable, outer SELECT re-ranks by raw_score * source_factor, inner LIMIT scales with offset to preserve pagination. searchTakes/searchTakesVector take full SearchOpts and apply the standard source-scope predicates (federated sourceIds[] wins over scalar sourceId, via the joined page's source_id) alongside the holder allow-list — parity SQL in postgres-engine.ts; pinned by test/e2e/think-source-isolation-pglite.test.ts. initSchema() calls applyForwardReferenceBootstrap() BEFORE replaying SCHEMA_SQL — probes for forward-referenced state the embedded blob needs (pages.source_id, links.link_source, links.origin_page_id, content_chunks.symbol_name, content_chunks.language, sources FK target, plus files.source_id, files.page_id, oauth_clients.source_id, oauth_clients.federated_read, sources.archived, sources.archived_at, sources.archive_expires_at, timeline_entries.event_page_id — column-only, migration v121 stays the source of truth for its FK + indexes) and adds only what's missing; threads the DDL connection from initSchema so probes run inside the advisory-lock scope; no-op on fresh installs and current brains (this is what keeps an upgrade from wedging on a forward-referenced column). getBrainScore returns 100/100 with full breakdown (35/25/15/15/10) when pageCount === 0 (vacuous truth — empty brain has no coverage problem); Pinned by test/brain-score-breakdown.test.ts empty-brain assertion + test/doctor-report-remote.serial.test.ts. disconnect() uses snapshot+early-null (snapshot _db/_lock, null instance fields BEFORE any await so a concurrent connect() can't see a partial mid-close state) wrapped in try/finally guaranteeing lock-release even if db.close() throws; KEEPS close-then-release order (release-then-close was rejected: it would widen the window where a sibling process connects to a still-closing brain); Pinned by test/pglite-engine-disconnect.serial.test.ts. PGlite.create() runs inside preservingProcessExitCode: PGLite's Emscripten runtime writes its own status into process.exitCode (99 at create; in-memory brains run initdb whose status lands on a later tick; the exit status at close — and assigning undefined cannot clear a prior value), which would otherwise silently clobber an errored op's exit 1 back to 0. The wrapper keeps the global tidy for external readers; db.close() stays unwrapped (its 0-write is baseline behavior test runners depend on). The CLI's exit verdict is immune either way — it lives in the gbrain-owned channel in cli-force-exit.ts and never reads process.exitCode back. Exports classifyPgliteInitError(message): 'bunfs' | 'wasm-abort' | 'corrupt' | 'unknown' + buildPgliteInitErrorMessage(verdict, original, platform?, ctx?) + stringifyPgliteInitError(err) + buildWalRepairNotice(receipt) + the PgliteInitRepairContext type, routing the catch-block hint by failure shape (bunfs matches literal $$bunfs OR ENOENT[\s\S]*pglite\.data co-occurrence, surfaces a paste-ready bun upgrade + Node fallback; corrupt — 58P01/internal_load_library/missing vector type, catalog corruption WAL repair can't fix — stays matched BEFORE the wasm arm and routes to reinit-pglite; wasm-abort matches the real production shapes Aborted()/RuntimeError/unreachable plus legacy signatures, names the corrupt-WAL root cause + the recovery ladder (pglite-repair → rebuild → engine switch) + what auto-repair did per ctx incl. the honesty-critical failed-not-restored arm, and links the upstream tracking issue; unknown is platform-gated). stringifyPgliteInitError also surfaces message-less Emscripten objects (ErrnoError (errno N)) instead of [object Object]. Pinned by test/pglite-init-classifier.test.ts + test/pglite-wal-repair.serial.test.ts + test/fix-wave-structural.test.ts. Implements deletePages(slugs, {sourceId}) + resolveSlugsByPaths(paths, {sourceId}) via slug = ANY(\$1::text[]) array-param binding, caller-chunking primitive throwing when input exceeds DELETE_BATCH_SIZE, deletePages returns RETURNING slug rows so callers filter pagesAffected to confirmed deletes. Implements the embedding-signature stale-detection quartet — sumStaleChunkChars({sourceId?, signature?}), setPageEmbeddingSignature(slug, {sourceId?, signature}), invalidateStaleSignatureEmbeddings({signature, sourceId?}), widened countStaleChunks({sourceId?, signature?}) (the signature opt widens via JOIN pages p ... WHERE cc.embedding IS NULL OR (p.embedding_signature IS NOT NULL AND p.embedding_signature <> $signature), NULL grandfathered); parity SQL with postgres-engine.ts. PGLite-specific DDL (pgvector, pg_trgm, triggers). Engine-path helper dependencies (retry, ontology, recency decay) avoid dynamic import(); the only lazy dynamic imports are ai/gateway.ts in initSchema and _upsertChunksOnce, line-marked because their local catches preserve compiled-default and stored-config fallbacks that eager module evaluation would bypass.

  • src/core/pglite-embedded-assets.ts + src/core/pglite-embedded-asset-paths.ts — PGLite runtime-asset supply for every run mode (compiled binary, source checkout, bun-global install). pglite-embedded-asset-paths.ts is the bundler ANCHOR: five literal with { type: 'file' } imports of the wasm/data/extension-tarball assets via repo-relative node_modules paths (the package's exports map hides ./dist/*) — keep the specifiers literal and byte-stable, an expression or indirection silently stops the bytes embedding and the compiled binary falls into the Bun-vfs ENOENT failure. resolvePgliteAssetPaths() in pglite-embedded-assets.ts is the tiered resolver: tier 1 dynamically import()s the anchor (the rejection under a hoisted install — bun-global upgrades dedupe @electric-sql/pglite to the global root, making the anchor's specifiers unresolvable — is EXPECTED and routes to tier 2, which is why the anchor is not a static import); tier 2 derives the dist dir via module resolution of @electric-sql/pglite (import.meta.resolve then createRequire — the SAME specifier the engine imports, so assets can never come from a different pglite copy than the loaded JS); tier 3 throws one actionable error naming everything tried, with the canonical GitHub reinstall command (never the npm-registry name, which is squatted — see the README install warning). getEmbeddedPgliteOptions() memoizes one build per process and hands PGLite compiled WebAssembly.Modules + fs-bundle Blob + materialized extension tarballs via PGliteOptions. Pinned by test/pglite-embedded-assets.test.ts + test/pglite-hoisted-install.serial.test.ts (real bun add -g-shaped hoisted layout).

  • src/core/pglite-lock.ts — advisory data-dir lock so two processes never open the same embedded PGLite (single-connection WASM Postgres) at once. Atomic mkdir of .gbrain-lock/ + a lock file carrying {pid, acquired_at, refreshed_at, command, subcommand}. A held lock HEARTBEATS its refreshed_at every 30s (.unref()ed timer; informational). A waiting acquirer reaps a holder ONLY on affirmative proof of death — ESRCH from kill-0, or a ps-/proc-read command line proving the PID was recycled by a non-gbrain program under AFFIRMATIVE same-namespace proof (on Linux the lock's recorded pid_ns must be readable and equal ours, boot_id too when recorded; legacy locks without markers are never cmdline-reaped; unreadable cmdline and same-process PID also read as alive) — a LIVE holder is NEVER stolen, regardless of how stale its heartbeat is. Every reap serializes on an atomic claim dir (tryReapLockDir) and re-validates the victim's ownership token while the lock is still in place, so concurrent reapers can't delete each other's freshly installed locks; a crashed claimant's dir is broken after a 30s TTL / dead PID. A live gbrain serve holder is identified from the parsed subcommand and reported immediately with separate CLI-retry and MCP-tool choices; other live holders keep the bounded wait. gbrain sync normally never reaches this error under a live serve — the cli.ts preflight (src/commands/sync-delegate.ts) probes the lock file read-only and delegates the sync through the serve's IPC socket instead. The heartbeat runs on the JS event loop, which is blocked during long synchronous WASM imports/CHECKPOINTs, so a working dream/embed holder can look stale while alive; a steal-on-stale-heartbeat grace would let a second OS process open the same data dir and corrupt the catalog + pgvector extension (58P01 / internal_load_library / type "vector" does not exist), recoverable only by wipe+restore. A wedged-but-alive holder is never stolen: serve-tagged holders report immediately, while other holders time out with a message naming the PID. Each holder carries an ownership token (<pid>:<acquired_at>); the heartbeat and releaseLock verify the on-disk lock is STILL theirs before touching it. In-memory engines take no lock. There is deliberately NO same-process reentrancy or same-PID special case: a second acquireLock from the process that already holds the lock waits out the timeout like any other live holder (the classic cause is a command double-connecting a second engine on the same data dir; the fix is to reuse the connected engine at the dispatch layer, never to soften the lock). LockHandle.reaped marks an acquisition that reaped a prior holder's lock (dead-PID reap, PID-recycling reap, or corrupt-lock-file removal — the only reaps that exist); the WAL auto-repair gate refuses to run surgery on a reaped acquisition since a corrupt lock file cannot prove its holder is dead. A corrupt-lock reap ALSO writes a persisted marker (<dataDir>.lock-reap.json, read via exported msSinceLastReap) so the NEXT process's clean acquisition is still repair-quarantined for 10 minutes — the in-process flag alone would let the reaper's successor run surgery under a possibly-live writer; dead-PID reaps (affirmative ESRCH verdict; EPERM reads as ALIVE) deliberately skip the marker so dead-holder recovery stays one-failed-command-plus-one-re-run. Heartbeat refreshes write via tmp+rename (a torn in-place write could be read mid-flight by a polling acquirer and misclassify a HEALTHY live holder as a corrupt lock). Pinned by test/pglite-lock.test.ts. A corrupted store surfaces a reinit-pglite recovery hint via classifyPgliteInitError's corrupt verdict in pglite-engine.ts. Exported inspectLockHolder(dataDir): LockHolderInfo is the READ-ONLY inspection seam for status surfaces (gbrain engine status, smoke-test): never reaps, never mkdirs, never blocks; a dead-PID holder reads as not-held, a live serve holder reads as serve: true so a probe can report locked_by_serve instead of hanging on the single-writer lock. peekLock(dataDir) is the pure read of the same lock — no mkdir, no acquisition side effect, never throws LiveServeLockError — so a third-party caller can tell "held by a live serve" apart from "available" without taking the lock itself; it reports not-held for a missing lock dir, a dead-PID holder, or an unreadable lock file, and is deliberately outside gbrain's own acquisition path, which is unchanged. Exported as gbrain/pglite-lock.

  • src/core/pglite-resetwal.ts — pg_resetwal for PGLite NodeFS data dirs, in TypeScript (a port of the electric-sql/pglite pg_resetwal contribution, PR #994, Apache-2.0). Validates the PG17 pg_control layout fail-closed (WalResetUnsupportedError on any unsupported shape — PG_VERSION ≠ 17, control ≠ 8192 bytes, control version ≠ 1700, bad seg/block size), removes stale postmaster.pid + old WAL segments + archive_status/summaries entries, writes a replacement shutdown-checkpoint WAL segment + CRC32C'd pg_control. Both file writes are atomic + durable (tmp cleared then opened 'wx' so a pre-planted symlink at the predictable tmp name can never redirect the write, + fsync(tmp) + rename + fsync(parent dir)); write order is segment-first/control-last so a mid-write kill leaves a state that still fails startup and the next attempt re-runs (idempotent — a torn pair can never claim success). WAL segment size is capped at 64MB (pglite ships 16MB; the Postgres-general 1GB bound would let a corrupt-but-plausible control field drive a 1GB allocation on the repair path). Exports the shared PG17 layout literals (PG_CONTROL_FILE_SIZE, isWalSegmentName) consumed by pglite-repair.ts. LAYOUT COUPLING: any pglite bump past PG17 must revisit this file together with the ./vector export blocker (TODOS.md "pglite upgrade blocker" entry). Pinned by test/pglite-resetwal.test.ts.

  • src/core/pglite-repair.ts — WAL-repair orchestrator wrapping the resetWal port with the safety layers that make it runnable automatically from connect(): validateWalRepairTarget (read-only, fail-closed; refuses symlinked dataDir/pg_wal/global/pg_control — lstat follows INTERMEDIATE symlinks, so global/ itself must be checked or surgery would write pg_control through it into a foreign dir; tolerates the in-dir .gbrain-lock), rename-based backup (the ENTIRE pg_wal/ dir + postmaster.pid renamed into a sibling <dataDir>.wal-repair-backup-<ts>/, only the 8KB pg_control copied — zero transient disk cost), restoreWalBackup (overwrite order: control first via atomic tmp+rename, then a pg_wal dir swap with the reset dir set ASIDE inside the backup — nothing is ever deleted during restore; mtime guard refuses when a foreign segment is newer than the backup; a missing/empty backup NEVER reports restored:true), WalRepairError (thrown when resetWal fails AFTER the backup — carries the receipt + the best-effort restore's REAL result so the seam's restored flag and the failed-restored/failed-not-restored message arms never lie), a cooldown sidecar <dataDir>.wal-repair-attempt.json (skip 'recently-failed' inside GBRAIN_PGLITE_WAL_REPAIR_COOLDOWN_SECONDS, default 3600 — bounds the autopilot/supervisor reconnect loops) with episode-scoped backups (attempts within one corruption episode REUSE the episode's first backup — the pre-damage forensic state, honored only when the sidecar's path is a real non-symlink <dataDir>.wal-repair-backup-* sibling since the sidecar is user-writable JSON; retention keeps the newest 3 episodes, never pruning the open episode's), and attemptWalRepairAndRetry — the engine seam that NEVER throws (gates: kill-switch → reaped-lock 'possibly-live-writer' → 10-minute reap-marker quarantine (msSinceLastReap, cross-process) → validation → cooldown; then repair → retry create once → restore-and-record on failure; prints a repair-start stderr line so a timeout-killed attempt is self-explaining). inspectPgliteDataDir is the read-only diagnosis for gbrain doctor + pglite-repair --dry-run. Imports runtime values only from pglite-lock.ts/pglite-resetwal.ts/node:fs — never from pglite-engine.ts (no cycle; the engine statically imports THIS file per the engine-live rule). Pinned by test/pglite-repair.test.ts + test/pglite-wal-repair.serial.test.ts.

  • src/commands/pglite-repair.tsgbrain pglite-repair: the manual surface for WAL repair (--dry-run | --yes | --json | --path <dir>; CLI_ONLY + SELF_HELP; returns an exit code via setCliExitVerdict, never process.exit). Never connects an engine — works when the DB won't open and when auto-repair is disabled. --dry-run is strictly read-only. Its confirmation prompt (and src/commands/reinit-pglite.ts's) writes to stderr so --json stdout stays clean, refuses non-TTY stdin in-prompt (defense-in-depth behind the caller-side "Non-TTY environment requires --yes" guard), resolves false on EOF/close instead of parking forever on a closed or piped stdin, and cleans up its listeners; --yes/-y stays the non-interactive path. The real run validates BEFORE locking (acquireLock mkdirs the data dir — a typo'd --path must not create directories), refuses a live lock holder (pre-lock diagnosis names the PID; a live gbrain serve is called out), refuses a reaped acquisition (refused_reaped_lock — no --force by design: force-removing .gbrain-lock would reopen the concurrent-writer hole), re-validates under the lock, repairs with episode-backup reuse, and records the attempt in the sidecar. Pinned by test/pglite-repair-command.serial.test.ts.

  • src/core/pg-access-classify.ts — Postgres ACCESS-error classifier, the reason/remediation layer of the db-availability loop (detect → GBRAIN_DB_ACCESS marker → skills/db-repair → gbrain db-repair). Two entry points, one diagnosis type: classifyPgAccessError(err, ctx?) for thrown errors (data-driven ordered REASON_ROWS table, first match wins — specific rows before general ones) and diagnoseDbConfig(ctx) for errorless config-plane reasons (no_url, env_shadowed — there is no error object at connectEngine's no-config exit or under an env shadow; never fake one). The PgAccessReason union (16 reasons) is APPEND-ONLY — a compatibility surface once the bundled skills are in the wild, like progress phase names; consumers switch exhaustively with a never sentinel. Invariants: message/remediation are ALWAYS pre-redacted (safe for transcripts/receipts/issues); every fix descriptor is hardcoded or derived from the CURRENT config URL (deriveSessionPoolerUrl/deriveDirectUrl), never from anything parsed out of error text, and run_command argv[0] is always gbrain; remediation copy has exactly ONE home: this module — db-repair, doctor, and MCP dispatch all render diagnosis.remediation, skills reference the command and never duplicate recipe text. TWO-AXIS design note (mirrored in retry-matcher.ts's header — do not "fix" one side to match the other): retry-matcher answers "should I retry?" and deliberately treats password authentication failed as retryable (auth race during DNS failover); this module answers "what went wrong?" and reports the same error auth_failed + transient: false — both correct on their own axis; unknown reasons defer transient to isRetryableConnError. Also exports DB_ACCESS_MARKER_PREFIX/formatDbAccessMarker (the single source of the marker literal that skills/db-repair pins in both description: and triggers:; the action a reader takes is ALWAYS the hardcoded gbrain db-repair, never a command parsed from the marker), isDbAccessFailure (the ACCESS-class subset fail-open recall arms use to tell "one arm degraded" from "the DB is down"), and Supabase enrichment {pooler, projectRef, pausedSuspect} (paused projects aren't a distinct wire error — modeled as a suspicion flag on dns/timeout/tenant reasons). Pinned by test/pg-access-classify.test.ts.

  • src/commands/engine-status.tsgbrain engine status [--json] [--probe] [--brain <id>]: the ENGINE-FREE detection primitive (CLI_ONLY, dispatched in handleCliOnly BEFORE connectEngine like pglite-repair; deliberately NOT an operations.ts op — an MCP server can't serve a status call when its DB is down). Zero round-trips without --probe: reports (JSON schema_version: 1) effective_engine vs config_file_engine (can differ under a transient env URL), db_url_source, env-shadow + both-env-URLs precedence note, redacted URLs only, brain id from the engine-free resolver (NOT get_brain_identity, which needs an engine), thin-client flag, and a Postgres pooler block (supabase-pooler detection, resolvePrepare, direct/session-pooler derivability, GBRAIN_DISABLE_DIRECT_POOL, pool sizes). --brain <id>/mount resolution reports the MOUNT's engine/URL, never the host's. --probe is a SINGLE bounded connect + SELECT 1 (the driver's built-in connect_timeout — never a custom race; noRetry, never the 3-attempt ladder); Postgres success also reports ConnectionManager.describeMode(); failure returns a classified PgAccessDiagnosis. PGLite probe is LOCK-AWARE: a live serve holding the single-writer data-dir lock reports locked_by_serve (healthy-with-note) instead of hanging ~30s and misreporting a healthy brain. Exit 0 healthy, 1 on config diagnosis or failed probe, 2 on flag errors. Pinned by test/engine-status.test.ts.

  • src/commands/db-repair.tsgbrain db-repair [--yes] [--apply-rewrites] [--json] [--force] [--undo-last-rewrite] [--dry-run]: ENGINE-FREE Postgres-access repair, sibling of pglite-repair (PGLite brains are redirected there). Never calls connectEngine; default invocation is DIAGNOSE-ONLY (--dry-run is an explicit alias). Consent is tiered and flag-gated, never TTY-dependent: auto tier under --yes (bounded reconnects, pending migrations in-process, CREATE EXTENSION vector, docker start of gbrain's own container via docker-postgres.ts); rewrite tier under --yes --apply-rewrites only (config-file database_url rewrites — pooler form, session pooler, ?sslmode=require — printed first, candidate connect-probed BEFORE persisting, receipted, undo-able); manual tier never applied (credentials/paused-project/env recipes printed — relaying the recipe IS the repair). --apply-rewrites without --yes is an error. Invariants: fix targets derive only from the CURRENT config URL, never error text; the prober uses exactly ONE connection (poolSize: 1 — diagnosing pool exhaustion with a 10-connection pool would worsen the outage); rewrite cooldown is ≤24h per (reason, action) with --force bypass while auto-tier fixes are NEVER cooldown-blocked (availability first); every rewrite stores the prior URL in the 0600 ~/.gbrain/db-repair-undo.json (a secret — never the redacted receipts) and --undo-last-rewrite (--yes-gated) restores it — undo runs INSIDE the advisory lock, emits the same --json envelope, validates the record's postgres:// scheme before restoring, and is itself reversible (the outgoing URL becomes the new undo record); an O_EXCL-created advisory lockfile blocks concurrent double-rewrites (check-then-write race closed; one stale-holder retry; other write failures stay fail-open); a healthy probe exits 0 "nothing to fix" (the forged-marker defense) and additionally REPORTS (never rewrites) stale DB-plane engine/database_url rows; the schema probe treats a ZERO-ROW pg_extension result as pgvector_missing (the query never errors on an absent extension); the docker arm matches the container's REAL inspected host port (deps.docker.hostPort) as well as the default 5434; a successful rewrite prints the "restart long-lived gbrain processes" note (config rewrites reach NEW processes only). Refusals: thin-client (no local DB), non-host brain resolution (a mount outage must never rewrite host config — mount-targeted repair is a filed TODO), PGLite (→ pglite-repair). Injectable DbRepairDeps prober/executor seam for tests; defaultDeps.probeAccess is also reused by the init ladder (one prober, one contract). Pinned by test/db-repair.serial.test.ts.

  • src/core/db-repair-receipts.ts — the shared seam between gbrain db-repair (writer) and doctor's db_repair_recurrence check (reader — it must run in doctor's dead-DB filesystem lane, exactly when repairs are repeating). JSONL at ~/.gbrain/db-repair-receipts.jsonl; row shape {ts, brain_id, reason, action, outcome: 'applied'|'refused'|'diagnose'} — only applied rows count toward the recurrence threshold (3+ same-reason per brain in 7 days). Redacted before write (the classifier redacts its own copy; actions are fixed strings), fail-open (a receipts problem never blocks a repair), capped at 200 rows ON EVERY WRITE (the recurrence check reads the file on every doctor run — it must not grow unbounded on exactly the machines having chronic problems) — EXCEPT applied rows inside an 8-day retention window, which are exempt from the flat cap (separately bounded at 200): a diagnose flood from an agent loop must never evict the cooldown/recurrence memory. The reader is per-line tolerant (one torn row never discards every other receipt). Also owns rewriteCooldownBlocked (24h per (reason, action), rewrite tier only).

  • src/core/degraded-engine.ts + src/core/degraded-marker.ts — degraded-mode serve, STARTUP-SCOPED by design: when Postgres is unreachable at gbrain serve startup, serve boots on createDegradedEngine (a concrete object whose method set is enumerated from PostgresEngine's prototype at construction — no dynamic get trap; kind is an explicit getter, read synchronously at ~29 branch sites; disconnect() is a no-op while dead so shutdown never triggers a reconnect) instead of dying. Every tool call attempts ONE reconnect (single-flight across concurrent callers, min-interval gated ~5s — no connect storms against a dying pooler); calls inside the window throw the STORED original error (stale-but-honest, refreshed on every real attempt) so MCP dispatch classifies the REAL error shape into the database_error envelope + marker. The reconnect callback is the FULL deferred connectEngine (config merge, gateway, migrations — guarded in cli.ts: a vanished config.json throws instead of connectEngine's process.exit killing the live server); first success swaps in the live engine permanently and fires recovery callbacks (serve re-runs deferred boot + sends tools/list_changed — clients that handshook degraded hold the reduced gate-hidden catalog and must refresh; the callback is a no-op once shutdown has started) — but the TRIGGERING call gets DegradedRecoveredRetryError, never a result: its source scope was resolved under the degraded fallback, so one client retry buys scope correctness (no call ever executes under the fallback scope against a live engine). Callers waiting on an in-flight attempt are latency-capped (callerWaitMs, default 2s — they get the stored error while the attempt finishes in the background, so a timeout-class failure can't stall the MCP handshake). Prototype GETTERS delegate too (throw the stored error while dead — never a silent undefined); DEGRADED_LAST_ERROR is a read-only accessor for the stored diagnosis (drives the HTTP /health degraded reason without consuming a reconnect). Serve-side degraded posture: source scope honors a validated GBRAIN_SOURCE (tier env) before seed_default, and stdioVisibleTools fail-closes every gated op WITHOUT touching the engine. The degraded gate keys on the RESOLVED brain's engine (a PGLite mount on a postgres host keeps die-on-startup). Mid-session outages ride PostgresEngine.reconnect() + per-call classified envelopes — degraded mode adds no mid-session state; PGLite startup failures keep die-on-startup (single-writer lock makes a lazy proxy wrong there; that lane's repair is pglite-repair). Kill switch GBRAIN_SERVE_DEGRADED=0 (or false); structured [gbrain-serve] DEGRADED/RECOVERED stderr lines. degraded-marker.ts is the dependency-free seam (Symbol.for-registered markers + isEngineDegraded/onEngineRecovered/degradedLastError) so server.ts/serve.ts/http-transport.ts can ask about degraded state without importing the heavy Postgres engine module.

  • src/core/docker-postgres.ts — gbrain's OWN docker Postgres container, the shared seam between the init ladder's docker rung and db-repair's conn_refused auto arm. GBRAIN_PG_CONTAINER = 'gbrain-postgres' / GBRAIN_PG_IMAGE = 'pgvector/pgvector:pg16' / GBRAIN_PG_HOST_PORT = 5434 live HERE and only here (DRY — the init rung and the repair arm must never carry two string literals that can drift). Ownership contract: gbrain starts and reuses this container, NEVER stops or removes it, and never touches a container whose credentials it cannot recover via inspectCredentials (docker inspect → POSTGRES_PASSWORD env + the container's ORIGINAL port mapping — a freshly generated password can never match a surviving container; unrecoverable → refuse with a recreate-with-consent recipe). Hardening: the port binds loopback-only (127.0.0.1:5434 — a single-machine brain store, not a network service); data lives on the named gbrain-pgdata volume so a container recreate never loses the brain; the password rides the child ENVIRONMENT (bare -e POSTGRES_PASSWORD — argv is world-readable in ps); docker run gets a 10-minute timeout (the first run synchronously pulls the image; 30s stays for cheap ps/inspect/start calls). isGbrainDockerUrl is the one predicate for "does this URL point at our container" (loopback + dedicated port — never error text).

  • src/commands/init-prefer-postgres.tsrunPreferPostgresLadder: the Postgres-first install ladder behind gbrain init --prefer-postgres [--allow-docker] [--allow-create-db] [--local-postgres] [--json] (the zero-config gbrain init default stays PGLite; this flag is the harness lane). Five rungs, first usable wins, every reachable-but-unusable rung prints a one-line note and falls through (only the PGLite floor is terminal): (1) env URL; (2) Supabase Management-API DISCOVERY via supabase-admin.tsSUPABASE_ACCESS_TOKEN (+ SUPABASE_PROJECT_REF on multi-project accounts, auto-selected on single-project ones) + SUPABASE_DB_PASSWORD (the API can't return it), ~10s timeouts, the discovered URL is a CANDIDATE connect-probed before anything persists, creation stays dashboard guidance, the token is never persisted/logged/echoed; (3) local Postgres, opportunistic (only when PG* env vars are set or --local-postgres — a blind localhost probe on peer-auth dev installs just burns a timeout) and detection-only (CREATE DATABASE gbrain needs explicit --allow-create-db); (4) docker via docker-postgres.ts (explicit --allow-docker; idempotent reuse recovers real credentials via docker inspect; readiness-polled); (5) initPGLite + explicit upgrade-later note. Calls initPostgresCore (typed InitPostgresFailure, no process.exit — the returnable core that makes rung fall-through possible) and reuses db-repair's defaultDeps.probeAccess prober. Prints the rung-entry criterion line (Postgres wins on concurrency/multi-machine/1000+ pages; PGLite keeps the per-turn hook lane) so the preference is informed, not silent. Guards: an ALREADY-CONFIGURED brain refuses the whole ladder (rung choice is environment-dependent, so a re-run during an outage would let the PGLite floor overwrite a healthy Postgres config — outages are db-repair's lane); a bare DATABASE_URL (vs the stated-intent GBRAIN_DATABASE_URL) is adopted only when the target is already a gbrain brain or holds no tables at all (classifyDbContent, fail-closed on foreign/unknown — deploy platforms export DATABASE_URL pointing at the APP's database); the docker rung content-guards BOTH reuse and fresh-create (a fresh container can attach a surviving gbrain-pgdata volume) and detects the surviving-volume auth mismatch (fresh password rejected → adoption/clean-start recipe instead of "never became reachable"); rung-note error text is redacted. --json emits ONE final {status, engine, ladder_rung, url_source} envelope and stdout is EXACTLY that document — withStdoutToStderr reroutes both console.log (Bun writes it to fd 1 directly) and bare process.stdout.write around the inner init cores.

  • src/commands/doctor.tsgbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit] [--no-migrate]: health checks. --no-migrate keeps doctor observational: the CLI connects with probeOnly so a clean-or-behind schema is reported as-is instead of being auto-migrated before the checks run. The file is a façade carrying buildChecks/runDoctor and output rendering; the check-function library lives in bundles under src/commands/doctor/checks/ plus four tail-cluster modules under src/commands/doctor/ (see that entry), all re-exported here so the full surface is unchanged — structural guards pin its source text via test/helpers/doctor-source.ts, never by reading this file alone. Checks include jsonb_integrity + markdown_body_completeness (reliability), schema_version (fails loudly when version=0, routes to gbrain apply-migrations --yes; a version AHEAD of this client's LATEST_VERSION warns "upgrade this client"; and when the ledger reads current, a read-only detectMissingColumns diff from src/core/schema-verify.ts runs INSIDE the ledger-current branch — a PgBouncer-swallowed ALTER TABLE can advance config.version over a physically narrower table, so missing live columns downgrade the ok to a warn naming them with the gbrain init --migrate-only hint; diff failure is best-effort and the ledger ok stands; positional wiring pinned by test/doctor-schema-column-diff.test.ts), upgrade_errors (async checkUpgradeErrors(engine) over ~/.gbrain/upgrade-errors.jsonl: the warn downgrades to an EXPLICIT status-ok line — never silence — only when the past failure is provably superseded, i.e. the running binary is at/past the failed target version AND the schema verifies current via engine.getConfig('version'); a missing engine or unverifiable schema keeps the warn, fail-closed, because self-upgrade swaps the binary before post-upgrade runs migrations and the binary alone can lie; pinned by test/doctor-upgrade-errors.test.ts + test/doctor-upgrade-errors-stale.test.ts), queue_health (Postgres-only: stalled-forever active jobs started_at > 1h, waiting-depth-per-name > threshold default 10 via GBRAIN_QUEUE_WAITING_THRESHOLD, and dead-lettered subagent jobs with last_error matching the prompt_too_long classifier in last 24h), sync_failures ([CODE=N, ...] breakdown for unacked-warn + acked-ok; severity comes from the shared decideSyncFailureSeverity in src/core/sync-failure-ledger.ts so the LOCAL and REMOTE/thin-client doctor surfaces can never drift — a stuck bookmark escalates to FAIL once an OPEN failure has blocked past the staleness window or ≥10 files block, while already auto_skipped rows stay a visible WARN), rls_event_trigger (healthy evtenabled set is ('O','A') only; fix hint gbrain apply-migrations --force-retry 35), graph_coverage (short-circuits to ok when SELECT COUNT(*) FROM pages WHERE type IN ('entity','person','company','organization') returns 0; WARN hint is gbrain extract all), embedding_column_registry (probes each declared column via Postgres format_type(atttypid, atttypmod) to catch dim mismatch with a paste-ready gbrain config set embedding_columns '{...}' hint, probes HNSW index presence via pg_indexes, computes default-column population via COUNT(*) FILTER (WHERE <col> IS NOT NULL) / COUNT(*) warning below 90% except empty brains where chunk_count=0 short-circuits to ok; PGLite parity via executeRaw), and skill_brain_first (walks SKILL.md via autoDetectSkillsDirReadOnly, calls analyzeSkillBrainFirst() from src/core/skill-brain-first.ts per file with structured Check.issues[]; warn states missing_brain_first/brain_first_typo, ok states compliant_callout/compliant_phase/compliant_position/exempt_frontmatter/no_external; snapshot+diff audit at ~/.gbrain/audit/skill-brain-first-YYYY-Www.jsonl). --fix delegates inlined cross-cutting rules to > **Convention:** see [path](path). callouts via src/core/dry-fix.ts (and MISSING_RULE_PATTERNS for the brain-first callout); --fix --dry-run previews. --index-audit (Postgres-only, informational, no auto-drop) reports zero-scan indexes from pg_stat_user_indexes. Every DB check runs under a progress phase; markdown_body_completeness runs under a 1s heartbeat. runDoctor uses autoDetectSkillsDirReadOnly (from src/core/repo-root.ts; install-path fallback so cd ~ && gbrain doctor finds bundled skills); --fix carries a D6 install-path safety gate that refuses auto-repair when detected.source === 'install_path' (would rewrite the bundled tree). The Lane D supervisor check at doctor.ts:1011-1043 consumes summarizeCrashes(events) from src/core/minions/handlers/supervisor-audit.ts (warn at >=1 real crash; ok message has clean_exits_24h=N; warn message has runtime=A oom=B unknown=C legacy=D per-cause breakdown) so OOM/runtime/unknown crashes are distinguishable from clean code=0 worker drains; cross-surface parity with gbrain jobs supervisor status is pinned by source-grep wiring assertions requiring the breakdown substrings in BOTH doctor.ts and jobs.ts. checkSyncFreshness (exported, in runDoctor local + doctorReportRemote thin-client) is a staleness probe: warns at 24h, fails at 72h or never-synced; future-last_sync_at warns ("clock skew") instead of falling through ok; env overrides GBRAIN_SYNC_FRESHNESS_WARN_HOURS/GBRAIN_SYNC_FRESHNESS_FAIL_HOURS (invalid fall back with once-per-process stderr warn via _resolveSyncFreshnessHours); failure messages embed source.id so the printed gbrain sync --source <id> matches. A source holding a LIVE, non-expired per-source sync lock (inspectLock(engine, syncLockId(source.id)) from src/core/db-lock.ts) is reported as actively syncing (the message names the holder pid + host) and counted in synced_recently_count, NOT flagged stale — the live lock is the only honest in-progress signal (checkpoint banking can't distinguish in-progress from wedged: a blocked sync banks its files but writes no anchor). A blocked/failed sync's process has exited (no lock row) and a wedged holder stops refreshing (TTL lapses), so either falls through to the stale path and is never masked; the dynamic db-lock import is swallowed to a no-op on a stub engine or pre-lock-table brain, so this can only ADD an in-progress verdict, never suppress a real stale one. The in-progress note is appended to whatever verdict the buckets produce and is empty when nothing is syncing, so steady-state messages stay byte-for-byte unchanged. It has a localOnly-gated git short-circuit (runDoctor passes localOnly: true; doctorReportRemote runs in the HTTP MCP server src/commands/serve-http.ts and keeps default false so that path never walks DB-supplied local_path via subprocess — trust boundary). The local predicate mirrors sync's "do work?" gate (HEAD == last_commit AND working tree clean via requireCleanWorkingTree: 'ignore-untracked' so a quiet repo with only untracked dirs is unchanged not SEVERE, AND chunker_version === CURRENT); the inline SELECT carries last_commit + chunker_version + newest_content_at. The REMOTE path computes lag via lagFromContentMs(newest_content_at, lastSync, now) from the stored column, NO git subprocess; LOCAL fall-through and the < 0 clock-skew check stay on raw wall-clock. Three-bucket count math populates Check.details = {unchanged_count, synced_recently_count, stale_count} with the invariant sum === sources.length. checkCycleFreshness is DELIBERATELY NOT git-short-circuited or content-relativized (last_commit == HEAD can't answer "did the full cycle complete?"; a sync can succeed while later cycle phases fail; different axis last_full_cycle_at). Pinned by test/doctor.test.ts (incl. the IRON-RULE guard banning stale verb names, the sync_freshness boundary matrix, the D4 guard verifying git probes are NEVER called when localOnly is unset/false, the three-bucket invariant, and the untracked-folders / remote-never-shells-out trust-boundary cases). pglite_data_dir check: fs-only check that runs when a PGLite brain FAILS to connect (!fastMode && !engine && config.engine === 'pglite', placed after orphan_clones, before the DB-checks gate): computePgliteDataDirCheck(dataDir, diagnosis) (exported pure fn, computeWorkerOomLoopCheck convention) maps the inspectPgliteDataDir verdict to a Check — corruption-likely/looks-healthy-but-unopenable/unsupported-layout → fail naming gbrain pglite-repair --dry-run/--yes or the rebuild path, live-lock/missing-dir → warn; all remediation_status: 'human_only' (Minion remediation needs the DB that is down). Escalates when ≥2 repair attempts failed inside 7 days (unclean-shutdown genesis still active → engine-switch pointer) and reports retained backup-dir inventory (orphan_clones disk-visibility class). Registered in doctor-categories.ts OPS_CHECK_NAMES. Pinned by test/doctor-pglite-datadir.test.ts. silent-failure checks: content_hash_duplicates (single GROUP BY over (source_id, content_hash) with FILTER aggregates — never N² — flagging hash groups that hold BOTH a bare and a path-prefixed slug, the wrong-import-root pattern; warn carries sample pairs + the pages deletepurge-deleted --older-than 0 remediation); undeclared_db_only_pages (per source with a local repo: markdown pages with no backing file outside every declared + derive-phase-default db_only prefix — the one check deliberately allowed to stat the repo); db_only_collector_collision (configured recipe output_paths inside a declared db_only dir — auto-gitignore means sync AND import silently skip the collector's files; same warning fires in sync's manageGitignore at config-write time). All warn-level, engine-parity pinned by test/e2e/doctor-silent-death-parity.test.ts; units in test/doctor-silent-death-checks.test.ts. graph_signals_coverage check wired into both runDoctor (local) and doctorReportRemote (HTTP/JSON thin-client path). Reads search.graph_signals config first, falls back to mode default; silent ok when disabled. Computes inbound link coverage on the page set; warns at <10% with gbrain extract all fix hint; ok at ≥30% ("fire on most queries") and 10-29% ("fire occasionally"), each with the percentage embedded. Pinned by cases in test/doctor.test.ts. subagent_provider check (layer 3 of 3). Resolves subagent model config in runtime order (models.subagent > models.default > models.tier.subagent > built-in default) and warns when the selected model lacks native tool-loop capability (message names the bad value + paste-ready fix gbrain config set models.tier.subagent anthropic:claude-sonnet-4-6); also warns when models.default would sneak subagent into a non-Anthropic provider via tier inheritance. OK when subagent tier resolves to Anthropic. Tests in test/doctor.test.ts. computeWorkerOomLoopCheck(engine) is the single authoritative OOM-loop signal, unioning supervised summarizeCrashes(readRecentSupervisorEvents(24)).by_cause.rss_watchdog (cross-week read via readRecentSupervisorEvents so a Monday window can't lose Sunday) + bare-worker minion_jobs error_text='aborted: watchdog' count (Postgres-only; the same source queue_health subcheck 3 reads). Cap comes from the latest rss_watchdog_loop breaker alert's max_rss_mb, else resolveDefaultMaxRssMb() fallback. fail at breaker-tripped or oomKills≥5, warn at ≥1, null otherwise. computePoolReapHealthCheck(engine) is the Postgres-only pool_reap_health check reading readRecentPoolRecoveries(1) — fail when reconnect failures>0 (reconnect throwing is the actionable signal), warn at ≥10 reaps/hr (pooler thrash), null otherwise. Both registered in buildChecks after the supervisor block. The supervisor causeStr carries rss=N (see worker_oom_loop) and queue_health's watchdog message cross-references worker_oom_loop. DoctorReport.top_issues + the cause-ranked render header. worker_oom_loop + pool_reap_health registered under ops in doctor-categories.ts. Pinned by test/doctor-worker-oom-loop.test.ts, test/doctor-pool-reap-health.test.ts. supervisor_singleton check, a SEPARATE check from supervisor (same split precedent as the niceness check) so a singleton-divergence warn can't clobber the crash/liveness precedence. Runs only when a started supervisor event was seen in the last 24h and a live engine is available. Reads the queue-scoped DB lock row (gbrain_cycle_locks WHERE id = supervisorLockId(queue)) and compares the lock holder (holder_host:holder_pid) against the local pidfile holder via the pure classifySupervisorSingleton. mismatch → warn (a second supervisor may be running with a different --max-rss; message names both holders, the effective cap from the started event's max_rss_mb, and the fix gbrain jobs supervisor stop); single → ok (names holder + cap); no_lock → no check emitted. Best-effort try/catch (silent skip on brains without the lock table). Registered under ops in doctor-categories.ts as supervisor_singleton. Pinned by test/supervisor-db-lock.test.ts + test/doctor.test.ts. checkBatchRetryHealth: batch_retry_health check surfacing Supavisor circuit-breaker incidents. Wired into both runDoctor (local) and doctorReportRemote (thin-client). Reads last 24h. States: ok (zero exhausted in 24h OR <3 from a single site), warn (>=3 same-site OR >=5 cross-site), fail (>=20 sustained breaker). Surfaces bad GBRAIN_BULK_* env at doctor startup. Corrupt-JSONL tolerant. Paste-ready fix hints in every warn/fail message. Also reads readRecentDbDisconnects(24) and appends Disconnect-call audit: N call(s) in 24h (most recent caller: <frame>). to ALL three message paths so connection-incident signal is greppable from one gbrain doctor --json call (module-import wrapped in try/catch so older brains without the audit file degrade silently). Pinned by test/doctor-batch-retry.test.ts (10 cases). three checks wired into runDoctor() and the JSON envelope, all warn-only with paste-ready fix hints. (1) checkSourceRoutingHealth(engine) scans up to 200 pages on federated brains and flags pages whose source_id doesn't match what resolveSourceWithTier() would have picked for their source_path; single-source brains short-circuit to ok; the 200-page cap is total across the brain so doctor stays under 5s. (2) checkOauthConfidentialHealth(engine) probes registered confidential clients for /token reachability. (3) checkAutopilotLockScope() (pure, no engine) compares the resolved lock path to $GBRAIN_HOME; warns when set but the lock lives elsewhere, with a PID-safe inspection hint (kill -0 <pid> before deletion). Pinned by test/doctor-v0_37_7_checks.test.ts. buildChecks(engine, args, dbSource, connectError?): Promise<Check[]> exported as a test seam; the optional connectError is the connect failure captured by the CLI's dead-DB fallback, which the null-engine path turns into a SYNTHESIZED classified connection fail entry (so checks[name=="connection"] exists in every failure shape — smoke-test branches on it). The connection check's failure path classifies via src/core/pg-access-classify.ts into a redacted message + details: {reason, transient, fix_hint} naming gbrain db-repair (deliberately NOT remediation[]/makeRemediationStep — that lane feeds --remediate, whose Minion jobs need the very DB that's down; applicator boundary: --remediate = brain-DATA quality with a live engine, db-repair = DB ACCESS engine-free). The URL-only pgbouncerPrepareCheck helper and the engine-free db_repair_recurrence check run BOTH before the connection check and in the dead-DB filesystem lane; pglite_scale (engine-fit, warn at ≥1000 pages on pglite) runs in the DB phase. computeDoctorReport(checks, extras?) accepts optional {engine, db_url_source} which land as additive DoctorReport.engine / DoctorReport.db_url_source JSON fields (schema_version stays 2). runDoctor is a thin wrapper: buildChecks → computeDoctorReport → render + process.exit. All 10 process.exit sites stay in the wrapper; the two early-return paths (no engine, connection failure) return partial check lists instead of inline exits (observable output identical). Pinned by test/doctor-behavioral.test.ts (13 cases: pure aggregation math over computeDoctorReport, orchestrator cases for --fast skip set + --json flag + no-engine partial path + snapshot of load-bearing check names) and test/doctor-cli-smoke.serial.test.ts (1 subprocess case spawning bun run src/cli.ts doctor --json against a fresh PGLite tempdir, asserting schema_version=2 envelope, status enum, non-empty checks array — the render-path coverage buildChecks-only tests miss; quarantined .serial because PGLite write-locks don't play with parallel runners). three checks wired into runDoctor() and the JSON envelope: oversized_pages (warns on pages exceeding content_sanity.bytes_warn), scraper_junk_pages (warns on live DB pages matching any junk pattern that escaped ingest), and content_sanity_audit_recent (reads the last 7 days of audit events, aggregates by pattern+source). Default scans the 1000 most-recent pages; --content-audit opts into a full scan. All three warn-only with paste-ready fix hints (junk → gbrain sources audit <id> + git rm source-of-truth, oversize → split or accept). two checks wired into runDoctor() + the JSON envelope: quarantined_pages (counts pages carrying the quarantine marker via engine.executeRaw JSONB ? existence, works on PGLite + Postgres; warn-only with a gbrain quarantine list hint) and flagged_pages (counts content_flag pages — searchable but odd; warn-only). Both skip gracefully (status ok, "Skipped") on engines/brains where the probe errors. Pinned by test/doctor.test.ts. home_dir_in_worktree: filesystem check walking up from gbrainPath() toward $HOME looking for a .git directory (main repo) or .git file (linked worktree pointer; Conductor + git-worktrees topology). Walk terminates at $HOME so a .git above the user's home doesn't false-positive. Honors GBRAIN_HOME (appends .gbrain to the override). Warn (not fail) with worktree-root path + paste-ready fix pointing at GBRAIN_HOME override or moving the brain. --remediation-plan [--json] [--target-score N] prints what would run (stable id, idempotency_key, severity, est_seconds, est_usd_cost, depends_on); --remediate [--yes] [--target-score N] [--max-usd N] submits each plan step as a Minion job in dependency order, re-checking score between steps. --target-score N defaults to 90; refuses to start when target exceeds maxReachableScore() and lists what's missing. --max-usd N is the cron-safety guard — submission refuses when the plan's est_total_usd_cost exceeds the cap. JSON envelope adds a Check.remediation field (additive, schema_version unchanged). Pinned by tests in test/doctor.test.ts. 4 checks: abandoned_threads, calibration_freshness, grade_confidence_drift (mitigation surface; math ships later), voice_gate_health. Schema health is a two-check pair: schema_version resolves through schemaVersionHealth (src/core/schema-version-health.ts, shared with the remote doctor report; a DB AHEAD of the client warns with an upgrade-this-client hint rather than suggesting migrations), and on ledger-current brains a read-only schema_columns check diffs live columns against the expected schema (detectMissingColumns, dynamically imported inside the ledger-current branch — positional guard in test/doctor-schema-column-diff.test.ts) and warns naming the missing columns with the gbrain init --migrate-only hint.

  • src/commands/doctor/ — the doctor module directory (the meat behind the doctor.ts façade). checks/ holds the check-function library in bundles grouped by concern (core-health.ts, queue-jobs.ts, extraction-sync.ts (its atom_provenance_drift check scopes drift to page-bound atoms and splits it into source_changed / source_gone; atoms with no source_slug — transcript-origin source_path-only rows, whose hash is over a file and whose page liveness cannot be resolved by slug — are reported as their own slug_unbound count and never enter the drift total or the WARN ratio), graph-embedding.ts, search-eval.ts, calibration.ts, consolidation-cycle.ts, pglite-worker.ts, routing-federation.ts, verbs-reflex.ts, stale-mentions.ts, engine-fit.ts — the last carrying pglite_scale + the engine-free db_repair_recurrence over the db-repair receipts); schema-pack-checks.ts, report-remote.ts, bootstrap-checks.ts, and skill-checks.ts are the four tail-cluster modules (schema-pack checks, the remote/thin-client doctor report, bootstrap checks, skill checks). All are re-exported by the façade. Structural guards that pin doctor source text load it through test/helpers/doctor-source.ts: doctorSource() concatenates the façade plus every src/commands/doctor/**/*.ts (façade first, then sorted) so a module move can never take a pinned string out of a guard's sight; doctorFileSource(rel) reads one named file for positional/ordering assertions; the helper also feeds the doctor-source-helper detector in scripts/classify-tests.ts. facadeExpansion in scripts/generate-flag-registry.ts keeps this directory on the doctor command's flag-scan surface.

  • src/commands/doctor/checks/search-eval.ts — search, model, and AI-config health checks. checkChatFallbackChainInert returns one warning when either the effective file/environment config or the DB plane has a non-empty chat_fallback_chain; it returns null when both are empty, so clean doctor reports add no line. The local and remote doctor registries both consume the check. checkSearchMode (search_mode) reports the resolved mode and its search.* overrides; when the only override is an explicit search.reranker.model row equal to the mode bundle's own default (a row an init may have written on Voyage installs) it is called redundant with the precise gbrain config unset search.reranker.model — never a --reset, which would also wipe every other tuned search knob — and overrides that keep a brain OFF a sunsetting bundle default are never reset-nagged (a reset would re-arm a dying provider).

  • src/core/default-source-path-check.ts — pure assessDefaultSourcePath(input) verdict for the default_source_local_path doctor check (no DB/FS access — caller supplies gathered inputs, the npm-squat-check.ts shape). INVARIANT: default.local_path: null is the DESIGNED fallback topology (write-through nests under sync.repo_path), NOT an error — the check warns ONLY when the null pointer provably breaks something: the fallback repo is another source's own working tree (the leak-guard collision: every unscoped default write-through silently skipped), or default has file-backed pages and no resolvable root. Everything else (no pages, resolvable fallback, deliberate DB-only brain) is ok. The repair it names is the non-destructive gbrain sources set-path default <path>.

  • src/commands/doctor/checks/default-source-path.ts — the gathering wrapper on the doctor surface: reads the default sources row, page counts (live + file-backed), sync.repo_path resolution, and the leak-guard collision, then delegates the verdict to assessDefaultSourcePath. Only gathers the fallback-topology inputs when the pointer is actually null; returns null for the skip verdict.

  • src/commands/doctor/checks/home-worktree.ts — the home_dir_in_worktree check: walks up from the gbrain home looking for an enclosing .git (dir or linked-worktree file), warning that a git add from the worktree root could stage the brain; stops at HOME.Bothanchors(GBRAINHOME,HOME. Both anchors (`GBRAIN_HOME`, `HOME) are path.resolve()d before the containment test, so a trailing-slash HOME=/home/user/(a common shell/launchd spelling) cannot make a brain inside a worktree grade ok.isValidGitMarkerstructurally validates a candidate before it counts (a.git/dir must contain HEAD; a.gitfile must start withgitdir:— git itself rejects anything else), and invalid candidates CONTINUE the walk so a valid repo higher up still warns. Cheap (stat + read, no subprocess) — runs on every doctor invocation including--fast`.

  • src/core/postgres-engine.ts — Postgres + pgvector implementation (Supabase / self-hosted). The facts/takes/code-edges/salience method clusters are implemented in narrow-deps modules under src/core/postgres-engine/ (see the engine-module-dirs entry below); the class methods delegate to them and the façade keeps its full public surface. addLinksBatch/addTimelineEntriesBatch/addTakesBatch pass the batch as one JSONB document — INSERT ... SELECT FROM jsonb_to_recordset((\$1::jsonb)->'rows') AS v(...) JOIN pages ... bound through executeRawJsonb({ rows }) — which encodes arbitrary free text safely (an unnest(${arr}::text[]) array-literal path would crash Postgres with "malformed array literal" on free text such as calendar/Zoom context) and sidesteps the 65535-parameter cap; takes declares native recordset column types (page_id int, weight real, active boolean, …) so no per-element casts; all three are batchRetry-wrapped. disconnect() runs drainBackgroundWorkBeforeDisconnect() before pool teardown (engine parity with PGLiteEngine — mode 'disconnect', so residual telemetry buffers drop symmetrically on both engines; guarded so a never-connected/already-torn-down engine skips the drain). searchKeyword/searchVector scope statement_timeout via sql.begin + SET LOCAL so the GUC dies with the transaction instead of leaking across the pooled postgres.js connection. getEmbeddingsByChunkIds uses tryParseEmbedding so one corrupt row skips+warns instead of killing the query. searchKeyword/searchKeywordChunks/searchVector apply source-aware ranking by inlining the source-factor CASE and NOT (col LIKE …) hard-exclude from src/core/search/sql-ranking.ts; searchVector is a two-stage CTE (HNSW-safe inner ORDER BY, source-boost re-rank in outer SELECT) carrying p.source_id inner→outer. _savedConfig retains the connect config; reconnect() tears down + recreates the pool from saved config (called by supervisor watchdog after 3 consecutive health-check failures, and by batchRetry on a retryable connection error). Concurrent callers share one in-flight _reconnectPromise (they await the single reconnect rather than racing a half-rebuilt pool); ownership re-samples through the atomic db.connect() token on the connect leg. reconnect(ctx?) accepts the triggering error and records a pool-recovery audit event (reap_detected/reconnect_other/reconnect_succeeded/reconnect_failed) for the pool_reap_health doctor check. executeRaw is a single-statement passthrough — no per-call retry (unsound for non-idempotent statements; recovery is supervisor-driven). connect() applies resolveSessionTimeouts() from db.ts as connection-time startup parameters (statement_timeout, idle_in_transaction_session_timeout) so orphan pgbouncer backends can't hold locks for hours. countStaleChunks()+listStaleChunks() server-side-filter on embedding IS NULL for embed --stale (eliminates ~76 MB/call client-side pull); upsertChunks() routes text-embedding writes through the column registry (both engines in parity): a caller-resolved opts.embeddingColumn descriptor wins; otherwise the DB-plane search_embedding_column + embedding_columns config rows resolve via resolveWriteColumnFromConfigRows to the SAME active column + cast the read side searches (legacy embedding::vector on pre-registry brains; so a registry-routed brain never writes with a dimension mismatch against the legacy column). The INSERT column list, the $N::vector|::halfvec(N) cast, and every embedding reference in the ON CONFLICT CASE branches use the resolved column; it resets both the active column AND embedded_at to NULL when chunk_text changes without a new embedding. Pinned by test/e2e/upsert-chunks-registry-column.test.ts (PGLite always; Postgres DATABASE_URL-gated). initSchema() calls applyForwardReferenceBootstrap() BEFORE replaying SCHEMA_SQL — a superset of PGLite's probe set: the shared column-only forward-reference cases (files.source_id, files.page_id, oauth_clients.source_id, oauth_clients.federated_read, sources.archived/archived_at/archive_expires_at, timeline_entries.event_page_id) plus Postgres-blob-only probes for state the PGLite blob doesn't carry (dream_verdicts.expires_at — see the forward-reference-bootstrap.ts entry); the entire probe path runs on the DDL connection threaded from initSchema (so concurrent bootstraps cannot race on a Supabase pooler). disconnect() is idempotent — _connectionStyle tracks whether the engine owns its pool (worker engines) or shares the module-level singleton; second call on an instance-pool engine is a no-op rather than clobbering the singleton; and a module-style engine only calls db.disconnect() when it owns the singleton (_ownsModuleSingleton, set from the db.connect() creation token), so a borrower probe engine's teardown leaves the cycle owner's connection intact. Pinned by test/e2e/postgres-engine-disconnect-idempotency.test.ts + test/postgres-engine-singleton-ownership.test.ts. getBrainScore empty-brain parity with PGLite — 100/100 with breakdown 35/25/15/15/10 when pageCount === 0 (both engines must agree to keep doctor-report-remote.serial.test.ts deterministic). Implements deletePages(slugs, {sourceId}): Promise<string[]> via DELETE FROM pages WHERE slug = ANY(\$1::text[]) AND source_id = \$2 RETURNING slug (single round-trip; caller chunks); resolveSlugsByPaths does SELECT slug, source_path FROM pages WHERE source_path = ANY(\$1::text[]) AND source_id = \$2; FK cascades through content_chunks/links/tags/raw_data/timeline_entries/page_versions, files.page_id+links.origin_page_id go SET NULL; throws when input exceeds DELETE_BATCH_SIZE (from src/core/engine-constants.ts); both short-circuit on empty input. Implements the embedding-signature stale-detection quartet (sumStaleChunkChars, setPageEmbeddingSignature, invalidateStaleSignatureEmbeddings, widened countStaleChunks, all accept optional signature extending "stale" to model/dims-swap drift via the pages.embedding_signature JOIN, NULL grandfathered; the embedding IS NULL server-side filter is preserved as the no-signature fast path); Pinned by test/e2e/engine-parity.test.ts. Retry classifiers, ontology/recency helpers, and disconnect/pool-recovery audit writers bind statically; only the two ai/gateway.ts fallback lookups stay lazy and line-marked, in parity with PGLite. insertFact + insertFacts do not hardcode tx.unsafe(\'${embedLit}'::vector`)for the embedding column.resolveFactsEmbeddingCast()(private) probespg_attributeonce per engine instance (cached in_factsEmbeddingCastSuffix) and returns '::halfvec'when migration v40 created the column as halfvec, else'::vector'; both insert paths use the cached suffix so the cast matches the actual column type (works on older pgvector that lacks implicit auto-cast). Test seam __resetFactsEmbeddingCastCacheForTest()clears the per-engine cache.withReservedConnectionroutes long-hold reserved work (CREATE INDEX CONCURRENTLY,transaction: falsemigration DDL, backfill write batches) to the DIRECT session lane when dual-pool is active, so multi-minute holds do not pin the worker's shared read pool — never rerouted inside an open transaction (same guard shape asexecuteRawDirect), semaphore-capped below direct_pool_sizewith deliberately NO minimum-1 floor (atdirect_pool_size=1a floor would let one long reserve consume the only direct session and starve the claim/renewLock heartbeats — the same starvation on the direct lane), overflowing to the shared read pool when the direct lane has no spare capacity or is unavailable; the permit is released on both fn throw and reserve failure.getPoolDiagnostics()(duck-typed, no BrainEngine change) surfaces theCheckoutGaugein-flight counters fromsrc/core/pool-gauge.tsat the raw/direct/reserved/tx seams — a tracked SUBSET (tagged-template traffic is untracked) thatdb-probe.tslabels honestly in health-probe failure lines. Pinned bytest/postgres-engine-reserved-routing.test.ts`.

  • src/commands/doctor/checks/integrations-memorable.tsmemorable_relay_health (OPS category), the doctor surface for the optional third-party relay. Engine-free (file-plane reads only), pushed unconditionally so it survives --fast/--scope. Rung ladder: gate off → quiet ok; enabled WITHOUT the gbrain disclosure stamp → FAIL naming the exact fix (the memorable enable out-of-band state); CLI-side consent absent → warn; no runnable binary → FAIL; last relay run failed → warn with the CLAMPED cause (the shared clampRelayCause; child text never lands in a doctor message verbatim) — EXCEPT the documented openclaw no_decisive_steps rejection, which becomes an ok-with-note (expected_openclaw_rejection, name-only capture refused until argument capture lands) held as a DEFERRED note that surfaces only after the codex rung below passes: on a mixed openclaw+codex host the rejection is the persistent last-relay state, and an early return would paint the check green while a trust-stale codex hook sits silently dead; receipts written but the child never reported → warn; codex hooks wired but no codex-harness receipt in the recent receipt window (tail-200 read) → warn (codex_hooks_never_fired — codex hooks fail silently on a stale trust entry); healthy → ok with structured details (out_of_band_settable: true always rides — the enable flag can be flipped by the external CLI, which is exactly why the gate also demands the stamp). Pinned by test/doctor-memorable.test.ts.

  • src/core/postgres-engine/ + src/core/pglite-engine/ — narrow-deps engine module directories, mirrored in lockstep (the engine-parity discipline applies to these dirs exactly as to the façades). Each holds facts.ts, takes.ts, code-edges.ts, and salience.ts: the corresponding BrainEngine method clusters implemented as free functions over a typed deps interface (e.g. PgliteFactsDeps) that the engine class satisfies, so each module depends only on the narrow slice it uses. The façade methods delegate to them via static top-level imports (the engine-live no-runtime-dynamic-import rule applies) and keep the full public surface.

  • src/core/postgres-engine/forward-reference-bootstrap.tsapplyPostgresForwardReferenceBootstrap(conn): the forward-reference bootstrap for the RAW SCHEMA_SQL replay path (probe + patch every column/table the embedded schema blob forward-references but older brains don't have yet — single probe round-trip, idempotent, fast no-op when nothing is missing). Peeled out of PostgresEngine so BOTH replay entrypoints run it before the blob: PostgresEngine.initSchema() AND the standalone module-singleton db.ts:initSchema() (replaying with no bootstrap would wedge an upgrade-boundary brain on CREATE INDEX over a column the replayed blob forward-references). Callers MUST hold the initSchema advisory lock (key 42) on conn so concurrent bootstraps can't race on a transaction pooler. Mirror of PGLiteEngine#applyForwardReferenceBootstrap in shape — keep in sync; covered by test/schema-bootstrap-coverage.test.ts (PGLite A2 static check + the Postgres-blob CREATE-INDEX class-closure gate, which parses SCHEMA_SQL and requires every migration-added blob-indexed column to have a probe here or an explicit exemption) + test/e2e/postgres-bootstrap.test.ts (Postgres side, live convergence cases). Probes are column-only (nullable ADD COLUMN, plus SET DEFAULT where hardening the upgrade window needs it — dream_verdicts.expires_at); the owning migration stays the source of truth for backfill/constraints/indexes: on the engine path it runs right after the bootstrap under the same advisory lock, while the standalone db.ts:initSchema path replays the blob WITHOUT running migrations — safe because the NULL-tolerant read predicates treat pre-backfill rows as valid until the next full engine init.

  • src/core/cjk.ts — Single source of truth for CJK detection. Exports CJK_RANGES_REGEX, CJK_SLUG_CHARS (character-class fragment for embedding inside other regexes), CJK_SENTENCE_DELIMITERS (。!?), CJK_CLAUSE_DELIMITERS (;:,、), CJK_DENSITY_THRESHOLD = 0.30, hasCJK(s), isCJKDominant(s) (the ONE density test: CJK chars ≥ 30% of non-whitespace chars), countCJKAwareWords(s) (routes through isCJKDominant — English docs with one Japanese term stay whitespace-tokenized; Chinese-dominant docs get char-counted), and escapeLikePattern(s) (escapes %, _, \\ for ILIKE ... ESCAPE '\\'). BMP-only ranges (Han / Hiragana / Katakana / Hangul Syllables). Consumers: expansion.ts, sync.ts:slugifySegment, operations.ts:validatePageSlug + validateFilename, chunkers/recursive.ts:countWords + DELIMITERS + extractTrailingContext (the overlap extractor branches on isCJKDominant so it counts the same unit countWords does), pglite-engine.ts:searchKeyword + searchKeywordChunks.

  • src/core/latin-fold.ts — leaf text primitive (no engine/graph imports): NON_DECOMPOSING_LATIN, the lowercase-keyed table of the eleven Latin stroke/bar/ligature letters Unicode gives no decomposition (đ ð ø ł ħ ŧ ı ß æ œ þ), + foldNonDecomposingLatin(s). Call it AFTER lowercasing and AFTER the combining-mark strip so composed forms reduce in one pass (ǿ → ø → o). Consumers: entities/resolve.ts:slugify (entity fallback slugs — pre-fold Đăng Example collided onto ang-example and an all-stroke name slugged to empty) and link-extraction.ts:normalizeBasename (basename-index keys + dir-hint candidates). Deliberately NOT applied to sync.ts:slugifySegment — the page-slug grammar keeps these letters (#3417) — so the dir-hint candidate step in makeResolver / the FS resolver tries both forms.

  • src/core/chunkers/recursive.ts — base chunker: 300-word chunks, 50-word sentence-aware overlap, 5-level delimiter hierarchy. Applies sanitizeRemoteBody before splitting so only world Facts and ordinary prose enter retrieval text; all Takes and malformed protected sections are omitted. Non-overlapping portions reassemble to that sanitized input. The overlap extractor counts chars (not whitespace tokens) for CJK-dominant chunks — same unit as countWords — and aligns to 。!? or a whitespace-followed ASCII .!?; the L4 char-slice fallback advances through safeSplitIndex so astral pairs (emoji, non-BMP CJK) are never halved. Both are gated on isCJKDominant, so English output is byte-identical (pinned by test/chunkers/recursive-cjk-overlap.test.ts). The markdown chunker version records the current index format.

  • src/core/chunkers/semantic.ts — embedding-based topic-boundary detection: embeds sentences, computes cosine-similarity valleys, smooths with a Savitzky-Golay filter (5-window, 3rd-order polynomial) to find chunk boundaries. Sanitizes the full body before sentence embedding and boundary detection.

  • src/core/chunkers/llm.ts — LLM-guided chunking: pre-splits into 128-word candidates via the recursive chunker, then asks a Haiku-class model "where does the FIRST topic shift occur?" per window.

  • src/core/search/dedup.ts — 4-layer result dedup + compiled-truth guarantee: (1) top 3 chunks per page by score, (2) drop chunks >0.85 Jaccard-similar to already-kept chunks of the SAME page (cross-page near-dups survive — two legitimately similar pages both return), (3) no page type exceeds 60% of results — applied only when the candidate set actually contains multiple page types; a homogeneous set passes through unchanged because dropping (for example) one of three distinct LongMemEval note sessions cannot introduce diversity and destroys multi-session recall (pinned by the homogeneous-set case in test/dedup.test.ts), (4) max 2 chunks per page (default; the two-pass structural expansion in hybrid.ts widens it), (5) ensure at least 1 compiled_truth chunk per page. Page identity is the composite pageKey() (source_id, slug) — the one canonical key helper every layer uses, so slug collisions across sources can't collapse recall.

  • src/core/audit-slug-fallback.ts — Weekly ISO-week-rotated audit JSONL at ~/.gbrain/audit/slug-fallback-YYYY-Www.jsonl. logSlugFallback(slug, sourcePath) fires when importFromFile falls back to a frontmatter slug because slugifyPath returned empty (emoji / Thai / Arabic / non-CJK exotic-script filenames). readRecentSlugFallbacks(days) reads the last N days for gbrain doctor's slug_fallback_audit check. Honors GBRAIN_AUDIT_DIR via the shared resolveAuditDir(). Separate surface from sync-failures.jsonl — that file carries bookmark-gating semantics that info events shouldn't trigger.

  • src/core/embedding-pricing.tsEMBEDDING_PRICING map keyed provider:model for the post-upgrade reindex cost estimate. Sibling to anthropic-pricing.ts; EMBEDDINGS only — chat/completion pricing lives in model-pricing.ts (different unit) and is never mixed in. Every entry carries its official source URL + the date it was last read. Entries: OpenAI text-embedding-3-large ($0.13/1M), 3-small ($0.02/1M), ada-002 ($0.10/1M); Voyage 4-large ($0.12/1M), 4 ($0.06/1M), 4-lite ($0.02/1M), legacy 3-large ($0.18/1M), 3 ($0.06/1M); ZeroEntropy zembed-1 ($0.05/1M), zerank-2 ($0.025/1M); Mistral mistral-embed ($0.10/1M); Perplexity pplx-embed-v1-4b ($0.03/1M), 0.6b ($0.004/1M). voyage-4-nano is deliberately unpriced (open-weight variant, no published hosted rate) so it degrades to "estimate unavailable" rather than a fabricated 0. lookupEmbeddingPrice(modelString) returns a tagged union (known with price + unknown with provider name); estimateCostFromChars(charCount, pricePerMTok) uses 3.5 chars/token. Unknown providers degrade to "estimate unavailable" instead of fabricating numbers.

  • src/core/post-upgrade-reembed.ts — Pure functions backing the gbrain upgrade chunker-bump cost prompt. computeReembedEstimate(engine, model) queries real SQL (COUNT(*) + COALESCE(SUM(LENGTH(compiled_truth)) + SUM(LENGTH(timeline)), 0)) on pages WHERE chunker_version < MARKDOWN_CHUNKER_VERSION. formatReembedPrompt(est, graceSeconds) is the stderr-line formatter. runPostUpgradeReembedPrompt(engine, model, opts) orchestrates the 10-second Ctrl-C window; TTY-only wait (non-TTY auto-proceeds for CI / cron); GBRAIN_NO_REEMBED=1 bails with a doctor-warning marker; GBRAIN_REEMBED_GRACE_SECONDS=0 skips the wait.

  • src/commands/reindex.tsgbrain reindex --markdown [--type PAGE_TYPE] [--limit N] [--dry-run] [--json] [--no-embed] [--repo PATH]. Walks markdown pages with stale chunker_version (plus unstamped contextual-retrieval mode when embedding is enabled) in 100-row id-keyset batches; a failed row cannot starve later batches in the same invocation. --type adds a bound-parameter pages.type = $N scope for focused backfills such as atom pages and is rejected by reindex modes that do not consume it. Current-version chunkless healing deliberately remains owned by the native, bounded embed --stale path. Rows with non-null source_path re-import via importFromFile; rows without fall back to importFromContent. Both paths pass forceRechunk: true to bypass importFromContent's content_hash short-circuit — without it the chunker version bump never reaches pages whose source content hasn't changed, AND the stripFactsFence privacy strip never applies to chunks stored without it. Wired into src/commands/upgrade.ts:runPostUpgrade after apply-migrations. The DB-only fallback (no source file on disk) does NOT pass body-only compiled_truth to importFromContent (that path re-parses with EMPTY frontmatter and OVERWRITES the page's real frontmatter/title/timeline); it getPage+getTags, reconstructs FULL markdown via serializeMarkdown(frontmatter, compiled_truth, timeline, {type, title, tags}), and re-imports THAT so re-chunking a DB-only page preserves everything while bumping chunker_version. Pinned by test/reindex-preserve-tags.test.ts and test/reindex.test.ts.

  • src/commands/reindex-code.tsgbrain reindex --code [--source ID] [--dry-run] [--yes] [--json] [--force] [--no-embed]. Walks pages WHERE type = 'code' in 100-row batches, replays through importCodeFile for chunk + embed + content_hash folding. Idempotent unless --force bypasses the content_hash early-return. Cost-preview model field reads getEmbeddingModelName() from the gateway so preview reflects what the gateway will actually embed with. An informational stderr nudge inside runReindexCode (so dry-run + execute both surface it): when the configured embedding model isn't code-tuned (allowlist {'voyage-code-3'}, case-insensitive bare match), prints a recommendation to switch to voyage:voyage-code-3; suppress with GBRAIN_NO_CODE_MODEL_NUDGE=1, --no-embed, or --json. Pure shouldNudgeCodeModel(bareName) returns a tagged NudgeDecision union (takes the bare model name, emits qualified voyage:voyage-code-3 for the paste-ready gbrain config set line). When --yes is absent and the caller is non-TTY or passed --json, the cost gate refuses (exit 2, no spend) via the pure exported buildCostRefusal({json, previewMsg, preview, costUsd, model}): {stdout?, stderr?} — JSON envelope only when --json is explicit, otherwise a human refusal on stderr (the spend guardrail is independent of the output format). spend.posture=tokenmax OR an explicit --max-cost off/unlimited makes the gate informational and proceeds; --max-cost off also disables the runtime BudgetTracker cap. Pinned by test/ai/voyage-code-3-recipe.test.ts, test/reindex-code-nudge.serial.test.ts, test/reindex-code-model-source.serial.test.ts (IRON-RULE guard for the cost-preview model source), test/reindex-cost-refusal.test.ts.

  • src/core/fts-language.ts — Single source for the Postgres text-search configuration name used by FTS. getFtsLanguage() resolves GBRAIN_FTS_LANGUAGE (default english), validates against /^[a-z][a-z0-9_]*$/ (tsvector config names can't be bound as parameters, so the value is interpolated into raw SQL — the allowlist regex is the injection guard; invalid values warn once and fall back to english), and caches on first read (resetFtsLanguageCache() is test-only). Consumed by both engines' searchKeyword/searchKeywordChunks (websearch_to_tsquery query side), the configurable_fts_language migration, and reindex-search-vector (write-side trigger functions). Pinned by test/fts-language.serial.test.ts + test/fts-language-migration.serial.test.ts (includes the '; DROP TABLE pages; -- injection cases).

  • src/commands/reindex-search-vector.tsgbrain reindex-search-vector [--dry-run] [--yes] [--json]. Escape hatch for changing GBRAIN_FTS_LANGUAGE after the configurable_fts_language migration has run (the migration shows applied and is skipped): recreates update_page_search_vector + update_chunk_search_vector with the configured language — bodies mirror the migration's and KEEP the SET search_path = pg_catalog, public hardening (CREATE OR REPLACE resets proconfig) — then backfills pages (UPDATE-to-self re-fires the trigger) and content_chunks (direct vector recompute) in id-keyset batches of BACKFILL_BATCH_SIZE (5000) via UPDATE … WHERE id IN (SELECT … LIMIT n) RETURNING id, streaming phases reindex_search_vector.pages/.chunks through the shared progress reporter (stderr). Confirmation gate: --yes, or an interactive TTY [y/N]; --json does NOT bypass the gate (non-TTY without --yes refuses with a ConfirmationRequired envelope, exit 2). Idempotent. Pinned by test/reindex-search-vector.serial.test.ts.

  • src/commands/sync.tsgbrain sync CLI + the performSync / performFullSync library entrypoints (consumed by the autopilot cycle and the Minion sync handler). performSyncInner resolves the source's persisted config.strategy (markdown/code/auto) when the caller passes no strategy — the autopilot lane, the dream cycle, the MCP sync op and the single-source CLI path all omit it, and without this the walk fell back to markdown, importing nothing from a code source while still advancing the anchor and soft-deleting its modified code pages; an explicit --strategy still wins, so the --all fan-out is unchanged. Pinned by test/sync-index-matches-tree.serial.test.ts (index == working tree across first/incremental/full sync). Six pure-function clusters live in src/core/sync-{cost-gate,git,anchor,lock,reconcile,status-report}.ts (see the grouped entry below) and are re-exported through this façade, so the full import surface is unchanged. performSync runs under a writer lock: per-source gbrain-sync:<sourceId> whenever opts.sourceId is set, wrapped in withRefreshingLock from src/core/db-lock.ts so long-running sources (250K+ chunks) don't lose the lock at the 30-min TTL mid-run; the bare no-source path uses the SAME refreshing lock; SyncOpts.lockId?: string is the explicit override. The lock refresh AND its health probe route through the DIRECT session pool so Supavisor transaction-pool exhaustion (EMAXCONNSESSION) can't kill renewal; takeover is heartbeat-aware (it will NOT steal a holder whose last_refreshed_at is within GBRAIN_LOCK_STEAL_GRACE_SECONDS, defending an alive-but-starved holder); the import loop yields the event loop every GBRAIN_SYNC_YIELD_EVERY files (setTimeout(0), not setImmediate — Bun starves the timers phase) so the refresh setInterval heartbeat fires mid-import. This lock-identity invariant prevents a sync --all per-source worker racing sync --source foo on the global lock from corrupting the same source. performSync throws a typed SyncLockBusyError when the writer lock is held; the Minion sync handler (src/commands/jobs.ts) catches it and marks the job SKIPPED (not failed) so a cron/autopilot tick defers to the holder without polluting crash metrics. performSyncInner is RESUMABLE (incremental path): it drains a PINNED target commit (lastCommit..pin), banking drained file paths via appendCompleted (append-only delta into the op_checkpoint_paths child table, migration v115 — one row per path, O(delta) rather than an O(N²) full-array rewrite), keyed by syncFingerprint({sourceId, lastCommit}) from src/core/op-checkpoint.ts (paths under op:'sync'; the pinned target under op:'sync-target'), and advances last_commit/last_sync_at ONLY at full import completion. Checkpoint writes route through the DIRECT session pool + bounded retry so they survive EMAXCONNSESSION; the flush cadence is first-file then every GBRAIN_SYNC_CHECKPOINT_EVERY (default 1000) files OR GBRAIN_SYNC_CHECKPOINT_SECONDS (default 10s), with a race-safe pendingCheckpointPaths delta (single-flight swap, re-merge on failure) under parallel workers; a SIGTERM banks the in-flight delta via a no-retry one-shot (appendCompletedOnce, ordered before lock release through registerCleanup); and sustained flush failure aborts the run with reason:'checkpoint_unavailable' after GBRAIN_SYNC_MAX_CHECKPOINT_FAILURES consecutive fails rather than importing work it can never bank (every partial/blocked exit logs the banked-file count). A sync killed mid-import banks its progress, leaves the anchor unmoved (the source stays correctly stale to the autopilot scheduler — last_sync_at is never bumped on a partial), and the next run resumeFilters the same fixed diff to skip done files. The pin is the checkpoint's stored target when still reachable from HEAD, else live HEAD (a history rewrite / reset re-pins); completion advances to the pin, NOT live HEAD, so commits landing past the pin are a clean next-sync diff and there is no cross-run staleness window. After import a pin-reachability gate (git merge-base --is-ancestor pin HEAD) decides: forward commits on top of the pin (e.g. a background enrich process committing to the same repo every ~2 min) are SAFE and do not block the run; only a real rewrite (pin not an ancestor of HEAD) blocks and discards the checkpoint. A file added in lastCommit..pin but gone from disk (deleted by a commit after the pin) is SKIPPED and marked completed, not recorded as a failure. Downstream extract/facts/embed are size-gated: inline only for totalChanges <= 100; large syncs defer to the resumable extract --stale watermark + embed --stale/backfill + the facts cycle phases, so a 44K-page facts/embed pass never re-blocks import convergence (sync convergence == import convergence). Worker engines wrap in try/finally so disconnect always fires; both PGLite-detection sites use engine.kind === 'pglite'. CLI accepts --workers N (alias --concurrency N) validated via parseWorkers (explicit bypasses the file-count floor; auto path defers to autoConcurrency()). The newest-first descending-lex order uses sortNewestFirst(addsAndMods) from src/core/sort-newest-first.ts (shared with gbrain import). gbrain sync --all runs a continuous worker pool: parseWorkers-validated --parallel N (default min(sourceCount, --workers, DEFAULT_PARALLEL_SOURCES=4)), long-lived async workers pulling from a shared FIFO queue (no head-of-line blocking), per-source withSourcePrefix(src.id, ...) so every slog/serr line carries [<source-id>]; --skip-failed/--retry-failed are scoped per source (acknowledgeFailures(sourceId); --all acks every source, single-source acks only its own) and run UNDER parallel — the failure ledger is per-(source_id, path) and serialized through withLedgerLock, so recovery syncs never need --serial; a connection-budget stderr warning fires when parallel×workers×2>16\text{parallel} \times \text{workers} \times 2 > 16 (the ×2perfilepool \times 2 \text{per}-\text{file} \text{pool} factor: each per-file worker opens its own PostgresEngine with poolSize=2). Exports resolveParallelism, syncOneSource, buildSyncStatusReport, printSyncStatusReport, SyncStatusReport back the gbrain sources status dashboard. --json envelope {schema_version: 1, sources, parallel, ok_count, error_count, skipped_count} on stdout; human banners route to stderr via humanSink so jq parses cleanly. Exit matrix: 0 all ok (sources skipped by --missing-path skip count as ok), 1 any error. --missing-path <fail|skip> (default fail) handles sources whose local_path does not exist on this machine — machine-specific state in a brain-wide table, so a brain registered from several machines fails every foreign source on every run; skip classifies them skipped_missing_path (⊘ line, envelope entry with local_path, excluded from error_count and the rc gate) via the exported pure helpers parseMissingPathMode + partitionMissingPathSources, pinned by test/sync-all-missing-path.test.ts; default fail stays loud because on a single-machine brain a missing path usually means an unmounted volume. (The non-TTY cost gate does not exit 2: a worker-backed engine can auto-defer, while a runtime without a worker surface reports a manual drain instead of claiming a queued job.) The dashboard SQL is content_chunks ch JOIN pages pg ON pg.id = ch.page_id WHERE pg.deleted_at IS NULL with archived = false at the caller; embedding column resolved via resolveEmbeddingColumn(undefined, cfg) from src/core/search/embedding-column.ts so Voyage/multimodal/non-default-column brains count against the column they use; errors propagate (no swallow-catch). Every sync delete lane (removed-file drain, full-sync reconcile, un-syncable-page cleanup, rename-stale reconcile) SOFT-deletes via engine.softDeletePages: deleted_at is set for a 72h recovery window, the autopilot purge phase owns the eventual hard delete, and a re-import within the window revives the page through putPage's upsert (lanes without a threaded sourceId fall back to DEFAULT_SOURCE_ID, matching deletePage's 'default' scope). The removed-file drain is interleaved per-batch resolve+soft-delete using engine.resolveSlugsByPaths + engine.softDeletePages from src/core/engine.ts (a 73K-delete commit takes ~292 SQL round-trips instead of ~146K, so one big-delete commit cannot jam every other source's sync); per-batch try-catch decomposes batch failures to one-element softDeletePages batches per slug (same primitive, per-slug isolation), unrecoverable per-slug failures land in failedFiles and the run continues; pagesAffected filters to slugs that actually transitioned (phantom slugs and already-soft-deleted rows are excluded by the primitive's deleted_at IS NULL predicate). The rename loop gets the same batched slug-resolve; a rename whose destination is un-syncable folds the source path into the delete set so the old page can't orphan. An entry-time bookmark-reachability guard distinguishes a gc'd anchor (cat-file fails → performFullSync) from a history-rewrite anchor that is merely no-longer-an-ancestor but still on disk: the latter is diffed tree-to-tree directly (git diff lastCommit..pin is an endpoint-tree compare, ancestry not required) so a force-push / mastermain consolidation imports only the real delta instead of re-walking the whole tree forever; an oversized or failed diff degrades to performFullSync. performFullSync is itself authoritative for deletes — after an advancing full import it soft-deletes file-backed pages (source_path != null AND strategy-aware isSyncable) whose source file no longer exists (same softDeletePages + 72h-window semantics as the incremental lanes; already-soft-deleted rows don't inflate the reconcile count), sparing put_page/manual pages (null source_path) and metafiles. The stale-file decision routes through the pure, exported planReconcileDeletes(rows, currentFiles, isSyncablePath): it normalizes path separators on both sides of the membership test (a Windows path.relative backslash path vs a git-derived forward-slash source_path would otherwise mark every page stale and wipe the source) and computes a mass-delete signal — when the reconcile would delete more than MASS_RECONCILE_RATIO (50%) of the file-backed pages the strategy manages, on a source holding more than MASS_RECONCILE_MIN_PAGES (20) of them, the delete is REFUSED with a loud stderr warning (that shape is almost always a path-comparison bug or the wrong repo path, not a genuine bulk deletion); GBRAIN_ALLOW_MASS_RECONCILE=1 restores the unguarded delete for genuinely intended bulk removals. Pinned by test/sync-reconcile-mass-delete.test.ts. Below the valve, stale pages are partitioned by git history via exported listEverCommittedPaths(repoPath) (one git log --all --no-renames --diff-filter=A --name-only pass; null on non-git dirs → unchanged behavior): a stale path that EVER existed in history was genuinely deleted → reconciled; a path with NO history is DB-only write-through (never committed/pushed, e.g. lost to a fresh clone) → the page is KEPT and its markdown re-exported to the working tree via writePageThrough, with a stderr hint to commit it ("absent from git" is the symptom of the missing write-through commit, not evidence the content is disposable). Pinned by test/sync-reconcile-db-only.serial.test.ts. resolveSlugByPathOrSourcePath (in src/core/sync-git.ts — see its dedicated entry below) delegates to engine.resolveSlugsByPaths when sourceId is set, keeping the legacy executeRaw fallback for the no-sourceId path. failedFiles is hoisted to the top of performSyncInner so both delete-decompose and import loops feed the same bookmark gate. The cost gate is the shared runInlineCostGate (one implementation on BOTH the --all and single-source paths; runs at the command layer, never inside performSync), mode-aware via resolveWorkerBackedSyncEmbedMode + posture-aware shouldBlockSync from src/core/embedding.ts. The DEFERRED path (v2 on, explicitly Postgres/worker-backed, and not --serial; source count controls fan-out only) is INFORMATIONAL (embedding goes to per-source embed-backfill jobs with their own $X/source/24h cap, default $25 via SPEND_CAP_CONFIG_KEY from embed-backfill-submit.ts; prints the cap + backlog + queued-job count, NEVER exits 2). The INLINE path (v2 off, --serial, or PGLite/unknown engine) gates on the DELTA estimate vs sync.cost_gate_min_usd (default $0.50): below floor proceeds; above floor in a TTY prompts [y/N]; above floor in a non-TTY/--json session auto-defers only when a worker-backed queue is actually available. Without that worker surface it exits 0 without wedging import convergence, writes no queue row, reports manual_drain_required with reason no_worker_surface, and emits one exact gbrain embed --stale --source <id> command per source; a worker-backed runtime with --no-auto-embed reports the distinct auto_submit_disabled policy reason without denying queue capability; intrinsic >100-file incremental deferral reaches the same final manual outcome on no-worker runtimes; spend.posture=tokenmax makes it informational and proceeds inline. The estimate MIRRORS EXECUTION instead of pricing the whole tree: estimateInlineNewTokens routes through the shared computeSyncDelta (src/core/sync-delta.ts) — fetch-first against origin/<branch>, prices only the committed delta (a dirty-but-caught-up tree → $0), with a full-tree CEILING only on the fail-open rungs (chunker drift, first sync, git-unavailable) honestly labeled; --full adds the stale backlog (full sync sweeps it inline). Return shape carries estimateKind: 'delta'|'ceiling'|'mixed'|'unchanged' + ceilingReasons. Helpers resolveCostGateFloorUsd(engine) + resolveBackfillCapUsd(engine) resolve via parseUsdLimit (off/unlimitedInfinity; floor accepts 0 = block-on-any-spend). JSON envelopes carry mode + gate discriminators (dry_run | deferred_notice | below_floor | auto_deferred_embeds | manual_drain_required | posture_tokenmax) + a paste-ready hint; terminal single-source and --all envelopes carry per-source embed_backfill outcomes so machine readers see queued/manual/skipped state; Infinity floors/caps render as the string 'unlimited' (never raw, which JSON-serializes to null); SyncStatusReportSource gains backfill_queued/backfill_active/backfill_last_completed_at; cost previews read getEmbeddingModelName() (no hardcoded OpenAI). Format splits on the explicit --json flag only (human text otherwise). SyncOpts.noSchemaPack (CLI --no-schema-pack, threaded through performSync AND syncOneSource) skips loadActivePack so pages fall back to legacy prefix typing — an escape hatch when a suspect pack regex wedges a sync. A per-file BEGIN heartbeat if (process.env.GBRAIN_SYNC_TRACE) serr('[sync] begin import: <path>') fires BEFORE importFile (the progress.tick fires only AFTER) so a stuck file is a begin-line with no matching completion. Triage doc: docs/architecture/serve-sync-concurrency.md (serve-delegated sync + the GBRAIN_SYNC_TRACE + --no-schema-pack recipes). On a PGLite host brain with a live gbrain serve, the cli.ts pre-connect hook routes gbrain sync through src/commands/sync-delegate.ts (delegation over the resolve-IPC socket; default-deny flag gate; 1s status polls; Ctrl-C → sync_abort; embeds always deferred to the serve's drain) — SyncOpts.onProgress is the seam performSync fires at phase boundaries and checkpoint flushes so the serve-side job record carries live progress; printSyncResult is exported for the delegated result path. Pinned by test/e2e/sync-status-pglite.test.ts (IRON-RULE: PGLite seeds 2 sources × pages × chunks, soft-deletes 1 page, archives 1 source, validates the SQL excludes both AND uses the active embedding column), test/sync-cost-gate.serial.test.ts, test/sync-cost-preview.test.ts. Runaway-sync protection: resolveSyncHardDeadline(args, {isTty, env, defaultNonTtySec?}) resolves a wall-clock hard deadline (precedence --no-hard-deadline > --hard-deadline <s> > --timeout <s>(non---all, which auto-arms the backstop) > GBRAIN_SYNC_MAX_RUNTIME_SECONDS env > non-TTY default 3600s > none; HARD_DEADLINE_GRACE_SEC=30). src/cli.ts installs the out-of-band watchdog (see src/core/process-watchdog.ts) for the sync command BEFORE connectEngine and disposes it in the dispatch finally, so even an event-loop-starved sync — or a connect-phase hang — is SIGTERM-then-SIGKILLed by the deadline instead of orphaning under cron. runSync registers a SIGINT handler that aborts an interrupt AbortController composed via composeAbortSignals(...) (an AbortSignal.any wrapper over the defined signals) with the per-source --timeout signal, so Ctrl-C returns a clean partial and releases the lock through the normal finally (process-cleanup.ts owns SIGTERM lock-release; the watchdog owns the hard kill). withRefreshingLock unref()s its refresh setInterval. The spin's own root cause is not yet pinned (leading lead is catastrophic-backtracking in a pack link-inference regex, bounded by the redos-guard); the watchdog heartbeat plus the existing [gbrain phase] breadcrumbs are the diagnosis surface. Pinned by test/sync-hard-deadline.test.ts (resolution precedence + composeAbortSignals). Monorepo subdir sources: --src-subpath <dir> (or a repo path that IS a subdir — auto-discovery via discoverGitRoot, i.e. git rev-parse --show-toplevel) splits the repo path into gitContextRoot (all git ops: pull/diff/rev-parse/cat-file) and syncScopeRoot (walk/import/delete/rename scope); scoped syncs use git-root-relative slugs + source_path (full sync threads slugRoot into runImport) so full and incremental agree; NAV-1/NAV-2 realpath containment rejects ../-traversal and symlinked scopes resolving outside the repo BEFORE any git op, and a per-file realpath guard (isPathSafe) refuses symlink-escape files in the incremental drain and rename reimport (fail-closed into failedFiles, so the bookmark can't advance past an escape); the full-sync reconcile is scope-restricted so a scoped sync never sweeps out-of-scope pages. --exclude <glob> (repeatable) filters scope-relative paths in both full and incremental paths; exclusion never deletes previously-imported pages (conservative, matching the metafile posture); an all-excluded run warns loudly (NAV-4). The persisted sync.exclude config key (comma- or newline-separated patterns; a trailing / normalizes to a <dir>/** subtree glob) is UNIONED with per-call --exclude on EVERY sync path — union, not override, so an ad-hoc flag narrows further but never silently re-opens a scope the operator persisted, and internal callers with no flag surface (autopilot, minion sync jobs, dream cycle) inherit the same indexing scope. CAVEAT: the key is BRAIN-GLOBAL — one exclude list scopes EVERY source, including sync --all (where the per-call flag is forbidden), so a pattern meant for repo A silently narrows repo B; per-source scoping is a filed follow-up. The union resolves at the top of performSyncInner, ABOVE the performFullSync early returns — position is load-bearing: the first sync's full walk is exactly where ignoring the persisted scope would permanently import excluded files that no later incremental revisits. Best-effort config read (an unreadable config never breaks a sync). A warn-and-continue internal git pull failure (non-timeout class — e.g. a local-path origin rejected by protocol.file.allow=never) still falls through to sync the local working tree, but a ZERO-import run after a failed pull returns partial with reason: 'pull_failed' instead of up_to_date: last_commit AND the last_sync_at heartbeat stay frozen (so doctor sync_freshness / sources status staleness fires), the single-source CLI exits non-zero, sync --all exits non-zero if any source hit it (JSON envelope carries the per-source reason), and the autopilot cycle's sync phase maps it to warn. Timeout-class partials keep their pre-existing exit-0 / phase-ok semantics (they converge on retry; a failing pull does not). Pinned by test/sync-pull-failed-anchor.serial.test.ts. resolveSlugByPathOrSourcePath is threaded into all 4 delete/rename call sites — see its dedicated src/core/sync-git.ts entry below for the resolution contract. Above the size gate (totalChanges > 100) the deferred link/timeline extraction is DURABLY QUEUED, not just hinted — the defer branch submits an extract Minion job {stale: true, sourceId?, deferred_commit: pin} keyed extract-stale:<sourceId|default>:<pin> (repeat submissions toward the same drained pin coalesce; deliberately NO maxWaiting — an unscoped payload's coalesce filter matches ANY waiting extract job and would silently drop the sweep), timeout_ms derived from extract.ts's exported STALE_TIME_BUDGET_MS + headroom. The returned row is verified to be a live {stale:true} job (waiting/delayed/active) before the log claims "queued"; a finished row occupying the key slot (a prior sweep toward the same pin that completed before this run's pages landed — the checkpoint-resume / blocked-advance re-sync case) triggers a fresh submission under a run-unique key so those pages never strand stale. Submission is best-effort (failure falls back to the hint; pages stay stale + doctor-visible, never mis-stamped). Pinned by test/sync-deferred-extract-queue.serial.test.ts. Unscoped default-write guard: when the resolver lands on tier seed_default for a single-source sync (not --all) and GBRAIN_ALLOW_DEFAULT_WRITE is unset, assessDefaultWriteGuard (src/core/source-resolver.ts) decides whether the brain's pages overwhelmingly live outside default; if so the run prints formatDefaultWriteRefusal('sync', …) and exits 1 — except under --dry-run, which prints the same text prefixed [dry-run] a real run would be refused: and still previews (a dry run writes nothing; refusing it only hides information). Escapes: --source default (tier flag), --all, or GBRAIN_ALLOW_DEFAULT_WRITE=1; a failed assessment is fail-open. Pinned by test/sync-default-write-guard.serial.test.ts + test/sync-default-write-guard-dry-run.test.ts.

  • src/core/sync-git.ts:resolveSlugByPathOrSourcePath — Resolves a slug by pages.source_path first (returns the stored slug for frontmatter-fallback pages whose path doesn't derive a slug), then falls back to resolveSlugForPath(path). Three call sites: the un-syncable cleanup (which passes opts.sourceId), and the bare no-sourceId delete and rename-source paths; when sourceId IS set the delete and rename loops batch through engine.resolveSlugsByPaths instead. The source_path lookup is scoped to the DEFAULT source on the bare path, matching every write that path performs (updateSlug, deletePage, the rename bookkeeping repair are all default-scoped) — an unscoped lookup could return a row from another source sharing the source_path, and the rename loop consumes this value directly as its cheap-move source slug. Without the helper, emoji-only / Thai / Arabic filenames whose slug came from frontmatter would orphan on delete/rename (that path would compute the wrong path-derived slug). Best-effort query — pre-migration brains fall through to the legacy path.

  • src/core/sources-ops.ts — Multi-source registration + clone-lifecycle ops (addSource, recloneIfMissing, defaultCloneDir, isOwnedClone, unownedHint). Reclone-ownership invariant (must-never-violate): gbrain may only delete/re-clone a clone it created, NEVER a user working tree. recloneIfMissing deletes local_path, so it gates on isOwnedClone(src) and throws a SourceOpError('unmanaged_path', ...) BEFORE any filesystem op when ownership is unprovable — fail-closed. Ownership is proven by config.managed_clone === true (written by addSource's --url path, covering default-location and --clone-dir clones) OR local_path === defaultCloneDir(id) (back-compat for pre-marker clones, via exact normalized-path equality, symlink-free). A row with remote_url + an unowned local_path (a user-registered working tree, e.g. sources add --path) is refused untouched; re-add with --url to regain auto-reclone. The reclone is EXDEV-safe: clone into a SIBLING temp of local_path (not the shared clones/.tmp, which may sit on a different mount than a --clone-dir target), then swap (move old aside → move new in → drop old) so local_path is never left missing-and-unrecoverable; on swap failure the original is restored, and if restore fails the error names the aside path so it's never reflexively deleted. A TOCTOU re-check re-confirms ownership immediately before the destructive move and rejects a symlink leaf swapped in after the entry check (symlink_escape). unownedHint(src, state) is the shared recovery message used by both the core error and the gbrain sync --source CLI error; gbrain sources restore special-cases unmanaged_path to print "DB row restored; gbrain syncs this path read-only" instead of the misleading "try sync to recover" guidance. SourceOpErrorCode includes unmanaged_path. Pinned by test/sources-ops.test.ts, test/sources-resync-recovery.test.ts. assertNoOverlappingPath(engine, id, path) is the ONE overlapping-path guard shared by every surface that binds a local_path (addSource, gbrain sources set-path): a path equal to, nested inside, or enclosing another source's local_path throws SourceOpError('overlapping_path') — overlapping trees make sync/write-through attribute files to the wrong source, so no add or repair path may skip it. The guard compares by spelling AND by realpath (both the candidate and each sibling local_path are realpathSync'd when they exist, fail-open to the spelling otherwise), so a symlink resolving into or over another source's tree cannot slip past it; the stored pointer and the error message keep the operator's spelling.

  • src/core/utils.ts — Shared SQL utilities used by both engines. Exports parseEmbedding(value) (throws on unknown input, used by migration + ingest paths where data integrity matters) and tryParseEmbedding(value) (returns null + warns once per process, used by search/rescore paths where availability matters more than strictness). isUndefinedColumnError(err) predicate — pattern-matches Postgres SQLSTATE 42703 / "column ... does not exist" with engine-driver shape variation tolerated; used by oauth-provider.ts so genuine errors (lock timeout, network blip, permission denied) propagate while column-missing falls through to the legacy fallback. validateSourceId(id) throws on anything outside ^[a-z0-9_-]+$, used by the per-source disk-layout fix in patterns.ts/synthesize.ts before any join(brainDir, '.sources', source_id, slug+'.md') so source_id can't traverse out of brainDir. rowToSearchResult projects email message_id / thread_id metadata and exposes source_subject only when a non-empty Message-ID proves the page is an email, so generated page titles never become authoritative email subjects. rowToPage populates the required Page.source_id from the SELECT projection (scripts/check-source-id-projection.sh enforces every projection feeding rowToPage includes the column). bigintToStringReplacer(key, value) — the JSON replacer turning bigint into strings (the postgres.js int8 wire shape), shared by cli.ts's local-result normalizer (which re-exports it) and the commands that stringify results themselves (gbrain call, extract --explain); commands import it from here, never from the dispatcher. Its doc comment deliberately carries no double-dash flag literals — the flag-registry generator harvests them from every module a command transitively imports, and utils.ts is imported by nearly all.

  • src/core/llm-json.ts — tolerant JSON extraction from LLM output. parseLlmJson<T>(raw, {array?}) walks a strategy ladder (strip ```json fences → direct parse → first {...}/[...] substring), then — only after the raw parse has failed — retries the same ladder with reasoning blocks stripped, so a payload that parses raw is untouched and a payload legitimately containing <think> is unaffected; returns null on any failure (adversarial throws swallowed). Exports stripReasoningBlocks(raw): removes closed <think>…</think> pairs AND a truncated never-closed <think> tail (output-budget exhaustion) — reasoning models draft their JSON inside the think block, which defeats the greedy first-to-last-brace substring scan. src/core/facts/extract.ts's hand-rolled parseExtractorJsonDetailed applies the same raw-first fallback ladder with the shared stripReasoningBlocks.

  • src/core/db.ts — Connection management, schema initialization. resolveSessionTimeouts() returns statement_timeout + idle_in_transaction_session_timeout (defaults 5min each, env-overridable via GBRAIN_STATEMENT_TIMEOUT/GBRAIN_IDLE_TX_TIMEOUT/GBRAIN_CLIENT_CHECK_INTERVAL). Both connect() (module singleton) and PostgresEngine.connect() (worker pool) consume the result via postgres.js's connection option, sending GUCs as startup parameters that survive PgBouncer transaction mode (setSessionDefaults kept as a back-compat no-op shim). connect() returns Promise<boolean>true iff THIS call created the module singleton, false if it joined an existing one; the decision is atomic (no await between the if (sql) null-check and the synchronous sql = postgres(...) assignment), so two concurrent module connects can't both claim creation. PostgresEngine stores the return as its _ownsModuleSingleton token and only the creating engine may db.disconnect() the singleton — a borrower probe engine (lint/doctor config-lift) no-ops its disconnect, so its teardown can't null the connection the long-lived cycle owner is still using (the dream-cycle "connect() has not been called" failure). The module sql is only ever nulled by db.disconnect() (postgres.js auto-reconnects its own internal pool and never touches our reference). disconnect() snapshots + nulls sql before awaiting the pool end so a concurrent connect can't join a pool that's already closing. The end routes through endPoolBounded(pool) — a gbrain-owned Promise.race of pool.end({ timeout: POOL_END_TIMEOUT_SECONDS }) against a hard timer — so a PgBouncer transaction-mode drain that never settles can't hang teardown — the CLI teardown contract (finishCliTeardown's computed-deadline backstop + flushThenExit's fence-and-grace exit in cli-force-exit.ts) bounds it and delivers pending stdout before exit. connection-manager.ts ends its direct + read pools concurrently through the same helper so the per-pool bounds don't stack. resolveMaxLifetimeSeconds(env?) — explicit client-pool max_lifetime for all four postgres() call sites (matches the postgres.js implicit 30-60min jittered default; GBRAIN_POOL_MAX_LIFETIME_S overrides, 0 disables; warn-once on invalid). Pinned by test/db-pool-max-lifetime.test.ts.

  • src/core/pool-gauge.tsCheckoutGauge: approximate in-flight counters at the engine's raw/direct/reserved/tx seams, surfaced via duck-typed PostgresEngine.getPoolDiagnostics() (no BrainEngine change, no PGLite stub). HONESTY CONTRACT in the module doc: tagged-template traffic is untracked; consumers must label counts as a subset and never derive waiter/available figures. Fail-open (clamped release, try/finally around sync-throwing runUnsafe). Consumed by db-probe.ts. Pinned by test/pool-gauge.test.ts.

  • src/commands/migrate-engine.ts — Bidirectional engine migration (gbrain migrate --to supabase/pglite). Copies the complete source catalog FIRST (copyMigrationSources — every sources row incl. archived rows and sync/routing metadata, ON CONFLICT (id) DO UPDATE, default ordered first) so every page write has a valid pages.source_id FK parent and the target preserves per-source behavior; pages copy afterward, tracked in the resume manifest by composite (source_id, slug) key. Link copy preserves each row's to_source_id (falling back to the origin source only for legacy rows without it), and failed-target filtering uses that same target composite key, so cross-source links migrate without being rebound to the origin source. The resume manifest is target-aware: migrationTargetId(config) hashes (engine, locator) (database_url for Postgres, resolved database_path for PGLite) and manifestMatchesTarget requires schema_version === 2 plus a matching target_id — a legacy engine-only manifest, or one from a DIFFERENT target of the same engine kind, starts fresh instead of skipping "completed" pages the new target never received. After links, copyMigrationFacts carries the facts table verbatim (conversation facts have no markdown fence to re-sync from): source∩target column intersection via information_schema (older source schemas still copy), delete-and-recopy so every re-run converges to source truth, two-pass superseded_by restore (the self-FK points old→newer id, so id-ordered single-pass inserts would violate it), embeddings round-tripped as text into whichever physical type (vector/halfvec) the target column has with a NULL-embedding retry on per-row cast failure, and a setval bump so post-migration inserts don't collide with copied ids; fact-row failures block the config flip and force a non-zero exit, same contract as page failures. Takes are deliberately NOT copied (fence-canonical, keyed on target-reassigned page_id; the next extract cycle rebuilds them). copyMigrationConfig then copies EVERY DB-plane config row except the explicit engine-local denylist MIGRATE_CONFIG_ENGINE_LOCAL_KEYS (engine, version = target-owned schema ledger, embedding_columns + search_embedding_column = physical-column registry the copy doesn't materialize); skipped keys print in the end-of-run per-table copied-count summary, so nothing is dropped silently. Pinned by test/migrate-engine-resume.test.ts (manifest identity) + test/e2e/migrate-engine-sources-postgres.test.ts (source catalog lands before overlapping-slug pages, PGLite → real Postgres) + test/e2e/multi-source-bug-class.test.ts (cross-source links) + test/migrate-engine-completeness.serial.test.ts (facts chains/embeddings/sequence, config copy-all + denylist, summary).

  • src/core/import-file.ts — importFromFile + importFromContent (chunk + embed + tags). importFromContent and importCodeFile stamp pages.embedding_signature via setPageEmbeddingSignature(slug, {sourceId, signature: currentEmbeddingSignature()}) when the import actually embedded (not --no-embed) so a model/dims swap is detectable as stale; importCodeFile only stamps when every chunk was freshly embedded this call (needsEmbedIndexes.length === chunks.length), mixed reuse-by-hash pages stay unstamped (reindex --code --force / embed --stale handle those). importFromContent's tag reconciliation is ADD-ONLY: it only addTag (idempotent, ON CONFLICT DO NOTHING). The tags table has no provenance column and frontmatter tags are stripped from stored pages.frontmatter (markdown.ts:118), so a frontmatter-origin tag can't be distinguished from a DB-enrichment tag (auto-tag / dream synthesize / signal-detector) at re-import — deletion is unsafe (would wipe enrichment under gbrain reindex --markdown). Accepted trade-off: removing a tag from frontmatter does not remove it from the DB on next sync (needs a tag_source provenance column). Pinned by test/reindex-preserve-tags.test.ts + test/import-file.test.ts. identity-based dedup pre-check at :427-490. Calls engine.findDuplicatePage?.(sourceId, {hash, frontmatterId}) (optional ? so test doubles compile). Posture: SKIP when frontmatter.id matches (true external duplicate from overlapping ingest roots), WARN-ALWAYS on content_hash collision with different/missing frontmatter.id (templates and daily logs may legitimately share text), FAIL CLOSED on lookup error, bypass via --force-rechunk. Soft-deleted pages excluded at the engine layer so tombstones don't block legitimate re-imports under new slugs. Pinned by test/import-dedup-frontmatter-id.test.ts (11 cases). importFromContent is the narrow waist every ingest path passes through (gbrain import, gbrain sync, put_page MCP, /ingest webhook). It runs a three-tier content-quality disposition via assessContentSanity from src/core/content-sanity.ts BEFORE chunking: (1) high-confidence junk (built-in Cloudflare/CAPTCHA interstitial patterns + operator literals) → QUARANTINE (stamps the quarantine frontmatter marker, writes ZERO chunks, hides the page from search) OR REJECT (throw → sync-failure) when content_sanity.junk_disposition is reject; (2) fuzzy markup-heavy (prose-vs-markup ratio above content_sanity.max_markup_ratio, warn-tier byte window, code pages exempt) → content_flag:markup_heavy marker (page stays fully searchable, marker rides search results + get_page to warn the agent); (3) oversize → embed_skip soft-block via buildEmbedSkipMarker() PLUS a content_flag:oversized marker, AND deletes any pre-existing chunks in the same transaction so search can't surface stale chunks. Gate-owned markers (quarantine, content_flag) are STRIPPED from untrusted (remote MCP, ctx.remote !== false) frontmatter so a write-scoped client can't hide pages or forge the warning channel; markers are excluded from content_hash so a flagged page doesn't re-embed every sync. gbrain import honors errors > 0 for non-zero exit. classifyErrorCode in src/core/sync.ts recognizes the PAGE_JUNK_PATTERN code so sync-failures.jsonl grouping bins these. extractEntityRefs (canonical; matches both [Name](people/slug) markdown links and Obsidian [[people/slug|Name]] wikilinks), extractPageLinks, inferLinkType heuristics (attended/works_at/invested_in/founded/advises/source/mentions), parseTimelineEntries, isAutoLinkEnabled config helper. Link candidates match any dir-shaped path (existence-checked at persist). Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts. Pinned by test/import-file-content-sanity.test.ts. importCodeFile writes source_path: relativePath on every code page (the same repo-relative path markdown imports record) so the full-sync reconcile — which only considers source_path IS NOT NULL rows — can retire a deleted code file's page; putPage COALESCEs the column, so rows imported before this keep NULL until the file changes, sync --force, or gbrain reindex-code --force.

  • src/core/sync.ts — Pure sync functions (manifest parsing, filtering, slug conversion). Exported pruneDir(name: string): boolean is the single source of truth for descent-time directory exclusion across walkers — blocks node_modules (no leading dot, so a naive walker would descend into it and inflate MISSING_OPEN counts via vendor packages), vendor/dist/build/venv, dot-prefix dirs, and *.raw sidecars — NOT ops/, which is ordinary user content (the bundled daily-task-manager stores ops/tasks there); isSyncable applies it per path segment, and walkMarkdownFiles in src/commands/extract.ts + listTextFiles in src/core/cycle/transcript-discovery.ts consult it BEFORE recursing to save the IO of walking thousands of vendor files. manageGitignore worktree discriminator matches the gitdir path segment (/modules/<name> = submodule, /worktrees/<name> = worktree, per Git's documented layout) so Conductor worktrees (first-class repos) get .gitignore management for storage-tiering. The sync-failure ledger (failure store, error classifier, the shared bookmark gate, and the doctor severity rule) lives in src/core/sync-failure-ledger.ts; sync.ts re-exports classifyErrorCode, summarizeFailuresByCode, loadSyncFailures, unacknowledgedSyncFailures, acknowledgeSyncFailures, recordSyncFailures, decideSyncFailureSeverity, applySyncFailureGate, and the SyncFailure type for backward-compatible imports — see its entry below. isSyncable factored through private classifySync(path, opts): SyncableReason | null; exported companion unsyncableReason(path, opts) returns the same tagged reason or null when syncable. SYNC_SKIP_FILES is a named export (the four canonical metafile basenames schema.md, index.md, log.md, README.md). SyncableReason union: 'metafile' | 'strategy' | 'pruned-dir' | 'include-glob-miss' | 'exclude-glob-hit' | 'malformed-path'. Malformed filenames are TWO-TIER: hasMalformedPathSegment (ADMISSION — control chars on any path; square brackets on .md/.mdx paths only, so code-strategy lanes keep indexing app/[id]/page.tsx framework layouts) vs isPoisonedPath (DESTRUCTION — only the injection signature ]( or control chars; sync's row-DELETING lanes gate on this so a bare-bracket markdown row already in the DB survives reconcile while its file exists). sanitizePathForDisplay scrubs control bytes + caps length before echoing such paths. The commands/sync.ts cleanup loop guards on unsyncableReason(path) being 'metafile' OR 'pruned-dir' so previously-indexed metafile pages AND deliberately put-created pages under pruned dirs survive every re-sync. Does NOT cover manifest.deleted (the upstream filter already strips metafiles). Pinned by test/sync-isSyncable-shape.test.ts (15 cases, duality contract) + test/sync-metafile-skip.serial.test.ts (3 PGLite cases incl. the renamed .md → .txt negative). pruneDir: pruneDir(name, parentDir?) extended with optional parentDir. When provided, additionally rejects directories containing .git as a FILE — the git submodule gitfile pattern (regular repos have .git as a DIRECTORY; submodules as a file pointing into the parent's .git/modules/). Sync + extract walkers thread parentDir so the gitfile-as-FILE check fires per descend step. Best-effort: statSync failures fall through and treat as a normal dir. Prevents phantom imports from a worktree-with-submodules sync walking into submodule trees. Pinned by test/sync-walker-submodule.test.ts.

  • src/core/sync-failure-ledger.ts — the bounded auto-skip sync failure ledger. A LEAF module (imports only fs/path/crypto/config) so sync.ts can re-export it without a circular dependency. State lives in ~/.gbrain/sync-failures.jsonl, one JSON object per line, keyed by (source_id, path) with a per-key attempts count and a 3-state machine: open (fresh/blocking) → auto_skipped (chronic, still doctor-visible) or acknowledged (human resolved via gbrain sync --skip-failed from either unresolved state). classifyErrorCode(errorMsg) regex classifier with 23 named codes (RENAME_RECONCILE, SLUG_MISMATCH, DB_DUPLICATE_KEY, STATEMENT_TIMEOUT, YAML_PARSE, YAML_DUPLICATE_KEY, MISSING_OPEN, MISSING_CLOSE, EMPTY_FRONTMATTER, NULL_BYTES, NESTED_QUOTES, INVALID_UTF8, FILE_TOO_LARGE, SYMLINK_NOT_ALLOWED, TAKES_TABLE_MALFORMED, TAKES_HOLDER_INVALID, EMBEDDING_NO_CREDS, EMBEDDING_NO_TOUCHPOINT, EMBEDDING_TIMEOUT, EMBEDDING_RATE_LIMIT, EMBEDDING_QUOTA, EMBEDDING_OVERSIZE, PAGE_JUNK_PATTERN) plus UNKNOWN (the default fallthrough); summarizeFailuresByCode(failures) returns sorted [{code, count}]; MISSING_OPEN/EMPTY_FRONTMATTER regexes match the markdown.ts validator strings; MISSING_CLOSE's regex is stale against current markdown.ts output — it matches the no-heading-hint case (No closing --- delimiter found) but not the heading-hint case (No closing --- before heading at line N), which falls through to UNKNOWN. FILE_TOO_LARGE covers import-file.ts:395, 1236, 1436 (markdown/text, plain-file, and code-file import paths), SYMLINK_NOT_ALLOWED covers :1231, 2038 (file and image import paths). EMBEDDING_INFRA_CODES is the subset (EMBEDDING_TIMEOUT/EMBEDDING_RATE_LIMIT/EMBEDDING_QUOTA) that means the embedding PROVIDER is unhealthy rather than the file being poison — isEmbeddingInfraCode(code) is consulted inside decideGateAction's chronic-eligibility loop so these three are never counted toward auto-skip (treated as fresh, forcing block instead of advance_then_autoskip); an explicit --skip-failed still advances past them like any other failure, since that check runs before the loop. All mutations run under withLedgerLock (cross-process file lock) with an atomic rename write. The auto-skip threshold resolves via resolveAutoSkipThreshold() from GBRAIN_SYNC_AUTOSKIP_AFTER (default DEFAULT_AUTOSKIP_AFTER = 3; 0 disables the valve = pure fail-closed). Two pure decision functions are the unit-test surface: decideGateAction({fileFailures, sentinels, attemptsByPath, threshold, skipFailed}) returns hard_block | block | advance | advance_then_autoskip (sentinels like <head> ALWAYS hard-block, even with --skip-failed, so a history rewrite can't auto-skip; any FRESH failure with attempts < threshold blocks fail-closed; only when ALL failures are chronic does it advance_then_autoskip), and decideSyncFailureSeverity({entries, nowMs, failHours}) returns the sync_failures doctor status (ok when zero unresolved; fail when ≥10 OPEN-blocking or the oldest OPEN failure's ts (last attempt, NOT first_seen) is older than failHours — a failure retried inside that window keeps refreshing ts and never trips this leg even if first_seen is much older; otherwise warnauto_skipped-only rows stay WARN-visible regardless of count because the bookmark already advanced). applySyncFailureGate(input) is the one orchestrator BOTH sync paths (incremental + full/runImport) call: it records/clears ledger rows, runs decideGateAction, then executes effects in the crash-safe order (advance the bookmark FIRST via the injected advance() callback, THEN auto-skip the chronic set) so a crash can never mark a file skipped while leaving sync wedged. isSkippablePath rejects <…> sentinels. Pinned by test/sync-failure-ledger.serial.test.ts + test/sync-failures.test.ts.

  • src/core/sync-cost-gate.ts + sync-git.ts + sync-anchor.ts + sync-lock.ts + sync-reconcile.ts + sync-status-report.ts — six pure-function clusters behind the src/commands/sync.ts façade, which re-exports them; facadeExpansion in scripts/generate-flag-registry.ts keeps exactly these six (NOT the other sync-* siblings, which are ordinary deps) on the sync command's flag-scan surface. sync-cost-gate.ts: the inline-embed cost gate + token estimation for gbrain sync. sync-git.ts: git plumbing — invocation building, repo discovery, baseline-commit self-heal, path-containment guards, and the delete/rename slug resolvers (resolveSlugByPathOrSourcePath — see its dedicated entry above — plus the verified resolveSlugsForRemovedPaths/resolveRemovedPathSlug). sync-anchor.ts: sync anchor + chunker-version state helpers (source-scoped vs legacy global-config storage). sync-lock.ts: the lock layer — typed lock-busy error, the rich busy message, --break-lock handling, the partial-result envelope (performSync itself stays in the façade). sync-reconcile.ts: full-sync reconcile planning (the mass-delete valve and ever-committed gate) + sync deadline/stall resolution. sync-status-report.ts: the per-source sync status report backing gbrain sources status and the get_status_snapshot op. Its queue context is engine-aware: only an explicitly worker-backed surface may promise job deferral; otherwise non-interactive cost deferral returns manual_drain_required with exact per-source gbrain embed --stale --source <id> commands and never writes or claims an embed-backfill row.

  • src/core/sync-embed-backfill.ts — Sync command’s engine-aware embedding-delivery seam. It derives separate worker-backed deferral and multi-source fan-out eligibility, keeps automatic-submission policy distinct from capability, invokes the shared cost gate for --all and single-source runs, propagates intrinsic large-sync deferral into final delivery outcomes, resolves backfill results through the centralized submitter, formats one truthful human status, and builds the single-source JSON envelope. Keeping these pieces together prevents cost-gate, queue, human, and machine surfaces from drifting while the size-ratcheted src/commands/sync.ts façade stays at its committed ceiling.

  • src/core/storage.ts — Pluggable storage interface (S3, Supabase Storage, local).

  • src/core/storage-config.ts — Storage tiering: loadStorageConfig reads gbrain.yml, normalizes deprecated keys (git_tracked/supabase_only) to canonical (db_tracked/db_only) with once-per-process deprecation warning, and runs normalizeAndValidateStorageConfig (auto-fixes missing trailing /, throws StorageConfigError on tier overlap). Path-segment matcher: media/x/ does NOT match media/xerox/foo. Uses a dedicated parser for the gbrain.yml shape rather than gray-matter (broken on delimiter-less YAML). Also carries DERIVE_PHASE_DB_ONLY_DEFAULTS (life/events/, atoms/, extracts/, dream-cycle-summaries/) + effectiveDbOnlyDirs — the engine's derive-phase output prefixes treated as implicitly-declared db_only by the undeclared_db_only_pages doctor check but deliberately NOT merged into loadStorageConfig (a global merge would auto-gitignore those dirs and silently kill ingestion for brains that file-back them) — and findDbOnlyCollisions (pure collector-output vs db_only overlap detector shared by the db_only_collector_collision doctor check and sync's manageGitignore warning). Pinned by test/storage-config.test.ts + test/doctor-silent-death-checks.test.ts.

  • src/core/disk-walk.tswalkBrainRepo(repoPath) returns Map<slug, {size, mtimeMs}> from one recursive readdirSync. Skips dot-dirs, node_modules, non-.md files. Used by gbrain storage status instead of per-page existsSync + statSync (~400K syscalls on 200K-page brains → tens).

  • src/core/git-head.ts — local git HEAD freshness probe for gbrain doctor. isSourceUnchangedSinceSync(localPath, lastCommit, opts?) returns true iff localPath is a git repo whose current HEAD matches lastCommit; when opts.requireCleanWorkingTree is true also requires a clean working tree (mirrors gbrain sync's force-walk gate at sync.ts:1075 so doctor and sync agree on "is there work to do?"). requireCleanWorkingTree is boolean | 'ignore-untracked' — in 'ignore-untracked' mode the clean probe runs git status --porcelain --untracked-files=no so a quiet repo with stray untracked dirs (?? companies/, ?? media/) is still "unchanged" (sync's incremental path keys off the commit diff and does not import untracked files by default — it counts them as uncommitted drift with a stderr warning; --working-tree opts into importing them); GitCleanProbe gains an ignoreUntracked? second arg. Two probe seams (_setGitHeadProbeForTests, _setGitCleanProbeForTests) keep unit tests R2-compliant (no mock.module). Uses execFileSync with array args so shell metachars in local_path cannot escape to a shell (the test runs real execFileSync against '/nonexistent/$(touch <sentinel>)/repo' and asserts the sentinel is never created). Fail-open on every error (missing path, not a git repo, git not installed, timeout, NULL inputs, dirty-probe errored → false) so the caller falls back to its time-based check. The chunker-version-match check lives in the caller (doctor.ts) because it depends on engine state (sources.chunker_version vs CHUNKER_VERSION from src/core/chunkers/code.ts). Pinned by test/core/git-head.test.ts (incl. the shell-injection guard).

  • src/core/env-number.ts — numeric env-var resolution with ONE shared warn-once memo. resolveEnvNumber(varName, fallback, {unit?}) (positive numbers only; zero/negative/garbage → warn once + fallback), resolveHoursEnv (hours unit pre-applied), warnOnceForEnv(varName, message) for disabled-unless-set vars that need the same memo without the fallback shape (e.g. GBRAIN_EXTRACTION_LAG_FAIL_PCT). Lives in core because source-health.ts needs the hours resolver for the staleness ceiling while doctor.ts already imports FROM source-health.ts — reaching back would cycle, and duplicating would fork the memo so one bad var warns twice. doctor.ts re-exports it as _resolveEnvNumber for sync.ts's dynamic import. _resetEnvNumberWarnedForTests() is a test seam.

  • src/core/source-health.ts — per-source health metrics for gbrain sources status + doctor's federation_health. Commit-relative staleness: newestCommitMs(localPath) = HEAD committer time via git log -1 --format=%ct (fail-open null; NO working-tree mtime parsing — committed content only, robust against the porcelain-mtime bug farm); lagFromContentMs(contentMs|null, lastSyncMs|null, nowMs, ceilingSeconds?) = remote/column comparator (null lastSync → null; negative wall-clock → skew passthrough; contentMs <= lastSyncmax(0, wallClock - ceiling); else/null-content → wall-clock). Core logic is pure; only the DEFAULT ceilingSeconds reads env, via resolveStalenessCeilingSeconds() (GBRAIN_STALENESS_CEILING_HOURS overriding GBRAIN_SYNC_FRESHNESS_FAIL_HOURS, default 72) — pass the 4th arg for determinism. The caught-up branch RAMPS rather than returning a flat 0: a flat 0 meant a source whose clone vanished reported fresh forever (the dead-daemon class), while a hard step to the ceiling would trip federation_health (24h) and sync_freshness (24h/72h) in the same instant and skip the warn tier — the alert-storm shape the caught-up branch exists to prevent. Ramping keeps the escalation ordered. computeAllSourceMetrics(engine, sources, {probeContent?}): LOCAL (probeContent:true, gbrain sources status) → isSourceUnchangedSinceSync(..., {requireCleanWorkingTree:'ignore-untracked'}) ? 0 : wall-clock (live commit-hash catches HEAD moving to an old-dated commit a timestamp compare would miss); REMOTE (default, federation_health on the HTTP MCP path) → lagFromContentMs(row.newest_content_at, ...), NO git subprocess (trust boundary). commitTimeMs(localPath, sha) is the newestCommitMs sibling pinned to an arbitrary commit (committer time via git show -s --format=%ct <sha>, fail-open null, execFileSync array args) — the resumable sync stamps newest_content_at against its pinned target commit, not whatever HEAD raced to. Pinned by test/source-health.test.ts.

  • src/core/npm-squat-check.ts — classifies gbrain PATH entries as real, foreign npm package, broken, or unknown for doctor's npm_squat check. On Windows it normalizes Git Bash/MSYS drive paths (/c/...C:/...) and tries the native .exe suffix before reporting a broken entry; non-Windows classification keeps the original single-candidate behavior. Pinned by test/npm-squat-check.test.ts.

  • src/core/git-remote.ts — SSRF-hardened git invocations for remote-source cloneRepo, pullRepo, and fetchRemote(repoPath, branch) (the last used by the sync cost-estimator's fetch-first path, so a cost preview / dry-run fetches through the same hardened flags + GIT_TERMINAL_PROMPT=0 as real sync rather than a less-protected route). Exports two distinct flag constants because git's argv grammar treats them differently: GIT_SSRF_FLAGS (3 -c config flags — http.followRedirects=false, protocol.file.allow=never, protocol.ext.allow=never) is global config, spread BEFORE the subcommand verb; GIT_SSRF_SUBCOMMAND_FLAGS = ['--no-recurse-submodules'] is subcommand-scoped, spread AFTER the verb (a combined array would spread --no-recurse-submodules before the verb where real git rejects it exit 129). cloneRepo argv: git <GIT_SSRF_FLAGS> clone <GIT_SSRF_SUBCOMMAND_FLAGS> --depth=1 [--branch X] <url> <dir>. pullRepo argv: git -C <dir> <durableSsrfFlags()> pull <GIT_SSRF_SUBCOMMAND_FLAGS> --ff-onlydurableSsrfFlags(), not the hardcoded GIT_SSRF_FLAGS: identical -c flags except protocol.file.allow honors the GBRAIN_GIT_ALLOW_FILE_TRANSPORT=1 escape hatch. fetchRemote uses the same helper. Pinned by test/git-remote.test.ts position-anchored guard (argv.indexOf('--no-recurse-submodules') > argv.indexOf(verb)). Also exports the durability-side helpers that power gbrain sources harden/pull: GIT_ENV_AUTH (the no-prompt env minus the askpass /bin/false overrides, so an auth'd push/fetch can consult the repo's configured credential helper while GIT_TERMINAL_PROMPT=0 still fails fast on a missing credential), divergenceSafePull(repoPath, branch) (fetch + pull --rebase; returns skipped_dirty on a dirty tree, conflict_aborted on ANY pull --rebase failure (the catch doesn't classify the cause — it's treated as a conflict) after up to two best-effort rebase --abort attempts (each swallows its own failure; conflict_aborted is still returned even if rebase state remains), else up_to_date/advanced), detectDefaultBranch (origin/HEAD → current branch → main), pushProbe(repoPath, branch) (authenticated push --dry-run that proves push access and classifies auth/protected/unreachable), and isWorkingTreeDirty. These auth'd paths route their protocol.file.allow through GBRAIN_GIT_ALLOW_FILE_TRANSPORT (default never; set =1 for self-hosted filesystem remotes), unlike cloneRepo, which still uses the hardcoded GIT_SSRF_FLAGS and stays strict.

  • src/core/brain-repo-durability.ts + src/commands/sources-harden.ts — brain-repo git durability. hardenBrainRepo(opts) makes a brain's working tree durable, idempotently: divergence-safe pull, a LOCAL untracked .git/hooks/post-commit auto-push safety net (never committed — a pulled commit can't rewrite executed code next to the token; installed into the active core.hooksPath dir and excluded via .git/info/exclude when that dir is tracked), a committed scripts/brain-commit-push.sh that refuses to exit 0 without a confirmed push and stages+commits BEFORE any pull so a dirty tree of modified pages (the write-through shape) can still be committed — the push-retry's rebase-on-reject handles a remote that advanced (hook + helper render from ONE bash push-retry template — DRY at the TS source, not by the hook sourcing a repo-controlled script), durability rules patched into the active resolver file (findResolverFile → RESOLVER.md > AGENTS.md; taxonomy rendered from the bundled _brain-filing-rules.json), a minimal DB-free pull cron (launchd/crontab running gbrain sources pull --path <dir> so it never opens the PGLite single-writer lock), and a push-probe verify (no heartbeat commit). Credential is REPO-scoped (acceptPat from --pat-file/GBRAIN_GITHUB_PAT, warns on loose perms; reuses an existing repo-local credential.helper, else a 0600 store wired via repo-local config); the token is redacted everywhere via redactSecretsInText and never enters the repo, remote URL, logs, or DurabilityReport. unhardenBrainRepo removes the cron/hook/credential wiring (ownership-fingerprinted); sources remove runs it only AFTER the source row's DELETE commits (a refused or raced delete leaves the scaffolding intact; post-commit teardown failure is loud but non-fatal). CLI: gbrain sources harden <id|--all> / pull <id>|--path <dir> / unharden <id>; auto-harden fires on sources add --url ... --pat-file for managed clones (--no-harden opts out). sources pull --path is dispatched in src/cli.ts BEFORE connectEngine so the cron stays DB-free. CLI-only (writes executables + an OS cron + a credential helper on the host); never exposed over MCP. Tests: test/brain-repo-durability.serial.test.ts, test/git-remote-durable.serial.test.ts, test/brain-durability-hook.serial.test.ts, test/durability-cron.test.ts. GBRAIN_PUSH_LOCK_WAIT_SECONDS (default 30, env-only) tunes the flock -w wait in the rendered push-retry block that both the committed helper and the local hook share; on lock timeout the synchronous helper returns rc 1 (fail-loud, no push was attempted) while the detached post-commit hook returns rc 0 (another holder is already pushing — the designed coalescing outcome).

  • src/commands/storage.tsgbrain storage status [--repo P] [--json]. Split into pure data (getStorageStatus) + JSON formatter + human formatter (ASCII-only) matching the orphans.ts pattern. PageCountsByTier and DiskUsageByTier are distinct nominal types so swaps fail at compile time.

  • gbrain.yml (brain repo root) — Optional storage tiering config. Top-level storage: section with db_tracked: and db_only: array-valued keys. gbrain sync auto-manages .gitignore for db_only paths on successful sync (skips on dry-run, blocked-by-failures, submodule context, or GBRAIN_NO_GITIGNORE=1). gbrain export --restore-only [--repo P] [--type T] [--slug-prefix S] repopulates missing db_only files from the database.

  • src/core/supabase-admin.ts — Supabase Management-API client (listProjects, discoverPoolerUrl, extractProjectRef — refs are lowercase ALPHANUMERIC, [a-z0-9]+). Consumed by the init --prefer-postgres ladder's rung 2 (DISCOVERY only — project creation stays dashboard guidance; discovered URLs are candidates, connect-probed before persisting) and by the classifier's Supabase hints. The access token is never persisted, logged, or written to receipts/config.

  • src/core/file-resolver.ts — File resolution with fallback chain (local -> .redirect.yaml -> .redirect -> .supabase).

  • src/core/chunkers/ — 3-tier chunking (recursive, semantic, LLM-guided). code.ts is a tree-sitter-based semantic chunker for 30 languages (plus SQL via DerekStride/tree-sitter-sql) with embedded-asset WASMs (src/assets/wasm/), @dqbd/tiktoken cl100k_base tokenizer, small-sibling merging. CHUNKER_VERSION is folded into importCodeFile's content_hash so chunker shape changes force clean re-chunks across releases. extractSymbolName has an inline SQL branch (extractSqlSymbolName) diving through DerekStride's statement wrapper into the inner DDL child (create_table/create_function/create_view/create_index/create_procedure/create_type/create_schema/create_database/create_trigger/alter_table/alter_view) and extracting the target identifier via the name field with identifier-shaped fallback; DML kinds (select/insert/update/delete/merge/with) deliberately return null so chunks emit unnamed (code-def is a DDL signal). normalizeSymbolType has parallel SQL branches mapping create_table → 'table', create_view → 'view', etc. DEF_TYPES (owned by src/core/chunkers/def-types.ts, re-exported by src/commands/code-def.ts — see that entry) carries the SQL kinds ('table' | 'view' | 'index' | 'procedure' | 'schema' | 'database' | 'trigger') so the new chunks surface in gbrain code-def <name> queries.

  • src/core/chunkers/def-types.tsDEF_TYPES: the ONE list of definition-shaped normalized (post-normalizeSymbolType) symbol types, shared by gbrain code-def's lookup allowlist (src/commands/code-def.ts re-exports it) and the code chunker's small-sibling merge guard so the two can't drift — a symbol type code-def can resolve never has its symbol_name erased by merging a small definition into a neighbor. Includes the fallthrough forms normalizeSymbolType emits verbatim for tree-sitter node types it doesn't canonicalize ('method declaration', 'struct specifier', 'record declaration', …) — most of an OO codebase's symbols arrive in those forms. Pinned by test/chunkers/code-merge-defs.test.ts.

  • src/core/errors.tsStructuredAgentError + buildError + serializeError. Every agent-facing surface (code-def, code-refs, usage errors) uses this envelope; matches the CycleReport.PhaseResult.error shape.

  • src/assets/wasm/ — 37 tree-sitter grammar WASMs + tree-sitter runtime. Committed to the repo so bun --compile embeds them deterministically via import path from ... with { type: 'file' }. The CI guard scripts/check-wasm-embedded.sh fails the build if the compiled binary ever silently falls through to recursive chunks. tree-sitter-sql.wasm (DerekStride/tree-sitter-sql @ c2e1e08db1ea20dc23bdb8d228a81a8756e9c450, built with tree-sitter-cli@v0.26.3 --abi 14) adds SQL coverage at 11 MB — larger than peers because the grammar covers PostgreSQL + MySQL + SQLite + T-SQL basics (40 MB generated parser.c); the compiled binary grows ~6%.

  • src/commands/code-def.ts + src/commands/code-refs.ts — symbol definition + references lookup. Query content_chunks.symbol_name or chunk_text ILIKE with page_kind='code' filter. Auto-JSON when stdout is not a TTY (gh-CLI convention). Bypass the standard searchKeyword DISTINCT ON (slug) collapse so multiple call-sites from the same file surface. The JSON envelope (CLI + the code_def/code_refs MCP ops) carries status + ready from src/core/code-graph-readiness.ts so a count:0 result is distinguishable as not_built (no code indexed) vs ready (genuinely no match); human output prints a one-line hint when not ready. Both commands resolve --source <id> (space or inline = spelling) and the ambient source scope through the shared code-scope.ts resolver, matching code-callers/code-callees; --all-sources restores the brain-wide read. The AND p.source_id = $N fragment comes from code-scope.ts's pushSourcePredicate(params, opts) (numbered off params.length so it composes with --lang and any other optional predicate; '' when spanning every source) — code-def's lookup, its filtered-types probe, and code-refs all use it.

  • src/commands/code-scope.ts — shared --source/--all-sources resolution for the four code-* CLI commands (code-def, code-refs, code-callers, code-callees), extracted so the four can't drift. positionalArgs skips value-taking flags AND their values (--source, --limit, --lang — a naive scan looked up the flag's value as the symbol); inline name=value spellings are one token and consume nothing. A bad .gbrain-source/GBRAIN_SOURCE pin is recognized through the shared isResolverUserError predicate (exported by src/core/source-resolver.ts, next to the messages it matches) and exits 2 with a clean invalid_source_pin envelope instead of an uncaught stack. Also exports pushSourcePredicate(params, opts), the source-scope SQL fragment builder code-def/code-refs share.

  • src/core/code-graph-readiness.ts — typed readiness signal shared by the code-* surfaces (code-def/code-refs/code-callers/code-callees, plus the code_blast/code_flow walk ops). resolveCodeReadiness(engine, {kind:'symbol'|'edge', count, sourceId?, allSources?, remote?}) returns {status:'not_built'|'no_symbols'|'indexing'|'ready'|'out_of_scope'|'unknown', ready, has_code, pending_edges, scoped_source_id?} (6-state CodeGraphStatus; scoped_source_id is set only for out_of_scope, naming the excluded scope). count>0 short-circuits to ready with no query; on empty it runs EXISTS probes against content_chunks JOIN pages (page_kind='code') — no page_kind index needed, and the pending probe rides the partial idx_content_chunks_edges_backfill. kind:'symbol' (code-def/refs) is 3-state + brain-wide, because symbol metadata is set at chunk time and edge resolution is irrelevant to it: no code chunks → not_built; code chunks but none carry symbol_name yet → no_symbols (chunks indexed before symbol extraction; hints at reindex-code); symbol-bearing chunks → ready (it never reports indexing). kind:'edge' (code-callers/callees) is 3-state + source-scoped (not_built/indexing/ready), with the pending predicate mirroring the resolver (edges_backfilled_at IS NULL OR < EDGE_EXTRACTOR_VERSION_TS from src/core/chunkers/symbol-resolver.ts) so a resolver-version bump never falsely reports ready. Both grains additionally share out_of_scope: when the scoped probe finds no code but an unscoped rerun finds code exists brain-wide (a chunk-existence check only — it doesn't verify symbol metadata or edge completeness), it's the caller's resolved scope that excludes it, not an unbuilt graph. That unscoped rerun runs ONLY for trusted local callers (remote === false); a scoped remote caller sees not_built instead, never a brain-wide code-existence disclosure. Probe scope matches each command's result-query deleted_at posture (def/refs don't filter deleted_at, so neither do the probes). Any DB error returns status:'unknown' (fail-open; never breaks the command). readinessHint(r) renders the human one-liner. Wired into code-def.ts/code-refs.ts (brain-wide), code-callers.ts/code-callees.ts (resolved sourceId/allSources), and all six code_* MCP op handlers in src/core/ops/code-intel.ts (operations.ts imports codeIntelOperations and spreads it into the canonical operations array, not a re-export). code_blast/code_flow stamp status/ready (+ scoped_source_id) onto the walk result through the module-private attachWalkReadiness helper: computed AFTER the traversal cache (readiness is never cached), not_found probes at symbol grain, ok/ambiguous at edge grain with count = nodes/candidates, and unsupported_language is passed through untouched. Pinned by test/code-graph-readiness.test.ts + readiness-envelope cases in test/e2e/code-intel-mcp-ops-pglite.test.ts.

  • src/core/search/ — Hybrid search: vector + keyword + RRF + multi-query expansion + dedup. searchKeyword/searchKeywordChunks/searchVector apply source-aware ranking at the SQL layer (curated content like originals/, concepts/, writing/ outranks bulk content like <fork>/chat/, daily/, media/x/). searchVector uses a two-stage CTE so source-boost re-ranking doesn't kill the HNSW index. Hard-exclude prefixes (test/, archive/, attachments/, .raw/ by default) filter at retrieval, not post-rank. Both gates honor detail !== 'high' so temporal queries surface chat pages normally.

  • src/core/search/query-intent.ts — Deterministic query classifiers (pure, no LLM): classifyQuery/classifyQueryIntent (entity/temporal/event/concept/general → auto-selects detail level, salience/recency/modality axes; concept fires on definitional-paraphrase OR landscape/quantifier cues with a proper-noun name-guard — capitalized names, quoted phrases, slugs, and sub-3-word queries never trigger — and ranks vector-lean via the RRF-k tilt in intent-weights.ts), isAmbiguousModalityQuery (LLM-escalation gate), and the concept-shape pair — looksConceptShaped (fuzzy-quantifier/landscape cues minus exact-identifier anti-signals, tuned to favor false-negatives; cues owned by other routers like "who are the"/find_experts and bare "anything"/salience are deliberately excluded) + conceptNudge (full one-line CLI hint string steering a concept-shaped search toward query; consumed by maybePrintConceptNudge in src/cli.ts on BOTH the local-engine and thin-client result paths, stderr-only, --quiet-gated). Pinned by test/query-intent-concept.test.ts + test/cli-concept-nudge.test.ts.

  • src/core/search/llm-intent.ts — opt-in LLM modality tie-break. classifyModalityWithLLM(query, fallback) routes through gateway.chat() with a fixed single-word-output system prompt; 1s timeout via AbortController. parseModality(raw, fallback) is the pure parser (tolerates trailing punctuation + casing). Fail-open on every error (gateway unavailable, timeout, parse failure, unrecognized output) — returns the fallback so a misbehaving LLM can never break search. Cost-bounded by isAmbiguousModalityQuery in query-intent.ts so the LLM call fires on only a small fraction of queries when on.

  • src/core/search/image-loader.tsloadImageInput(input, opts) accepts a local path, data: URI, or http(s):// URL. Magic-byte sniff for PNG/JPEG/WebP. Hard size cap (default 10 MB, configurable via search.image_query.max_bytes). URLs route through fetchWithSSRFGuard so DNS rebinding + redirect chains are defeated; shared guarded transport pins validated DNS answers and streams decoded bodies with a 2 MiB remote cap (10 MiB local cap). ImageLoadError with discriminated code (INVALID_FORMAT / OVERSIZED / INVALID_URL / FETCH_FAILED / TIMEOUT / SSRF_BLOCKED / NOT_FOUND).

  • src/core/search/by-image.tssearchByImage(engine, input, opts). Always runs the image branch (embedQueryMultimodalImage + searchVector(embedding_image)). Hybrid intersect: when the caller provides an optional query, runs a parallel text branch via embedQueryMultimodal(query) and merges via rrfFusionWeighted with effectiveRrfK(baseRrfK, weight) from the resolved mode's refinement weights. Widens to the unified column when search.unified_multimodal=true (transparently upgrades retrieval quality post-reindex).

  • src/core/ssrf-validate.ts — DNS-rebinding-defended URL validation. validateAndResolveUrl(url) resolves the hostname via dns.lookup({all: true, family: 0}), checks EVERY A and AAAA record against the internal-IP deny list, and returns the resolved IP so callers fetch by IP (validation IP === fetch IP defeats DNS rebinding). fetchWithSSRFGuard(url, opts) does redirect-aware fetching with per-hop re-validation (max 3 hops by default). Reusable across all URL-fetching features. Test seam __setDnsLookupForTests for hermetic tests.

  • src/core/spend-log.ts — per-OAuth-client paid-API spend tracking against the mcp_spend_log table. checkBudget(engine, clientId, capCents) is the pre-flight gate; throws BudgetExceededError when today's spend has hit the cap. recordSpend(engine, entry) is best-effort post-call. UTC day-aligned aggregation so caps roll over deterministically regardless of server timezone. Local CLI callers (no clientId) bypass the gate; brains without the table fail open to spend=0. VOYAGE_MULTIMODAL_3_PER_IMAGE_CENTS = 0.12 cents per image embed.

  • src/commands/reindex-multimodal.tsgbrain reindex --multimodal [--limit N] [--dry-run] [--cost-estimate] [--no-embed] [--yes] [--json]. Walks content_chunks WHERE embedding_multimodal IS NULL, batches via embedMultimodalSafe (partial-failure-aware), persists. Lock via tryAcquireDbLock (360min) so a concurrent autopilot embed phase can't race it. Cost prompt + Ctrl-C grace window in TTY. GBRAIN_NO_REEMBED=1 bypass. Checkpoint at ~/.gbrain/reindex-multimodal-checkpoint.json for resume. Auto-flip prompt at coverage=100% completion (TTY: interactive; non-TTY: stderr hint with a paste-ready command).

  • src/core/backfill-registry.ts — registry of idempotent data backfills. The modality backfill flips modality to 'image' on image-asset chunks the ingest path missed; its SQL filter requires chunk_source='image_asset' AND embedding_image IS NOT NULL AND (modality IS NULL OR modality != 'image') — the chunk_source guard ensures a non-image chunk that happens to have embedding_image populated is never flagged. A second run finds zero rows.

  • src/core/search/eval.ts — Retrieval eval harness: P@k, R@k, MRR, nDCG@k metrics + runEval() orchestrator.

  • src/core/eval/ranked-docs.ts — dependency-free page-level ranking primitive shared by the retrieval eval harness, NamedThingBench, and the qrels correctness gate. dedupeRankedKeys() preserves the first/best occurrence of each ${source_id}::${slug} or slug key before any cutoff, so repeated chunks neither inflate relevance gain nor consume document ranks. Precision retains its fixed-k denominator; Recall and nDCG are structurally bounded to [0,1]. Pinned across test/eval.test.ts, test/retrieval-quality-harness.test.ts, test/bench/qrels-file.test.ts, and test/bench/correctness-gate.test.ts.

  • src/core/search/source-boost.ts — Source-type boost map keyed by slug prefix. DEFAULT_SOURCE_BOOSTS (originals/ 1.5, concepts/ 1.3, writing/ 1.4, people/companies/deals/ 1.2, daily/ 0.8, media/x/ 0.7, /chat/ 0.5, archive/ 0.5, extracts/ 0.3) and DEFAULT_HARD_EXCLUDES (test/, attachments/, .raw/). archive/ is DEMOTED (findable, ranked below curated), not hard-excluded — archive holds high-signal history users expect to retrieve; the demote is a prior at the SQL/fusion layer and the cross-encoder reranker can still promote a strongly-matching archive page. parseSourceBoostEnv/parseHardExcludesEnv parse comma-separated prefix:factor pairs from GBRAIN_SOURCE_BOOST/GBRAIN_SEARCH_EXCLUDE. resolveBoostMap and resolveHardExcludes merge defaults + env + caller SearchOpts.exclude_slug_prefixes/include_slug_prefixes. The surviving exclude policy is auditable via the hidden_by_search_policy doctor check (src/commands/doctor.ts, local + remote paths) which counts chunked pages withheld per active exclude prefix, reusing resolveHardExcludes + buildVisibilityClause + the exported escapeLikePattern.

  • src/core/search/sql-ranking.ts — Pure SQL string builders. buildSourceFactorCase(slugColumn, boostMap, detail) emits a CASE with longest-prefix-match wins (returns literal '1.0' when detail === 'high' for temporal-bypass parity with COMPILED_TRUTH_BOOST). buildHardExcludeClause(slugColumn, prefixes) emits NOT (col LIKE 'p1%' OR col LIKE 'p2%') — OR-chain wrapped in NOT, NOT NOT LIKE ALL/ANY (those quantifiers don't express set-exclusion). LIKE meta-character escape covers all three of %, _, AND \ (backslash is Postgres LIKE's default escape char). Single-quote doubling on SQL string literals so injection-style inputs are inert text. buildBestPerPagePoolCte(...) is the shared per-page max-pool CTE both engines' searchVector inject — instead of returning the single best chunk per page from an inner ORDER BY embedding <=> vec LIMIT N (which lets a page lose to a neighbor on ONE weak chunk while its strong chunk sits just below the inner cut), the CTE pools the BEST chunk score per (source_id, slug) composite key so a page surfaces on its strongest evidence; composite key (not bare slug) keeps multi-source brains correct; single source of truth so the two engines can't drift.

  • src/core/search/title-match.ts — pure, zero-I/O title-phrase matcher shared by the production title boost AND NamedThingBench (no drift). isTitlePhraseMatch(query, title) returns true when the normalized query is a contiguous token run inside the title with >= MIN_CONTENT_TOKENS=2 non-stopword tokens, OR an exact full-title match (covers deliberate 1-word chosen names like "Helios"). Token-boundary matching (never raw substring, so "art" doesn't match "Bartholomew"); small conservative English stopword set excluded from the content-token floor (guards against promoting generic pages on stopword-y queries); NFKC normalize so CJK / width variants converge. Exports tokenizeTitle + __test__ internals.

  • src/core/search/alias-normalize.ts — ONE normalizer shared by the WRITE path (ingest projects frontmatter aliases: into page_aliases) and the READ path (search matches query against page_aliases), so stored aliases can't silently fail to match queries via divergent normalization (same single-source posture as cjk.ts). normalizeAlias(raw) does NFKC + lowercase + whitespace-collapse + trim + strip one layer of wrapping quotes/brackets; returns '' for empty (callers MUST skip empty aliases). normalizeAliasList(value) coerces a frontmatter scalar / array / comma-list / garbage into a deduped list of normalized non-empty aliases — used by both the ingest projection and the reindex --aliases backfill.

  • src/core/search/evidence.ts — the agent-facing why-it-matched contract (so an agent never reads one blended score, decides "no strong match, safe to create", and writes a duplicate over a fully-developed page). classifyEvidence(r, opts?) names the strongest signal (precedence: alias_hit > exact_title_match > high_vector_match (real query↔chunk cosine SearchResult.cosineDEFAULT_HIGH_COSINE_FLOOR=0.8, overridable via EvidenceOpts.cosineFloor / config search.evidence_cosine_floor — never the blended score, so a keyword+boost pile-up can't earn the label; keyless/hermetic runs have no cosine and degrade to keyword-based labels; legacy HIGH_MATCH_FLOOR=0.85 stays exported for back-compat only) > keyword_exact (base ≥ SOLID_MATCH_FLOOR=0.6) > weak_semantic). createSafetyFor(evidence) derives the don't-duplicate hint (exists/probable/unknown) the agent keys off INSTEAD of a raw threshold (a blended RRF/cosine score is not a calibrated probability). stampEvidence(results, opts?) stamps evidence + create_safety in place once at pipeline end (after the alias hop, before slice); idempotent.

  • src/core/search/private-visibility.ts — shared page-visibility resolver and SQL predicates. Explicit local trust disables page filtering; operator env/config opt-outs retain their existing semantics and failed config reads enforce privacy. privatePagesFilterFragment authorizes concrete page rows; link origins and timeline projections use independent ID-based predicates. Legacy slug-set helpers are for name enumeration, never authorization of data from another row/source. ops/context.ts:readPolicyOpts combines canonical source precedence, resolved privacy and holder permissions. Both engines filter content/history/graph/identity/analytics before limits or aggregation; current and historical visibility are checked independently while page privacy is enforced. Semantic response caching remains disabled, so visibility-key folding alone never authorizes reuse.

  • src/commands/search-diagnose.tsgbrain search diagnose "<query>" --target <slug> [--json] [--source <id>]: Phase-0 retrieval diagnostic. Traces WHERE a target page surfaces (or fails to) across keyword / vector (per-page max-pool) / alias / hybrid layers and names the layer responsible for an incident, so an operator can pin whether the fix is max-pool/innerLimit (vector) vs title/alias. The verdict names the layer that DOES surface the target (or "none"). Pinned by test/search/search-diagnose.test.ts.

  • src/commands/reindex-aliases.tsgbrain reindex --aliases [--limit N] [--dry-run] [--json] [--source <id>]: backfills the free-text alias layer for EXISTING pages whose frontmatter aliases: predate the alias table (the import-time projection covers new + changed pages). Reads each page's frontmatter aliases:, writes via engine.setPageAliases. Idempotent + convergent (setPageAliases replaces a page's alias set) so no op-checkpoint needed; walks listAllPageRefs (cheap cross-source enumeration), --source narrows. Pinned by test/search/reindex-aliases.test.ts.

  • src/eval/retrieval-quality/harness.ts + src/commands/eval-retrieval-quality.ts + test/fixtures/retrieval-quality/namedthing.jsonl + test/fixtures/retrieval-quality/namedthing/corpus.ts — NamedThingBench, the retrieval-quality eval that pins the named-thing-miss failure class. Seven query families, each a distinct failure class: title-substring (the named-thing miss itself), generic-to-named (tourist label → named thing), alias-synonym (declared alias / romanization → canonical), multi-chunk-dilution (one strong chunk among many weak — stresses max-pool), short-vs-rich, graph-relationship (guardrail), hard-negative (precision guard, must NOT return a page). gbrain eval retrieval-quality <fixture.jsonl> runs it with hard gates (e.g. title-substring Hit@1 ≥ 0.95, alias Hit@1 ≥ 0.98, multi-chunk-dilution Hit@3 = 1.0). Pure: caller injects a SearchFn (CLI uses hybridSearch, tests stub) so it's engine-agnostic. Metric glossary entries (hit@1/hit@3) added to src/core/eval/metric-glossary.ts. Pinned by test/eval-retrieval-quality.test.ts + test/retrieval-quality-harness.test.ts. The seed corpus (NAMEDTHING_CORPUS, seedNamedThingCorpus(engine, { embed? }), loadNamedThingQuestions()) lives in test/fixtures/retrieval-quality/namedthing/corpus.ts — the ONE brain the fixture is written against (sibling of relational/corpus.ts), seeded without vectors by the hermetic gate and with real vectors by the R1 A/B script, so a paid receipt and the CI gate describe the same pages. The committed fixture is 12 queries — 11 non-relational core queries plus one graph-relationship query, which R1 scores together with relational/corpus.ts's 38 RELATIONAL_QUESTIONS as the 39 relational questions. seedNamedThingCorpus seeds exactly what the gate always seeded (putPage with compiled_truth = the chunks joined, one upsertChunks row per chunk with token_count: 10, lower-cased setPageAliases when declared) and, when embed is supplied, embeds every chunk text in ONE batch and stores the vectors (a count mismatch throws); the result reports pages, chunks, embedded, embedded_chars for spend accounting. Change pages, chunk boundaries or aliases only together with the fixture and its gate expectations.

  • scripts/r1-namedthing-rerank-ab.ts — Phase C′ rule R1 receipt: balanced reranker ON vs OFF on NamedThingBench, paired per query, in ONE in-memory PGLite brain seeded ONCE with real embeddings (seedNamedThingCorpus; --relational adds relational/corpus.ts + its 38 RELATIONAL_QUESTIONS, which with the fixture's own graph-relationship query are the 39 relational questions the receipt reports beside the 11 non-relational core queries). Arms are applied with engine.setConfig (ARM_PINS: OFF = search.mode balanced, search.reranker.enabled false, search.autocut false; ON adds search.reranker.enabled true + search.reranker.model voyage:rerank-2.5; autocut pinned off in both by default — a rerank-only comparison — and --autocut on|off is an overlay applied to BOTH arms through applyArmPins(engine, arm, overlay) so the ON arm can run in the pre-R2 shipped balanced shape (autocut on) without changing the pinned defaults the tests pin), so bare hybridSearch resolves them the way a production balanced brain would; scoring reuses runRetrievalQuality/evaluateGate (no local hit@k). Per query it records deduped top-3, the rank-1 evidence/create_safety tier, finite-rerank_score row count and the degraded stamp. INVARIANTS: readiness (rerankerReadinessForEngine) is checked BEFORE any spend and the ON arm must show no reranker_skipped/rerank_passthrough stage and ≥1 reranked row per non-empty query, else exit 2 (fail-open would silently turn ON into OFF); embedding degradation (embed_unavailable/embed_timeout/vector_arm_failed) in a live arm is exit 2 too; --embed-cache PATH (installEmbedCache) makes both arms see byte-identical query vectors and the receipt says whether that held (identical_query_vectors). r1Verdict is pure: PASS iff hit@1 losses == 0 and hit@3 losses ≤ 1, losses counted per query (OFF hit, ON miss) and never offset by wins; create_safety downgrades are counted and named, not gated. A pass-through rerank transport tees the API-echoed model id + usage into the receipt. --stub-embed is the hermetic dry run (embed transport throws, OFF arm only, ON reported as skipped — needs VOYAGE_API_KEY). --json carries _meta.metric_glossary (hit@1, hit@3, mrr, create_safety); exit 0 PASS/dry-run · 1 FAIL · 2 integrity/usage. Receipt (shipped balanced, voyage:rerank-2.5): the 11 core queries show 0 hit@1 / 0 hit@3 losses in every cell; the 39 relational queries WITHOUT the relational re-pin fell hit@1 21→3 (19 losses) and hit@3 27→5 (22 losses) — R1 FAIL; WITH search.relational_rerank_pin at its default 3 (measured with --autocut on, the shape that shipped before rule R2 turned autocut off) 0 losses, hit@1 21/39 and hit@3 27/39 — R1 PASS, the balanced reranker stays ON. --relational-pin N|off overlays search.relational_rerank_pin on BOTH arms: the no-pin cell is --relational-pin off (or 0); the default cell omits the flag and resolves the bundle default 3 exactly as production does. Pinned by test/r1-namedthing-rerank-ab.test.ts.

  • docs/architecture/RETRIEVAL.md + docs/incidents/RETRIEVAL_MAXPOOL_INCIDENT.md — retrieval-pipeline architecture reference + the named-thing-miss incident write-up (root cause, the five-layer fix, the eval that pins it).

  • src/eval/brainbench/ + src/commands/eval-brainbench.ts — BrainBench, the cross-harness memory conformance suite (gbrain eval brainbench; methodology in docs/eval/BRAINBENCH.md). types.ts carries the PUBLISHED interchange shapes (fixture/gold/result/baseline — mirrored as JSON Schemas in evals/brainbench/schema/; breaking changes bump the schema versions). fixtures.ts: strict loader/validator + corpus fixtures_hash (covers fixture AND gold files); a gold key inside a fixture turn is a validation error — gold is SEALED in the gold dir and adapters only ever see sanitized PublicTurns. seed.ts: fail-fast hermetic seeding (importFromContent noEmbed + NULL-embedding insertFact; any non-imported status ⇒ SeedError ⇒ fixture seed_failed ⇒ run exit 2). adapters/shared.ts: ONE runReflexPipeline all three adapters drive with declarative config (pointer budget, suppression mode) — cross-harness comparability is structural; openclaw.ts (seam production, the shipped pipeline), claude-code.ts (seam production; drives the shipped gbrain hook user-prompt path end-to-end — fixture turns become UserPromptSubmit stdin JSON, synthesized Claude Code JSONL transcripts feed the real window parse + cross-turn dedupe, and resolution rides a run-scoped resolve-IPC server with the real shared secret; bench-pinned deviations disclosed in docs/eval/BRAINBENCH.md: generous userPromptDeadlineMs, push-failure banner suppressed), codex.ts (seam contract; static entity-index preamble whose slugs deliberately don't count as injections + ≤1 per-turn fragment; fixture conversations round-trip through the real rollout format + the shipped parser src/core/transcripts/codex.ts for turn selection — fragment DELIVERY remains harness-shaped until a shipped codex injection path lands). metrics/: know-to-ask (+false-fire anti-gaming companion), push (micro-averaged P/R), write-back (drives the PRODUCTION conversation→facts pipeline via the injectable-extractor seam; gold extractor in CI, real extractor under --llm), continuity (writer→reader pairs on a shared brain through DIFFERENT adapters; pointer-injection OR stored-fact keyword probe). harness.ts: ONE in-memory PGLite per run + resetTables between fixtures (longmemeval engine-sharing pattern); read-only suites share one seeding across all adapters; emits per-(harness×suite) cells + re-scoreable turn rows; source_isolation_violations counted per turn and gated at zero. scoreboard.ts: markdown render, canonical diff-stable committed baseline (4-decimal rounding, sorted keys, receipts excluded), compareBaselines with main-baseline governance — same-hash count-aware gate vs corpus-bless mode (the committed baseline must byte-match the run; regressions vs main require a justification). The CLI brings its own PGLite (cli.ts routes before connectEngine), writes --out as the canonical CI artifact, and terminates via an explicit grace-tick process.exit(verdict) (0 pass / 1 regression / 2 error) because PGLite stomps process.exitCode and Bun discards queued stdout on exit. runBrainBenchCore() is the in-process entry eval run-all uses (one record per sweep, EvalRunRecord schema_version 3, mode: 'n/a'). Pinned by test/brainbench-*.test.ts + test/eval-brainbench-e2e.slow.test.ts.

  • evals/brainbench/ — the committed BrainBench corpus: 141 fixtures (135 generated + 6 hand-authored spike) / 241 gold-annotated turns across 7 categories (kta-pos/kta-neg/push/write-back/continuity/multi-source/adversarial), ~15% holdout (excluded from the CI gate, scored in published --include-holdout runs). generator/gen.ts rebuilds the corpus byte-identically (Mulberry32, seed 42; whole-cloth fictional universe from curated synthetic name pools so scenario privacy is structural; prose is template-synthesized with PRNG-selected variants — deliberately no LLM pass, difficulty stays controlled; several know-to-ask variants intentionally exercise documented v1 reflex limits so the baseline measures the roadmap). gold/ is sealed; schema/ is the foreign-runner contract (gbrain-evals drives the suite as a subprocess via --fixtures DIR --gold DIR --json --out FILE); baselines/main.json is the committed gate baseline; _ledger.json records counts/seed/rebuild command. CI: the .github/workflows/test.yml brainbench job + scripts/ci-brainbench-gate.sh (fetches MAIN's baseline via git show origin/master:… — a PR cannot rewrite what it's compared against; first-landing path runs ungated) + scripts/render-brainbench-delta.ts (compact step-summary/PR-body delta block from the --out artifact). Privacy: scripts/check-synthetic-corpus-privacy.sh scans evals/brainbench/{fixtures,gold} in bun run verify.

  • src/core/types.ts extension + src/core/operations.ts:search + src/core/import-file.ts + src/cli.ts + src/core/search/telemetry.ts — the wiring layer for the retrieval-quality stack. SearchResult carries evidence, create_safety, title_match_boost, alias_hit (all optional; evidence/create_safety reference the union types in evidence.ts). The search MCP op uses a cheap-hybrid path by default and accepts a per-call mode (conservative|balanced|tokenmax) honored ONLY for trusted/local callers (resolvePerCallMode(ctx, ...) — remote callers use the configured mode so a remote provider can't force tokenmax spend); every search path stamps evidence fail-soft. importFromContent projects frontmatter aliases: into page_aliases via normalizeAliasList + engine.setPageAliases so new + changed pages register aliases at ingest. src/cli.ts carries the gbrain search diagnose dispatch (lazy import) and reconciles the search CLI path with the cheap-hybrid op. src/core/search/telemetry.ts carries in its rollup the rank-1 base_score drift signal (sum/count + 3 coarse buckets, aggregate not per-query), surfaced via gbrain search stats, backed by migration v111's search_telemetry columns. Tests: test/cli-search-dispatch.test.ts, test/search/per-call-mode.test.ts, test/search/telemetry-rank1.test.ts, test/search/title-boost-stage.test.ts, test/search/alias-hop.test.ts, test/search/evidence.test.ts, test/search/searchvector-maxpool.test.ts, test/search/pre-migration-failopen.test.ts.

  • src/commands/eval.tsgbrain eval command: single-run table + A/B config comparison. Sub-subcommand dispatch on args[0] routes gbrain eval export + gbrain eval prune + gbrain eval replay into session-capture handlers; bare gbrain eval --qrels … fall-through preserves the legacy IR-metrics flow. gbrain eval cross-modal is in the dispatch (the user-facing path is the cli.ts no-DB branch — src/commands/eval.ts:cross-modal only fires when callers re-enter with an existing engine).

  • src/commands/eval-cross-modal.ts — multi-model quality gate. Three different-provider frontier models score the OUTPUT against the TASK on a 5-dim list. Verdict pass (exit 0) / fail (exit 1) / inconclusive (exit 2; <2/3 model successes). Reuses src/core/ai/gateway.ts:chat() so config/auth/aliasing comes from the gateway recipe registry — no parallel provider stack. Self-configures the gateway (configureGateway(loadConfig() + process.env)) since the cli.ts dispatch bypasses connectEngine(). Default cycles 3 in TTY, 1 in non-TTY (partial cost guardrail) via the shared resolveCycleDefault(explicit, isTty) in src/core/eval/cycle-default.ts; the cost-estimate banner appends cycleDefaultSuffix(...) (for 1 cycle(s) (non-interactive default; --cycles N for more)) when the value is the silent non-TTY fallback, so the 1-vs-3 difference isn't hidden. Receipts land at gbrainPath('eval-receipts')/<slug>-<sha8-of-output>.json. --batch <jsonl> [--limit N] [--concurrent N] [--max-usd FLOAT] [--yes] fans out cross-modal scoring across a LongMemEval-shape JSONL; mutually exclusive with --task (fail-fast usage error if both set); filters kind: "by_type_summary" rows; pre-flight cost estimate refuses if > --max-usd without --yes (default cap 5.00 USD). Semaphore-bounded fan-out via inline runWithLimit<T>(items, limit, fn)$ (\text{exported} \text{for} \text{unit} \text{tests}): \text{max} \text{N} \text{questions} \text{in}-\text{flight} \times 3 \text{model} \text{slots} = \text{ceiling} \text{of} 3\text{N} \text{parallel} \text{API} \text{calls} (\text{default} $--concurrent 3 → 9). Per-question receipts land in a per-batch tempdir and are deleted at end of run; the summary receipt inlines per-question verdicts as JSON, not file paths. Exit precedence (batch-level policy, NOT inherited from aggregate.ts): ERROR > FAIL > INCONCLUSIVE > PASS. DI seam: runEvalCrossModal(args, opts?: {runEval?: typeof runEval}) mirrors runEvalLongMemEval(args, {client?}); tests pass opts.runEval to bypass real LLM calls AND the gateway availability check. Pinned by test/eval-cross-modal-batch.test.ts.

  • src/core/eval/cycle-default.ts — single source of truth for the eval cycle-count default. Exports DEFAULT_CYCLES_TTY = 3, DEFAULT_CYCLES_NONTTY = 1, resolveCycleDefault(explicit, isTty): {cycles, usedNonTtyDefault}, and cycleDefaultSuffix(r) (returns (non-interactive default; --cycles N for more) only when the non-TTY default was applied, else ''). Consumed by eval-cross-modal.ts, eval-takes-quality.ts (run + regress), and takes-quality-eval/runner.ts (core uses only the constant — library stays TTY-agnostic; the CLI owns the TTY=3 upgrade + banner annotation). eval-suspected-contradictions.ts applies the same transparency to its $5/$1 budget default via a budgetUsdExplicit flag (the budget is overwritten in-place so explicitness can't be inferred post-hoc). Not shared with resolveWorkersWithClamp (different domain, no engine, no dedup). Pinned by test/eval/cycle-default.test.ts, test/eval-suspected-contradictions-budget-default.test.ts.

  • src/core/cross-modal-eval/json-repair.tsparseModelJSON(raw) named export with a 4-strategy fallback chain (direct parse → fence-strip → trailing-comma + single-quote + embedded-newline repair → regex nuclear option). Adversarial input throws rather than fabricating scores — the aggregator treats a throw as "this model contributed nothing this cycle" so the gate stays correct at >=2/3 successes.

  • src/core/cross-modal-eval/aggregate.ts — pure verdict logic. Judge dimension keys are normalized with trim + lowercase before cross-model roll-up, so CORRECTNESS and correctness cannot split into separate one-model dimensions. Pass criterion: (successes >= 2) AND (every dim mean >= 7) AND (every dim min across models >= 5). Inconclusive when <2/3 models returned parseable scores (guards the Object.values({}).every(...) === true empty-array PASS trap).

  • src/core/cross-modal-eval/runner.ts — orchestrator. The judge prompt pins the exact scores JSON keys per dimension (dimensionScoreKey = the label before the em-dash) so judges cannot invent spellings; aggregate.ts's normalization is the backstop. buildPrompt wraps the task and the candidate in <task_to_grade>/<candidate_output> data blocks, and neutralizeClosingTag rewrites every closing form of those tags inside the untrusted text (case-insensitive, whitespace-tolerant) to the visibly-escaped <\/tag, so a candidate carrying its own closing delimiter cannot terminate the block and land text outside the data boundary — the post-candidate grading-only instruction stays the last thing the judge reads. callSlot sends EVALUATOR_SYSTEM_PROMPT as system and the bounded prompt as the single user turn. Each cycle runs Promise.allSettled([gwChat(slotA), gwChat(slotB), gwChat(slotC)]) (bare allSettled, no rate-leases for the CLI path). Stops early on PASS or INCONCLUSIVE; runs up to 3 cycles. Default slots: openai:gpt-5.2 / anthropic:claude-opus-4-7 / deepseek:deepseek-v4-pro. estimateCost() prices via the canonical model-pricing table; test/cross-modal-default-slots.test.ts pins recipe support, pricing coverage, and three distinct providers.

  • src/core/cross-modal-eval/receipt-name.ts — receipt filename binds (slug, SKILL.md sha-8). findReceiptForSkill(skillPath, receiptDir) returns 'found' | 'stale' | 'missing'. Skillify-check surfaces the status as informational; the audit does NOT fail on missing/stale receipts.

  • src/core/cross-modal-eval/receipt-write.ts — wraps fs.writeFileSync with mkdirSync({recursive:true}) ahead of every write (gbrainPath() does NOT auto-mkdir).

  • src/commands/eval-export.ts — streams eval_candidates rows as NDJSON to stdout with schema_version: 1 prefix on every line. EPIPE-safe, progress heartbeats on stderr, stable id-desc tiebreaker so --since windows never dupe/miss rows.

  • src/commands/eval-prune.ts — explicit retention cleanup. Requires --older-than DUR. --dry-run reports would-delete count.

  • src/commands/eval-replay.ts — contributor-facing replay tool. Reads NDJSON from gbrain eval export, re-runs each captured query / search op against the current brain, computes set-Jaccard@k between captured + current retrieved_slugs, top-1 stability rate, and latency Δ. Stable JSON shape (schema_version: 1) for CI gating; human mode prints a regression table. Pure Bun, zero new deps. The dev-loop half of BrainBench-Real. See docs/eval-bench.md. parseNdjson skips lines where _kind === 'baseline_metadata' so gbrain bench publish baselines parse cleanly without the metadata header polluting row counts. Exports replayCore(engine, opts): Promise<{summary, results}> + ReplaySummary type so gbrain eval gate calls replay in-process (NOT subprocess — avoids gbrain-version-drift for source-tree CI). CLI runEvalReplay wraps replayCore.

  • src/core/bench/baseline-file.ts + src/core/bench/qrels-file.ts + src/core/bench/correctness-gate.ts + src/commands/bench-publish.ts + src/commands/eval-gate.ts — the eval loop. gbrain bench publish --from <captured.ndjson> --to <X.baseline.ndjson> writes a baseline (stamps stable query_hash per row; metadata header carries _kind: 'baseline_metadata' + thresholds + source_hash + baseline_mean_latency_ms; deterministic sort by (tool_name, query_hash); strict: empty=fail, dupes=fail with paste-ready hint, --to exists=refuse without --force). gbrain eval gate [--baseline X] [--qrels Y] is the two-gate dispatcher (regression gate via in-process replayCore, correctness gate via bare hybridSearch for determinism, both must pass when both flags set, exit 0 PASS / 1 FAIL / 2 USAGE). Source-id-aware: bench publish dedup key is (tool_name, source_ids, query_hash); qrels compare keys are ${source_id}::${slug} everywhere (so multi-source brains are keyed correctly at the file-shape layer). Latency math: (baseline + delta) / baseline <= multiplier. Fail-closed: ANY in-process throw flips verdict to fail with named breach in breaches[] — never silently exit 0. .qrels.json preserves the 12-row test/fixtures/eval-baselines/qrels-search.json fixture (slug-only relevant_slugs + first_relevant_slug auto-promote to source_id='default') AND supports the federated shape (explicit relevant: [{source_id, slug}] + expected_top1). correctness-gate.ts runs each qrels query via bare hybridSearch; per-query throw recorded as errored: true and flagged as gate failure. Audit JSONL at ~/.gbrain/audit/bench-publish-YYYY-Www.jsonl. Hermetic mode: --embedder deterministic (correctness gate ONLY — rejected with --baseline, requires --qrels) embeds each query as the qrels fixture's basis vector via src/eval/deterministic-embed.ts (basisEmbedding unit vectors; FNV-1a-derived fallback dim for off-fixture query texts) and threads it into bare hybridSearch through the queryEmbedFn seam — no API keys, no network; bare hybridSearch neither reads nor writes the semantic query cache (both live in hybridSearchCached), so deterministic runs cannot poison cached production results. scripts/run-eval-canary.ts (check:eval-canary package script, on-demand; in CI the same runner executes via test/eval-canary.test.ts in the unit matrix, not in the verify battery) is the hermetic CLI retrieval canary built on it: boots a throwaway PGLite brain under a temp GBRAIN_HOME, seeds the qrels fixture corpus (the expected-top1 page carries its query text in timeline too — page-grain FTS indexes title(A) + timeline(C) only, compiled_truth is deliberately unindexed), spawns the REAL CLI with engine-reroute/provider env stripped, and asserts exit 0 + metric floors; --record additionally appends an EvalRunRecord-shaped line to .gbrain-evals/eval-results.jsonl. Honest scope: the canary gates the hybrid ranking pipeline (keyword/title/alias arms + RRF against gold qrels) with synthetic vectors — semantic-embedding regressions remain the keyed eval suites' job. Pinned by test/bench/baseline-file.test.ts, test/bench/qrels-file.test.ts, test/bench/correctness-gate.test.ts, test/bench-publish.test.ts, test/eval-gate.test.ts, test/eval-canary.test.ts, test/eval-replay-metadata-skip.test.ts, test/cycle/nightly-probe-adapters.test.ts, test/autopilot-nightly-probe-wiring.test.ts, test/e2e/eval-loop.test.ts.

  • src/core/cycle/nightly-probe-adapters.ts — bridges the autopilot's object-shape NightlyProbeDeps to the argv-shape runEvalLongMemEval + runEvalCrossModal CLI functions. Cross-modal adapter argv MUST include --output summaryPath (without it the summary lands at the default receipt path and the adapter reads nothing from summaryPath). In-process invocation (NOT subprocess) — avoids gbrain-version-drift for source-tree CI. Pinned by test/cycle/nightly-probe-adapters.test.ts (incl. the argv-shape guard for the --output requirement).

  • test/eval-replay-gate.test.ts + test/fixtures/eval-baselines/qrels-search.json — hermetic retrieval qrels gate running in the standard PR unit-shard CI matrix (.github/workflows/test.yml, NOT the fixed-file E2E workflow). Uses the canonical PGLite block (test-isolation R3+R4) and the basis-vector embedding pattern from test/e2e/search-quality.test.ts:23-28 for fully hermetic retrieval. The qrels fixture (12 queries) uses PLACEHOLDER names only (alice-example, widget-co-example, etc. — privacy rule) and embeds each query at a deterministic basis dimension so retrieval is reproducible. Each query lists relevant_slugs[] + first_relevant_slug; the test computes top1_match_rate (top-1 == first_relevant) and recall@10 (fraction of relevant_slugs in top-10), asserting both meet floors (defaults >= 0.80 and >= 0.85). Env-overridable floors GBRAIN_REPLAY_GATE_TOP1_FLOOR / GBRAIN_REPLAY_GATE_RECALL_FLOOR (via withEnv() per R1). Refresh discipline: when ranking changes intentionally move expected slugs, edit qrels-search.json directly with a Why: line in the commit body or the gate degrades to rubber-stamp. Pinned by test/eval-replay-gate.test.ts (incl. a privacy-grep guard against real names).

  • src/core/cycle/nightly-quality-probe.ts + src/core/audit-quality-probe.ts + test/fixtures/longmemeval-nightly.jsonl + test/nightly-quality-probe.test.ts — opt-in nightly cross-modal quality probe. The phase runs gbrain eval longmemeval --by-type against the committed 10-question placeholder fixture, pipes output through gbrain eval cross-modal --batch --max-usd 5 --yes, and writes one event per run to ~/.gbrain/audit/quality-probe-YYYY-Www.jsonl (ISO-week-rotated, mirrors audit-slug-fallback.ts; honors GBRAIN_AUDIT_DIR). Default DISABLED — opt-in via gbrain config set autopilot.nightly_quality_probe.enabled true (prevents surprise API spend). 24h rate limit (pure shouldRunNightly(now, recentEvents, windowMs?)) skips with audit row outcome: rate_limited. Embedding-key short-circuit: longmemeval needs gateway.embedQuery(), so the phase exits early with outcome: no_embedding_key + stderr warn when no provider configured. Full DI surface via NightlyProbeDeps (isEnabled, hasEmbeddingProvider, resolveMaxUsd, resolveRepoRoot, runLongMemEval, runCrossModalBatch, now) so the unit test stubs every external effect. Cost ceiling: $5/run × 30 nights ≈ $150/month worst-case; expected ~$10.50/month. The nightly_quality_probe_health doctor check (src/commands/doctor.ts, right after slug_fallback_audit) reads last 7 days: SKIPPED when flag off (with enable command); OK when enabled + all PASS; WARN on any FAIL / ERROR / BUDGET_EXCEEDED with per-outcome counts. Pinned by test/nightly-quality-probe.test.ts.

  • src/commands/eval-trajectory.ts + src/commands/founder-scorecard.ts + src/core/trajectory.ts — temporal trajectory + founder scorecard. gbrain eval trajectory <entity> shows the chronological typed-claim history (mrr/arr/team_size/etc) with regressions auto-flagged inline; gbrain founder scorecard <entity> rolls up claim_accuracy / consistency / growth_trajectory / red_flags into one JSON. Pure-function math in trajectory.ts: detectRegressions(points, threshold) walks consecutive metric-value pairs per metric (10% drop default, env override GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD); computeDriftScore(points) returns 1 - mean(cosine(emb[i], emb[i-1])) over existing embeddings (null when <3 embedded points). Backed by BrainEngine.findTrajectory(opts) — both Postgres and PGLite, single SQL query, deterministic ORDER BY valid_from ASC, id ASC. Source-scoped via the sourceId scalar / sourceIds array dual pattern; visibility-filtered for remote callers. MCP op find_trajectory (read scope, NOT localOnly) registered after find_experts. Migration v67 adds optional typed-claim columns (claim_metric, claim_value, claim_unit, claim_period) + a partial index on (entity_slug, claim_metric, valid_from) WHERE claim_metric IS NOT NULL. Fence widens from 10 to 14 cells when any row has typed data; renderer stays at 10 cells when none do (no churn diff on existing fences). Metric labels normalize to lowercase snake_case via normalizeMetricLabel (15-entry seed map). The consolidate cycle phase does semantic upsert keyed on (page_id, claim, since_date) (so re-running the cycle after extract_facts clears consolidated_at cannot append duplicates via MAX(row_num)+1) and writes chronological valid_until on each cluster's older facts. The extract_facts cycle phase batch-embeds via gateway.embed() before insert AND threads pages.effective_date as the pageEffectiveDate fallback for valid_from (precedence: fence-row > pageEffectiveDate > now()). The contradiction probe MUST NOT write valid_until — grep guard at test/eval-contradictions/no-valid-until-write.test.ts. Haiku extraction lives in src/core/facts/extract.ts (not the extract-facts.ts cycle phase); its output cap is config facts.extraction_max_tokens (default 4000), a stopReason: 'length' response retries once at 2× the cap, and persistent truncation warns loudly on stderr instead of silently extracting zero facts. Mixed extractor arrays salvage valid candidates and warn with the dropped malformed count; all-malformed output still fails retryably, while an explicitly empty facts array remains a successful empty result. pageEffectiveDate is OPTIONAL because fence-write.ts callers have no Page object. Migration v89 adds a nullable event_type TEXT column on facts so the substrate carries event-shaped rows (event_type='meeting' / 'job_change' / 'location_change') alongside metric rows. TrajectoryPoint.event_type: string | null projected by both engines. TrajectoryOpts.kind?: 'metric' | 'event' | 'all' filter (default 'all'); founder-scorecard + eval-trajectory pass kind: 'metric' explicitly. Back-compat pinned by test/regressions/v0_40_2_0-trajectory-backcompat.test.ts (byte-identical computeFounderScorecard + computeTrajectoryStats with and without event rows); engine parity in test/engine-parity-event-type.test.ts.

  • src/core/trajectory-format.ts — shared formatTrajectoryBlock(points, entitySlug, opts) consumed by both gbrain think (production) and the LongMemEval harness (benchmark). Groups by (metric ?? event_type), per-metric cap 20, total cap 100, knowledge_update intent annotates value-change rows with (superseded prior). Emits a <trajectory entity="..."> XML envelope — INJECTION_PATTERNS in src/core/think/sanitize.ts escapes </trajectory>, <trajectory ...> open tags, and attribute injection so adversarial fact text can't break out. Pinned by test/trajectory-format.test.ts.

  • src/core/think/intent.ts + src/core/think/entity-extract.ts — pure classifyIntent(question) returns 'temporal' | 'knowledge_update' | 'other' (regex-first, no LLM, 'other' fast path short-circuits with zero SQL). extractCandidateEntities(question, retrievedSlugs) pulls high-precision candidates from retrieved entity-prefix slugs (people/, companies/, organizations/) and medium-precision noun phrases. Stop-word boundaries + leading-verb stripper handle "When did I last meet Marco" → marco. Both consumed by runThink and the LongMemEval harness so the two paths cannot drift. Pinned by test/think-intent.test.ts and test/think-entity-extract.test.ts.

  • src/commands/eval-suspected-contradictions.ts + src/core/eval-contradictions/{judge,runner,types,date-filter,cost-tracker,cache,severity-classify,cross-source,trends,calibration,judge-errors,auto-supersession,fixture-redact}.tsgbrain eval suspected-contradictions [run|trend|review]. Probe samples top-K retrieval pairs per query (cross-slug + intra-page chunk-vs-take), date pre-filters (3-rule layered — same-paragraph-dual-date overrides separation rule), LLM judge (query-conditioned; UTF-8-safe truncation; confidence-floor double-enforcement; resolution_kind output drives paste-ready commands), persistent cache keyed on (chunk_a_hash, chunk_b_hash, model_id, prompt_version, truncation_policy) (prompt edits cleanly invalidate prior verdicts), Wilson 95% CI calibration on the headline percentage with small_sample_note when n<30, judge_errors as first-class typed counters (parse_fail/refusal/timeout/http_5xx/unknown — avoids bias from silent skip), trend writes to eval_contradictions_runs, source-tier breakdown reuses DEFAULT_SOURCE_BOOSTS prefix logic, deterministic sampling (combined_score DESC + lex tiebreaker for stable cache hit-rate). Hermetic via judgeFn + searchFn DI in the runner; never touches the real gateway in tests. Engine surface: BrainEngine.listActiveTakesForPages (batched), writeContradictionsRun + loadContradictionsTrend, getContradictionCacheEntry + putContradictionCacheEntry + sweepContradictionCache. Schema migrations v51 + v52. MCP op find_contradictions (read scope, NOT localOnly, NOT in subagent allowlist — user-initiated only). Doctor check surfaces high-severity findings with paste-ready resolution commands; synthesize phase pre-fetches the latest probe's top-5-by-severity findings and threads them into buildSynthesisPrompt as an informational block. Architecture doc: docs/contradictions.md.

  • src/core/think/index.tsrunThink builds its internal LLMClient via a small adapter wrapping gateway.chat() from src/core/ai/gateway.ts (not new Anthropic() directly) so stdio MCP launches (Claude Desktop, Cursor) that don't inherit shell env still find a key set via gbrain config set anthropic_api_key (the gateway reads ~/.gbrain/config.json AND env). Test seam: opts.client?: ThinkLLMClient injection works (test/think-pipeline.serial.test.ts, test/think-gateway-adapter.test.ts); opts.stubResponse short-circuits before any LLM call. When neither key nor client is available, the "no LLM available" stub fires with NO_ANTHROPIC_API_KEY. Trajectory injection (default ON): runThink orchestrates classifyIntent(question)extractCandidateEntities(question, retrievedSlugs)findTrajectory (5s Promise.race timeout per candidate, concurrency cap 3) → formatTrajectoryBlock. buildThinkUserMessage (in src/core/think/prompt.ts) has a trajectory?: ThinkTrajectoryBlockOpts slot honoring BOTH prompt orderings (calibration mode: retrieval → calibration → trajectory → question; default mode: question → retrieval → trajectory → instruction). The MCP think op handler maps sourceScopeOpts(ctx) onto RunThinkOpts via thinkSourceScopeOpts(ctx) (operations.ts), and runThink threads the scope into runGather (src/core/think/gather.ts) — so every gather stream (hybrid retrieval, takes keyword + vector via the engines' scoped searchTakes/searchTakesVector, graph walk via traversePaths) AND trajectory resolution stay within the caller's source grant (federated sourceIds[] wins over scalar sourceId); pinned by test/e2e/think-source-isolation-pglite.test.ts. Config key think.trajectory_enabled (default true). Any error in the trajectory path degrades to "no block injected" + TRAJECTORY_INJECTION_FAILED warning — the think call never crashes from trajectory. Production path skips fallback_slugify resolutions (avoid querying invented slugs); the LongMemEval harness accepts them. Pinned by test/think-trajectory-injection.test.ts. Debug: GBRAIN_THINK_DEBUG=1 gbrain think "..." prints the spliced prompt to stderr.

  • src/commands/eval-longmemeval.ts + src/eval/longmemeval/{harness,adapter,sanitize,extract,intent,reader,emit,capture,gateway-client,trajectory-route}.tsgbrain eval longmemeval <dataset.jsonl> runs the public LongMemEval benchmark against gbrain's hybrid retrieval and is the repo's receipt producer for the strict retrieval metric and, with --judge, for judged answer accuracy (both reproduction paths are in docs/eval-bench.md). One in-memory PGLite per run via createBenchmarkBrain + withBenchmarkBrain; between questions, TRUNCATE over runtime-enumerated pg_tables with the infrastructure tables (sources, config, gbrain_cycle_locks, subagent_rate_leases) preserved — which is why the run's pins, written once via engine.setConfig, hold for every question. cli.ts pre-dispatch bypass skips connectEngine(), so ~/.gbrain is never opened. Metric (metrics.ts): recall joins on RAW dataset session ids through a per-question slug→raw map; recall_all@k (every gold session among the distinct sessions of the top-k CHUNK rows returned at limit: k) is the headline, recall_any@k the diagnostic, and the per-row recall_hit is a deprecated alias of recall_any_hit; _abs abstention questions are emitted with abstention: true but stay out of the recall denominators unless --include-abstention; a slug collision touching a gold id aborts that question with an error row. Pins: --mode, --reranker on|off, --autocut on|off, --expansion (OFF for every mode — the per-call setting wins over the bundle, so --mode tokenmax alone never expands), --expansion-variant-budget legacy|(0,4], --search-pin KEY=VALUE (repeatable; any search.* key, written verbatim via engine.setConfig); precedence is explicit flag > --search-pin > injected RunOpts.searchConfigSnapshot > bundle, so a flag wins over a pin on the same key. The raw --search-pin map folds into retrieval_config_hash (only when non-empty, so pin-free runs keep their hash identity) while the knobs hash covers only the resolved mode knobs. The reranker gate keys on the RESOLVED pin (flag, --search-pin, snapshot or bundle), not only on an explicit --reranker on: a run that resolves to reranker on preflights rerankerReadinessForEngine and exits 2 with the fix when the reranker cannot run (a balanced run with no VOYAGE_API_KEY exits with the fix text — --reranker off or set the key — instead of quietly scoring un-reranked rows). Gates (exit 1): a run in which every question errored, a row that fell through un-reranked under a resolved reranker-on pin (reranker_skipped_rows), a row whose vector arm silently degraded to keyword-only on a non---keyword-only run (vector_degraded_rows: vector_enabled:false / embed_unavailable / embed_timeout), an --expansion row that did not expand as configured (expansion_failed_rows), an --expansion-replay miss, and --by-type-floor F breaches — which gate on recall_all by default (--by-type-floor-metric recall_any selects the lenient rate). The run-end gates and GBRAIN_LME_DEBUG=1 prints per-question wall time to stderr; --record run through ONE finishRun from both the main path and the no-op resume path. Rows: retrieved[] (every returned chunk row: slug, chunk_id, RAW session_id, rank, score, rerank_score?, alias_hit?), retrieved_session_ids, distinct_sessions_in_top_k, gold_total/gold_found, search_meta (vector_enabled, expansion_applied, degraded, derived reranked = some finite rerank_score and no skip stage, autocut), retrieval_config_hash, the reader pins on every answered row (reader_model = the requested id, reader_model_snapshot = the provider-reported id when it differs — a dated API snapshot — else null, reader_prompt_sha, reader_max_tokens; --retrieval-only rows carry retrieval_only: true instead so a judge backfill can refuse them), expansion_variants when expansion ran (--expansion-replay FILE serves them back and stamps expansion_replayed, so every cell differs only in its knobs), and with --capture-pool a rerank_pool — the exact pre-autocut returnPool hybridSearch hands to applyAutocut (via HybridSearchOpts.onRerankPool; unscored alias/exact-lookup injections included; rrf_rank, pool_rank, est_tokens per row; autocut_kept_keys when the returned rows are the kept set) for the offline autocut-floor replay. Summary: --by-type emits a schema_version: 2 by_type_summary (per-type + aggregate {total, all_hit, all_rate, any_hit, any_rate}, excluded_abstention, mean_distinct_sessions, legacy_rows, gold_missing_from_haystack, slug_collisions, and run_config: every pin, embedder model@dims, dataset sha256 + question count, knobs_hash + KNOBS_HASH_VERSION, retrieval_config_hash, the embed-cache receipt, the degradation counters); resume-replace keeps exactly ONE summary at the tail (emit.ts:emitByTypeSummary: the summary is always the FINAL line — any prior by_type_summary is removed before the new one is appended, and the file rewrite is atomic, <path>.summary.tmp + rename, so a kill mid-write never truncates the paid rows — its _meta.metric_glossary is the ONE glossary block per response and names exactly the metrics carried, recall_all@k, recall_any@k, and qa_accuracy when the judge lane ran; a CR inside any emitted line throws rather than splitting a JSONL record). Reader (reader.ts): READER_SYSTEM_TEXT is a module constant — every per-question input (question, Current Date: {question_date} when the dataset row carries one, the trajectory block, the sanitized <chat_session> blocks) lives in the USER message — so READER_PROMPT_SHA (sha256 of the system text) is a run-level pin and two rows with equal shas saw the identical instruction; READER_MAX_TOKENS is 512 (official 500). Disclosed deviations from the official run_generation.py prompt: an abstention instruction (say the information is not available / "I don't know" when the retrieved sessions lack it — without it the 30 _abs questions are answered and judged wrong by construction), the #4338 data-boundary framing + pattern stripping, and the 512 cap. generateAnswer returns {text, response_model}, response_model being the provider-reported snapshot when it differs from the requested id (the harness's gateway client maps ChatResult.responseModel ?? model into the Anthropic-shaped message.model). Judge lane (--judge): implies --by-type; --judge --retrieval-only is a usage error. Preflight (judge-lane.ts:judgePreflight): no usable chat provider for the judge model → exit 1; --max-usd against an unpriced model → exit 2 (pass --max-usd off); estimate over the cap without --yes → exit 2. The estimate (judge.ts:estimateJudgeRunUsd) assumes READER_MAX_TOKENS per live hypothesis and the stored hypothesis for backfill rows; the BudgetLedger soft-stops at the cap and the remaining rows are stamped judge_skipped: 'budget'. Live rows are judged inline after each reader call. --judge --resume-from FILE is the judge-only backfill: selectBackfillRows picks every prior row with a hypothesis and no settled verdict (judge_error rows are re-judged, --retrieval-only rows are refused, rows absent from the dataset are left unjudged with a WARN); a same-file resume ALWAYS appends (makeEmitter(outputPath, append) has no atomic-rewrite mode — appending is the only write path) — judged backfill rows and retries land as newer duplicates the moment they finish, so a timeout or kill loses at most the in-flight question — and compactJsonlByQuestionId (emit.ts) rewrites the file atomically (<path>.compact.tmp + rename) to one row per question_id (last wins, first-seen order) before the summary is emitted; readJsonlRows / loadResumeSet read appended files last-wins and --judge-concurrency N parallelizes the backfill. Every judged row carries judge_config_hash — the judge pins (model, JUDGE_PROMPT_VERSION, max_tokens, temperature) plus the reader pins the row was produced under; a prior row hashes from its OWN recorded reader_model / reader_prompt_sha / reader_max_tokens, so a file answered by another reader is never relabelled as this run's, and rows already judged under a different hash are refused unless --allow-mixed-run-config. Run end: qa_accuracy (qa-accuracy.ts) is rebuilt from ALL rows (prior + new, last row per question_id wins) whenever the lane ran or any row carries a verdict; a run with judge_errors > 0, skipped_budget > 0 or unjudged > 0 (the qa_accuracy.complete predicate) prints a FAIL … NOT publishable line and exits 1 unless --allow-incomplete-judgments (WARN, exit 0) — the fix is --judge --resume-from FILE until all three are 0; a row whose judge call threw is stamped judge_error: 'provider_error' by the backfill, never silently left unjudged. Judge row fields: exactly one of judge_correct / judge_error (+ secret-redacted judge_error_detail) / judge_skipped, plus judge_model, judge_model_snapshot, judge_raw (first 200 chars), judge_cost_usd, judge_attempts, judge_prompt_kind, judge_prompt_version, judge_config_hash. RunOpts.judgeClient / judgeBackoffMs are the test seams. Embed cache: on by default at ~/.cache/gbrain-eval/longmemeval-embed.sqlite (--embed-cache FILE / --no-embed-cache; never installed on --keyword-only), installed through the gateway embed-transport seam so every arm sees byte-identical vectors; the cache transaction wraps only the embed-producing section of a question (import + search), never a reader call. Resume: --resume-from FILE re-scores prior rows from retrieved[] + the dataset's gold (stored booleans are never trusted) and refuses a file whose rows carry a different retrieval_config_hash unless --allow-mixed-run-config; error rows without a hypothesis are retried. --question-ids FILE restricts the run to a listed slice (unknown ids or an empty file exit 1); --record appends a secret-redacted EvalRunRecord (schema 3, suite longmemeval, params = run_config) via persistRunRecord. Flags live in ONE table, LME_FLAGS, that drives both parseArgs and printHelp, so the flag-registry scan sees every literal and help cannot drift from the parser. Sanitization parity: harness.ts reuses INJECTION_PATTERNS from src/core/think/sanitize.ts; retrieved chat content is wrapped in <chat_session id="..." date="..."> and the answer-gen system prompt declares it UNTRUSTED. RunOpts seams — client, extractorClient/extractorModel, engine, searchConfigSnapshot, expandFn, embedTransport, rerankerReadiness, recordDir — let the full pipeline run hermetically without an API key. Trajectory routing (default on; --no-trajectory bypasses BOTH the extractor and the intent routing, the like-for-like retrieval setting): extract.ts runs the Haiku claim extractor over each haystack session into the benchmark brain's facts table (content-hash cache, per-question alias map, fail-open on every error path), intent.ts prefers the dataset's question_type before the SHARED regex set from src/core/think/intent.ts, temporal/knowledge_update questions splice a findTrajectory block into the reader prompt, and rows carry intent, trajectory_points, entity_resolved, resolution_source, methodology_note (extractor=haiku-preprocess-full-haystack-v1 — that number is "gbrain + Haiku-preprocess", not "gbrain alone"). Pinned by test/longmemeval-metrics.test.ts, test/eval-longmemeval-mixedcase.slow.test.ts (raw-id join, strict/any split, abstention, pins, gates over the placeholder fixture test/fixtures/longmemeval-mixedcase.jsonl), test/eval-longmemeval-cli-smoke.test.ts, test/longmemeval-embed-cache.test.ts, test/longmemeval-judge.test.ts + test/eval-longmemeval-judge.slow.test.ts (the judge lane, reader pins and qa_accuracy denominators), test/eval-longmemeval.slow.test.ts, test/eval-longmemeval-e2e.slow.test.ts, test/eval-longmemeval-search-config.test.ts, test/longmemeval-extract.test.ts, test/longmemeval-intent.test.ts, test/longmemeval-trajectory-routing.test.ts.

  • src/eval/longmemeval/metrics.ts — LongMemEval strict-recall metrics: the raw-id join, recall_all@k / recall_any@k, per-type buckets, the schema-v2 by_type_summary, and the per-question row assembler. PURE: no engine, no I/O, no LLM, so the harness and the tests score the same bytes. The join is on RAW session ids: haystackToPages lowercases and hyphenates ids to build slugs, so a slug tail compared against answer_session_ids never matches on the public _s split; buildSlugToRawMap inverts the slug construction per question, distinctRetrievedSessions joins through it, detectSlugCollisions / collisionsTouchingGold name the ambiguous slugs (a gold-touching collision makes the harness emit an error row), goldMissingFromHaystack counts dataset defects, and normalizeSessionId exists only for slug construction — never for the gold compare. k semantics: recall_*@k is scored over the DISTINCT sessions among the top-k CHUNK rows returned at limit: k (the caller slices first; scoreRecall treats k as a guard); empty gold scores both hits false, not vacuously true, and such rows stay out of the denominator. addRowToBucket folds a v2 row into total/all_hit/any_hit; a row carrying only the deprecated any-only recall_hit counts toward total + any_hit and bumps legacy_rows (its all_rate is therefore a lower bound); buildByTypeSummaryV2 emits sorted type keys, null rates on empty buckets, mean_distinct_sessions, and the caller's run_config. buildRow assembles the JSONL row — scored fields over results.slice(0, k), retrieved[] + retrieved_session_ids over EVERY returned row so replay can re-score at any smaller k — and harness passthrough extra keys never override the scored fields. Pinned by test/longmemeval-metrics.test.ts.

  • src/eval/longmemeval/run-config.ts — the resolved retrieval pins, retrieval_config_hash, the run_config receipt block, secret redaction, and the --question-ids loader. Pure given its inputs (the harness resolves the mode bundle and the embedder string and passes them in). RetrievalPins = {mode, keyword_only, reranker: {enabled, model}, autocut, expansion, expansion_variant_budget, embedder (model@dims or 'unconfigured'), top_k, trajectory}; retrievalConfigHash(pins, {knobs_hash, knobs_hash_version}) is sha256 over stableStringify (sorted keys at every level) of the pins PLUS the resolved knobsHash, so two runs hash equal regardless of flag order, dataset, output path or clock, and a resume file written under a config snapshot that differs in ANY result-shaping knob — not just a pin — is a different run. redactSecrets (URL userinfo, bearer tokens, secret-looking key=value pairs, provider key prefixes, ?password=) is applied to every error string BEFORE it lands in a receipt row or the eval ledger. loadQuestionIds reads one id per line with # comments, dedupes, and throws on a missing or empty file. buildRunConfig stamps the summary's run_config (pins, expansion_replay, dataset sha256 + count, question_ids_file, both hashes, the CacheReceipt {path, hits, misses, bypassed, infra_faults, canonical_sha256, sha256} or cache: null + cache_skipped reason, and the counters reranker_skipped_rows, vector_degraded_rows, expansion_failed_rows, expansion_replay_miss, gold_missing_from_haystack, slug_collisions, excluded_abstention, errors).

  • src/eval/longmemeval/resume.ts — reading a prior run's JSONL back: row scanning, the mixed-run check, expansion-variant replay maps, and per-type bucket re-seeding. INVARIANT: recall is RECOMPUTED, never trusted — seedBucketsFromRows re-derives both metrics for every prior row from retrievedIdsAtK (prefers retrieved[] sliced to k; falls back to the older retrieved_session_ids shape) joined against the dataset's gold, so a file written before the raw-id join simply scores false instead of poisoning the summary, and there is no legacy any-only counter. readJsonlRows skips corrupt lines (a SIGKILL tail); isScoredQuestionRow excludes summaries and hypothesis-less error rows (those are retried); checkResumeConfigHash reports rows stamped with a foreign retrieval_config_hash (refused by the harness unless --allow-mixed-run-config) and tolerates unstamped rows; loadExpansionReplay maps question_id → recorded expansion_variants. Pure given its inputs (file reads are the only I/O).

  • src/eval/longmemeval/judge.ts + judge-lane.ts + qa-accuracy.ts — the LongMemEval LLM-as-judge lane behind gbrain eval longmemeval --judge. judge.ts is a faithful port of the official evaluate_qa.py::get_anscheck_prompt: the per-type instruction (standard for single-session-user / single-session-assistant / multi-session; temporal-reasoning adds the off-by-one-days clause; knowledge-update; the single-session-preference rubric) and the abstention instruction for _abs ids (the id suffix beats the type; an unknown type falls back to standard), ONE user message per question, DEFAULT_JUDGE_MODEL openai:gpt-4o, JUDGE_TEMPERATURE 0, JUDGE_MAX_TOKENS 16 (the OpenAI API's minimum; the official 10 is rejected, and a one-token yes/no verdict is unaffected), and the official verdict rule (parseJudgeVerdict: 'yes' substring of the lowercased completion). Disclosed deviations, carried verbatim in JUDGE_METHODOLOGY_NOTE on every summary: (1) the question / reference / response sit inside <judge_input> data-boundary framing under JUDGE_DATA_BOUNDARY_INSTRUCTION (#4338) — escapeJudgeData neutralises tag closures inside the data and the response text is otherwise unaltered, so the judge grades what the reader actually said; (2) the judge_error class — classifyJudgeResponse returns null for a completion that is neither a yes nor a standalone no (malformed, re-judged) instead of a silent no; (3) abstention detected by the _abs suffix; (4) the unknown-type fallback. JUDGE_PROMPT_VERSION bumps whenever any instruction text, framing or field label changes. judgeConfigHash = sha256 over the stable JSON of {judge_model, prompt_version, max_tokens, temperature, reader_model, reader_prompt_sha, context: {k, max_tokens}}. judgeRow judges one row from its stored hypothesis: budget-skips on the PROJECTED cost (estimateJudgeCallUsd) BEFORE spending, records actual cost after, never throws on a transport failure, and stripJudgeFields removes every prior judge_* key first so a re-judge leaves no stale field. judge-lane.ts is the harness-side orchestration, pure except runJudgeBackfill (whose only effects are the injected client and in-place row updates; no engine, no file I/O): parseMaxUsd (N, or off / none / unlimited → no cap), makeJudgeConfigHasher (per-row hasher — a prior row's recorded reader pins win over this run's; k is the run's --top-k, already gated by retrieval_config_hash), hasJudgeAttempt / hasSettledVerdict, selectBackfillRows (candidates = rows with a hypothesis and no settled verdict; counts settled, mismatched + the foreign hashes, retrievalOnly, missingFromDataset), judgePreflight (availability → pricing → estimate → cap: exit 1 no provider, exit 2 unpriced-with-budget or over-cap without --yes) and runJudgeBackfill (runWithLimit at --judge-concurrency; the dataset's answer is the reference, the row's own answer the fallback). qa-accuracy.ts builds the qa_accuracy block from ALL rows (pure; last row per question_id wins): accuracy_headline (alias accuracy) = correct / total_questions over EVERY question including _abs — a judge_error, a budget-skipped row, a reader-error row and a never-judged row all score INCORRECT, so it is never more lenient than the official scorer; accuracy_excluding_errors = correct / judged (secondary); accuracy_470 = the headline rule over the non-_abs questions (the retrieval-metric denominator); by_type and an abstention sub-block; judge_error_classes; ci95_bootstrap (percentile bootstrap over the headline 0/1 vector, label: 'question-sampling only'); complete (no error, no skip, no unjudged row — the publishability bit the run-end gate reads); mixed_judge_config; est_cost_usd, actual_cost_usd (summed over the rows' own judge_cost_usd across resumes), run_cost_usd (this run's ledger); methodology_note. Pinned by test/longmemeval-judge.test.ts and test/eval-longmemeval-judge.slow.test.ts.

  • src/eval/shared/judge-runner.ts + src/eval/shared/bootstrap.ts — dataset-agnostic judge mechanics shared by the LongMemEval lane and any later judged eval. judge-runner.ts owns the chat-client seam (JudgeChatFn — the gateway chat in production, a canned fn in tests), bounded retries with exponential backoff (default 2 retries, base 500 ms, sleep seam), the closed judge_error vocabulary (timeout | rate_limit | empty | refusal | malformed | provider_error, exported as JUDGE_ERROR_CLASSES), usage summed over every attempt, per-call cost through the ONE canonical pricing table (canonicalLookup; an unpriced model yields cost_usd: null, never 0, and isJudgeModelPriced lets a budget refuse it), and the BudgetLedger (maxUsd === null = no cap; canAfford(next) is PROJECTED so a run never overshoots by more than the calls already in flight; unpriced calls are counted, not summed; snapshot() for receipts). INVARIANT: a judge malfunction is an error outcome, never a verdict: 'incorrect'runJudge never throws on a transport failure (only an already-aborted AbortSignal or a bug in the caller's parse can), classifies 429 / rate-limit prose → rate_limit, abort / timeout → timeout, everything else → provider_error, and maps a refusal / content_filter stop, an empty completion and a null from parse to refusal / empty / malformed; the caller decides how the headline scores them. It passes temperature and maxTokens through ChatOpts and records ChatResult.responseModel as response_model. bootstrap.ts: bootstrapMeanCi(xs, {resamples = 10000, seed = 42, alpha = 0.05}) is a seeded (mulberry32) percentile bootstrap of the mean, byte-reproducible; n = 0 → nulls, n = 1 → a degenerate [x, x]. INVARIANT: every block carries label: 'question-sampling only' — the interval quantifies how much the number would move under a different draw of questions and says nothing about reader / judge nondeterminism, dataset revision or prompt drift. Pinned by test/longmemeval-judge.test.ts.

  • src/eval/longmemeval/diagnostics.ts + scripts/lme-miss-diagnostics.ts — LongMemEval miss diagnostics: locate WHERE a strict miss is lost before any fix is chosen. Input is a harness receipt (ndjson) plus the dataset; for every question with recall_all_hit=false (every scored question under --all) the question's brain is re-created EXACTLY as the harness built it (one in-memory PGLite per run, resetTables per question, haystackToPagesimportFromContent, the embed cache installed through the same seam so every page / question vector is a cache hit) and each gold session missing from the receipt's top-k is located per arm: vector_rank (first chunk row of the session in engine.searchVector, paged by offset to depth, default 200, because one call caps at MAX_SEARCH_LIMIT), keyword_rank (engine.searchKeyword, OR-fallback as hybrid), title_rank (engine.searchTitles), fused_rank_* (the pre-rerank RRF order of ONE hybridSearch call at fusedLimit, default 50, under the same pins, captured via onRerankPool(pool, preRerank)), post_rerank_* (that call's post-rerank pool) and final_rank_rows (what the call returned after autocut / limit). Frozen classes (classifyMiss): (i) absent from every arm's top-depth, (ii) in an arm pool but outside the fused top-k pre-rerank, (iii) in the fused top-k pre-rerank but reranked out, (iv) ceiling (more gold sessions than k); two observational classes keep the receipt honest — rerun_hit (the re-created run places the gold inside the top-k: the receipt's miss did not reproduce under these pins) and autocut_dropped / post_fusion_dropped (the gold survived fusion + rerank and a later trim removed it). Frozen hypothesis probes: h1Signature (≥ 2 gold, one at fused session rank 1–3, the missing one at 6–15), the counterfactual clause sub-queries (splitClauses / splitClausesDetailed over the frozen pattern list how_many_between | between | first_or | before_after, with the content-token and quoted-span guardrails; H1 is supported when a clause's own vector top-5 contains the missing gold) and the h3Candidates split — H3a candidate generation (gold in the vector top-depth but outside the pre-fusion pool, computeInnerLimit) vs H3b reranker depth (in the fused pool but beyond reranker_top_n_in). splitMembership tags each question with the committed evals/longmemeval/splits-seed42.json splits (dev40 | decision430 | halfA430 | halfB430 | halfA470 | halfB470). INVARIANT: the clause sub-query embeds BYPASS the embed cache, so a diagnostics run never adds rows to the shared like-for-like cache (its canonical hash is a receipt field). INVARIANT: every helper except runDiagnostics (the one engine-touching entry point; applyPins writes the pins) is pure over plain data — classifyMiss, splitClauses, h1Signature, h3Candidates, the rank builders, parseReceipt / pinsFromReceipt / receiptTopKSessions, summarizeDiagnostics, renderDiagnosticsMarkdown — so the tests pin them without PGLite, and every printed metric routes through the shared glossary (glossFor, with LOCAL_GLOSSARY for the diagnostics-only names). The script owns argv, gateway bootstrap, file I/O and printing (<receipt.ndjson> --dataset FILE [--splits FILE] [--mode M] [--reranker on|off] [--autocut on|off] [--expansion-variant-budget legacy|B] [--embed-cache PATH | --no-embed-cache] [--k N] [--depth 200] [--fused-limit 50] [--all] [--question-ids FILE] [--limit N] [--out-ndjson FILE] [--out-md FILE] [--json]); pins default to the receipt's flat run_config on the by_type_summary line (mode, reranker.{enabled,model}, autocut, expansion, expansion_variant_budget, topK, embedder; a legacy nested pins block is still accepted) and explicit flags win; it is NOT a gbrain subcommand, so its flags sit outside the CLI flag registry by design. Spend: with the receipt's cache every page / question embed is a hit; the ≤ 2 clause sub-query embeds per miss and, under --reranker on, one rerank call per miss are the only paid calls. Exit 0 ok · 1 bad input / run error · 2 gateway or reranker not ready. Pinned by test/longmemeval-diagnostics.test.ts.

  • src/eval/shared/embed-cache.ts — content-addressed embedding cache for fixed-corpus evals (dataset-agnostic; the LongMemEval harness is its first consumer). INVARIANT: a cached vector is served ONLY for the exact (model@dims, text, side) it was computed for, and every row is integrity-checked on read (declared dims == stored bytes / 4 == vector length); a mismatch is a HARD EmbedCacheIntegrityError naming the file and the key — never a silent re-embed, because a silently corrupted cache makes every arm's vectors non-comparable. Key = ${model}@${dims} (+ #query for query-side asymmetric embeds, so zembed/Voyage query vectors can never be served a document-side row) + : + sha256(text). Storage: bun:sqlite, WAL journal + synchronous=NORMAL, busy retry, one transaction per question's embeds; local filesystem only. installEmbedCache installs it through the gateway's __setEmbedTransportForTests seam (gateway.ts sits at its module-size ceiling; a named setEmbedTransport() hook is a filed follow-up) and restores the caller-supplied realTransport ?? null on uninstall. Stats: hits, misses (genuine misses PLUS every value of a batch that fell open on an infrastructure fault), bypassed (SDK model id ≠ the installed model — never cached), infra_faults (file deleted mid-run / table dropped / disk error → uncached re-embed or lost write-back, so a run that lost its cache can never show a clean misses: 0). The canonical hash (PRAGMA wal_checkpoint(TRUNCATE) then sha256 over the sorted (key, dims, sha256(vector)) rows) goes into run_config.cache.canonical_sha256 so two runs can prove they saw the same vectors. Pinned by test/longmemeval-embed-cache.test.ts.

  • src/eval/shared/autocut-replay.ts + scripts/replay-autocut-floor.ts — pure replay of the autocut weak-top-floor sweep over captured rerank pools, and its CLI. INVARIANT: the replay uses the SAME applyAutocut the live path uses, over the SAME pre-autocut reranked pool (gbrain eval longmemeval --capture-poolrerank_pool, unscored alias/exact-lookup rows included — dropping them would change decision.total), with the SAME preserve predicate (alias_hit === true || exact_lookup === true), and slices to k AFTER autocut exactly like hybrid.ts, so validateLive(rows, floor) can demand byte-for-byte agreement with the recorded live decision (search_meta.autocut, plus autocut_kept_keys when present) before any other floor cell is trusted. Every cell — including floor off — comes from ONE capture; no second reranker call. Metric semantics mirror the harness (distinct sessions among the first k kept chunk rows; recall_all = gold ⊆ distinct; recall_any = non-empty intersection), with the benefit metrics mean_returned_results / mean_returned_est_tokens (why autocut exists) and recall as the guardrail; pairedDelta gives wins/losses/net per type vs the first floor, splitHalf(rows, seed) gives a seeded half-A/half-B selection/confirmation split, topScoreHistogram publishes the reranker's top-score distribution. normalizePoolRow throws on a row without slug/session_id (a silently dropped row shifts every cliff). The script owns argv (<capture.ndjson> --floors off,0.10,… [--k 5] [--validate-live F] [--split-half SEED] [--jump 0.2] [--min-keep 1] [--json]; --validate-live takes exactly one floor), file I/O and printing; every metric it prints routes through src/core/eval/metric-glossary.ts (REPLAY_GLOSSARY_KEYS); exit 0 ok · 1 validate-live mismatch or bad input · 2 usage. Pinned by test/replay-autocut-floor.test.ts.

  • scripts/eval-spend-guard.sh — hard spend cap for paid eval runs: scripts/eval-spend-guard.sh <cap_usd> <estimate_usd> -- <command...>. INVARIANT: a paid command never launches when ledger total + estimate would exceed the cap, and every launch is appended to the ledger whether or not it succeeded, so the running total can only under-state spend if the command itself lies about its cost. FAILS CLOSED on anything it cannot account for: a missing ledger (a missing file is NOT a $0 ledger — the first run sets GBRAIN_EVAL_SPEND_LEDGER_INIT=1, which prints a loud NEW LEDGER line), an unparseable ledger line (named by line number), a signed or malformed amount (a negative estimate would drive the ledger backwards). Ledger $GBRAIN_EVAL_SPEND_LEDGER (default ~/gbrain-lme-receipts/spend.jsonl): every launch writes TWO rows sharing a run_id — a status: running reservation appended BEFORE the command starts (cost = the estimate, so a concurrent guard already counts it) and a status: done reconciliation after it exits (actual cost, exit code); the sum is reconciled rows + unreconciled reservations, so a guard killed by INT/TERM/HUP (trap-reconciled at the estimate) or SIGKILL (reservation left standing) never under-counts, and an unwritable ledger refuses the launch — the child gets TERM, then KILL after GBRAIN_EVAL_SPEND_GUARD_KILL_GRACE_SECONDS (default 10). Legacy single-row lines {ts, estimate_usd, cost_usd, exit_code, command} still count. The recorded cost comes from $GBRAIN_EVAL_ACTUAL_COST_FILE (exported to the child as a scratch path when unset; bare unsigned number or JSON with cost_usd) only when positive, else the estimate (over-stating is the safe direction). Amounts normalize to %.6f; awk/grep/sed only — no jq or bun at runtime. Exit: the wrapped command's code · 2 usage · 3 refused. Pinned by test/eval-spend-guard.test.ts.

  • evals/longmemeval/splits-seed42.json + evals/longmemeval/dev-slice-seed42.txt — the committed, seeded question-id splits that carry the LongMemEval decision discipline (ids only, never question text or session content). schema_version: 1; dataset_sha256 + scored: 470 / abstention_excluded: 30 pin the exact cleaned _s corpus the splits were drawn from; prng = mulberry32 over question_ids sorted ascending with Fisher-Yates; seeds dev: 42, halves470: 4242, halves430: 4243. Lists: dev40 (the 40-question dev slice, per-type counts in dev40_type_counts), decision430 (the held-out remainder every pre-registered success rule is decided on), halfA470/halfB470 (235 each) and halfA430/halfB430 (215 each) for select-on-A / confirm-on-B decisions; type_counts records the per-type totals of the 470. dev-slice-seed42.txt is dev40 one id per line for gbrain eval longmemeval --question-ids. Invariant: mechanisms are chosen on the dev slice or half A and decided on decision430 or half B; a full-470 row published alongside is labelled as including the dev slice.

  • docs/eval-bench.md — contributor guide for using captured data to benchmark retrieval changes before merging. Linked from CONTRIBUTING.md under "Running real-world eval benchmarks (touching retrieval code)".

  • src/core/eval-capture.ts — op-layer capture wrapper called from src/core/operations.ts query + search handlers (catches MCP + CLI + subagent tool-bridge from one site). Fire-and-forget; failures route to engine.logEvalCaptureFailure so gbrain doctor sees drops cross-process. Capture is off by default — isEvalCaptureEnabled resolution: explicit config.eval.capture (true/false) wins, else process.env.GBRAIN_CONTRIBUTOR_MODE === '1', else off. Contributors set export GBRAIN_CONTRIBUTOR_MODE=1. PII scrubber gate is independent and defaults to true regardless of CONTRIBUTOR_MODE.

  • src/core/eval-capture-scrub.ts — zero-deps PII scrubber: emails, phones, SSN, Luhn-verified credit cards, JWT-shaped tokens, bearer tokens.

  • src/core/search/hybrid.tsPromise<SearchResult[]> return shape. onMeta?: (m: HybridSearchMeta) => void callback so op-layer capture records what hybridSearch actually did; callers that don't need it leave it undefined. HybridSearchOpts.queryEmbedFn?: (text) => Float32Array | Promise<Float32Array> is the hermetic eval seam: when set, the TEXT vector arm's query embedding comes from this function instead of the gateway's query-embed path AND the no-embedding-provider keyword-only short-circuit is bypassed, so deterministic eval canaries (gbrain eval gate --embedder deterministic, scripts/run-eval-canary.ts) run the vector arm with no provider key; never set on production paths — absent, the production path is untouched, and bare hybridSearch never touches the semantic query cache so the seam can't poison query_cache. HybridSearchOpts.types?: PageType[] (on SearchOpts) threads a multi-type filter into per-engine searchKeyword + searchVector + searchKeywordChunks as AND p.type = ANY($N::text[]) (primary consumer gbrain whoknows, filters to ['person','company']); AND-applies alongside the single-value type filter. hybridSearch resolves the embedding column at the boundary via resolveColumn(loadRegistry(cfg), opts.embedding_column, cfg) from src/core/search/embedding-column.ts, threads the ResolvedColumn descriptor (not a raw string) into per-engine searchVector, and uses isCacheSafe(resolved, cfg) for the cache-skip decision so a repointed embedding builtin doesn't leak across vector spaces. cosineReScore calls engine.getEmbeddingsByChunkIds(ids, resolved.name) so rerank uses vectors from the active column, not the hardcoded OpenAI embedding, and hydrates each result's raw query↔chunk cosine onto SearchResult (the calibrated signal evidence and --explain consume). SearchOpts.onVectorPoolMeta is the engines' out-channel for searchVector's bounded pagination escalation (one dense page filling the inner candidate pool escalates the pool ×4 up to 3 times; HNSW-backed columns additionally cap at the ef_search ceiling, while exact-scan columns above the index dim ceiling are bounded by the escalation count alone); hybrid passes the collector and owns the emit, fired when the loop ends with the pre-DISTINCT pool still full — at the substrate cap or after the escalation budget. The query MCP op accepts embedding_column for per-call A/B; search (keyword-only) rejects it. Two post-fusion stages + evidence stamp: applyTitleBoost(results, query, titleBoost, floorThreshold) multiplies a result's score by the resolved title_boost when isTitlePhraseMatch fires, stamps title_match_boost, inherits the floor-ratio gate so a title match can't shove a much-stronger page below it; applyAliasHop(engine, results, query, opts) normalizes the query, calls engine.resolveAliases, and on exact normalized-alias match surfaces that page at top-of-organic + epsilon with alias_hit=true; stampEvidence(...) runs LAST (after the alias hop, before slice) on every path — keyword-only, no-embed, and full hybrid — so MCP callers and --explain read the same evidence + create_safety contract. title_boost resolved from the mode bundle and threaded in. runPostFusionStages has a 4th stage (graphSignalsEnabled, onGraphMeta, onScoreDistribution). base_score stamped at function entry idempotently (captured ONCE before any boost stage mutates score). Each post-fusion stage stamps its multiplier: applyBacklinkBoostbacklink_boost, applySalienceBoostsalience_boost, applyRecencyBoostrecency_boost. applyReranker (earlier in the pipeline) stamps reranker_delta as a rank delta (positive = improved). applyExactMatchBoost in src/core/search/intent-weights.ts stamps exact_match_boost when fired. Per-stage attribution powers gbrain search --explain — every boost surface carries its own field so formatResultsExplain reads them all without coupling to internal stage ordering. with src/core/search/sql-ranking.ts + src/core/operations.ts + src/core/types.ts: agent-warning channel. SearchResult.content_flag?: {reason, detail} (optional field in types.ts) is stamped post-fusion by stampContentFlags (the stampEvidence precedent) in hybridSearch AND in the keyword-only search MCP op so both retrieval paths surface the marker. get_page returns a top-level content_flag parallel field via getContentFlag(page.frontmatter). buildVisibilityClause (sql-ranking.ts) ANDs in QUARANTINE_FILTER_FRAGMENT so quarantined pages are excluded from all six search call sites (alongside soft-delete + archived-source filters). Pinned by test/sql-ranking.test.ts + test/e2e/quarantine-search-exclusion.test.ts. Cross-modal routing at the embed step: effectiveModality resolves per-call opts.crossModal (literal 'auto' → undefined) → suggestions.suggestedModality'text'. Image route: embedQueryMultimodal + searchVector({embeddingColumn: 'embedding_image'}), skipping expansion + keyword. 'both' route: parallel text + image vector searches merged via rrfFusionWeighted with effectiveRrfK(baseRrfK, weight) from the configured cross-modal weights. Unified routing fires when search.unified_multimodal is true — bypasses dual-column branching, runs embedQueryMultimodal + searchVector({embeddingColumn: 'embedding_multimodal'}), fail-open on zero rows (non-strict falls through to dual-column). LLM modality escalation fires only when no explicit per-call opt is set AND the regex returned 'text' AND search.cross_modal.llm_intent is on AND isAmbiguousModalityQuery fires; fail-open on every error. compiledTruthBoost(result, applyBoost) is exported for direct predicate tests: a synthetic chunkless title row (chunk_id === 0 AND blank chunk_text) never receives the 2x compiled-truth boost (test/search/compiled-truth-boost.test.ts). The reranker pass-through callback is typed with RerankPassThroughReason imported from rerank.ts — one union, not a re-declaration. RRF inputs are assembled by ONE composeFusionLists call (next entry) over role-tagged vectorArms built with pushVectorList at every assembly site (unified, image-only, text/both, and the allSettled salvage path — where the original role additionally requires the original's searchVector to have succeeded); after expandFn, queries[0] is re-enforced to be the caller's query and repeats are deduped, so a variant can never take the anchor role. HybridSearchOpts.onRerankPool (eval-only, best-effort) fires immediately before applyAutocut with the exact pre-autocut returnPool — post alias-hop / exact-lookup / adaptive-return, unscored injected rows included, even when autocut is off — plus the deduped pre-rerank order, so an offline autocut replay is byte-faithful. Immediately after applyReranker — only when it actually reordered (reranked !== deduped) and never for image modality — pinRelationalRows (relational-rerank-pin.ts) re-pins the relational arm's rows above the reranked text rows bounded by resolvedMode.relational_rerank_pin, stamps relational_pinned, and emits HybridSearchMeta.relational_rerank_pin; ensureRelationalEvidenceSlot still runs after autocut (usually a no-op on relational queries now — it remains the page-1 guarantee for pin 0, reranker fail-open, and arm rows dropped before the pin).

  • src/core/search/fusion-lists.ts — the ONE composition point for hybridSearch's RRF inputs. Every vector recall list is a ROLE-tagged arm (VectorArm {list, role: 'original'|'variant'|'clause'|'image'}, appended only via pushVectorList) — never a parallel index-aligned array: the Promise.allSettled salvage path drops failed arms and the both-mode image branch fails open, so any positional rule ("last list is the image", "index 0 is the original") mis-tags lists under partial failure. composeFusionLists({arms, keywordFusionList, titleFusionList, relationalList, includeRelational, ks, knobs}) returns the complete weighted list set (FusionListEntry {list, k, weight?}) in the fixed order vector arms → keyword (keywordK) → title (keywordK, only if non-empty) → relational (baseRrfK, only if non-empty and not image modality): text arms fuse at textRrfK and the image arm at imageRrfK only when BOTH kinds are present, otherwise every arm at vectorK (text, image-only, unified, and a both-mode whose image branch fell open). variant/clause arms share expansionVariantBudget as total RRF weight — weight_i = b / n_voting_arms over the NON-EMPTY expansion arms (an empty list casts no vote), the original arm always weight 1; a null budget emits no weight key, byte-identical to unweighted fusion; when the original is missing (its embed or searchVector failed) every surviving text arm is a variant sharing the budget. rrfFusionWeighted (hybrid.ts) scores weight / (k + rank) — the literature weighted-RRF form, a list-level multiplier that holds at every rank (a k-penalty would fade at deep ranks). textArmsNonEmpty is the role-based text-arm health gate behind the OR-relaxed keyword demotion (image arms never count, so a fell-open image branch can't veto the lexical rescue). normalizeExpansionVariantBudget is the single range contract for the knob: finite (0, 4] (or a string parsing to one) → number, null/legacy/null-literal → null, anything else → undefined (fall through to config → bundle); shared by mode.ts:loadOverridesFromConfig and both per-call seams in hybrid.ts, so an invalid per-call value never reaches fusion or the cache key. Pure, no engine/IO. Pinned by test/search/fusion-lists.test.ts (knife-edge arithmetic at budget 1.0, legacy deep-equals the pre-role text mapping, role alignment under rejected arms), test/search/expansion-variant-budget.test.ts (hermetic directional PGLite test), and the both-mode demotion-gate regression in test/keyword-relaxed-fusion.serial.test.ts.

  • src/core/search/relational-rerank-pin.ts — relational-arm rows bypass reranker DEMOTION (ranker wave, receipt R1 in scripts/r1-namedthing-rerank-ab.ts: on NamedThingBench's 39 graph-relationship questions the shipped balanced default lost hit@1 21/39 → 3/39 and hit@3 27/39 → 5/39 with the reranker on — 19 and 22 paired losses — because the cross-encoder scores chunk TEXT and an edge-derived answer's text need not mention the query's entity; with the pin at its default 3, measured with --autocut on, the shape that shipped before rule R2 turned autocut off, the same paired run shows 0 hit@1 / 0 hit@3 losses, 21/39 and 27/39, and the 11 non-relational core queries show 0 losses — one hard-negative gain, hit@1 10/11 → 11/11). pinRelationalRows(reranked, relationalList, {max, fusedOrder?, onPin?}) is a pure PERMUTATION of the post-rerank pool: relational rows (page key (source_id, slug) in the arm's list; ONE row per page — its first, highest-ranked occurrence) claim min(fused_rank, reranked_rank) (fused rank = position among relational rows in fusedOrder, the pre-rerank deduped pool; arm order for rows absent from it), claims sort ascending with ties resolved to the FUSED order then the reranked position, the first max claimants form the top block in that order, every other row follows in reranked order (an unpinned row lands at its reranked position or up to pinned.length lower, never higher). Pinned rows are shallow copies stamped relational_pinned: true; no row is added or removed (re-injecting rows dropped upstream stays ensureRelationalEvidenceSlot's job). Every no-op path (max <= 0, empty pool, empty arm, no relational page in the pool) returns the input array itself. normalizeRelationalRerankPin is the ONE range contract for the knob (non-negative integer <= 10 or a string parsing to one → that integer; the literals off / false (any case) or boolean false → 0; anything else → undefined = fall through), shared by mode.ts:loadOverridesFromConfig and both per-call seams in hybrid.ts; DEFAULT_RELATIONAL_RERANK_PIN (3) and RELATIONAL_RERANK_PIN_MAX (10) are exported. RelationalRerankPinDecision ({max, relational_in_pool, pinned:[{slug, source_id, from_rank, to_rank, fused_rank}], moved}) is surfaced as HybridSearchMeta.relational_rerank_pin for --explain. Wired in hybridSearch immediately after applyReranker and before the alias hop, gated on the reranker having actually reordered (reranked !== dedupedapplyReranker returns its input on every fail-open / skip / pass-through path, and the fused order already carries the arm) and on non-image modality; autocut's scoreOf ignores pinned rows and its preserve predicate keeps them, so text-row autocut is byte-identical to the pre-pin behavior. Known cost: a false-positive arm (relational shape parsed AND a real seed resolved, but wrong/stale edges) now puts up to max edge pages at ranks 1..max instead of one at limit; mitigations are the arm's fallback_slugify confidence gate, the tier-2 resolution-margin TODO, config search.relational_rerank_pin off, and per-call SearchOpts.relationalRerankPin. Pinned by test/search/relational-rerank-pin.test.ts (pure contract + tie policy) and test/search/relational-rerank-pin-hybrid.serial.test.ts (hermetic PGLite on the relational corpus through the REAL gateway rerank path behind __setRerankTransportForTests with an inverted-relevance stub: pin 0 reproduces the regression, pin 3 restores hit@1/hit@3 on all 8 who-invested questions, non-relational queries and reranker-off runs are byte-identical).

  • src/core/search/metadata-boost-gate.ts — the ONE parse + decision contract for search.metadata_boost_gate (always | lexical; every bundle is lexical, DEFAULT_METADATA_BOOST_GATE stays always so a knobs literal without the field keeps its pre-wave hash identity). lexicalArmsVoted answers "did a strict keyword, title-phrase or relational row fuse?" from the composed fusion lists (relaxed OR-fallback rows do not count — they are already demoted); decideMetadataBoosts({gate, lexicalVoted}) returns {gate, lexical_voted, boosts_applied, reason: 'gate_always'|'lexical_voted'|'vector_only_voter'|'image_modality'}. Image-modality queries are exempt from the gate: hybridSearch never runs the keyword / title arms for an image query and excludes the relational arm, so their lexical arms never vote by construction and lexical would silently disable the boosts for the whole modality — the caller passes modality (from effectiveModality) and an image decision applies the boosts as before (reason image_modality). Under lexical, hybridSearch passes skipMetadataBoosts into runPostFusionStages so the backlink, salience, recency (+chronicle), graph-signal and alias-resolved boosts are skipped when the vector arm was the only voter; supersede downrank, exact-match boost, title-phrase boost, compiled-truth boost, cosine re-score, dedup, reranker and autocut are untouched either way. Receipt (Cat 13 conceptual recall, sibling repo): the E1 localization showed hub pages carrying 1.03–1.12x backlink / graph-adjacency / recency boosts over gold concept pages that carried none whenever both lexical arms were empty; the pre-registered E3 held-out arm moved gbrain from 53.0 to 57.8 nDCG@5 (rule ≥ 57.0) with NamedThingBench, BrainBench, the retrieval canary and the LongMemEval dev slice byte-identical, so the gate flipped to lexical in all three bundles. The decision is surfaced as HybridSearchMeta.metadata_boost_gate (--explain); knobs-hash part mbg=; override chain per-call metadataBoostGate → config → bundle (normalizeMetadataBoostGate is the shared parser, garbage falls through). Pinned by test/search/metadata-boost-gate.test.ts (pure contract, hash participation, resolution chain) and test/search/metadata-boost-gate-hybrid.test.ts (hermetic PGLite hub-vs-gold corpus: always reproduces the hub-first regression, the default yields gold-first, config and per-call planes, unparseable falls through).

  • src/core/search/arm-confidence.ts — arm-confidence-weighted fusion for the keyword arm (search.keyword_arm_confidence_floor; null = off in every bundle — the pre-registered Cat 13 E2 mechanism FAILED its held-out rule at the calibrated floor 0.6121 (53.0 → 53.0), so the knob ships default off for operators and the receipt is published). keywordArmConfidence(rows) reads the strict keyword arm's raw score column into a scale-free statistic (top, second, margin_ratio; TypeScript-side — engines still return raw ts_rank × sourceFactor, no SQL change); decideKeywordArmWeight({keywordList, floor, vectorArmVoted, relationalQuery}) down-weights the keyword AND title fusion entries to KEYWORD_ARM_WEAK_WEIGHT (0.5) only when the floor is set, the keyword list is non-empty, a text vector arm voted, the query is not relational, and margin_ratio < floor; otherwise it emits no weight (byte-identical fusion). It is called from composeFusionLists (fusion-lists.ts) so the weight lands through the same per-list weight primitive as the expansion budget, never a k change. normalizeKeywordArmConfidenceFloor is the ONE range contract (number in (0, 1] or a string parsing to one; null/false/offnull; anything else → undefined = fall through), shared by mode.ts:loadOverridesFromConfig and both per-call seams in hybrid.ts. The decision ({top, second, margin_ratio, downweighted}) is surfaced as HybridSearchMeta.keyword_arm_confidence even with the floor off, so a calibration run can pick a floor from per-probe receipts; knobs-hash part kacf=. Pinned by test/search/arm-confidence.test.ts (pure contract) and test/search/arm-confidence-hybrid.test.ts (hermetic PGLite: a weak keyword arm is down-weighted only with the floor set; off is byte-identical).

  • docs/eval-capture.md — stable NDJSON schema reference for gbrain-evals consumers.

  • test/public-exports.test.ts — runtime contract test (R2). Imports each of the 17 public subpaths via package name and pins a canary symbol per module. Paired with scripts/check-exports-count.sh.

  • src/core/embedding.ts — OpenAI text-embedding-3-large, batch, retry, backoff. BATCH_SIZE=100 (per-recipe pre-split + recursive halving + adaptive shrink-on-miss live in the gateway; the outer paginator is for progress-callback granularity, not batch protection). estimateEmbeddingCostUsd(tokens) prices against the currently-configured model's rate via currentEmbeddingPricePerMTok() (resolves the per-1M-token rate via lookupEmbeddingPrice(gatewayGetModel()) from embedding-pricing.ts, falling back to the OpenAI 3-large rate 0.13 only when the gateway is unconfigured or the model is unknown to the pricing table). EMBEDDING_COST_PER_1K_TOKENS retained for back-compat with direct importers/tests. currentEmbeddingSignature(): string returns the embedding-provenance signature <provider:model>:<dims> (e.g. openai:text-embedding-3-large:1536) stamped onto pages.embedding_signature at every embed-write site; DELIBERATELY excludes the chunker version (tracked separately via pages.chunker_version) — this signature is strictly the EMBEDDING space, so a model OR dimension swap makes the stored signature differ from current and a page becomes stale. Same unconfigured-gateway fallback as the cost helpers. (See src/core/sync-delta.ts + src/core/spend-posture.ts for the cost-gate supporting modules.) willEmbedSynchronously({v2Enabled, serialFlag, noEmbed}): SyncEmbedMode retains the package-exported compatibility contract for downstream TypeScript and JavaScript consumers. The command-internal resolveWorkerBackedSyncEmbedMode({deferEligible, noEmbed}) uses the engine-aware admission result, so PGLite/unknown runtimes stay inline while worker-backed Postgres preserves v2 deferral even with one runnable source; source count separately controls fan-out. shouldBlockSync(costUsd, floorUsd, mode, posture='gated'): boolean is the pure cost-gate decision: blocks ONLY when mode === 'inline' && costUsd > floorUsd — deferred mode never blocks (the backfill's $X/source/24h cap is the real money gate), and posture === 'tokenmax' never blocks (the operator declared cost isn't the constraint; an off/unlimited floor is Infinity and so is never exceeded). Pinned by test/sync-cost-preview.test.ts + test/embedding-signature-stale.test.ts.

  • src/core/embed-retry.ts — embed retry/backoff primitives + the title-tier restamp helper, kept in core because src/core/import-file.ts and src/core/embed-stale.ts consume these (a core→commands import would make every import-file consumer eagerly load the embed COMMAND module — one value import away from a real ESM cycle). embedBatchWithBackoff wraps embedBatch with rate-limit-aware retry: detects 429s via the wrapped error's cause.status (message-match fallback), also retries transient gateway 502/503/504, parses provider retry-delay hints, jitters ±30% so concurrent workers don't resynchronize, passes maxRetries: 0 through so the AI SDK's own retry stack doesn't multiply attempts, and threads an external AbortSignal into both the sleep and the in-flight HTTP call. restampIfDemotedToTitleTier restamps a per_chunk_synopsis page's CR state to 'title' after a plain re-embed so contextual_retrieval_mode keeps describing the vectors actually in the column (the reindex sweep restores the synopsis tier later). src/commands/embed.ts re-exports everything here (façade rule) so import sites and tests never chase the peel.

  • src/core/embedding-input-limit.ts — per-model embedding INPUT token caps. Some hosted encoders reject any single input over a hard per-input limit far below their batch budget (a 512-token model would otherwise leave ~35% of a typical vault's chunks permanently unembeddable, re-failing every sweep). resolveMaxChunkTokens(env?) resolves the effective chunk-token cap for the ACTIVE embedding model so the chunkers SPLIT (never truncate) at a size the model accepts: GBRAIN_MAX_CHUNK_TOKENS env (escape hatch for models not in a recipe; warn-once per distinct invalid value) → recipe per-model max_input_tokens × EMBED_INPUT_SAFETY (0.6 — covers the cl100k-based estimateEmbedTokens overestimate vs BERT-wordpiece tokenizer mismatch) → DEFAULT_MAX_CHUNK_TOKENS (2000 for every other provider). MIN_CHUNK_TOKENS floor of 64 so a tiny declared limit can't produce confetti chunks; fail-open on any resolver error (a resolver bug must never change chunking for unaffected installs). maxInputTokensForModel(recipe, modelId) mirrors model_dims' case-fold lookup rule (exact match first, then case-insensitive scan). Consumed at import-file.ts's chunk step. Pinned by test/embedding-input-limit.test.ts.

  • src/core/embed-oversize-heal.ts — heals ALREADY-STORED chunks that exceed the active embedding model's per-input token cap (new imports respect resolveMaxChunkTokens(), but --stale re-embeds existing chunk_text rows chunked under looser caps, which fail every sweep forever). healOversizedChunks splits oversized rows in place (never truncates; char-based hard split as the pathological fallback), re-indexes contiguously, carries each piece's modality/code-symbol metadata, and leaves embeddings unset so upsertChunks preserves vectors for unchanged (chunk_index, chunk_text) pairs and NULLs the split/shifted ones; healOversizedPageChunks is the load-split-upsert-reload wrapper. healedChunksToStaleRows(chunks, slug, sourceId) remaps a post-heal chunk list back into the stale-row shape — the shared seam between src/commands/embed.ts:embedAllStale and src/core/embed-stale.ts:embedStaleForSource so the two drains cannot drift; chunk_source passes through UNCHANGED (coercing fenced_code to compiled_truth would make wrapChunkTextsForStoredMode prefix code chunks, violating the D20-T4 never-wrap convention). Pinned by test/embed-oversize-heal.test.ts.

  • src/core/stall-env.ts — shared env resolver for the stall-watchdog knobs (resolveStallAbortSecondsFromEnv(envVar, defaultSec)): one parameterized implementation behind BOTH GBRAIN_SYNC_STALL_ABORT_SECONDS and GBRAIN_EMBED_STALL_ABORT_SECONDS so the two surfaces' semantics cannot drift — unset/empty/garbage → the caller's default; any finite number returned as-is (<= 0 disables the watchdog). Deliberately a leaf module (neither the sync nor the embed cluster imports the other's machinery just to parse an env var); env-only incident knobs by design, no config-dashboard surface.

  • src/core/embed-stall.ts — progress-keyed stall watchdog for embed drains (mirror of sync's stall watchdog; bounds embed drains that hang without a pinned root cause). Two clocks, one trigger: LIVENESS ticks via noteEmbedApiResponse() on EVERY settled embed API attempt — success, error, or retry (hooked in embed-retry.ts:embedBatchWithBackoff; diagnostic only, distinguishes "fully wedged" from "live but failing" in the abort message, scoped to the armed run); the TRIGGER is no SUCCESSFUL forward progress (readProgress(), wired to chunks embedded) for GBRAIN_EMBED_STALL_ABORT_SECONDS (env-only knob; default 900 via DEFAULT_EMBED_STALL_ABORT_SEC, <= 0 disables, garbage falls back to default) — a retry storm that never lands a chunk trips it BY DESIGN. A stall produces an ERROR RESULT (reason: 'stall_timeout'), never a process.exit: only the CLI wrapper (src/commands/embed.ts:runEmbed) maps it to a non-zero exit, and minion handlers convert it to a failed job via assertEmbedNotStalled. The abort path runs an idempotent, deadline-bounded cleanup (EMBED_STALL_CLEANUP_DEADLINE_MS) that releases only the run's self-acquired single-flight locks and banks partial progress, so the next run resumes cleanly. The check interval is deliberately NOT unref'd — for non-single-flight callers it can be the only handle keeping a lost-promise hang loud; stop() clears it on every normal path. Pinned by test/embed-stall.test.ts.

  • src/core/sync-delta.ts — the single "what changed since last_commit" helper, consumed by BOTH performSyncInner (sync executor) and estimateInlineNewTokens (cost estimator) so the gate's dollar figure can't drift from what the sync imports. computeSyncDelta(repoPath, fromCommit, toCommit, {detachedManifest?, detached?}){status:'ok', manifest} | {status:'unavailable', reason:'anchor_missing'|'diff_failed'}. Anchor reachability via git cat-file -t (a gc'd bookmark is anchor_missing, but a present-but-non-ancestor bookmark is still diffed tree-to-tree), then git diff --name-status -M from..to parsed by buildSyncManifest; merges the detached working-tree manifest when detached (buildDetachedWorkingTreeManifest). NO dirty/untracked probe in the estimator — pricing dirty files would create phantom costs on a busy brain. The executor imports only the commit diff on attached HEADs BY DEFAULT, but counts uncommitted working-tree files (untracked + dirty) as SyncResult.uncommitted drift and, with --working-tree / config sync.include_working_tree, merges the working-tree manifest into the delta so attached repos can import uncommitted state (the estimator does not price that opt-in). execFileSync array-args (shell-injection safe), 30s / 100 MiB budget. Test seam _setGitRunnerForTests. Pinned by test/sync-delta.test.ts.

  • src/core/spend-posture.ts — spend-control surface. resolveSpendPosture(engine): 'gated'|'tokenmax' (DB-plane spend.posture, fail-open gated); tokenmax makes every cost gate informational across sync/reindex/enrich/onboard (spend still ledgered — removes the ceiling, not the accounting). parseUsdLimit(raw, def, {allowZero?}) accepts off/unlimited/noneInfinity; formatUsdLimit(n) renders Infinity as the string 'unlimited' (never raw — JSON.stringify(Infinity) is null); usdLimitToCap(n) maps Infinityundefined at the BudgetTracker boundary so ledger rows never serialize null. normalizeSpendPosture/isValidSpendPosture back the config set validation. Doc: docs/operations/spend-controls.md. Pinned by test/sync-cost-preview.test.ts + test/spend-off-switch.test.ts.

  • src/core/ai/dims.ts — per-provider providerOptions resolver for embed-time dimension passthrough; the single source of truth for "which provider needs which knob to produce vector(N)". Exports dimsProviderOptions(implementation, modelId, dims) (called by embed() in gateway.ts), VOYAGE_OUTPUT_DIMENSION_MODELS (private const — the 7 hosted Voyage models that accept output_dimension: voyage-4-large, voyage-4, voyage-4-lite, voyage-3-large, voyage-3.5, voyage-3.5-lite, voyage-code-3 — nano deliberately excluded), VOYAGE_VALID_OUTPUT_DIMS = [256, 512, 1024, 2048] as const, supportsVoyageOutputDimension(modelId), isValidVoyageOutputDim(dims). Voyage path uses the SDK-supported dimensions field ({ openaiCompatible: { dimensions: N } }), NOT Voyage's output_dimension wire-key — the voyageCompatFetch shim in gateway.ts:541 translates dimensions → output_dimension before the HTTP body is built (the AI SDK's openai-compatible adapter doesn't recognize the wire-key, so sending it from here would be silently dropped and Voyage would return its default 1024-dim). Runtime guard: when a Voyage flexible-dim model is configured with dims outside VOYAGE_VALID_OUTPUT_DIMS, throws AIConfigError with a paste-ready gbrain config set embedding_dimensions <256|512|1024|2048> hint at the embed boundary (most common trigger: embedding_model: voyage:voyage-4-large without embedding_dimensions, falling back to DEFAULT_EMBEDDING_DIMENSIONS=1536, an OpenAI default not a Voyage one). Every lookup in this module folds through the private modelMatchKey (trim + lowercase) so cased hub-form ids (Qwen/Qwen3-Embedding-4B) match the all-lowercase tables — the folded key is a MATCH key only, never sent to a provider (wire model ids are case-sensitive; error messages keep the original id so they stay paste-ready); the Qwen3-Embedding native-width table lives at module scope alongside the other dim tables. Consequence, deliberate: a cased Voyage/ZeroEntropy/Perplexity config matches and fails loudly at init when its dims is invalid, rather than skipping validation and silently producing wrong-width vectors.

  • src/core/ai/types.ts — provider/recipe types. EmbeddingTouchpoint has optional chars_per_token (default 4, matching OpenAI tiktoken on English) and safety_factor (default 0.8, budget-utilization ceiling), both consulted only when max_batch_tokens is also set; Voyage declares chars_per_token=1 + safety_factor=0.5 to handle dense payloads (CJK/JSON/base64). Pre-split budget = max_batch_tokens × safety_factor / chars_per_token. EmbeddingTouchpoint.multimodal_models?: string[] model-level allow-list for recipes mixing text-only + multimodal models under one touchpoint (Voyage's 12 models share supports_multimodal: true but only voyage-multimodal-3 accepts /multimodalembeddings); when omitted, recipe-level supports_multimodal is sufficient. AIGatewayConfig.embedding_multimodal_model?: string lets embedMultimodal() route to a different model than embedding_model (OpenAI text + Voyage images without flipping the primary pipeline). AIGatewayConfig.embedding_image_ocr_model?: string is its OCR sibling: generateOcrText() routes to it instead of the expansion model; a direct provider:model string, never models.tier-resolved. EmbeddingTouchpoint.trust_custom_dims?: true — passthrough tier for a user-declared --embedding-dimensions on local / bring-your-own-backend recipes (ollama, llama-server, litellm) where the model catalog can't be enumerated; consumed by isCustomDimValidForProvider in src/core/embedding-dim-check.ts AFTER Tier 1 (recipe dims_options) and Tier 2 (provider Matryoshka allowlists), so a recipe that declares fixed options (openrouter) is still governed by those and fixed-dim hosted providers (openai/voyage/zeroentropy) stay fail-closed; the provider's /embeddings response-dim validation catches a genuine mismatch pre-storage. Recipe.default_headers?: Record<string, string> (static) and Recipe.resolveDefaultHeaders?(env) (env-templated) seam for per-recipe headers riding alongside auth on every openai-compat touchpoint; mutually exclusive (declaring both throws AIConfigError at gateway-configure time); keys conflicting with the resolved auth header (Authorization, the resolver's custom header) rejected at applyResolveAuth call time so defaults can't shadow auth. Used by OpenRouter for the HTTP-Referer + X-OpenRouter-Title + X-Title attribution triple.

  • src/core/ai/defaults.ts — leaf module holding the embedding/reranker default constants (no gateway import, so schema + registry helpers can read them without loading provider SDKs). Split-default: NEW_INSTALL_DEFAULT_EMBEDDING_MODEL (voyage:voyage-4) + NEW_INSTALL_DEFAULT_EMBEDDING_DIMENSIONS (1024) feed every new-install surface — init auto-pick canonical tiebreak, the interactive picker default, the no-keys hint, keyless fresh-install schema sizing (passed as an explicit init param), and all recommendation copy (playbook, banners, doctor fix-hints, advisor). DEFAULT_RERANKER_MODEL (= NEW_INSTALL_DEFAULT_RERANKER_MODEL = voyage:rerank-2.5) is the ONE runtime/mode-bundle reranker default: the three MODE_BUNDLES.*.reranker_model values and the gateway's rerank() fallback import it, so a brain with no search.reranker.model row reranks with Voyage on VOYAGE_API_KEY. LEGACY_DEFAULT_RERANKER_MODEL (zeroentropyai:zerank-2) is not a default anywhere; it names the RERANKER_SUNSETS row (which STAYS while the recipe exists, so an explicit zeroentropyai:* config still short-circuits past the date instead of hanging 5s per query), the short-circuit tests and the migration copy. Init (init.ts:writeNewInstallRerankerDefault, shared by the PGLite + Postgres paths) writes NO reranker row when the default is ready (reranker-readiness.ts against the file-plane + process env — an explicit row equal to the bundle value would only earn doctor's search_mode reset nag), writes explicit search.reranker.enabled false for keyed installs WITHOUT a Voyage key (no key for the default; silence beats a no_key audit row per process), writes nothing for keyless installs (the documented keyless-recovery re-init must find virgin reranker config; keyless brains take the no-embedding search path, which never reaches applyReranker), treats a ZeroEntropy embedding pick as any other keyed non-Voyage install (evaluated against env > file > DB-plane keys via loadConfigWithEngine, so a --force re-init sees a config-table key), and never clobbers an existing explicit choice. DEFAULT_EMBEDDING_MODEL (zeroentropyai:zembed-1) + DEFAULT_EMBEDDING_DIMENSIONS (1280) serve ONLY as the configless runtime fallback for brains with no embedding_model in file config — their stored vectors live in ZE's 1280d space, so flipping the fallback under them would break retrieval before the provider dies; the fallback is slated for removal after the sunset, at which point unmigrated configless brains hard-error with the migrate command. ZEROENTROPY_SUNSET_DATE ('2026-09-04') is the single source of truth for the upgrade banners, the provider_sunset doctor check, and the ZE recipe's sunset metadata.

  • src/core/ai/reranker-readiness.ts — pure leaf answering "is the reranker actually going to run?" for gbrain search modes (buildModesReport.reranker_readiness), doctor's reranker_health, and init's reranker-default write. rerankerReadiness(model, env, { now?, baseUrlOverrides? }){model, provider, modelId (alias-canonical), recipeKnown, hasTouchpoint, modelListed, requiredKey (the first MISSING key, else the first required key — this is how Reranker: … VOYAGE_API_KEY present renders; null only for keyless recipes), keyPresent, sunset, sunsetPassed, selfHosted, sunsetBlocks, ready}; the env snapshot comes from the CALLER — the leaf never reads process.env and never imports the gateway (init runs before configureGateway and passes mergedProviderEnv(cfg, process.env) where cfg is loadConfigFileOnly() merged with the brain's DB-plane provider keys via loadConfigWithEngine — file plane alone if that read fails — so a --force re-init sees a Voyage key that lives only in the config table). A recipe with a custom resolveAuth needs no env key (mirrors the gateway); a provider_base_urls override marks the provider self-hosted so a passed sunset does not block. describeRerankerFix(r) renders the paste-ready one-liner (sunset switch > unknown-model > missing key + disable command). src/core/ai/reranker-readiness-engine.ts is the engine-plane wrapper doctor and gbrain search modes share: rerankerReadinessForEngine(engine, model, { now? }) reads the LIVE gateway snapshot (env + base_urls — exactly what rerank() consults, so the verdict cannot disagree with search and a mounted brain's DB plane cannot steer it) and falls back to env > file > DB-plane provider keys + provider_base_urls via loadConfigWithEngine only when no gateway is configured; redactReadinessForRemote (modes-report.ts) strips required_key/key_present/fix from the search_modes op for untrusted callers. test/ai/reranker-readiness.test.ts pins agreement with isAvailable('reranker', m) on an env × model matrix (recipeKnown && hasTouchpoint && keyPresent) so the two predicates cannot drift.

  • src/core/ai/gateway.ts — unified seam for every AI call. embedQuery(text, opts?) and isAvailable(touchpoint, modelOverride?) accept a model override so the resolved-column path embeds via the column's provider (Voyage / ZeroEntropy / OpenAI) instead of the global default; the hybrid path passes {embeddingModel: resolved.provider, dimensions: resolved.dimensions} and the gateway resolves the matching recipe via instantiateEmbedding(). isAvailable('embedding', 'voyage:voyage-3-large') checks the override's recipe (not the default) so hybrid skips vector search only when the active column's provider is actually down. zeroEntropyCompatFetch shim (sibling to voyageCompatFetch) handles ZE's non-OpenAI-compatible wire shape — rewrites the request URL /embeddings → /models/embed, injects input_type (default 'document'; the threaded 'query'|'document' crosses the SDK boundary via the module-level __embedInputTypeStore AsyncLocalStorage populated in embedSubBatch(), because the AI SDK's openai-compatible adapter strips input_type from providerOptions before building the wire body; voyageCompatFetch injects it opt-in the same way, and openAICompatAsymmetricFetch is the fallthrough shim for every other openai-compat recipe — llama-server/litellm/ollama — a strict pass-through when nothing was threaded) and explicit encoding_format: 'float', and rewrites the response {results: [{embedding}], usage: {total_bytes, total_tokens}}{data: [{embedding, index}], usage: {prompt_tokens, total_tokens}} so the SDK's openai-compatible Zod schema validates. Layer 1 (Content-Length) + Layer 2 (per-embedding) OOM caps via tagged ZeroEntropyResponseTooLargeError (kept separate from VoyageResponseTooLargeError because test/voyage-response-cap.test.ts does structural source-text greps pinning the Voyage name). Wired in instantiateEmbedding() via the recipe.id === 'zeroentropyai' branch. gateway.rerank() native HTTP path (no AI-SDK reranking abstraction): resolves the EFFECTIVE reranker as input.model ?? getRerankerModel() ?? DEFAULT_RERANKER_MODEL (imported from ai/defaults.ts; voyage:rerank-2.5), posts to the recipe's reranker path (touchpoints.reranker.path, default /models/rerank; Voyage declares /rerank) with bearer auth — the request's top-N key is recipe-pluggable via touchpoints.reranker.top_param (default top_n; Voyage top_k) — and returns RerankResult[] sorted by relevance. warnSunsetOnce(recipe, touchpoint): once-per-(recipe,touchpoint) stderr DEPRECATED warning for recipes carrying sunset metadata, fired on actual use (embedding resolution + the rerank path) so brains still riding a dying provider hear about it on every process, not only at upgrade time; prints the sunset date plus the migrate / search.reranker.model fix, never throws, _resetSunsetWarningsForTest() is the test seam. Past a RERANKER_SUNSETS date (rerankerSunset() from ai/defaults.ts), rerank() short-circuits BEFORE the HTTP call: the check runs where the EFFECTIVE model is resolved (input.model ?? getRerankerModel() ?? DEFAULT_RERANKER_MODEL — the main case is an absent per-call model landing on the configured/legacy default) and throws RerankError('sunset_short_circuit') so applyReranker fails open at once instead of burning the 5s timeout per query; suppressed under a base_urls recipe override (self-hosted wire-compatible endpoints outlive the hosted shutdown, same rule as warnSunsetOnce); traceability is ONE sunset_short_circuit audit row per process per model (written by the gateway to the rerank-failures JSONL, feeding doctor's reranker_health) plus one stderr line with the replacement-model switch command — per-query rows would flood the audit file until the user migrates; __setSunsetClockForTests is the injected-clock seam for date-matrix tests, reset alongside _resetSunsetWarningsForTest(). no_key preflight: after the sunset check and requireConfig(), a recipe without a custom resolveAuth whose auth_env.required key is absent from cfg.env throws RerankError('no_key') BEFORE any HTTP — noKeyOnce() (mirror of sunsetShortCircuitOnce MINUS the stderr line; memo _noKeyNoticed, cleared by _resetSunsetWarningsForTest()) writes ONE no_key audit row per process per model and nothing is printed (shell-per-query agents would otherwise see a line per search). auth therefore means "key present but rejected" (HTTP 401/403). RerankError.reason classifier: auth | no_key | rate_limit | network | timeout | payload_too_large | sunset_short_circuit | unknown. 5s default timeout (search hot path). Pre-flight payload guard rejects bodies over recipe.touchpoints.reranker.max_payload_bytes with reason: 'payload_too_large'. _rerankTransport test seam mirrors _embedTransport. embedQuery(text) threads inputType: 'query' through dimsProviderOptions() (4-arg). getRerankerModel() returns only the EXPLICITLY configured model (callers need not pre-check availability — rerank() fails no_key itself; the sync readiness predicate for dashboards is reranker-readiness.ts) + isAvailable('reranker') branch; configureGateway + reconfigureGatewayWithEngine thread reranker_model; applyResolveAuth + defaultResolveAuth widen touchpoint param to include 'reranker'. embedMultimodalOpenAICompat() routes recipes with implementation: 'openai-compatible' (LiteLLM, Anyscale, vLLM, Gemini multimodal via proxy) through the standard /embeddings endpoint with content arrays carrying image_url entries; the Voyage /multimodalembeddings path is unchanged (gateway selects by recipe implementation tag). Runtime dimension validation throws AIConfigError (with model id + observed + expected) before the vector reaches storage when the provider returns a width that doesn't match the recipe's default_dims or the brain's embedding_dimensions. Pinned by test/openai-compat-multimodal.test.ts. Module-scoped _embedTransport defaults to AI SDK embedMany, with __setEmbedTransportForTests(fn) test seam so tests drive embed() with a stubbed transport. splitByTokenBudget and isTokenLimitError exported @internal (pure functions reused by the test file). Module-level _shrinkState: Map<recipeId, {factor, consecutiveSuccesses}> halves the recipe's effective safety_factor on token-limit miss (floor 0.05) and heals back ×1.5 after SHRINK_HEAL_AFTER=10 consecutive successes. configureGateway() walks every registered recipe at construction and emits a once-per-process stderr warning for any embedding touchpoint missing max_batch_tokens (excluding the canonical OpenAI fast-path). resetGateway() clears _shrinkState, the warned-set, and restores the real transport. embedMultimodal() reads cfg.embedding_multimodal_model first (falls back to cfg.embedding_model); after the recipe-level supports_multimodal fast-fail, validates the resolved model against touchpoint.multimodal_models when declared (closes the Voyage-text-only-into-multimodal-endpoint footgun before any HTTP call). getMultimodalModel() accessor mirrors getEmbeddingModel / getChatModel. Exported VoyageResponseTooLargeError tagged class: voyageCompatFetch's two OOM-defense caps (Layer 1 Content-Length, Layer 2 per-embedding base64) throw it; the inbound response-rewriter's try/catch (which swallows parse failures so misshaped responses fall through to the SDK parser) checks instanceof VoyageResponseTooLargeError and rethrows so the cap is actually effective (an assertion in test/voyage-response-cap.test.ts pins the instanceof ⇒ throw err line). AI SDK v6 toolLoop compat (gbrain skillopt rollouts AND production background subagent jobs both route through chat() / toolLoop): in chat(), tool defs wrap the raw JSON Schema with the SDK's jsonSchema() helper (inputSchema: jsonSchema(t.inputSchema)) — v6's asSchema() treats a bare {jsonSchema: ...} object as a thunk and throws "schema is not a function"; exported pure toModelMessages(messages: ChatMessage[]): unknown[] converts gbrain's provider-neutral ChatMessage[] into v6 ModelMessage[] — tool results (pushed by toolLoop as role:'user' with bare-value tool-result blocks) become a dedicated role:'tool' message with structured output:{type:'json'|'text'|'error-text', value} parts; null output preserved as {type:'json', value:null} (not dropped); text/tool-call blocks pass through with v6 field names (toolCallId/toolName/input); applied at the generateText call (messages: toModelMessages(opts.messages)). The converter is load-bearing for the production subagent path, not just skillopt. Pinned by test/gateway-model-messages.test.ts. Companion: src/core/skillopt/rollout.ts builds tool schemas through the shared paramDefToSchema from src/mcp/tool-defs.ts (single source of truth, recursive on items/enum/default), never an inline converter. Provider-agnostic plumbing: resolveNativeBaseUrl(provider, cfg) normalizes a configured ANTHROPIC_BASE_URL / OPENAI_BASE_URL to carry the /v1 suffix and is passed explicitly at every native createAnthropic / createOpenAI site (chat/expansion/embedding), so an env-injected bare host doesn't 404; returns undefined when unset so the SDK default is preserved (Google deferred until its native suffix is verified). diagnoseEmbedding fails closed with user_provided_dims_unset when a user-provided / zero-default recipe (litellm/llama-server) has no configured embedding_dimensions. configureGateway does not backfill embedding_dimensions (readers default it themselves), keeping the "no dims set" signal honest for that guard and the multimodal skip. withBudgetTracker: gateway-layer enforcement via AsyncLocalStorage<BudgetTracker>. withBudgetTracker(tracker, fn) installs the tracker on the module-internal store; every gateway.chat / embed / rerank call inside the scope auto-composes (reserve before, record in try/finally). Outside-scope calls are budget no-ops. Nested scopes restore the outer tracker on exit. getCurrentBudgetTracker() is the test seam. The chat path uses the pessimistic fallback on error paths; the embed path estimates input tokens from char count × recipe's chars_per_token because the AI SDK doesn't surface per-batch embed token usage; the rerank path estimates char count of query+docs. Pinned by 6 unit cases. reconfigureGatewayWithEngine(engine) (async, called from cli.ts after engine.connect(), before every command except CLI_ONLY no-DB commands) re-resolves expansion + chat defaults through resolveModel() so models.tier.* and models.default overrides apply to both. DEFAULT_CHAT_MODEL is anthropic:claude-sonnet-4-6. ChatOpts.temperature is threaded verbatim to the AI SDK generateText call (unset → the provider's default; the LongMemEval judge pins 0, the official evaluate_qa.py setting). ChatResult.responseModel carries the model id the PROVIDER reported (response.modelId, e.g. a dated snapshot) when the SDK surfaced one and is absent otherwise, while model stays the requested provider:modelId; eval receipts pin the two side by side (reader_model_snapshot, judge_model_snapshot). __setChatTransportForTests mirrors __setEmbedTransportForTests so tests drive chat() with a stubbed transport. toolLoop per-turn permit hook: optional acquireTurnPermit() acquires a provider permit before EVERY round-trip and releases in finally; a lease-full throw propagates without consuming the turn (the subagent path wires it to the rate leases with provider-derived keys — Anthropic models share anthropic:messages, others get <recipeId>:chat). ToolLoopStopReason includes 'length': a zero-tool-call turn that hit the output cap propagates 'length' instead of folding into 'end', so truncation is never reported as a clean finish. Per-part provider state: ChatBlock variants carry optional providerMetadata captured from SDK parts in chat() and re-emitted as providerOptions on the rebuilt parts in toModelMessages() (attached only when present — metadata-free blocks stay byte-identical); this is the Gemini 3.x thoughtSignature echo. isThinkingByDefaultModel(modelStr) (exported) matches Claude 5-family ids behind any provider-prefix chain with a letters-only family segment (never claude-3-5-*); isThinkingModel(modelStr) (exported) is that regex OR the recipe's chat thinking_by_default capability (fail-closed: unknown/chat-less recipes are non-thinking) and drives defaultMaxOutputTokens's 32k thinking headroom for chat()/toolLoop() callers that omit maxTokens (e.g. gbrain skillopt rollouts) and the subagent handler's resolveMaxOutputTokens; think/index.ts shares the regex and makes the same capability check. Pinned by test/ai/gateway-thinking-headroom.test.ts. expand() and generateOcrText() also record on the ambient tracker (they call generateObject/generateText directly and never pass through chat()'s _recordBudget): record-only, no reserve — a breach surfaces on the NEXT reserving call, matching chat()'s swallow of BudgetExhausted from record(). Successes record normalized SDK usage via normalizeSdkUsage (the ONE home for the v6/legacy usage shapes, used by chat's success path too; first FINITE field wins so a NaN v6 field can't shadow a real legacy value or poison the running total); failures record pessimistically under gateway.expand.failed / gateway.ocr.failed (a rejected attempt still billed provider tokens — one expand() can legitimately produce two records when the structured-output attempt fails and the text fallback runs). A recipe whose declared structured-output support is rejected at call time is remembered for the process lifetime (_structuredOutputRejectedRecipes) so the rejected attempt isn't re-paid on every call. OCR's input estimate is prompt text + a fixed per-image token constant, never base64 length (bytes are not tokens). generateOcrText() routes to getImageOcrModel()embedding_image_ocr_model when set, else the expansion model — gated by isAvailable('expansion', <ocr model>) so a misconfigured OCR model (provider without an expansion touchpoint, or unkeyed) fails closed to '' instead of silently OCRing with the expansion model; the accessor mirrors getMultimodalModel(), configureGateway/buildGatewayConfig thread the field, and an unconfigured gateway stays a silent '' no-op. Pinned by test/ai/ocr-model-routing.test.ts. __setGenerateObjectTransportForTests mirrors the generateText seam. + test/core/budget/expand-records-budget.test.ts + test/ai/gateway-ocr-budget.test.ts

  • src/core/ai/recipes/zeroentropyai.ts — ZeroEntropy openai-compatible recipe declaring BOTH embedding (zembed-1, 7 Matryoshka dims: 2560/1280/640/320/160/80/40) AND reranker (zerank-2 flagship + zerank-1 + zerank-1-small, 5MB payload cap) touchpoints. implementation: 'openai-compatible' (pinned by test/ai/zeroentropy-recipe.test.ts). base_url_default: 'https://api.zeroentropy.dev/v1' already ends with /v1, so the zeroEntropyCompatFetch URL rewrite /embeddings → /models/embed produces …/v1/models/embed (NOT …/v1/v1/… — pinned there too). chars_per_token: 1 + safety_factor: 0.5 match Voyage's dense-content hedge. Carries sunset metadata (ZEROENTROPY_SUNSET_DATE + replacement models from ai/defaults.ts) that drives init picker/auto-pick exclusion, the gateway's once-per-process warn-on-use, and every gbrain providers rendering via the shared sunsetMarker in src/commands/providers.ts (list status cell, ⚠ explain rows, and the env deprecation block that replaces the signup funnel); the recipe itself is slated for removal after the sunset date.

  • src/core/ai/recipes/llama-server-reranker.ts — sibling of llama-server (the embedding recipe) for llama.cpp in --reranking mode. Distinct recipe rather than dual-touchpoint extension because --reranking and --embeddings are mutually exclusive at server-launch time, so the two backends need independent base URLs (default 8081 here vs 8080 there). Declares reranker touchpoint with models: [] (user-provided id matching the --alias the user launched with), path: '/rerank' (leaf-only; consumes RerankerTouchpoint.path override; gateway concatenates with base_url_default which ends in /v1, producing …/v1/rerank), default_timeout_ms: 30_000 (consumed by src/core/search/mode.ts's reranker timeout chain — CPU-only first-call warmup headroom; the 5s mode-bundle default would fail-open as timeout), cost_per_1m_tokens_usd: 0 (recognized by FREE_LOCAL_RERANK_PROVIDERS in src/core/budget/budget-tracker.ts so --max-cost callers don't hard-fail on local rerank). Setup hint emphasizes --alias because llama-server's /v1/models defaults model id to the gguf file path without it. Covers Qwen3-Reranker via llama.cpp AND self-hosted ZE weights via llama.cpp — same recipe, different --model at launch. Pinned by test/ai/recipe-llama-server-reranker.test.ts. Voyage / Cohere / vLLM rerankers stay out of scope (different wire shapes). Companion surfaces: path?: string + default_timeout_ms?: number on RerankerTouchpoint in src/core/ai/types.ts; consumed by the URL build at src/core/ai/gateway.ts:rerank() and by mode-resolution at src/core/search/mode.ts:resolveSearchMode (precedence: per-call > config-key > recipe touchpoint default > mode bundle); LLAMA_SERVER_RERANKER_BASE_URL env passthrough in src/cli.ts:buildGatewayConfig; FREE_LOCAL_RERANK_PROVIDERS set in src/core/budget/budget-tracker.ts:lookupPricing (rerank-kind-only zero-pricing for the local provider prefix); doctor-fix at src/commands/models.ts:probeRerankerConfig reads search.reranker.model via loadSearchModeConfig + resolveSearchMode (so doctor and live search read the same resolution and the file plane / DB plane cannot diverge — the field-plane getRerankerModel() reads nothing writes); probeRerankerReachability reads the recipe's default_timeout_ms so CPU-only cold-start doesn't false-fail.

  • src/core/ai/recipes/nan.ts — nan.builders openai-compatible reranker recipe. POST {base}/v1/rerankresults[{index, relevance_score}], leaf-only (base_url_default ends /v1; bare /rerank and /compatible-api/v1/reranks both 404). Model id is the literal rerank (a Qwen3-Reranker-8B deployment). auth_env.required: ['NAN_API_KEY'].

  • src/core/ai/recipes/openrouter.ts — OpenRouter openai-compatible recipe: single key, many providers via openrouter:<provider>/<model> strings. base_url_default: 'https://openrouter.ai/api/v1'. Embedding touchpoint: default model openai/text-embedding-3-small; per-model model_dims carries verified native widths (text-embedding-3-small 1536, text-embedding-3-large 3072, qwen/qwen3-embedding-8b 4096, bge-m3 + baai/bge-m3 1024) with default_dims: 0 so an UNLISTED proxied id has NO silent default — it errors until the user supplies explicit dims (--embedding-dimensions / embedding_dimensions), which trust_custom_dims: true accepts (gemini-embedding-2-preview is deliberately unlisted — width unverified). Matryoshka dims_options: [512, 768, 1024, 1536] still governs the default model's shrink steps; max_batch_tokens: 300_000 = OpenAI's aggregate-per-request token cap (NOT per-input). Chat touchpoint declares 8 curated entry points (gpt-5.2, gpt-5.2-chat, gpt-5.5, claude-haiku-4.5, claude-sonnet-4.6, claude-opus-4.7, gemini-3-flash-preview, deepseek-chat) but openai-compat tier accepts any model ID; deliberately no max_context_tokens because OR's catalog spans 128K to 1M+. supports_subagent_loop: false is enforced by classifyCapabilities() in src/core/ai/capabilities.ts (verdict unusable:no_subagent_loopenforceSubagentCapable() in src/core/model-config.ts consumes the verdict: tool-less/unknown models fall back to TIER_DEFAULTS.subagent with a warn; tool-capable providers without prompt caching run with a once-per-model cost warn); the legacy Anthropic-direct path additionally gates on isAnthropicProvider() in src/core/model-config.ts when agent.use_gateway_loop is off. Declares resolveDefaultHeaders(env) returning OR's three attribution headers: HTTP-Referer (required for OR app-attribution), X-OpenRouter-Title (preferred), X-Title (back-compat alias); defaults to https://gbrain.ai / gbrain; forks override via OPENROUTER_REFERER / OPENROUTER_TITLE env vars. Smoke-tested by test/ai/recipe-openrouter.test.ts (incl. the shape guard: every model in the chat list matches ^[a-z0-9-]+\/[a-z0-9._-]+$).

  • src/core/ai/openrouter-families.ts — single source of truth for which OpenRouter model families may drive the subagent (tool) loop: OPENROUTER_SUBAGENT_FAMILIES (currently anthropic/ + deepseek/, each backed by a live abort/retry replay pin under test/e2e/) + openrouterModelSupportsSubagentLoop(modelId) (takes the bare OpenRouter id, no openrouter: prefix). Shared by the openrouter recipe's predicate and the subagent handler's auto-route so the two can never disagree; a new family gets its own live pin before joining the list.

  • src/core/ai/recipes/lmstudio.ts — LM Studio local OpenAI-compatible recipe (embedding touchpoint only; default base URL http://localhost:1234/v1, env LMSTUDIO_BASE_URL). Ships models: [] like llama-server — the model identity is whatever the user loaded in the app, so --embedding-model lmstudio:<id> + --embedding-dimensions <N> are required (the wizard refuses implicit defaults). Its own provider id keeps openai: pointed at OpenAI (complement of the keyless OPENAI_BASE_URL route). Documented operational failure mode: a degraded loaded instance passes the reachability probe but misses the query-embed deadline — queries degrade to keyword-only with the embed_timeout stamp, reload the model to recover.

  • src/core/ai/recipes/ollama.ts — the Ollama local recipe. thinking_by_default is a per-family predicate over the model id (qwen3 with a boundary so qwen2.5-* is never swallowed and a -coder exclusion — the instruct-only variant has no thinking mode; deepseek-r*, gpt-oss, magistral, phi*-reasoning including the -mini-reasoning tags), not a recipe-wide boolean, so non-reasoning local models keep the conservative default.

  • src/core/rerank-audit.ts — failure-only JSONL audit at ~/.gbrain/audit/rerank-failures-YYYY-Www.jsonl (ISO-week rotation, mirrors src/core/audit-slug-fallback.ts). Exports logRerankFailure({reason, model, query_hash, doc_count, error_summary}) + readRecentRerankFailures(days). The sunset_short_circuit and no_key reasons are written ONCE per process per model by the gateway itself (not per query by applyReranker): the reranker's hosted API passed its announced shutdown date / the provider key is absent, so calls are skipped without HTTP and results pass through unreranked; auth means key present but rejected. Deliberately no logRerankSuccess: writing once per tokenmax search is hot-path I/O churn AND success events leak query volume + timing into a local audit file. gbrain doctor's reranker_health check (doctor/checks/calibration.ts) resolves enablement + model through resolveSearchMode(loadSearchModeConfig) — the plane search reranks with — and rerankerReadiness before reading the audit: readiness is evaluated against the DB-merged config plane (loadConfigWithEngine, so DB-plane provider keys and provider_base_urls self-host overrides count, exactly like the gateway the CLI configures): a brain with NO embedding provider (isAvailable('embedding') false on a configured gateway) is ok even when not ready — search runs keyword-only there and never reaches applyReranker, so there is nothing to fix; otherwise enabled-but-not-ready (key absent / sunset passed without a self-host override / unknown model) warns with the paste-ready fix; only audit rows for the RESOLVED model count (rows for any other model never make the live default warn or send the operator to verify the wrong key); no_key/sunset_short_circuit skip rows are informational once ready (those processes ran unreranked; a long-lived worker started before the key was added keeps skipping until restarted) and never outlive the fix as a warn; then the auth / payload / budget / transient / unknown ladders on the non-skip rows; disabled → ok (with an enable hint when the default is ready). Query text SHA-256-prefix-hashed (8 hex chars) for privacy. GBRAIN_AUDIT_DIR env override honored via the shared resolveAuditDir().

  • src/core/search/embedding-column.ts — single source of truth for "which content_chunks.* column does this query rank against?" Pure functions, no engine I/O: loadRegistry(cfg) walks the embedding_columns config (DB plane, JSON map keyed by column name with {provider, dimensions, type} entries), seeds the OpenAI embedding builtin when unset, validates everything before it lands (column-name regex, type ∈ vector | halfvec, dims in [1, 8192], provider format) using Object.create(null) + Object.hasOwn so a key like constructor rejects instead of resolving to Object.prototype.constructor. resolveColumn(registry, override?, cfg) is the boundary call: returns a frozen ResolvedColumn descriptor ({name, provider, dimensions, type}) honoring per-call override → search_embedding_column config → 'embedding' default; throws UnknownEmbeddingColumnError with the list of registered names on miss. isCacheSafe(resolved, cfg) compares the full embedding SPACE (provider + dimensions + name) against cfg's default so a repointed embedding builtin doesn't serve OpenAI-shaped cache rows. validateResolvedColumn(descriptor) re-validates hand-rolled descriptors that bypass the registry (internal-SDK passthrough) so the SQL-injection escape hatch through the descriptor field is closed. Consumed by hybridSearch, gateway.embedQuery(text, {embeddingModel, dimensions}), cosineReScore, and the query MCP op (per-call embedding_column param). Pinned by test/search/embedding-column.test.ts (prototype-pollution, descriptor passthrough, env-only Postgres install, empty-brain coverage gate, cache-space comparison). Write side: resolveWriteColumnFromConfigRows({searchEmbeddingColumn, embeddingColumnsJson}) resolves upsertChunks's active write target from the raw DB-plane config rows through the same registry/validation chain (legacy embedding descriptor when neither row routes elsewhere; embedding_image falls back to the legacy text column since image vectors travel in their own INSERT column; malformed registry JSON is forgiven like loadConfigWithEngine; unregistered names throw the paste-ready error), and vectorCastSuffix(resolved) is the placeholder-free '::vector' | '::halfvec(N)' cast shared by the read fragment builder and the write path so the two casts can't drift. Pinned by test/e2e/upsert-chunks-registry-column.test.ts.

  • src/core/search/rerank.ts — the call-site abstraction. applyReranker(query, results, opts) slots between dedupResults() and enforceTokenBudget() in src/core/search/hybrid.ts. Slices opts.topNIn (default 30) by current RRF order, caps each document via capRerankDoc (RERANK_MAX_DOC_CHARS=6000 then a measured-ratio shrink to RERANK_MAX_DOC_TOKENS=1400 — a 2048-ubatch llama-server minus query/template headroom minus a Qwen-tokenizer margin; every cut through truncateUtf8 so no lone surrogate reaches the JSON body; docs under 1400 chars skip the tokenizer; applies to hosted providers too, so code/CJK chunks at the chunker ceiling lose part of their tail while prose is untouched), sends to gateway.rerank(), reorders by relevanceScore desc, appends the un-reranked tail unchanged (recall protection). Fail-open on every RerankError.reason: any error logs via logRerankFailure and returns the input array unchanged — except the two SKIP classes sunset_short_circuit and no_key (RerankSkipReason), which return the input immediately WITHOUT a per-query audit row (the gateway already wrote the one per-process-per-model row) and fire opts.onSkip(reason); hybrid.ts passes an onSkip that stamps {stage: 'reranker_skipped', reason} into HybridSearchMeta.degraded (closed vocabulary in types.ts), which --explain renders as degraded: reranker_skipped (no_key) via formatDegradedSummary — the only place a silently skipped reranker is visible from the CLI. types.ts classifies the stage as ranking-only: RANKING_ONLY_DEGRADED_STAGES (= {reranker_skipped}) + affectsRecall(entry) — consumers that ask "was recall impaired?" filter through it (the CLI No results. line in cli.ts:describeEmptyRetrieval and the MCP empty-result block in dispatch.ts:buildEmptyRetrievalBlock keep saying "clean miss"; the degraded cache-TTL rule below uses the same predicate), while consumers that report what did not run (--explain, telemetry, eval rows) keep it; pinned by test/degraded-stages-recall.test.ts. hybridSearchCached excludes reranker_skipped from the short degraded-TTL rule (it is a config state, not a transient limp: a keyless balanced brain keeps the full cache.ttl_seconds, and the stored meta still carries the stamp). Stamps rerank_score onto reordered items so downstream telemetry sees the new ordering signal. topNOut: null is the explicit "don't truncate" signal — semantically distinct from undefined ("fall through to mode bundle"). Test seam: opts.rerankerFn stubs gateway.rerank without the network. Document cap pinned by test/rerank-doc-cap.test.ts.

  • src/core/search/return-policy.ts (default OFF) — intent-aware adaptive return-sizing. Pure, dependency-light module that trims the final ranked candidate set to an intent-driven cap instead of returning the full top-K. entity intent gets a tight cap; temporal/event/general get a recall-preserving cap (concept is coerced to general by hybrid.ts before the call — concept queries want breadth). A minKeep failsafe (≥1) guarantees a human never gets a silent blank when candidates exist. WHY a cap, not a score-cliff detector: PrecisionMemBench instrumentation (gbrain-evals) measured the rank1→rank2 RRF gap is ~identical whether rank-1 is correct (0.602) or wrong (0.569) — mechanical decay, not a separatrix; rank-1 is right in 94% of single-answer cases, so "return a tight set" is the whole win and cliff-cutting just adds noise. Exports AdaptiveReturnConfig, DEFAULT_ADAPTIVE_RETURN (frozen: enabled=false, entityMax=2, otherMax=6, minKeep=1), AdaptiveReturnDecision ({applied, intent, cap, kept, total}), AdaptiveReturnInput (boolean | Partial<AdaptiveReturnConfig> | undefined), adaptiveReturnFromConfig(cfg), resolveAdaptiveReturn(perCall, fromConfig) (defaults → config → per-call merge), adaptiveReturnEnabled(...) (cache-skip gate check), applyAdaptiveReturn(results, intent, cfg) (the trim). Config knobs (DB or file plane): search.adaptive_return (master switch), search.adaptive_return_entity_max, search.adaptive_return_other_max, search.adaptive_return_min_keep (each clamped ≥1). Wired into hybridSearch AFTER applyReranker, BEFORE the limit slice, and ONLY on the first page (offset===0) — paginating a confidence-gated set is incoherent, so paginated calls fall through to the fixed limit. Stamps the decision onto HybridSearchMeta.adaptive_return for gbrain search --explain. hybridSearchCached SKIPS the cache when the gate is on (a trimmed set must not be served to a gate-off lookup and vice versa). SearchOpts.adaptiveReturn + HybridSearchMeta.adaptive_return declared in src/core/types.ts. Agent-facing: the query op (src/core/operations.ts) exposes an adaptive_return boolean param whose description instructs the agent WHEN to set it (single-answer → on; breadth/exploration → off; pass limit:1 for a hard single-answer cap), threaded into hybridSearchCached — end users never touch the config knob; their agent decides per query (same pattern as salience/recency). Pinned by test/search/return-policy.test.ts (mechanism) + test/search/query-op-adaptive-return.test.ts (agent surface: param exists + description teaches both directions + the never-empty contract).

  • src/core/search/autocut.ts (OFF in every bundle since rule R2; search.autocut true re-enables it where the reranker runs) — Weaviate-style autocut: score-discontinuity result-sizing on the cross-encoder rerank separatrix. applyAutocut(results, scoreOf, cfg) normalizes the reranker scores, finds the largest consecutive gap, and cuts there when it clears jumpRatio (default 0.20); robust to unsorted provider output (cuts on a sorted copy, keeps items in INPUT order via a score threshold), guards top<=0/non-finite, never returns empty, and no-ops when <2 results carry a finite rerank_score (covers the reranker fail-open path). WHY rerank_score and NOT RRF/cosine: gbrain measured (see return-policy.ts) that the RRF rank1→rank2 gap is ~flat whether rank-1 is right or wrong — not a separatrix; the cross-encoder score IS. So autocut, when enabled, runs ONLY where the reranker ran (balanced+tokenmax; conservative is a documented no-op). Weak-top floor: when the top rerank score is below minTopScore (default 0.35; config search.autocut_min_top; scale-calibrated to the default reranker; a default-reranker change must re-tune it), cliff trimming is skipped entirely so a low-confidence list returns the full cluster instead of collapsing to one result. Exports AutocutConfig, DEFAULT_AUTOCUT (frozen: enabled=true, jumpRatio=0.20, minKeep=1, minTopScore=0.35), AutocutDecision ({applied, signal:'rerank'|'none', cut, kept, total, gapRatio}), AutocutInput, autocutFromConfig, resolveAutocut, applyAutocut. Cache-key integration (clean path, not the adaptive-return cache-skip hack): enable+sensitivity flow through ModeBundleResolvedSearchKnobsknobsHash exactly like graph_signals. mode.ts adds autocut/autocut_jump/autocut_min_keep (autocut false in every bundle since rule R2 — the LongMemEval replay from the captured post-rerank pool found no floor that beat autocut off; jump 0.20 stays the sensitivity when re-enabled; floor 1 in every mode — search.autocut_min_keep sets the minimum result count a cut may leave, resolved through the same bundle → config → per-call chain) AND sets reranker_top_n_in = searchLimit for reranked modes (so the reranker scores the full returned set; there is no un-scored tail for autocut to wrongly drop). Autocut folds into knobsHash as its own parts entry (mode.ts:KNOBS_HASH_VERSION is the single source of truth for the current hash version; every bump is a one-time global cache cold-miss on upgrade). Wired into hybridSearch AFTER adaptive-return, BEFORE the limit slice, first page only; emits HybridSearchMeta.autocut. BOTH the cache-miss finalMeta and cache-HIT cachedMeta rebuilds carry autocut+adaptive_return+mode+embedding_column. Preserves alias-hop exact matches: applyAutocut takes an optional preserve predicate; hybrid passes r => r.alias_hit === true || r.exact_lookup !== undefined || r.relational_pinned === true so a canonical page injected by applyAliasHop after reranking (no rerank_score), an exact-lookup tier hit, or a relational row re-pinned by pinRelationalRows (low cross-encoder score by construction; ALSO excluded from the cliff computation through scoreOf, so text-row autocut is unchanged by the pin) is never cut. Agent surface: query op autocut boolean (ceiling override — false forces full top-K); SearchOpts.autocut; --explain shows per-result rerank_score, formatAutocutSummary renders the decision when search meta is threaded; gbrain search modes attribution; metric glossary autocut.signal/autocut.gap_ratio. Config: search.autocut, search.autocut_jump, search.autocut_min_keep. The DEFAULT_AUTOCUT module constant is unchanged (enabled=true) for operators who re-enable it; the cliff logic is backed by an in-repo eval gate — test/search/autocut-eval.test.ts (also bun run eval:autocut) measures precision-lift-without-recall-regression over labeled qrels fixtures with modeled cross-encoder distributions (no API key, no sibling repo; runs in CI): mean precision 0.33→0.94, recall 1.00→0.95, ZERO recall loss on enumeration queries. Env-overridable floors. Pinned by test/search/autocut.test.ts (pure-fn), test/search/query-op-autocut.test.ts (agent surface), test/search/autocut-integration.serial.test.ts (IRON-RULE behavioral via rerankerFn DI seam: cliff trims, flat doesn't, no-reranker no-ops, autocut:false ceiling, composes with adaptive-return), test/search/autocut-eval.test.ts (the precision/recall gate), and the knobsHash assertions in test/search-mode.test.ts.

  • src/core/ai/recipes/voyage.ts — Voyage AI openai-compatible recipe, home of the new-install default stack. Embedding touchpoint declares default_model: 'voyage-4' + default_dims: 1024 — the canonical pick for every "choose a model for the user" surface (models[0] stays voyage-4-large in quality order; the new-install default is voyage-4 for price/quality balance and the shared v4 embedding space — see NEW_INSTALL_DEFAULT_EMBEDDING_MODEL in ai/defaults.ts). Reranker touchpoint (the mode-bundle default via DEFAULT_RERANKER_MODEL, same VOYAGE_API_KEY as embeddings) allowlists rerank-2.5 ($0.05/M) + rerank-2.5-lite ($0.02/M) and Voyage's preview rerank-3 ($0.05/M) + rerank-3-lite ($0.02/M) (2.5 pair verified 2026-08-15, rerank-3 pair 2026-09-06); default_model stays rerank-2.5 deliberately — the array is the opt-in surface for every enforcement point (gateway.rerank()'s tp.models.includes guard, gbrain models doctor's reranker_config probe, rerankerReadiness), and flipping the default would migrate every install that resolves it instead of an explicit config row. All four share one wire: path: '/rerank', top_param: 'top_k' (Voyage's response wire matches ZE's {results: [{index, relevance_score}]}; only the request's top-N key differs), 5MB byte-proxy payload cap. Declares chars_per_token=1 + safety_factor=0.5 so the gateway pre-splits Voyage batches at a 60K-character budget (50% of 120K-token cap with the dense-tokenizer ratio), since tiktoken-grounded budgeting undercounts Voyage's actual token usage. Declares multimodal_models: ['voyage-multimodal-3'] so the gateway rejects text-only Voyage models pointed at the multimodal endpoint with a clear AIConfigError instead of waiting for Voyage's HTTP 400. The hosted flexible-dim models that accept output_dimension live in VOYAGE_OUTPUT_DIMENSION_MODELS in src/core/ai/dims.ts (v4 trio, voyage-code-4, voyage-3-large, voyage-3.5, voyage-3.5-lite, voyage-code-3; valid widths 256/512/1024/2048); voyage-4-nano is the open-weight variant fixed at 1024-dim that does NOT accept the parameter (negative assertion in test/ai/gateway.test.ts: dimsProviderOptions returns undefined for voyage-4-nano). voyage-code-3 is the recommended embedding model for gstack per-worktree code brains (Topology 3 in docs/architecture/topologies.md; voyage-code-4 is the flexible-dim hosted code model, $0.12/M); discoverability surfaces: decision-tree branch in docs/integrations/embedding-providers.md, Topology 3 "Recommended embedding model" subsection, runtime nudge from gbrain reindex --code against non-code-tuned models. Recipe shape pinned by test/ai/voyage-code-3-recipe.test.ts.

  • src/core/ai/recipes/anthropic.ts — Anthropic recipe (chat + expansion touchpoints). Canonical id is claude-sonnet-4-6 (no date suffix); a reverse alias claude-sonnet-4-6-20250929 → claude-sonnet-4-6 keeps stale user configs working (rescues facts.extraction_model and models.dream.synthesize). Recipe shape pinned by test/anthropic-model-ids.test.ts.

  • src/core/ai/providers/claude-cli-language-model.ts (+ recipe src/core/ai/recipes/claude-cli.ts) — ai-sdk LanguageModel adapter that shells out to the locally-installed claude CLI in print mode (OAuth-subscription lane, no API key; claude-cli:<model> is config-portable with anthropic:<model>). Tool use is system-prompt-instructed JSON emission: the recipe teaches the model the <use_tools>[{name,input}]</use_tools> format and the adapter parses those blocks back into ai-sdk tool-call parts. An entry that carries its arguments flat beside name (no input wrapper, the Anthropic tool_use shape with the wrapper dropped) is accepted: every key other than name/input and the Anthropic tool_use leftovers (type: "tool_use", a toolu_* id) becomes the input — any other type/id value is a real argument (e.g. list_pages' type filter) and stays — and a present input wins verbatim over stray sibling keys. Tool-call ids are ALWAYS gbrain-minted (toolu_claude_cli_<uuidv7>) — never model-authored: each doGenerate is a fresh subprocess replayed from an id-stripped transcript, so the model structurally cannot keep ids unique across turns (a model-authored id echoes the prompt's example entropy-free and collides); the prompt does not ask for an id, and a stray id field is deliberately ignored (nothing round-trips it — the loop pairs results in-memory within one turn). doStream not implemented; callers (gateway.toolLoop) use doGenerate. The recipe declares chat + expansion touchpoints (no embedding; the expansion list leads with the cheap haiku id and carries the same 30s default_timeout_ms as chat for the subprocess cold start); gateway.expand() routes claude-cli through the schemaless viaText path because the adapter ignores responseFormat (generateObject would throw NoObjectGeneratedError on the fenced-JSON text). Pinned by test/claude-cli-recipe.test.ts + test/ai/claude-cli-expansion.test.ts.

  • src/core/ai/providers/claude-cli-scratch.ts — the claude-cli provider's per-PID scratch dirs (claudeCliCwdDir/claudeCliConfigDir off the CLAUDE_CLI_CWD_PREFIX/CLAUDE_CLI_CONFIG_PREFIX basename constants, plus sweepDeadClaudeCliScratchDirs), split into a dependency-light module so transcript discovery (src/core/transcripts/discover.ts) can call isClaudeCliSelfTranscriptPath(path) WITHOUT importing the provider's @ai-sdk surface. Each claude --print subprocess runs with cwd = a per-PID scratch dir (context isolation — no local CLAUDE.md auto-discovery), and Claude Code records a session transcript for that cwd under ~/.claude/projects/<slugified-cwd>/; those sessions are gbrain's OWN internal LLM calls, and importing them back is a self-ingestion feedback loop (prompt scaffolding + page content re-entering as "conversations"). The prefix is lowercase [a-z-] only, so it survives Claude Code's cwd slugification verbatim and a substring match on the discovered path is a reliable self-exclusion fingerprint. Pinned by test/transcripts-self-exclusion.test.ts.

  • src/core/model-pricing.ts — single source of truth for paid-cloud CHAT/completion model pricing (USD per 1M tokens, input | output). CANONICAL_PRICING is a provider:model-keyed table (Anthropic Opus 5/4.8/4.7/4.6 $5/\$25, Sonnet 4.6 $3/\$15, Haiku 4.5 $1/\$5 both dateless + dated, plus OpenAI / Google / Together / DeepSeek panel models). canonicalLookup(modelId) resolves bare (claude-opus-4-8), colon (anthropic:claude-opus-4-8), and slash (anthropic/...) forms — bare ids default to the anthropic: provider; nested OpenRouter ids (openrouter:anthropic/...) intentionally MISS so OpenRouter markup isn't repriced as the inner vendor; after the exact match, a case-insensitive fallback folds BOTH sides (some canonical keys carry cased model tails verbatim), safe only while no two canonical keys collide case-insensitively (pinned by the drift guard). Every other chat-pricing table is a DERIVED view of this one (NOT a hand-copied duplicate), so cross-table price drift is structurally impossible. Embeddings live separately in embedding-pricing.ts (different unit). Pinned by test/model-pricing.test.ts whose drift guard asserts each derived view equals canonical and that the cross-modal panel models are all present.

  • src/core/ai/chat-usage.ts + src/core/ops/usage.ts — durable per-call chat usage accounting. gateway.chat() calls recordChatUsage at its SUCCESS boundary (production provider path AND test-transport path) with the answering model + token usage; the record lands in the chat_usage_log table (migration v140) via an injected sink (setChatUsageSink — no engine import here, so the gateway imports this leaf statically without deepening a cycle). Phase attribution is best-effort via AsyncLocalStorage (withChatPhase): the minion worker AND the cycle's inline drain (inline-drain.ts) both wrap handler execution in job:<name>, a cycle phase that meters its own spend wraps itself in phase:<name> (dream synthesize does), and direct callers record phase NULL. The innermost wrap wins, which is the point — a child drained INSIDE a wrapped phase must carry its own job: tag or the phase tag absorbs every child's spend and the ledger double-counts (one ledger per surface: minion_jobs stays the child-spend authority, the phase row is the orchestrator's own calls). Pricing resolves through CANONICAL_PRICING (estimateChatCostUsd, cache_read/cache_write at provider cache rates when the table carries them); unknown models record cost_usd = NULL, never a fake 0. Accounting is strictly fail-open + fire-and-forget: a sink error must never break a chat call. The get_usage op (admin scope, NOT read — chat_usage_log has no source dimension, so a source-restricted/federated token must not see brain-wide spend telemetry) returns per-model/per-phase aggregates only, plus an explicit coverage block stating what the ledger does NOT capture (subagent raw-SDK path, embeddings, pre-sink/pre-v140 calls, failed calls); budget-tracker.ts remains the pessimistic in-flight spend gate — this table is the after-the-fact ledger.

  • src/core/anthropic-pricing.ts — bare-keyed Anthropic VIEW of model-pricing.ts (the anthropic: canonical entries with the prefix stripped). Kept distinct because many callers look up by bare Claude id and because estimateMaxCostUsd(modelId, inTokens, maxOutTokens) carries the null-on-miss contract the dream-cycle budget gate depends on (non-Anthropic ids return null; BudgetMeter tries canonicalLookup first and only falls back here, so it warns BUDGET_METER_NO_PRICING and runs unbounded only when canonical has no rates either). estimateMaxCostUsd routes bare/colon/slash ids through splitProviderModelId. Do NOT hand-edit prices here — the map is derived from canonical, so it cannot drift. ANTHROPIC_PRICING is consumed by budget/budget-tracker.ts, minions/batch-projection.ts, and cycle/budget-meter.ts.

  • src/core/takes-quality-eval/pricing.ts — fail-closed budget pricing for eval takes-quality run --budget-usd N. MODEL_PRICING is a curated provider:model allowlist (default panel + likely overrides) whose VALUES are derived from model-pricing.ts via canonicalLookup; an allowlisted id missing from canonical throws at module load. Schema is {input_per_1m, output_per_1m}. A model NOT on the allowlist aborts the run with an actionable error rather than guessing (distinct from cross-modal-eval/runner.ts, which silently estimates zero on unknown models — both now source numbers from canonical).

  • src/core/budget/budget-tracker.ts — the keystone budget primitive. One typed error (BudgetExhausted with reason: 'cost' | 'runtime' | 'no_pricing'), one schema-stable audit JSONL at ~/.gbrain/audit/budget-YYYY-Www.jsonl. Contracts: record() throws when cumulative spend exceeds cap (the cap is a real ceiling, not a suggestion); reserve() admission checks cumulative + OUTSTANDING spend — each admitted projection is held until its record() settles it, so N concurrent callers (skillopt's validation gate) can't all pass against the same cumulative and breach --max-cost-usd by (N-1)×per-call cost; reserve() hard-fails with reason: 'no_pricing' when maxCostUsd is set AND the model is missing from pricing maps (warn-once preserved when cap is unset); lookupPricing normalizes recipe aliases through resolveRecipe before consulting the pricing tables, so claude-cli:haiku prices exactly like claude-cli:claude-haiku-4-5-20251001 at reserve(), record() and isModelPriceable() alike (the gateway reserves with the pre-resolution string the user configured); extractUsageFromError(err, fallback) returns err.usage when the SDK provides it, else the pessimistic fallback (caller passes maxOutputTokens, not the optimistic pre-call estimate). onExhausted(cb) fires once synchronously BEFORE the throw propagates so callers can persist checkpoints. Shared by brainstorm, cycle/budget-meter, and eval-contradictions; BudgetMeter keeps its public shape over it (schema_version: 1 stamped on every dream-budget audit line). Pinned by 18 unit cases.

  • src/core/audit-week-file.ts — single source of truth for ISO-week audit JSONL filename math. Exports isoWeek(d), isoWeekFilename(prefix, now?), resolveAuditDir() (honors GBRAIN_AUDIT_DIR). Year-boundary correctness pinned by tests at 2020-W53 (the 53-week year), 2025-W01 rolling in from 2024-12-30 (Monday), 2026-W01. Four call sites: src/core/minions/handlers/shell-audit.ts, src/core/facts/phantom-audit.ts, src/core/audit-slug-fallback.ts, src/core/cycle/budget-meter.ts. Each keeps its compute<X>AuditFilename thin wrapper.

  • src/core/diarize/payload-fitter.ts — generic fit-arbitrarily-large-items-into-per-call-token-budget utility. 'batch' strategy is deterministic token-budgeted chunking with no LLM calls. 'summarize' strategy embed-clusters into ceil(items/4) groups via cheap deterministic nearest-neighbor on cosine, Haiku-summarizes each cluster via Promise.allSettled at parallelism=4. Each Haiku call composes the active BudgetTracker via the AsyncLocalStorage. Quality gate: when success_ratio < min_success_ratio (default 0.75), result is flagged degraded: true — the fitter preserves the successful subset; the caller decides whether to surface a partial result or abort.

  • src/core/brainstorm/checkpoint.ts — crash-resilient checkpoint for gbrain brainstorm and gbrain lsd. Persists FULL idea bodies (~50KB/run) so resume MERGES pre-crash ideas with post-resume ideas before the judge runs (a resume that produces only second-run output is silent partial output). run_id = sha256(question + profile + sort(close_slugs) + sort(far_slugs)).slice(0,16) — NO embedding bits, stable across embedding-model swaps. Atomic write via .tmp + rename. ONE resume flag (--resume <run_id> covers both failed AND never-attempted crosses); --list-runs prints run_ids mtime-newest-first; --force-resume bypasses the 7-day staleness gate. Cycle purge phase (gbrain dream --phase purge) GCs checkpoints older than 7 days via gcStaleCheckpoints(7). Pinned by test/e2e/brainstorm-resume.test.ts (20 unit + 3 E2E cases incl. the merge contract).

  • src/core/remediation-checkpoint.tsdoctor --remediate checkpoint at ~/.gbrain/remediation/<plan_hash>.json. plan_hash = sha256(JSON.stringify(sorted recommendation ids)).slice(0,16). Schema-versioned, atomic .tmp + rename. gbrain doctor --remediate --resume <plan_hash> (no arg picks newest matching) loads it and skips completed steps. Mismatched plan_hash refuses with a paste-ready message. Cleared on clean completion. Pinned by 13 unit cases.

  • src/core/model-config.ts — Model-string resolution (the seam every internal LLM call walks through). Four-tier system (ModelTier = 'utility' | 'reasoning' | 'deep' | 'subagent') with TIER_DEFAULTS (utility→haiku-4-5, reasoning→sonnet-4-6, deep→opus-4-7, subagent→sonnet-4-6) and tier?: ModelTier on ResolveModelOpts. resolveModelDetailed() runs the 8-step chain and reports WHICH step won (ResolveSource): cliFlag → config key → deprecated key → models.tier.<tier>models.default → env var (GBRAIN_MODEL) → key-aware tier default → caller fallback (the tier key sits ABOVE models.default — tier-specific beats generic, so setting a cheap utility tier is honored even when models.default is also set); resolveModel() is the thin wrapper for callers that only want the string. Step 7 is KEY-AWARE: resolveTierDefault(tier, env?) walks PROVIDER_TIER_DEFAULTS (anthropic first — zero change for keyed installs; openai second so an OPENAI_API_KEY-only install resolves servable defaults) over the merged provider env (mergedProviderEnv, config-file keys folded, env wins, empty strings dropped); injected env is used EXCLUSIVELY (no config read — hermetic for tests); when neither anthropic nor openai key is present, a SERVABLE file-plane pin for the tier (utility→expansion_model, other tiers→chat_model, checked via providerKeyReady, PIN_KEY_BY_TIER is the single tier→key map) wins over the floor so a single-provider install (DeepSeek-only, OpenRouter-only, ...) routes every bare-default caller — extract_atoms, facts classify, page-summary — to the provider it actually has a key for; no key and no servable pin → TIER_DEFAULTS unchanged. The openai entry carries NO literal model pins: it resolves per call through the latest-model discovery cache (src/core/ai/openai-latest.ts, account-discovered, priced-only) with openaiStaticTierFallback() — the openai recipe's chat list ranked by the same grammar — as the offline floor, so the recipe is the single human-updated source and the account is the runtime source. The gpt alias resolves dynamically through the same path (the map entry is a documentation floor). resolveEffectiveChatModel(fileCfg, env) / resolveEffectiveExpansionModel are the ENGINE-FREE shared effective-model resolvers (GBRAIN_MODEL > servable file pin per providerKeyReady (recipe auth_env.required) > key-aware tier default; unservable pins warn once and fall through) — used by BOTH reconfigureGatewayWithEngine's fallback layer and detectCapabilities' extraction probe so runtime routing and the capability report cannot diverge; they read RAW loadConfig() output, never gateway state (the boot fold stamps defaults, making explicit pins indistinguishable there). isAnthropicProvider(modelString) checks provider:model prefix OR claude- bare-id pattern (routes through splitProviderModelId from src/core/model-id.ts so slash-form ids like anthropic/claude-sonnet-4-6 classify correctly). enforceSubagentCapable() is the layer-2 runtime guard: tier === 'subagent' resolutions are classified via classifyCapabilities()unusable:no_tools/unknown warn once and fall back to TIER_DEFAULTS.subagent; degraded:no_caching (e.g. OpenAI) passes with a once-per-(source, model) cost warn. _resetDeprecationWarningsForTest() clears all three warn memos (deprecation, subagent, unservable-pin). Pinned by test/model-config.serial.test.ts.

  • src/core/ai/model-resolver.ts — Recipe-touchpoint validator. assertTouchpoint(recipe, touchpoint, modelId) checks the PROVIDER's capability only (anthropic has no embedding touchpoint; voyage/ollama have no chat) and never gates on the model id. Recipe models: arrays are informational — default-model selection (models[0] for --model <provider> shorthand and env-ready pickers), guard-test fixtures for the repo's own hardcoded defaults, and gbrain providers list display — NOT a runtime allowlist, so frontier models newer than a recipe work without a recipe PR. A nonexistent id surfaces as the provider's own model_not_found at call time; gbrain models doctor live-probes the configured models for a pre-flight check. Exception: gateway.rerank() keeps its own model-list check because each listed reranker id maps to a known request/response wire shape. embeddingDimsForModel matches recipe model_dims keys exactly first, then case-insensitively with BOTH sides folded (configured ids arrive cased, and user-editable recipe tables can carry cased keys; without the fold a cased id would fall through to default_dims and gbrain init would build a wrong-width column), before falling back to default_dims.

  • src/commands/models.tsgbrain models [--json] read-only routing dashboard: prints tier defaults (utility/reasoning/deep/subagent), the resolved value for each (re-walking the resolution chain), every per-task override (14 PER_TASK_KEYS, including provider-neutral models.contextual_synopsis with legacy-key/env attribution and models.dream.extract_atoms which reports via its own caller-specific resolver — resolveExtractAtomsModel() in extract-atoms.ts — rather than the generic chain, so the report can't diverge from what the phase actually calls), the alias map, and a source-of-truth column (default / config: <key> / env: <VAR>). gbrain models doctor [--skip=<provider>] [--json] fires a 1-token gateway.chat() probe against each configured chat + expansion model and classifies failures into {model_not_found, auth, rate_limit, network, unknown}. The probe timeout resolves per model via resolveChatProbeTimeoutMs — the recipe touchpoint's default_timeout_ms when declared, else the flat 5000ms default (mirrors the reranker probe's recipe-default fallback; claude-cli declares 30s because its claude -p subprocess cold start routinely outruns 5s and would false-fail every run as unknown). Wired into cli.ts dispatch + CLI_ONLY set. A zero-token embedding_config probe runs FIRST, before any chat/expansion probes spend money: probeEmbeddingConfig() reads getEmbeddingModel() + getEmbeddingDimensions() and (for Voyage flexible-dim models) checks isValidVoyageOutputDim(dims) against VOYAGE_VALID_OUTPUT_DIMS. ProbeStatus variant 'config' + optional fix?: string on ProbeResult surface a paste-ready gbrain config set ... line in human + JSON output; touchpoint label 'embedding_config' joins 'chat' and 'expansion'.

  • src/core/init-embed-check.ts — embedding-key validation at gbrain init. runInitEmbedCheck(opts) runs a config-only diagnoseEmbedding (catches a missing key for ANY provider) plus a best-effort liveTestEmbed (1-token gateway.embed(['probe'], {inputType:'query', abortSignal}), 5s AbortController timeout, never throws — catches an invalid/expired key). Loud warning to stderr; init still exits 0 (--no-embedding is the deferred-setup escape; --skip-embed-check / GBRAIN_INIT_SKIP_EMBED_CHECK=1 skip the check). Builds the effective env (process.env + every file-plane provider key buildGatewayConfig folds — openai/anthropic/voyage/zeroentropy/dashscope/google — from loadConfigFileOnly() + opts.apiKey) and configures the gateway via buildGatewayConfig before diagnose/probe, so the check sees the same keys AND provider base URLs runtime will (no false "missing key" for config.json-keyed users; the probe hits the right endpoint). Init-specific warning text names --no-embedding / --skip-embed-check, not the sync-flavored --no-embed. Wired into initPGLite + initPostgres in src/commands/init.ts, with the result added to the --json envelope as embedding_check {ok, reason?, live_ok?}. Pinned by test/init-embed-check.test.ts (hermetic via the gateway embed-transport seam + withEnv).

  • src/commands/jobs.ts:refreshGatewayForJob + src/core/ai/gateway.ts:refreshGatewayEnvFromFilePlane — long-lived-worker staleness boundary, three tiers: (1) DB-plane model config re-resolves per gateway-refresh job (reconfigureGatewayWithEngine); (2) FILE-plane config (~/.gbrain/config.json, incl. provider API keys) re-folds per job via the env-ONLY refreshGatewayEnvFromFilePlane — never a full configureGateway(buildGatewayConfig(loadConfig())), which would clobber DB-plane-merged fields (base_urls, chat options) with file-plane-only values; (3) true process env vars are fixed at worker start and need a restart. gbrain config set *_api_key writes the DB plane, which loadConfigWithEngine() deliberately never merges for key fields — those writes do NOT reach workers (TODO filed to reroute them to the file plane). facts-absorb sits in GATEWAY_REFRESH_JOB_NAMES; its handler converts execution-time chat_unavailable in a KEYED worker into a typed retryable failure (factsAbsorbShouldRetry) while a keyless worker completes the job as a calm skip. Pinned by test/jobs-gateway-refresh.serial.test.ts.

  • src/core/ai/openai-latest.ts — latest-model discovery: OpenAI defaults are NEVER pinned. refreshLatestOpenAIModels() (called from reconfigureGatewayWithEngine, TTL 24h, 3s-bounded, fail-open, GBRAIN_MODEL_DISCOVERY=off|0 kill switch — the test preload sets it) fetches the account's own GET /v1/models, ranks ids through a conservative grammar (parseOpenAIChatId: bare family aliases gpt-N.M + known tier suffixes sol/pro/terra/luna/nano/mini; dated snapshots, -chat Instant-class, realtime/image and UNKNOWN future suffixes are ignored — rot degrades to newest-known-shape, never a wrong pick), and maps the newest family onto the tier ladder (utility→cheap, reasoning/subagent→mid, deep→top). ONLY ids with a canonical pricing row are eligible (rankOpenAIChatModels's priced filter): BudgetTracker fails closed (no_pricing) under a cost cap, so an unpriced discovered default would brick budget-capped backfills — a newer-unpriced family warns once naming the missing model-pricing.ts row. Result lands in <configDir()>/model-cache.json (atomic tmp+rename); latestOpenAITiers() is the SYNC read (stale cache beats static fallback — TTL gates refresh, never use) consumed by resolveTierDefault's openai entry and the dynamic gpt alias. Pinned by test/openai-latest.serial.test.ts.

  • src/core/ai/provider-env.tsmergedProviderEnv(cfg, env): THE canonical provider-key/env fold. Maps file-plane config keys (openai/anthropic/zeroentropy/openrouter/voyage/dashscope/deepseek/litellm/together/google/azure_openai + azure endpoint/deployment/entra) to the env names recipes read, merges env on top (env wins ONLY for keys carrying a real value — '' and undefined dropped), then applies the GEMINI_API_KEY → GOOGLE_GENERATIVE_AI_API_KEY alias (canonical env name > alias > config fallback). Three consumers: buildGatewayConfig (gateway env), detectCapabilities (capability probe), resolveTierDefault/resolveEffectiveChatModel (key-aware model resolution) — so gateway env and the capability probe cannot drift.

  • src/core/ai/build-gateway-config.tsbuildGatewayConfig(c: GBrainConfig): AIGatewayConfig, re-exported by src/cli.ts. Lets core modules (init-embed-check.ts) reuse it without importing the CLI entrypoint. Delegates the file-plane API-key fold + env merge to mergedProviderEnv (src/core/ai/provider-env.ts); keeps ownership of threading local-server *_BASE_URL env vars into base_urls. process.env wins EXCEPT empty-string / undefined values are dropped before the merge, so an injected empty ANTHROPIC_API_KEY='' (Claude Code neuters subprocess LLM calls this way) can't clobber a valid config-plane key; '0' / 'false' are preserved. Pinned by test/ai/build-gateway-config.test.ts.

  • src/core/skill-trigger-index.ts — Shared loader that unions per-skill SKILL.md frontmatter triggers: with curated RESOLVER.md / AGENTS.md rows from skillsDir AND the parent dir (preserves the OpenClaw workspace-root layout). UNION semantics: explicit RESOLVER.md rows ADD to frontmatter triggers (don't replace). Dedup keyed on (skillPath, trigger.trim().toLowerCase()). Three consumers fold through this primitive — checkResolvable, runRoutingEvalCli, mounts-cache.composeResolvers — so fixing frontmatter reaches all of them. Exports loadSkillTriggerIndex(skillsDir): SkillTriggerEntry[], entriesToResolverContent(entries): string (synthesizes a markdown-table resolver string for runRoutingEval's string-content API), findPrimaryResolverPath(skillsDir): string | null, the FRONTMATTER_SECTION constant, and _resetWarnedSkillsForTests. Skip rules: non-directory entries (a symlink dirent is followed via statSyncreaddirSync dirents report the link's OWN type without following, so skill dirs symlinked in from a shared store, e.g. ~/.agents/skills/<name>, still load), _*/.* prefixes, conventions/+migrations/ subdirs, skills with no SKILL.md (deprecated install/ graceful-skipped), no triggers: array, or malformed YAML (warn-once + skip). Reuses parseSkillFrontmatter from src/core/skill-frontmatter.ts, including block and wrapped flow-sequence arrays. Pinned by test/skill-trigger-index.test.ts (18 hermetic cases). CI gate bun run check:resolver (= bun src/cli.ts check-resolvable --strict --skills-dir skills/) wired into bun run verify.

  • src/core/skill-catalog.ts — host-repo skill catalog backing the MCP list_skills / get_skill ops. Lets a thin MCP client (Codex desktop, Claude Code, Claude Cowork, Perplexity) DISCOVER + FOLLOW the agent repo's fat-markdown skills over gbrain serve — a skill is prose, so "using" one = fetching its body then calling the gbrain MCP tools the server already exposes. Read-scope, NOT localOnly (defensible only via the full mitigation stack): (1) publish gateassertPublishEnabled(ctx, publishSkills); remote callers require mcp.publish_skills === true, default-OFF so an upgrade never silently grants existing read tokens host-skill read; local callers (ctx.remote === false) always pass. (2) path confinementassertSkillNameShape rejects separators/../null/space before any FS access; the client name is a manifest LOOKUP KEY (via loadOrDeriveManifest), never a raw path segment; confineManifestPath does realpath + relative-containment + SKILL.md-regular-file check on EVERY entry (defeats poisoned manifest.json path, symlink/.. escape). (3) frontmatter allowlistGetSkillResult.frontmatter projects a safe subset; private writes_to + sources dropped. (4) prose-only + 256KB cap (MAX_SKILL_MD_BYTES, env GBRAIN_MAX_SKILL_MD_BYTES), size-checked twice (statSync + UTF-8 byte length). (5) no install_path serve for remote — remote callers use autoDetectSkillsDir (no install-path tier) so a hosted gbrain with no agent repo returns storage_error; local callers use autoDetectSkillsDirReadOnly. (6) MCP rate-limiter caps call rate. Config reads honor BOTH planes: readMcpPublishSkills / readMcpSkillsDir prefer the DB plane (engine.getConfig) over the file plane (ctx.config.mcp). Tool-honesty: crossReferenceTools(declared, ctx) splits a skill's declared tools: into usable_tools vs unavailable_tools; buildSkillCatalog's instructions envelope (SKILL_CATALOG_INSTRUCTIONS) carries the "these are prose, follow-then-call-tools" protocol. Skills are host-filesystem repo-global — sourceScopeOpts(ctx) / ctx.brainId deliberately do NOT apply. buildSkillCatalog is resilient (one malformed/escaping skill is skipped, never throws). Config keys in src/core/config.ts: GBrainConfig.mcp?: { publish_skills?, skills_dir? } + KNOWN_CONFIG_KEYS entries mcp.publish_skills/mcp.publish_skills_prompted/mcp.skills_dir + mcp. prefix in KNOWN_CONFIG_KEY_PREFIXES. src/commands/init.ts writes config.mcp = { publish_skills: true, ... } for new installs (existing config wins on re-init). src/commands/upgrade.ts:runPostUpgrade adds a one-time consent prompt (gated by mcp.publish_skills_prompted; existing installs stay OFF until owner opts in). Three ops register in src/core/ops/skills-catalog.ts (spread into the operations.ts façade): list_skills with optional section filter + cliHints:{name:'skills'}; get_skill taking name (+ source_id for brain-resident packs) + cliHints:{name:'skill', positional:['name']}; list_brain_skillpack. They dynamically import this module to avoid the import cycle (skill-catalog statically imports the operations array). Descriptions in src/core/operations-descriptions.ts (LIST_SKILLS_DESCRIPTION, GET_SKILL_DESCRIPTION, SKILL_CATALOG_INSTRUCTIONS, SKILL_CLIENT_GUIDANCE), pinned by test/operations-descriptions.test.ts. CLI: gbrain skills / gbrain skill <name>. Pinned by test/skill-catalog.test.ts, test/skill-catalog-security.test.ts (path-confinement / poisoned-manifest / symlink-escape), test/skill-catalog-transports.test.ts (publish-gate + remote-vs-local) over test/fixtures/skill-catalog/.

  • src/core/check-resolvable.ts — Resolver validation: reachability, MECE overlap, DRY checks, structured fix objects. CROSS_CUTTING_PATTERNS.conventions is an array (notability gate accepts conventions/quality.md and _brain-filing-rules.md). extractTriggers() delegates to the shared SKILL.md parser, so MECE gap detection and the trigger index agree on block lists, single-line flow sequences, wrapped flow sequences, and CRLF input. extractDelegationTargets() parses > **Convention:**, > **Filing rule:**, and inline backtick references. DRY suppression is proximity-based via DRY_PROXIMITY_LINES = 40. parseResolverEntries accepts BOTH the markdown table AND a compact list format (- **skill-name**: trigger1 | trigger2 | trigger3 or - skill-name: trigger1 | trigger2); shapes can mix in one file, folded by the multi-resolver merge. Skill name MUST be kebab-lowercase (regex [a-z][a-z0-9-]+) so prose bullets like - **Note**:/- **Convention**:/- **TODO**: don't false-match as skill rows. skillPath is ALWAYS derived as skills/<name>/SKILL.md: an optional → \skills/path`(or ASCII->) suffix is stripped from the trigger but NOT honored as the path — two consumers (routing-eval.ts:skillSlugFromPath, the manifest lookup) assume the convention; use the table format for non-conventional paths. Multi-trigger rows fan out to one entry per trigger sharing the same skillPath; checkResolvablededupes so the reachability count counts each skill once. Pinned bytest/check-resolvable.test.ts(resolver shapes plus trigger array syntax cases) +test/check-resolvable-openclaw-compact.test.ts(8 cases overtest/fixtures/openclaw-compact-resolver/andtest/fixtures/openclaw-mixed-merge/). Tutorial: docs/guides/scaling-skills.md(three-tier scaling: ~300-skill agent to ~4K tokens/turn from ~25K).checkResolvable(skillsDir, opts?)takes an optionalskillsDirSource(the detection tier fromautoDetectSkillsDir, threaded by both callers: doctor.ts and the check-resolvable command; an explicit --skills-dirpassesnull). An unreachableissue downgrades fromerrortowarningonly when ALL hold: the directory was found via the ungatedcwd_walk_uptier, no resolver file contributes rows (noRESOLVER.md; a generic AGENTS.mdwith zero table rows counts as absent), unreachable skills outnumber reachable ones, and nomanifest.jsonexists on disk. This keeps a foreign tool'sskills/dir walked up to from cwd from being scored as a broken gbrain skillpack, while a real skillpack (RESOLVER.md present, higher-confidence tier, dense trigger coverage, or any manifest.json, even a corrupted one) stays strict. A foreign dir with zerotriggers:and no resolver file still hard-fails via themissing_file` branch (separate follow-up).

  • src/core/repo-root.ts — Shared findRepoRoot(startDir?): walks up from startDir (default process.cwd()) looking for skills/RESOLVER.md. Zero-dependency, imported by doctor.ts and check-resolvable.ts; parameterized startDir makes tests hermetic. Read-path / write-path split: autoDetectSkillsDir (shared, read+write-safe) has tier-0 $GBRAIN_SKILLS_DIR operator override ahead of the 4-tier chain. autoDetectSkillsDirReadOnly wraps it with a tier-5 install-path fallback that walks up from fileURLToPath(import.meta.url) and gates on isGbrainRepoRoot so unrelated repos can't false-positive. Read-path callers (doctor, check-resolvable, routing-eval) use the read-only variant; write-path callers (skillpack install, skillify scaffold, post-install-advisory) stay on the shared function so install-from-~ can't retarget the bundled gbrain skills/ instead of the user's workspace. SkillsDirSource variants 'env_explicit', 'install_path'; AUTO_DETECT_HINT_READ_ONLY documents the extra tier. The --fix safety gate in doctor.ts + check-resolvable.ts refuses auto-repair when detected.source === 'install_path'.

  • src/core/skills-integrity.ts — Tamper-evidence manifest for the bundled skills/ tree; NOT a signature system. Pure functions over node:crypto sha256: computeSkillsManifest(dir) (recursive, sorted '/'-relative paths, excludes the manifest itself, skips symlinks), renderSkillsManifest(dir) (2-space JSON + trailing newline, deterministic), verifySkillsManifest(dir, manifest){modified, missing, extra}. Committed manifest lives at skills/skills.lock.json (SKILLS_MANIFEST_FILENAME); regenerate via bun run scripts/generate-skills-manifest.ts. Consumers: the warn-only skills_manifest_integrity doctor check in src/commands/doctor.ts (ok/skip when no manifest is present — user workspaces and compiled-binary installs are not drift) and the CI freshness guard scripts/check-skills-manifest-fresh.sh (bun run check:skills-manifest, in bun run verify). Pinned by test/skills-integrity.test.ts.

  • src/commands/check-resolvable.ts — Standalone CLI wrapper over checkResolvable(). Exports parseFlags, resolveSkillsDir, DEFERRED, runCheckResolvable. Exit rule: 1 on any issue (warnings OR errors), stricter than doctor's ok flag. Stable JSON envelope {ok, skillsDir, report, autoFix, deferred, error, message} — same shape on success and error. --fix runs autoFixDryViolations BEFORE checkResolvable (same ordering as doctor). scripts/skillify-check.ts subprocess-calls gbrain check-resolvable --json (cached per process) and fails loud on binary-missing. AGENTS.md workspaces resolve natively (see src/core/resolver-filenames.ts). DEFERRED[] is empty. Resolver lookup is the multi-file merge in src/core/check-resolvable.ts — entries collected from every RESOLVER.md/AGENTS.md across the skills dir AND its parent, deduped by skillPath (first occurrence wins). Uses autoDetectSkillsDirReadOnly so cd ~ && gbrain check-resolvable finds bundled skills via the install-path fallback; --fix carries the same install-path safety gate (refuses to write when detected.source === 'install_path').

  • src/core/resolver-filenames.ts — central list of accepted routing filenames (RESOLVER.md, AGENTS.md). Shared by findRepoRoot, check-resolvable, and skillpack install so every code path walks the same fallback chain.

  • src/commands/skillify.ts + src/core/skillify/{generator,templates}.tsgbrain skillify scaffold <name> creates all stubs for a new skill: SKILL.md, script, tests, routing-eval.jsonl, resolver entry, filing-rules pointer. gbrain skillify check <script> runs the 10-step checklist (LLM evals, routing evals, check-resolvable gate, filing audit) against a candidate skill before it lands.

  • src/commands/skillify-check.tsgbrain skillpack-check agent-readable health report. Exit 0/1/2 for CI gating; JSON for debugging. Wraps check-resolvable --json, doctor --json, and migration ledger into one payload. Required item 12 (brain_first_compliance) calls analyzeSkillBrainFirst() on the candidate SKILL.md; exits 1 when the verdict is missing_brain_first (external-lookup pattern present, no callout, no brain_first: exempt). The scaffold path in src/core/skillify/templates.ts pre-inserts the canonical Convention callout into new SKILL.md files so freshly-scaffolded skills pass item 12.

  • src/commands/book-mirror.tsgbrain book-mirror --chapters-dir <path> --slug <slug> [flags]. Submits N read-only subagent jobs (one per chapter; allowed_tools: ['get_page', 'search']), waits for all via waitForCompletion, reads each child's job.result, assembles two-column markdown CLI-side, writes a single operator-trust put_page to media/books/<slug>-personalized.md. Trust narrowing happens at the tool-allowlist layer (subagents can't call put_page) so untrusted EPUB content can't prompt-inject any people page. Cost-estimate prompt before launching; refuses to spend in non-TTY without --yes. Per-chapter idempotency keys (book-mirror:<slug>:ch-<N>) for retry-friendly re-runs. Partial-failure: assembles completed chapters + a ## Failed chapters section. Pinned by test/book-mirror.test.ts (9 cases).

  • src/commands/skillpack.ts + src/core/skillpack/{bundle,scaffold,reference,migrate-fence,scrub-legacy,harvest,harvest-lint,copy,apply-hunks,diff-text,installer}.tsinstall/uninstall are not supported (they exit non-zero with a hint to the replacement). Surface: scaffold (one-time additive copy via copyArtifacts in copy.ts; refuses to overwrite; partial-state fills missing paired sources declared in SKILL.md frontmatter sources:), reference (read-only diff lens + --apply-clean-hunks two-way auto-apply via pure-JS unified-diff parser/applier in apply-hunks.ts + diff-text.ts), migrate-fence (one-shot strip of legacy fence; cumulative-slugs receipt → row-parsing fallback; preserves rows verbatim as user-owned routing), scrub-legacy-fence-rows (opt-in row cleanup with skill-present + non-empty-triggers gate), harvest (host→gbrain inverse with symlink-reject + canonical-path containment via a validateUploadPath-style gate + default-on privacy linter in harvest-lint.ts against ~/.gbrain/harvest-private-patterns.txt plus built-in a built-in fork-name pattern + email + Slack-channel patterns; rollback on match). Paired-source declarations live in each SKILL.md's frontmatter sources: array (validated by loadSkillSources in bundle.ts). autoDetectSkillsDir (in src/core/repo-root.ts) has a cwd_walk_up tier ahead of ~/.openclaw/workspace ($OPENCLAW_WORKSPACE precedence preserved). gbrain skillpack check --strict exits non-zero on drift (CI gate); top-level gbrain skillpack-check keeps exit-1-on-issues for cron. Companion editorial skill skills/skillpack-harvest/SKILL.md drives the genericization checklist. Doc: docs/guides/skillpacks-as-scaffolding.md. Test coverage across test/skillpack-{copy,scaffold,reference,reference-apply,apply-hunks,migrate-fence,scrub-legacy,harvest,harvest-lint,frontmatter-sources}.test.ts + 9-case E2E in test/e2e/skillpack-flow.test.ts. installer.ts + test/skillpack-install.test.ts remain because gbrain skillpack diff uses diffSkill from there.

  • src/core/skillpack/{personas,bridge-state,harness-bridge}.ts + src/commands/skillpack/{shared,scaffold,reference,harness}.ts — the harness skill bridge. src/commands/skillpack.ts is a peeled FAÇADE (dispatch + HELP_TOP + the install/uninstall removal errors; module-size ratchet); per-subcommand handlers live in src/commands/skillpack/ and the flag registry scans the dir via the façade's facadeExpansion entry — spell foreign CLI flags dash-less in these modules' comments/strings (prose-bleed class). scaffold --harness <claude-code|openclaw|codex|opencode> installs a persona-curated set into the harness's native skills dir: personas live in skills/plugin-lanes.json#personas (personas.ts is the SINGLE validation implementation — scripts/generate-plugin-tree.ts imports it; membership ⊆ the plugin lane set, so lane-excluded slugs are refused with their recorded reason); slugs path-resolve via bundle.ts's universe:'manifest' (the 8 lane additions are all absent from openclaw.plugin.json#skills). Safety: frontmatter fail-loud gate pre-write (a frontmatterless SKILL.md bricks Codex sessions), target-side confinement in assertTargetsConfined (deepest-existing-ancestor realpath — copy.ts confines SOURCES only), refuse-overwrite, and written-only ownership in bridge-state.ts (~/.gbrain/skillpack-bridge-state.json, schema gbrain-skillpack-bridge-v1, fail-open load, install-time sha256 per written file — never touches skillpack-state.json, whose loader drops unknown keys). --stub renders cold-pull pointers (frontmatter verbatim + <!-- gbrain-skill-stub v1 --> marker; ships the shared-dep closure AND sibling aux files — get_skill serves only the SKILL.md body) behind a three-check preflight in src/commands/skillpack/harness.ts (publish gate dual-plane, per-slug servability via verifySlugsServable, best-effort local surface warn — the module deliberately does NOT import src/mcp/surface.ts, whose comments would bleed serve flags into the allowlist; get_skill ∉ STARTER_OPS is pinned by a contract test). reference --harness is a stub-aware three-way lens (local_edit vs upstream_drift vs unknown — no install-time hash, never auto-applied — via the hash ledger; marker fallback survives state loss both directions) + --apply-clean-hunks (refuses stub files); remove --harness deletes ledger-owned files only; skillpack status renders an installed-bridges section from collectBridgesStatus. openclaw delegates to runScaffold({skillSlugs}); codex/opencode require an explicit dest until observation runs. Claude-code dirs come from host-specs.ts (claudeUserSkillsDir/claudeProjectSkillsDir, HOME-env-first). Tests: test/skillpack-{personas,bridge-state,harness-bridge,reference-harness,scaffold-harness}.test.ts. Doc: the harness-bridge section of docs/guides/skillpacks-as-scaffolding.md.

  • src/core/skillpack/{manifest-v1,tarball,state,remote-source,trust-prompt,bootstrap-display,scaffold-third-party,registry-schema,registry-client,rubric,doctor,init-scaffold,pack-publish,endorse,audit}.ts + examples/skillpack-reference/ + docs/skillpack-anatomy.md + scripts/build-skillpack-anatomy.ts — third-party skillpack ecosystem. gbrain skillpack scaffold <owner/repo|https-url|./tgz|./local-dir> resolves the spec via classifySpec, fetches through SSRF-hardened git-remote.ts (git) or extracts the tarball into ~/.gbrain/skillpack-cache/<host>/<owner>/<repo>/<sha>/, validates skillpack.json (api_version gbrain-skillpack-v1), checks gbrain_min_version, surfaces a TOFU first-install identity-confirm prompt (author + source + pinned commit + tarball SHA + tier; non-TTY requires --trust), records the pin in machine-owned ~/.gbrain/skillpack-state.json (schema gbrain-skillpack-state-v1, atomic .tmp + rename, isAlreadyTrusted skips re-prompt on author+pin match), runs through enumerateScaffoldEntriescopyArtifacts (one-time additive, refuses to overwrite), then DISPLAYS runbooks/bootstrap.md WITHOUT executing (deliberately does not auto-execute). Registry catalog at garrytan/gbrain-skillpack-registry split into registry.json (PR-able, gbrain-registry-v1) + endorsements.json (maintainer-only overlay, gbrain-endorsements-v1); effectiveTier merges. registry-client.ts fetches both via If-None-Match etag with 1h soft-TTL + stale-fallback (origins fresh_fetch | cache_warm | cache_soft_stale | cache_hard_stale); hard-fail only on no-cache + no-network. CLI: gbrain skillpack {search,info,registry,doctor,init,pack,endorse}. Doctor walks SKILLPACK_RUBRIC_V1 (10 binary dimensions: 5 required CORE — manifest_valid, skills_have_skill_md, routing_evals_present ≥5 intents, skills_have_unique_triggers MECE, changelog_present_and_current — and 5 quality BADGES — unit_tests_present, e2e_tests_present, llm_eval_present ≥3 cases, bootstrap_runbook_present, license_present); tier eligibility: endorsed needs all 10, community needs core + ≥3 badges, experimental needs core only, blocked when any core fails. --quick ~5s structural sweep; --fix --yes auto-scaffolds auto_fixable: true dimensions and refuses to overwrite files whose mtime is newer than skillpack.json. gbrain skillpack init <name> lands 11 files (skillpack.json, SKILL.md, routing-eval.jsonl, test/example.test.ts, e2e/example.e2e.test.ts, evals/example.judge.json, runbooks/{bootstrap,uninstall,upgrade-template}.md, CHANGELOG, README, LICENSE); freshly-init'd scores 10/10; --minimal skips test/e2e/evals. gbrain skillpack pack packs a deterministic tarball via GNU tar (--sort=name --mtime=@0 --owner=0 --group=0 --numeric-owner + GZIP=-n + TZ=UTC); refuses on tier_eligibility === 'blocked'. Extract caps (5000 files / 100MB total / 1MB per file / 255-char paths / 100:1 compression ratio); rejects symlinks/hardlinks/devices/FIFOs. gbrain skillpack endorse <name> [--tier ...] [--push] [--dry-run] runs in a clone of the registry repo: validates the pack in registry.json, mutates endorsements.json via pure applyEndorsement, stable-key-orders the write, commits endorse: <name> -> <tier>, optionally pushes. JSONL audit at ~/.gbrain/audit/skillpack-YYYY-Www.jsonl (ISO-week rotated, honors GBRAIN_AUDIT_DIR). examples/skillpack-reference/ is a 10/10 reference pack pinned by test/e2e/skillpack-third-party.test.ts. docs/skillpack-anatomy.md auto-generated via scripts/build-skillpack-anatomy.ts (--check for CI drift). CLI dispatch in src/commands/skillpack.ts disambiguates third-party (contains /, ://, .tgz) from bundled-skill kebab; kebab routes bundled-first, registry-fallback. Tests: test/skillpack-{manifest-v1,tarball,state,remote-source,trust-prompt,registry-schema,registry-client,rubric,doctor,init-scaffold,pack-publish,endorse,audit,scaffold-third-party}.test.ts + test/e2e/skillpack-third-party.test.ts. Spec at docs/designs/SKILLPACK_REGISTRY_V1_SPEC.md.

  • src/core/archive-crawler-config.ts — safety gate for the archive-crawler skill. Refuses to run unless archive-crawler.scan_paths: is explicitly set in the brain repo's gbrain.yml. Mirrors the storage-config.ts parsing pattern (sibling file, separate concern from storage tiering). loadArchiveCrawlerConfig(repoPath) throws ArchiveCrawlerConfigError(missing_section | empty_scan_paths | invalid_path | parse_error). normalizeAndValidateArchiveCrawlerConfig rejects relative paths and .. traversal; ~ is expanded; paths are stored resolved and terminated with the PLATFORM separator (path.sep) so error output reads natively on each OS. isPathAllowed(candidate, config) is the runtime per-file gate (scan_paths prefix-match with directory-boundary correctness; deny_paths overrides). Candidate, scan_paths and deny_paths all funnel through the private toComparablePrefix() before the prefix test — on Windows it folds \/ and lowercases (NTFS is case-insensitive, so a deny_path spelled Private must still match private, else the gate fails OPEN); on POSIX it is identity apart from the trailing separator, deliberately NOT folding, since \ is a legal filename character and paths are case-sensitive. Storing a native separator while appending a hardcoded / is the mixed-separator bug that made isPathAllowed deny every real path on Windows; the two functions must stay symmetric or the prefix test is meaningless. Pinned by test/archive-crawler-config.test.ts (26 cases, platform-selected fixtures + it.if-gated win32/POSIX comparison semantics).

  • test/helpers/tty-harness.ts + scripts/dx-explore.ts — the repo's single real-PTY layer, on pure Bun.spawn({terminal:}) (Bun 1.3.10+; engines.bun pin in package.json; no node-pty). launchTty spawns any CLI under a true pseudo-terminal with the hermetic env contract from test/helpers/agent-harness.ts (hermeticChildEnv; dropEnv strips pass-through auth keys), records timestamped output frames, and exposes waitFor/waitForAny/mark/sendKey/waitForQuiet/waitForExit/close. Lifecycle rule: only close() clears the wall-clock kill timer — always call it in a finally. Pure helpers (stripAnsi, computeStalls, renderStallsReport, parseDriveCommand, buildClaudeTuiSeed) are unit-tested in test/tty-harness.test.ts; that file's live-PTY smokes are describe.skipIf(!ptySupported())-gated. Transcript writes are structurally redacted: redactSecrets (secrets ≥ MIN_REDACT_SECRET_LEN[REDACTED:<name>]) runs at every write site, coalesceSecretStraddles merges frames so a secret split across a frame boundary can't bypass redaction, and saveTranscript takes an explicit redact map (seam — dx-explore builds it, tty-harness stays import-free of it). scripts/dx-explore.ts is the DX-exploration driver built on it — a developer instrument, not a test: nothing asserts, transcripts land gitignored under .context/dx-runs/ (doc: docs/guides/bootstrap.md). The interactive gbrain init pickers are asserted for real in test/init-picker-pty.serial.test.ts (serial lane, so it runs in required CI).

  • src/core/skillpack/{init-brain-pack,brain-pack-advisory,brain-pack-lint,brain-resident-locate,nag-state}.ts — brain-resident skillpacks. manifest-v1.ts gains optional brain_resident + schema_pack (additive). runInitBrainPack scaffolds a pack beside brain content (brain_resident:true, exact gbrain_min_version, 5-section machine-parseable README); applyWritePlan is factored out of init-scaffold.ts for the shared refuse-overwrite loop. brain-pack-lint.lintBrainPackTools validates each skill's tools: against the serving op set (E6 version-skew). Topology A: src/commands/sources.ts runAdd prints brain-pack-advisory to stderr after opsAddSource, fail-open; nag-state.ts (~/.gbrain/skillpack-nag-state.json) keys declines by (source-repo brain_id, source_id, pack_name) with pure decideNagAction (first/reminder/version-bump/ceiling) — declines count ONLY on CLI-interactive displays. Topology B: brain-resident-locate.loadResidentPacksForServer (source-scoped via sourceScopeOpts) backs the list_brain_skillpack op; getResidentSkillDetail backs get_skill source_id; scaffold_spec is the git source, never a server FS path. Tests: test/skillpack-{init-brain-pack,nag-state,brain-resident-locate}.test.ts + the brain-resident cases in test/skillpack-manifest-v1.test.ts.

  • src/core/advisor/{types,run,render,recommended-set,history,apply,collect-*}.ts + src/commands/advisor.tsgbrain advisor: read-only ranked actions from brain state. run.runAdvisor executes the hardcoded COLLECTORS array (version [cache-only], migration, schema-pack, stalled-jobs [absent-table tolerant], usage-shape, setup-smells, uninstalled-brain-pack, uninstalled-bundled, chronicle, mcp-client-fit), each in its own try/catch; collect-mcp-client-fit.ts (E3) flags full-surface MCP clients whose 30d usage fits STARTER_OPS (exact rescope-client --surface starter fix; ≥10-call threshold; automation-shaped clients excluded per D12) plus STARTER_OPS drift (top-used ops missing; starter members unused 90d) via the shared src/core/mcp-usage.ts reader — starter membership is judged against the exported ALWAYS_INCLUDED_STARTER_OPS (surface.ts) so the always-included lane never reads as unused, and the missing-from-starter arm excludes localOnly ops (never proposable for a network surface, mirroring derive-starter-ops) — REMOTE runs redact client identifiers to aggregate counts (amendment 29), and its dismiss/snooze rides the nag-state engine with its own ~/.gbrain/advisor-usage-nag-state.json (local runs only); rankFindings orders critical>warn>info then collector order, caps the info tail, and drops workspace_dependent findings when remote (A1). render.ts is the shared =-bar renderer used by the advisor AND post-install-advisory.ts (generalized to a single current-state recommended-set.RECOMMENDED, installscaffold). history.ts appends bounded ~/.gbrain/advisor-history.jsonl (no DB migration) for since-last-run deltas; local-only. apply.resolveApplyTarget is the allowlist+injection guard for commands/advisor.ts --apply <id> (structured argv, never a shell; local-only). The advisor op (operations.ts) is read-scoped, NOT localOnly, gated by mcp.publish_advisor (config.ts; default off) and strictly read-only on remote. CLI wired in cli.ts (CLI_ONLY + dispatch). Bundled skill skills/gbrain-advisor/ + weekly cron recipe. Tests: test/advisor-{core,apply,op-gate,ranking-eval}.test.ts.

  • src/core/chronicle/{eligibility,config,backstop,extract-events,ontology,narrative}.ts + src/eval/chronicle/harness.ts + src/commands/eval-chronicle.ts — Life Chronicle: the temporal spine. eligibility.isChronicleEligible decides which pages auto-emit events (meeting/conversation/calendar-event + directory rescue; diary and event pages NEVER eligible — privacy + anti-loop). backstop.runChronicleBackstop is the put_page hook body (fires ONLY on status==='imported' + the auto-link trust gate + the default-OFF auto_chronicle flag; enqueues a chronicle_extract minion job — LLM never runs on the write path). extract-events.runChronicleExtract is the job body: deterministic when/who, injectable judge (default = chat gateway; output cap 4000 tokens by default, operator override chronicle.judge_max_tokens), an ALL-or-nothing parse barrier (isValidProposal requires a real parseable date — a malformed batch writes NOTHING), then content-addressed life/events/ pages + a timeline_entries projection via engine.upsertEventProjection (dedup (event_page_id, date); idempotent re-runs). An unusable judge response is never recorded as no_events: a stopReason: 'length' truncation or a no-JSON-array response (parseJudgeJson returns null on parse failure; [] only for a legitimate empty array) surfaces as status: 'skipped' with reason judge_truncated / judge_parse_failed. ontology.ts carries the deterministic pieces of the bi-temporal per-entity ontology that RIDES THE facts TABLE (migration v122 adds dimension/value/value_hash/dim_status): valueHash (normalized, timestamp-free → crash-retry idempotent), normalizeDimension (seed alias lexicon), isNovelDimension (novel → quarantined, excluded from resolution/context until confirmed). The engine methods (mergeOntologyFact — corroborate on same value, forward-supersede via valid_until+superseded_by on a new value, backdated conflicts kept + flagged; getOntology with --asof valid-time travel; discoverOntologyDimensions; findOntologyConflicts — currently-open rows only) live in BOTH engines; both engines are on the R8 valid_until write allow-list (engine-layer, dimension IS NOT NULL rows only). Chronicle reads (getTimelineForDate/getSince/getLastSeen/getOnThisDay) JOIN the depth page (deleted_at IS NULL), hide soft-deleted event projections at READ time, and order by event effective_date for intra-day sequence. Ops: chronicle_day/chronicle_since/chronicle_last_seen/chronicle_on_this_day/ontology_*/volunteer_chronicle (agent orientation via src/core/context/chronicle-context.ts)/chronicle_backfill (admin, localOnly). Diary privacy: fail-closed for ctx.remote !== false callers — diary-sourced ontology + conflict values are redacted, the four timeline read ops (chronicle_day/chronicle_since/chronicle_on_this_day + volunteer_chronicle's recent_timeline) drop rows whose depth page, event page, or source provenance lives under life/diary/, and chronicle_last_seen answers the never-seen shape (last_date/last_event_slug/days_ago all null) when the latest sighting is diary-sourced or a diary page is queried as the entity. Page visibility rides the same policy: ontology_get/ontology_conflicts/volunteer_chronicle's ontologies resolve through readPolicyOpts, and both engines' getOntology/findOntologyConflicts apply privateProvenanceFilterFragment (an observation whose provenance page, looked up in the fact's own source, is visibility: private is dropped BEFORE per-dimension resolution and before the conflict HAVING, so an untrusted caller resolves the newest value they may see rather than a private value or a hole). The chronicle reads' event-page LEFT JOIN carries the caller's source scope in BOTH engines, so a scoped read can't surface another source's event pages through the join. Search: applyChronicleTypeBoost in search/hybrid.ts (bounded [1.0,1.25], fires only inside the recency !== 'off' post-fusion branch → non-temporal search bit-for-bit unchanged). Advisor collector collect-chronicle.ts (conflicts + coverage gap); doctor chronicle_projection_health (BRAIN category). Eval: gbrain eval chronicle — deterministic, own in-memory PGLite, 6 gold tasks (day order, last-seen, supersession, asof, conflict, isolation), exit 0 iff 6/6. Tests: test/chronicle-*.test.ts, test/eval-chronicle.test.ts.

  • src/core/skill-manifest.ts — parser for skill-manifest.json records. Used by skillpack installer to detect drift between the shipped bundle and the user's local edits, so updates merge instead of overwriting.

  • src/commands/routing-eval.ts + src/core/routing-eval.tsgbrain routing-eval catches user phrasings that route to the wrong skill. Reads skills/<name>/routing-eval.jsonl fixtures ({intent, expected_skill, ambiguous_with?}). Structural layer runs in check-resolvable by default (zero API cost). --llm is a placeholder for a future LLM tie-break layer; today it emits a stderr notice and runs structural only. Uses autoDetectSkillsDirReadOnly and the same multi-file resolver merge as check-resolvable, so on OpenClaw layouts (skills/RESOLVER.md + ../AGENTS.md) all three commands see the same trigger index. RESOLVER.md rows carry the full frontmatter triggers: arrays so the structural matcher sees realistic phrasings; ambiguous-fixture annotations cover deliberate skill chains like enrich → article-enrichment.

  • src/core/filing-audit.ts + skills/_brain-filing-rules.json — Check 6 of check-resolvable. Parses writes_pages: / writes_to: frontmatter on skills and audits their filing claims against the filing-rules JSON (error severity). Internal parseFrontmatter is a thin wrapper over the shared src/core/skill-frontmatter.ts parser so both filing-audit and skill-brain-first read the same shape (tools?, triggers?, brain_first?: 'exempt', typed brain_first_typo) from one source of truth.

  • src/core/skill-frontmatter.ts — shared content-based SKILL.md frontmatter parser. Array fields (writes_to, tools, triggers) use js-yaml with the failsafe schema, accepting block lists and single-line or wrapped flow sequences while preserving the tolerant legacy regex fallback for partially malformed YAML; CRLF is normalized before fence parsing. Recognizes the brain_first: 'exempt' declarative opt-out and surfaces near-miss declarations (brain-first, BrainFirst, quoted values, unknown values) as a typed brain_first_typo field so doctor can emit a paste-ready hint rather than fail silently. Single canonical form: snake_case brain_first: exempt, lowercase, unquoted.

  • src/core/skill-brain-first.ts — pure analyzer. analyzeSkillBrainFirst(skillPath, content): SkillBrainFirstResult walks the compliance ladder for every SKILL.md: (1) absent external-lookup pattern → no_external; (2) brain_first: exempt frontmatter → exempt_frontmatter; (3) canonical > **Convention:** see [conventions/brain-first.md](...) callout → compliant_callout; (4) explicit ## Phase 1: Brain heading → compliant_phase; (5) first gbrain search/query/get_page reference precedes first external pattern in the BODY (frontmatter stripped) → compliant_position; (6) else missing_brain_first warn. External pattern set: word-boundary regex over web_search, web_fetch, exa, perplexity, happenstance, crustdata, captain_api, firecrawl. Position scan is BODY-ONLY so a tools: [web_search] frontmatter declaration doesn't false-flag the skill. The 40-name FORMERLY_HARDCODED_EXEMPT list is preserved so doctor can emit a "this used to be auto-exempt, declare brain_first: exempt if still appropriate" hint. Consumed by 3 surfaces: doctor check, skillify-check item 12, dry-fix MISSING_RULE_PATTERNS.

  • src/core/skill-fix-gates.ts — shared safety primitives for dry-fix.ts. getWorkingTreeStatus(file) 3-state ('clean' | 'dirty' | 'not_a_repo'); isInsideCodeFence(content, offset); findAfterH1Paragraph(content) (canonical insertion offset for the auto-inserted Convention callout). Both REPLACE expanders (DRY violations) and the INSERT expander (MISSING_RULE_PATTERNS) consume from here so the install-path refusal and dirty-tree gates apply uniformly. src/core/dry-fix.ts re-exports them.

  • src/core/audit-skill-brain-first.ts — snapshot+diff JSONL audit at ~/.gbrain/audit/skill-brain-first-YYYY-Www.jsonl (ISO-week rotated, honors GBRAIN_AUDIT_DIR via shared resolveAuditDir()). recordBrainFirstRun(results) reads the previous snapshot at ~/.gbrain/audit/skill-brain-first-snapshot.json, diffs against current results, writes transition events (detected | resolved | fixed) one line per change, then atomically overwrites the snapshot via .tmp + rename. Transition-only writes — a stable brain produces 0 audit lines per doctor run. readRecentBrainFirstEvents(days) is the readback path for the future skill_brain_first_trend doctor check. Snapshot file is last-writer-wins under concurrent doctor runs; subsequent runs reconcile.

  • src/core/dry-fix.tsgbrain doctor --fix engine. autoFixDryViolations(fixes, {dryRun}) rewrites inlined rules to > **Convention:** see [path](path). callouts via three shape-aware expanders (bullet / blockquote / paragraph). Five guards: working-tree-dirty (getWorkingTreeStatus() 3-state 'clean' | 'dirty' | 'not_a_repo'), no-git-backup, inside-code-fence, already-delegated (40-line proximity, consistent with detector), ambiguous-multi-match, block-is-callout. execFileSync array args (no shell, no injection surface). EOF newline preserved. Safety primitives are in src/core/skill-fix-gates.ts (back-compat re-exports preserved). MISSING_RULE_PATTERNS INSERT pattern type lives alongside REPLACE patterns — same auto-fix entry point + git-safety gates, but places a canonical callout at a target offset (after-h1-paragraph only). First INSERT pattern is brain_first, auto-inserting > **Convention:** see [conventions/brain-first.md](../conventions/brain-first.md) for the lookup chain (search → query → get_page → external). on any flagged SKILL.md whose verdict is missing_brain_first. Idempotent — re-runs detect the existing callout and skip.

  • src/core/backoff.ts — Adaptive load-aware throttling: CPU/memory checks, exponential backoff, active hours multiplier.

  • src/core/retry.ts — canonical retry primitive for transient connection errors. Exports withRetry<T>(fn, opts) execution wrapper + BULK_RETRY_OPTS constant ({maxRetries:3, delayMs:1000, delayMaxMs:10000, jitter:'decorrelated'}, tuned for Supabase Supavisor's 5-10s circuit-breaker recovery) + BATCH_AUDIT_SITES typed const (closed enum of every audit-emission site) + resolveBulkRetryOpts(env) (reads GBRAIN_BULK_MAX_RETRIES/GBRAIN_BULK_RETRY_BASE_MS/GBRAIN_BULK_RETRY_MAX_MS with >=0 validation, throws on bad input with a paste-ready hint) + abortableSleep(ms, signal?) + RetryAbortError (tagged error for clean shutdown) + computeNextDelay() (pure-fn for 3 jitter modes: 'none', 'full', 'decorrelated'). The execution wrapper is consumed by postgres-engine.ts + pglite-engine.ts batch primitives (addLinksBatch / addTimelineEntriesBatch / upsertChunks) so every caller inherits retry as part of the data-primitive's contract. CI guard scripts/check-no-double-retry.sh fails the build on withRetry(...engine.batch...) patterns (prevents 3×3=9 retry amplification); scripts/check-batch-audit-site.sh validates every string-literal auditSite: '...' against the closed BATCH_AUDIT_SITES enum. Decorrelated jitter (AWS-style: uniform(base, prevDelay*3) capped at delayMaxMs) — 'full' jitter would allow near-zero retries that re-hit the recovering breaker. WithRetryOpts has an optional reconnect?: () => Promise<void> callback awaited in the catch branch AFTER isRetryableConnError classification but BEFORE the inter-attempt sleep — lets engine-level callers rebuild a dead pool/singleton between attempts. PostgresEngine.batchRetry injects () => this.reconnect() (the race-safe _reconnecting guard kicks in). Fail-loud: a reconnect throw PROPAGATES as the new error, replacing the symptomatic "No database connection". onRetry callbacks are awaited (sync arrows work identically; async callbacks correctly delay the sleep). Pinned by test/core/retry.test.ts (37 cases), test/core/retry-stress.slow.test.ts$ (5 \text{cases}, 100 \text{batches} \times 30% \text{blip} \text{rate}, \text{asserts} \text{zero} \text{row} \text{loss}), $test/core/retry-reconnect.test.ts (5 cases), test/e2e/db-singleton-shared-recovery.test.ts (3 DB-gated cases).

  • src/core/process-watchdog.ts — out-of-band hard-deadline killer for gbrain sync. A spinning sync (synchronous catastrophic-regex / ReDoS in pack link-inference) STARVES the main event loop, so the existing SIGTERM handler (process-cleanup.ts), --timeout setTimeout, and abort-flag checks can't fire — the process becomes unkillable-by-SIGTERM and, under cron, orphans pile up for 24h+. installProcessWatchdog({deadlineMs, graceMs?, label?, heartbeatMs?, onWarn?}): WatchdogHandle spawns a Bun worker_threads Worker via new Worker(code, {eval: true, workerData}) — its own OS thread + event loop fires even while main is in an unyielding sync loop. At deadlineMs it process.kill(process.pid, 'SIGTERM') (clean-shutdown chance if responsive); at deadlineMs+graceMs process.kill(process.pid, 'SIGKILL') (uncatchable — guaranteed death under starvation). Signaling SELF has NO PID-reuse footgun (current PID never reused while alive — the reason the detached-child-watches-parent design was rejected). eval: true bakes the worker body into the bun build --compile binary with no separate-file embedding. Empirically validated on Bun 1.3.13 (worker timer + SIGKILL killed a while(true){}-starved process). handle.dispose() (clean-exit finally) worker.terminate()s it; unref()'d so it never keeps the process alive. Pure watchdogDecision(elapsedMs, deadlineMs, graceMs) → 'wait'|'sigterm'|'sigkill' extracted for unit tests, and pure exported clampWatchdogTimers(deadlineMs, graceMs) + MAX_WATCHDOG_TIMER_MS clamp BOTH worker timers so the deadline AND the deadline+grace SUM stay ≤ 2312^{31}−1 — setTimeout overflow-fires above that at ~1ms, which for the sum timer would be an instant SIGKILL of a healthy process (unit-tested as pure arithmetic only; never arm a real max-deadline worker in-suite — its firing SIGTERMs the test runner itself). Optional heartbeatMs emits periodic [<label>] parent alive Ns, hard-kill in ~Ms lines (visible in cron logs even under starvation — the diagnosis surface). Fallback: if new Worker throws, degrades to an in-process timer with a loud warning that it can't fire under starvation. Two adopters: gbrain sync (below) and the opt-in PGLite disconnect watchdog (pglite-engine.ts entry above); autopilot/cycle are follow-up candidates. Pinned by test/process-watchdog.test.ts (pure decision matrix + clamp arithmetic + handle contract) + test/process-watchdog.serial.test.ts (Bun-pinned spawn integration: a starved harness process IS killed ~deadline+grace, a no-watchdog control does NOT self-exit, clean dispose never kills). Wired into src/cli.ts sync dispatch BEFORE connectEngine (so a connect-phase hang is bounded too); deadline resolved by resolveSyncHardDeadline in sync.ts (precedence: --no-hard-deadline > --hard-deadline > --timeout(non---all) > GBRAIN_SYNC_MAX_RUNTIME_SECONDS env > non-TTY default 3600s > none).

  • src/core/process-cleanup.ts — cleanup registry + abnormal-termination handlers (SIGTERM/SIGHUP/SIGPIPE, uncaughtException/unhandledRejection, EPIPE-on-stdout) that release locks before exit. registerCleanup(name, fn) returns a deregister handle; tryAcquireDbLock auto-registers. installSignalHandlers() is idempotent and called from INSIDE cli.ts's import.meta.main seam (first statement before main()), NOT at module load — installing at import time would leak a process-wide SIGTERM→process.exit(143) handler into any process that merely imports cli.ts (a bun test runner would die mid-suite when a test emits a synthetic SIGTERM, misreading rc=143 as an external kill). Spawned/compiled CLI processes are entrypoints, so they still install. Every attached listener ref is recorded so _resetForTests() DETACHES them (clearing flags alone would leave the exit(143) listener live on the shared runner). Tests that must emit synthetic signals strip foreign listeners around the emit (see test/run-child-entry.test.ts).

  • src/core/preferences.ts — preferences.json + the migration ledger (migrations/completed.jsonl append/read helpers; appendCompletedMigration, loadCompletedMigrations). Path resolution delegates to config.ts:gbrainPath(), so GBRAIN_HOME follows the ONE canonical convention: it is a PARENT dir and .gbrain is appended (GBRAIN_HOME=/tmp/x/tmp/x/.gbrain/migrations/...). copyForwardLegacyFile migrates a legacy layout that placed these files directly under $GBRAIN_HOME once per file on first touch: atomic (temp + linkSync, EEXIST = concurrent winner), JSON-validated for prefs, chmod 0600, copy-not-move for binary rollback, once-per-process warning; a valid-but-uncopyable legacy file (read-only home) is READ IN PLACE so a transient failure can't drop a minion_mode opt-out or migration history. One-shot by design — mixed-version writers diverging post-snapshot is an accepted, documented limitation.

  • src/core/audit/batch-retry-audit.ts — JSONL audit primitive for batch-retry events, built on audit-writer.ts. Schema: {ts, site, batch_size, attempt, outcome: 'success' | 'exhausted', delay_ms, error_message_summary, error_code?}. Privacy: NEVER logs slugs / page IDs / content (mirrors shell-audit.ts). logBatchRetry fires per successful retry recovery; logBatchExhausted fires when retries exhaust and rows are lost. readRecentBatchRetryEvents(hours=24) returns {events, corrupted_lines, files_scanned, files_unreadable} — corruption + permission errors surface to doctor, not silently swallowed. pruneOldBatchRetryAuditFiles(daysToKeep=30) deletes old files, called from gbrain dream --phase purge. File: ~/.gbrain/audit/batch-retry-YYYY-Www.jsonl (honors GBRAIN_AUDIT_DIR). summarizeError routes error messages through the shared redactConnectionInfo helper from src/core/audit/redact-connection-info.ts BEFORE truncation so DSNs / hostnames / credentials / IPv4 octets can't leak into operator-shared JSONL dumps. Pinned by test/audit/batch-retry-audit.test.ts (12 cases) + test/audit/batch-retry-redaction.test.ts (3 privacy cases).

  • src/core/audit/lock-renewal-audit.ts — JSONL audit primitive for per-job lock-renewal faults. Sibling of batch-retry-audit.ts, built on audit-writer.ts. Four outcomes: failure (single renewLock throw, counter incremented), success_after_failure (recovery; emits the recovery count), gave_up (time-based deadline exceeded; abort fired), executeJob_rejected (the second unhandledRejection vector — the stored executeJob(...).finally(...) promise itself rejected, e.g. failJob threw during the same DB outage). Schema: {ts, job_id, job_name, attempt?, outcome, error_message_summary?, error_code?} plus additive telemetry fields (cause?, lateness_ms?, overlap_skips?, load1?, cores?, via?, deadline_deferred?) threaded via an optional trailing ctx param on the sink (compactCtx copies only DEFINED fields so absent telemetry stays absent from the JSONL). Privacy: NEVER logs lock_token or job.data; error summaries route through redactConnectionInfo BEFORE truncation. Defense-in-depth: every audit call inside the lock-renewal tick's catch block is wrapped in its own inner try/catch so a misbehaving audit-writer can't surface as an unhandledRejection. readRecentLockRenewalEvents(hours=24) walks current + previous ISO week with corrupted-line tolerance. pruneOldLockRenewalAuditFiles(daysToKeep=30) is ready for future dream-cycle purge wiring. File: ~/.gbrain/audit/lock-renewal-YYYY-Www.jsonl. Pinned by test/audit/lock-renewal-audit.test.ts (incl. ctx round-trip + telemetry-less-line readback).

  • src/core/audit/redact-connection-info.ts — Shared pure helper. redactConnectionInfo(text: string): string strips Postgres connection info before any audit JSONL write: postgres:///postgresql:// URLs, host=foo, user=foo, password=foo, pwd=foo, IPv4 octets — each match becomes <REDACTED:kind>. Negative-lookbehind/lookahead [\w.@-] on the IPv4 pattern defeats version-string false positives (v3.1.4.0, tree-sitter@0.26.3.1) while still matching real IPs in PG errors ((192.168.1.42)). Order-sensitive pattern set: URL forms first so substrings inside URLs don't get double-redacted. Idempotent, pure (no I/O), hot-path-safe (regex compiled at module load). Wired into lock-renewal-audit.ts, batch-retry-audit.ts, and cli.ts's doctor DB-fallback stderr note (layered with url-redact.ts:redactUrlsInText). Known limitations: bare-quoted hostnames (at "db.example.com") and usernames (for user "postgres.foo") are NOT caught — the highest-value leak (the IP in those shapes) IS caught. Pinned by test/audit/redact-connection-info.test.ts (15 cases: all 5 patterns + Supabase fixture + ENOTFOUND fixture + version-string false-positive defense).

  • src/core/url-redact.ts — Postgres-URL + free-text credential redaction, sibling of redact-connection-info.ts. redactPgUrl(url) strips userinfo from a single postgres/postgresql URL, preserving scheme/host/port/db/query (non-URL input collapses to <redacted-url>). redactUrlsInText(text) sweeps free text (error messages, log lines) for credential shapes of any word-character scheme (the pattern is \w+://, so scheme names containing +/-/. are not matched — postgres shapes are the target): the userinfo match is greedy up to the LAST @ in the token so a raw @ inside a password can't leak its tail (over-redacts toward safety), and libpq keyword/value forms (password= / sslpassword=, including quoted values with spaces) are masked too; text without a credential shape passes through untouched. redactDeep(value) recursively redacts postgres URLs inside structured payloads about to be stringified. Consumers: the upgrade-errors + connection-events audit JSONL sites, doctor's connection_routing check output, and cli.ts's doctor DB-fallback stderr note (layered with redactConnectionInfo). CI guard: scripts/check-pg-url-redaction.sh fails the build when a new code path emits an unredacted postgres URL. Pinned by test/url-redact.test.ts.

  • src/core/minions/lock-renewal-tick.ts — Pure function behind MinionWorker.launchJob's setInterval body, structured so a renewal failure can never surface as an unhandledRejection, carrying the verify-before-evict doctrine. Exports runLockRenewalTick(deps, state) → Promise<TickResult> + resolveLockRenewalKnobs(env, lockDuration, intervalMs?) → LockRenewalKnobs + RenewalCallTimeoutError + LockRenewalTelemetryCtx. Doctrine: a thrown/timed-out renewal is NOT evidence of loss — when the NEXT tick would land past the soft deadline (lease - safetyMargin, cadence-aware: sinceLastSuccess + intervalMs >= deadline), the tick runs ONE bounded fenced VERIFY renewal. Fenced-true → starved-but-ours, lease re-extended, counter reset (audit success_after_failure with via: 'verify'); fenced-false → CERTAIN loss → lock_lost (the only certain signal); verify unreachable → defer + reconnect-once (audit failure with deadline_deferred: true), aborting only past hardEvictMs — a LOCAL decision under uncertainty bounding blind external side effects during a total outage. Four env knobs, positive-int parsed with stderr-warn-once + default fallback, then RELATIONALLY validated (margin < lease/2, callTimeout ≤ cadence, hardEvict ≥ soft deadline — warn-once clamps): GBRAIN_LOCK_RENEWAL_MAX_FAILURES (default 3, audit-labeling only), GBRAIN_LOCK_RENEWAL_CALL_TIMEOUT_MS (default min(lease/3, 15s)), GBRAIN_LOCK_RENEWAL_SAFETY_MARGIN_MS (default min(lease/6, 30s)), GBRAIN_LOCK_RENEWAL_HARD_EVICT_MS (default 2×lease, floored to the soft deadline; setting it TO the deadline approximates abort-at-deadline). Telemetry: tick lateness (now - lastTickFiredAt - intervalMs, the primary local-starvation signal — interval callbacks COALESCE under a blocked loop so missed-tick counters can't measure starvation), overlapSkips (tickInFlight re-entrancy skips only), failure-cause classification via the named RenewalCallTimeoutError (call-timeout | refused | fenced-lost), optional deps.loadSnapshot (try/caught — telemetry must never throw into control flow) and deps.onRenewalSuccess (worker resets its event-loop-delay histogram). Elapsed-time arithmetic runs on the injected deps.now, which production binds to performance.now() (monotonic) — the local clock only schedules WHEN to verify, never WHETHER to evict. The race timeout also aborts the in-flight call via AbortSignal threaded to renewLock (best-effort — the fence is the correctness authority). The tick checks state.cancelled() at every await boundary (entry, post-resolve, post-throw, post-verify). Result is a tagged union: should_abort carries {cause, latenessMs, sinceLastSuccessMs, overlapSkips, load1?, cores?} and lock_lost carries {cause: 'fenced-lost', via: 'renewal'|'verify'}; the worker stashes the result as per-launch abortMeta so the grace-evict log (30s later) reports the classified cause. The in-tick verify is reconciled in-code with queue.ts's no-background-retry rationale: it is synchronous, cancelled()-guarded, and callTimeoutMs-bounded — both UPDATEs are same-token idempotent lease extensions, so a fenced row cannot gain two holders. Pinned by test/worker-lock-renewal.test.ts (hermetic state-machine suite incl. a fake-time starvation replay, the cadence-quantization pin, hard-backstop timelines, relational-clamp cases) + test/e2e/worker-lock-renewal-starvation.test.ts (real-PG foundations). LockRenewalDeps.renewLock carries an optional per-call {signal}; the timeout race aborts it so the losing UPDATE releases its slot.

  • src/core/minions/worker-exit-codes.ts — single source of truth for reserved worker process exit codes, shared by the worker (sets them) and supervisor/CLI (classify them). Exports WORKER_EXIT_RSS_WATCHDOG = 12. The RSS watchdog drain must be self-identifying: a code-0 exit is indistinguishable from a healthy queue-drain, so a code===0 → clean_exit classifier would never count it and a respawn loop would stay invisible. A distinct code makes the drain likely_cause=rss_watchdog. Code 12 is deliberately outside {0 clean, 1 runtime_error} and the 128+N signal range. Also reserves the jobs run-child codes: 13 usage/PGLite, 14 not-claimed/token-mismatch, 15 result-write-failed (result-file presence, not the exit code, classifies the normal path).

  • src/core/minions/rss-default.ts — cgroup-aware auto-sized default for the worker RSS watchdog cap. resolveDefaultMaxRssMb(opts?) / describeDefaultMaxRss(opts?) (provenance for the startup log) / readCgroupMemLimitBytes(readFile?). Used at every spawn site (jobs work, jobs supervisor, autopilot, MinionSupervisor). Formula clamp(round(0.5 × basisMB), 4096, 16384) where basis = min(cgroupLimit, totalmem). LOAD-BEARING NUANCE: plain os.totalmem() reports HOST RAM, so in a 4GB cgroup on a 126GB host it would pick 16GB, the watchdog would never fire, and the kernel OOM-killer would SIGKILL at 4GB. The cap MUST sit below the real ceiling so the graceful drain (distinct exit code, loud log) beats the kernel's silent kill; the 4096 floor applies only when it stays below the basis. Reads cgroup v2 /sys/fs/cgroup/memory.max (literal max = unlimited) then v1 /sys/fs/cgroup/memory/memory.limit_in_bytes. Explicit --max-rss (including 0 to disable) always wins. Pinned by test/rss-default.test.ts.

  • src/core/cycle/extract-atoms-drain.ts — pure single-hold bounded drain for the silent lens-phase backlog. runExtractAtomsDrain(deps, opts) over injected deps (withLock, runBatch, countRemaining, now, optional onBatch) loops bounded batches under ONE continuous lock hold, rediscovering eligibility each batch (idempotent NOT-EXISTS-on-source_hash, so content mutated by a concurrent process simply doesn't match — no cross-window stale cursor), until the backlog is empty OR the time window elapses. Returns {phase, status, extracted, skipped, remaining, batches, stopped}. Backs gbrain dream --phase extract_atoms --drain. Takes the SAME cycleLockIdFor(sourceId) the routine cycle takes (a concurrent autopilot tick genuinely defers with cycle_already_running); NO release/reacquire-between-windows primitive. The shared wiring helper runExtractAtomsDrainForSource(engine, {sourceId, windowSeconds, brainDir?, maxBatches?, onBatch?}) owns the lock+batch+count+defer wiring (dynamic imports of db-lock/cycle/extract-atoms keep the pure loop cheap to unit-test) and is the ONE drain path for three callers — gbrain dream --drain (which calls it), the extract-atoms-drain Minion handler, and autopilot auto-drain — so lock id / window / defer-on-busy can't drift. sourceId: undefined → legacy gbrain-cycle lock + 'default' extraction; a real id → gbrain-cycle:<id>. LockUnavailableError propagates to the caller (each reports the busy case its own way). Pinned by test/extract-atoms-drain.test.ts. The summary's last_error always passes through sanitizeFailureText (secret/DSN redaction, whitespace collapse, bounded): typed failure records use the source/reason caps, and the count-only firstError compatibility path is sanitized to the combined bound, so a provider payload cannot ride either path into --json output. formatDrainProviderFailure(result) renders the Minion handler's provider_failure throw (batches/remaining + last_error), so the dead-lettered job's error_text names the cause.

  • scripts/check-worker-lock-renewal-shape.sh — CI guard wired into bun run verify. Two invariants on src/core/minions/worker.ts: (1) the bug pattern lockTimer = setInterval(async ...) must NOT appear (narrowed via lockTimer = prefix so unrelated setInterval(async) calls — like the stall detector — don't false-fire), (2) runLockRenewalTick must remain referenced so the pure-function test seam survives refactors. Bug-pattern-specific by design — a future refactor to setTimeout-recursion or AbortController-based scheduling passes as long as the bug pattern stays absent. POSIX ERE + [[:space:]] for BSD-grep portability. Honors GBRAIN_LOCK_RENEWAL_SHAPE_TARGET env override for fixture-based meta-tests. Pinned by test/scripts/check-worker-lock-renewal-shape.test.ts (5 cases).

  • src/core/doctor-cause-rank.ts — pure cause-ranking for gbrain doctor. rankIssues(checks) returns non-ok checks ordered fail-before-warn then root-before-symptom then name (deterministic). ROOT_CAUSE_CHECKS / SYMPTOM_CHECKS are ORDERING ONLY — tier membership asserts no causality. downstream_of is set ONLY from a small map of KNOWN grounded edges (queue_health / supervisorworker_oom_loop, since they read the same aborted: watchdog / rss_watchdog source) AND only when the named root is itself failing — never a root×symptom cartesian (co-occurrence never implies causality). fix prefers details.fix_hint else the message. CAUSE_GRAPH_NAMES + allKnownCheckNames() back a drift guard asserting every graphed name is a real check. Consumed by computeDoctorReport (top_issues field, additive, schema_version stays 2) + the "Top issues (ranked by cause)" header in outputResults. Pinned by test/doctor-cause-rank.test.ts.

  • src/core/audit/pool-recovery-audit.ts — reap/reconnect audit on the shared audit-writer primitive. Events: reap_detected (CONNECTION_ENDED), reconnect_other (network/auth/health-check), reconnect_succeeded, reconnect_failed. readRecentPoolRecoveries(hours=1) returns {reaps, recoveries, failures, others, events}. Error summaries route through redactConnectionInfo before truncation (DSN/host/IP safe). Emitted ONLY from PostgresEngine.reconnect(ctx?) (the rare reap-retry path, near-zero hot-path cost); reconnect() classifies the threaded error via isConnectionEndedError (in retry-matcher.ts) so only true pooler reaps are labeled reap_detected. The retry callback in retry.ts threads the triggering error as (ctx?: {error?}) => Promise<void>. Pinned by test/audit/pool-recovery-audit.test.ts.

  • src/core/audit/db-disconnect-audit.ts — JSONL audit for every call to db.disconnect() and PostgresEngine.disconnect(). Built on audit-writer.ts. Schema: {ts, engine_kind: 'postgres'|'pglite'|'unknown', connection_style: 'module'|'instance'|'unknown', caller_stack, command, pid}. caller_stack captured via new Error().stack truncated to ~20 frames so operators identify the offending caller without inflating JSONL. Privacy: stack frames carry file paths but NO SQL content / row data / user strings. File: ~/.gbrain/audit/db-disconnect-YYYY-Www.jsonl (honors GBRAIN_AUDIT_DIR). readRecentDbDisconnects(hours=24) walks current + previous ISO week and returns {count, most_recent_caller, files_scanned}. Wired into src/core/db.ts:disconnect and src/core/postgres-engine.ts:disconnect, logging BEFORE the early-return branches so even no-op disconnects on never-connected engines are recorded (that case may itself be a caller-side bug). Pinned by test/db-disconnect-audit.test.ts (6 cases: round-trip, stack truncation, sort order, empty-dir nulls, stable feature name, EROFS best-effort).

  • src/core/facts/queue.ts:FactsQueue.drainPending — method drainPending({timeout?: number}): Promise<{drained, unfinished}>. Semantically distinct from shutdown() (which calls this.internalAbort.abort() and would abort the very facts:absorb worker trying to log its post-completion event). Drain lets in-flight finish; only the wait is bounded. Default timeout 1000ms so commands that don't enqueue facts pay one fast 0ms check before exit. src/cli.ts op-dispatch finally block awaits getFactsQueue().drainPending({timeout: 1000}) BEFORE engine.disconnect(). Lazy-import keeps the facts-queue module off the hot path for ops that never touch it. Prevents a trailing 'No database connection' line after gbrain capture (a post-page-write facts:absorb outliving the CLI process). Pinned by test/facts-queue-drain-pending.test.ts (4 cases: empty fast-path, in-flight settled without abort, unfinished count on timeout, default timeout = 1000ms).

  • scripts/check-no-double-retry.sh + scripts/check-batch-audit-site.sh — CI lint guards wired into bun run verify. The former greps src/ for withRetry(...engine.{addLinksBatch|addTimelineEntriesBatch|upsertChunks}) patterns and fails the build on hit (prevents 3×3=9 retry amplification on incomplete reverts); its single-line pattern crosses arrow-callback parens (the canonical banned shape is withRetry(() => engine.addLinksBatch(...)) — a paren-stopping regex cannot see it), and its multi-line fallback runs under perl (always present locally and in CI). Both are proven fail-able by the guard self-test fixtures under test/fixtures/guards/. The latter extracts every string-literal auditSite: '...' from src/ and validates each appears in the BATCH_AUDIT_SITES const in src/core/retry.ts (typo guard — prevents fragmented doctor output).

  • src/core/fail-improve.ts — Deterministic-first, LLM-fallback loop with JSONL failure logging and auto-test generation.

  • src/core/transcription.ts — Audio transcription: Groq Whisper (default), OpenAI fallback, ffmpeg segmentation for >25MB.

  • src/core/enrichment-service.ts — Global enrichment service: entity slug generation, tier auto-escalation, batch throttling. Write path is trust-gated: enrichEntity / enrichEntities / extractAndEnrich take EnrichmentTrustOptions { trusted?, sourceId? }; only an explicit trusted: true writes authoritative people/ / companies/ stubs. Anything else (undefined/false — fail-closed, mirroring OperationContext.remote) creates the stub with the extraction quarantine markers from src/core/extraction-review.ts and reports quarantined: true in EnrichmentResult. The ONLY sanctioned op surface is extract_entities (operations.ts), which grants trusted solely for ctx.remote === false callers passing --trusted-extraction.

  • src/core/extraction-review.ts — Extraction quarantine lane markers, sibling of src/core/quarantine.ts / embed-skip.ts (frontmatter-key pattern, no schema migration). Auto-extracted stubs from untrusted input carry the PAIR provenance: 'auto-extracted' + status: 'unverified' (both required — user pages with their own status/provenance never match). Exports quarantineMarkers(), isUnverifiedExtraction() (JS predicate) and unverifiedExtractionFragment(alias) — the single SQL source of truth consumed by buildSourceFactorCase (namespace source-boost guard), both engines' getUnverifiedExtractionPageIds, the extraction_pending op, and the unverified_extractions doctor check, so filter and marker keys can never drift. Consequences: unverified stubs are excluded from the compiled-truth fusion boost + the people//companies/ source-boost (rank as ordinary content), stamped unverified: true in search results (stampUnverifiedExtractions, hybrid.ts), listed by extraction_pending, promoted (status → verified, provenance kept for audit) or rejected (soft-delete) by the owner-only extraction_review op. Pinned by test/extraction-review.test.ts (PGLite) + test/e2e/extraction-review-postgres.test.ts (live Postgres parity).

  • src/commands/enrich.ts + src/core/enrich/thin.ts + src/core/cycle/enrich-thin.tsgbrain enrich --thin: batch-develops stub (thin) pages via brain-internal grounded synthesis. gbrain's model tooling sees only brain-internal context (search / get_page / facts / backlinks), not the web, so enrich consolidates what the brain ALREADY knows about an entity (scattered across meetings, other pages, deals, facts) into one cited page via ONE gateway.chat call per page; web research stays the agent-driven enrich SKILL's job. runEnrichCore(engine, opts, signal) (strict per-source; multi-source iteration is the caller's job) drives enrichOne per candidate: withRefreshingLock('enrich:<src>:<slug>')getPage → deterministic retrieve (hybridSearch + getBacklinks + facts + raw_data, source-scoped, sanitized via INJECTION_PATTERNS) → assessGrounding gate (skip < MIN_CONTEXT_CHARS, no LLM) → buildEnrichPrompt (grounded dossier, [Source: slug] citations, SKIP sentinel) → synth → put_page handler (remote:false, auto-link + write-through) stamping enriched_at + enriched_by:'cli:enrich'. Candidate selection is the SQL-native engine.listEnrichCandidates(opts) (src/core/engine.ts interface + EnrichCandidate/EnrichCandidatesOpts/ENRICH_ORDER_SQL in src/core/types.ts + pg/pglite impls): thin-filter + per-page source-correct inbound count (to_page_id = p.id, mentions excluded) + enriched_at recency guard + whitelisted ORDER BY + LIMIT, lightweight projection (NO bodies). Resume via src/core/op-checkpoint.ts (local enrichFingerprint); budget via BudgetTracker + withBudgetTracker (best-effort under --workers > 1runSlidingPool aborts new claims on BUDGET_EXHAUSTED but does NOT cancel in-flight gateway.chat; pin --workers 1 for a hard ceiling). sanitizeContext (thin.ts) neutralizes the <context>…</context> data-envelope delimiters (injection escape, mirrors the </trajectory> convention); the --background multi-source fan-out idempotency key carries the run fingerprint via exported backgroundIdempotencyKey(sid, args) (a bare enrich:${sid} would return stale completed jobs); runEnrichCore flags budget_exhausted post-hoc when tracker.totalSpent > tracker.cap even when the gateway swallowed the final-call throw (via read-only BudgetTracker.cap getter); body() flushes the checkpoint on BudgetExhausted before it propagates so resume doesn't re-charge. The opt-in enrich_thin cycle phase (default OFF via cycle.enrich_thin.enabled) trickles max_pages_per_tick (default 3) per source with per-source cost cap enforced as min(per_source_cap, brain_wide_remaining) + brain-wide total + walltime caps. Wired into cycle.ts (CyclePhase/ALL_PHASES between conversation_facts_backfill and skillopt/embed; PHASE_SCOPE='source'; NEEDS_LOCK; dispatch), cli.ts (CLI_ONLY + CLI_ONLY_SELF_HELP + THIN_CLIENT_REFUSED_COMMANDS + dispatch), jobs.ts (Minion enrich handler, strict per-source, NOT in PROTECTED_JOB_NAMES). DI seam opts.synthesizeFn keeps tests hermetic (no API key, no mock.module). Pinned by test/enrich/thin.test.ts, test/enrich/idempotency.test.ts, test/enrich-cycle-phase.test.ts, test/e2e/enrich-pglite.test.ts (grew-cited, skip, ordering, multi-source, recency, resume, budget abort + checkpoint flush, final-call overage, lock-skip, provenance), test/e2e/engine-parity.test.ts (listEnrichCandidates pg↔pglite parity).

  • src/core/data-research.ts — Recipe validation, field extraction (MRR/ARR regex), dedup, tracker parsing, HTML stripping.

  • src/commands/embed.tsgbrain embed [--stale|--all] [--slugs ...]. --stale first calls engine.countChunklessPagesWithContent() (chunkless-page safety net: a page written directly via putPage — e.g. an enrichment-generated stub — that never went through chunking has ZERO content_chunks rows, so it has no row to go stale and is otherwise invisible to this command forever; the predicate excludes quarantined/embed_skip pages, both intentionally chunkless). When found, healChunklessPages chunks them locally (mirrors embedPage's chunkless branch: same chunkText calls over compiled_truth/timeline) with embedding = NULL, folding the new rows into the SAME pass; immediately before writing it re-fetches the LIVE page via getPage (chunks CURRENT content, not the batch-list snapshot) and re-checks getChunks, narrowing (NOT fully closing — accepted residual risk, see the function's docstring) the check-then-write window: a concurrent writer landing chunks in the gap between that re-check and the upsertChunks call can still have them overwritten with this sweep's stale-content chunks, the same window embedPage's single-page chunkless branch has. Each page's work is try/caught (a bad chunkless page records a failure via the same EmbedResult.failures/recordFailure path as every other embed failure and the sweep moves on — it never aborts the whole --stale run before the normal stale-chunk pass even starts). listChunklessPagesWithContent's default batch is 50 (not the 2000-row default elsewhere in this file) because each row carries a full page body. It honors the caller's pacer, mirrors --catch-up (removes its cap entirely, matching the main loop below), and otherwise shares ONE GBRAIN_EMBED_TIME_BUDGET_MS wall-clock budget with the main stale loop (both measure from the same overallStartedAt, not two independent 30-minute windows) so a large damaged brain can't run the combined --stale pass unbounded; an abort during healing stops the whole function before falling through to invalidateStaleSignatureEmbeddings. Pinned by test/embed-stale-chunkless-pages.serial.test.ts + test/e2e/engine-parity.test.ts. Then --stale calls engine.countStaleChunks() (single SELECT count(*) WHERE embedding IS NULL, ~50 bytes wire) so a fully-embedded brain short-circuits with no further reads beyond those two counts. When stale chunks exist, engine.listStaleChunks() returns just the chunks needing embeddings (slug + chunk_index + chunk_text + metadata, no vector(1536) payload); caller groups by slug, embeds, re-upserts via upsertChunks. Every re-embed merge carries per-chunk metadata through the ONE shared carryChunkMetadata(chunk, loaded) field list in src/core/embed-stale.tsmodality plus the code fields (language, symbol_name, symbol_type, start_line). This list is load-bearing: upsertChunks overwrites from EXCLUDED (not COALESCE), so any re-embed path that omits a field resets it — omitting modality flips every image chunk to modality='text' and silently zeroes the image search arm (its filter is cc.modality = 'image'). Never hand-roll a per-path field list. Pinned by test/embed-modality-preserved.test.ts. All console.log/console.error call sites use slog/serr from src/core/console-prefix.ts so when runEmbedCore runs inside a per-source withSourcePrefix scope (installed by the gbrain sync --all worker pool) every line carries the [<source-id>] prefix; standalone callers see identical output because slog/serr fall through to bare console fns outside the wrap. Every embed-write path stamps pages.embedding_signature via engine.setPageEmbeddingSignature(slug, {sourceId, signature: currentEmbeddingSignature()}) so a later model/dims swap is detectable as stale. The per-slug path (embedPage, used by gbrain embed <slug> AND sync's post-import embed step) and the full-re-embed path (embedAll) stamp per page when every chunk embedded cleanly. The stale path (embedAllStale) first calls invalidateStaleSignatureEmbeddings on a live run so signature-drifted pages flow through the NULL cursor, then stamps each page through stampIfPageProvenanceComplete in src/core/embed-stale.ts (shared with the minion embed-backfill drain, embedStaleForSource): the stamp is judged from DB state — every chunk carries an active-column vector whose model matches the signature and whose embedded_text_hash matches md5(chunk_text) — never from the batch subset, because listStaleChunks pages by row with no page alignment, so a page straddling a batch boundary is stamped by whichever batch lands its last chunk (a partially-stale page keeping preserved chunks of another model, or a NULL model/hash, stays unstamped rather than falsely marked current; embed --all fully re-embeds + stamps those). dry-run never mutates: it counts signature-drift via the widened countStaleChunks({signature}) predicate without NULLing anything. --include-null-signature lifts the NULL-signature grandfather clause: threads includeNullSignature: true into the invalidation + counts so pages that predate the v108 stamp re-embed too after a model swap (both engines' countStaleChunks/sumStaleChunkChars/invalidateStaleSignatureEmbeddings accept the flag; predicate becomes sig IS NULL OR sig <> current). Without the flag, a live stale run that just invalidated drifted rows probes for left-behind NULL-signature chunks and emits a loud stderr warning naming the count + the fix — mixed embedding spaces in one index are never silent. Pinned by test/embedding-migration.test.ts + test/e2e/migrate-embeddings-postgres.test.ts. Embed failures are never silent: all three page paths embed via embedPageTexts, which tries the page's chunks in one batch and, on a PERMANENT request-shaped failure (non-429, non-AITransientError, non-auth), retries once per chunk so one bad chunk costs one chunk instead of darkening the whole page (failed chunks stay embedding IS NULL for the next --stale pass; a partially-failed page is never signature-stamped). Rate-limit/outage/auth failures do NOT fan out (cost bounding — embedBatchWithBackoff already owns 429 backoff). Failed chunk counts land on EmbedResult.failures + capped failure_samples, and src/cli.ts's embed case sets a non-zero exit verdict on failures > 0 (mirror of the import errors>0 guard). Pinned by test/embed-partial-failure-3037.serial.test.ts + test/embed-exit-code-3037.serial.test.ts (real spawned CLI). applies the embed-skip filter at all 5 stale-chunk sites: runEmbedCore --stale, runEmbedCore --all, the embed-stale Minion helper, plus both engines' listStaleChunks + countStaleChunks via EMBED_SKIP_SQL_FRAGMENT. A soft-blocked page is queryable by title/slug but its chunks never enter the embed sweep. The shared helper from src/core/embed-skip.ts is the single implementation — no per-site ad-hoc filter allowed. Pinned by test/embed-skip.test.ts. both inline sliding-pool sites (embedAll simple at :458-467 and embedAllStale paginated + AbortSignal at :586-632) call runSlidingPool from the shared worker-pool helper. Invariant-level contract preserved (counts + cost + AbortSignal propagation + per-batch rate-limit retry via embedBatchWithBackoff); byte-equality on progress-event ORDERING is NOT promised. The GBRAIN_EMBED_CONCURRENCY || 20 default is preserved and embed bypasses resolveWorkersWithClamp because the 20-worker default would otherwise silently change every brain's embed hot path. Pinned by test/embed-helper-migration.test.ts (asserts the helper is wired in AND no inline let nextIdx = 0 + Promise.all(Array.from({length: numWorkers}, ...)) pool shape remains). wires --background as the reference integration for the maybeBackground() helper. gbrain embed --stale --background submits as a Minion job, prints job_id=N to stdout, exits 0. Composable: JOB=$(gbrain embed --stale --background | grep -oE 'job_id=[0-9]+' | cut -d= -f2); gbrain jobs follow $JOB. runEmbedCore accepts an optional signal threaded down both the --stale and --all paths (embedAllStale/embedAll/embedPage); each composes it with the internal wall-clock budget via anySignal and checks isAborted/effectiveSignal.aborted in every per-slug loop, page-claim pool, and embedBatch call, so a worker abort (wall-clock timeout / lock loss / SIGTERM) stops embedding within a batch. Pinned by test/embed.serial.test.ts. Keyless brains (embedding_disabled: true): the exported pure predicate isKeylessStaleRefusal(args, embeddingDisabled) gates a CLEAN refusal at the top of runEmbed — a bare stale run prints a stderr hint and returns a zero-failure result (exit 0), because the documented agent-scheduler chain gbrain sync ... && gbrain embed --stale must stay green on a keyless install; explicit asks (a slug, a slugs list, the all flag) and dry-run keep exiting 1 via EmbeddingDisabledError, mirroring the dispatch precedence where a slugs list wins over stale. Pinned by test/embed-keyless-guard.test.ts + test/agent-scheduler-contract.serial.test.ts. The stale path's signature invalidation is PROBE-GATED — a drift pre-count (two countStaleChunks calls) followed by one live probeEmbedder embed call (from src/core/embed-stale.ts; validates a real vector whose dims match the signature's trailing :<dims>) must succeed before anything is NULLed, so a misresolved config (temp GBRAIN_HOME → default model, no key) can never strip vectors it cannot replace; a failed probe warns loudly on stderr and degrades to NULL-only staleness. Every live stale run also calls engine.invalidateContentDriftEmbeddings (not probe-gated — blast radius is bounded by real content edits) so chunks whose embedded_text_hash no longer matches md5(chunk_text) re-embed from their CURRENT text; NULL hash (pre-v133 rows) is grandfathered. Pinned by test/embed-stale.serial.test.ts (probe-gate + content-drift blocks). The --stale single-flight lock HEARTBEAT (interval GBRAIN_EMBED_LOCK_HEARTBEAT_MS, default 5 min — test seam) is refresh-bounded per tick: each refresh() races a per-tick timeout (GBRAIN_EMBED_LOCK_HEARTBEAT_TIMEOUT_MS, default 30s via DEFAULT_EMBED_LOCK_HEARTBEAT_TIMEOUT_MS) with its own AbortController, so ONE never-settling refresh call can't silence the heartbeat forever; a slow tick never stacks on the next (beating guard), a refresh returning false (stolen/released lock) aborts the drain immediately as lock_lost, and 3 CONSECUTIVE tick failures (timeouts or throws) also abort as lock_lost rather than running without mutual exclusion. The interval is deliberately NOT unref'd (a lost drain promise stays a LOUD hang instead of a silent exit-0 that leaks the locks) while the per-tick timeout timer IS unref'd. runEmbedCore --stale also arms the progress-keyed stall watchdog from src/core/embed-stall.ts (GBRAIN_EMBED_STALL_ABORT_SECONDS, default 900 — see that entry): on stall it aborts the drain, releases the run's self-acquired locks, flushes the summary, and returns reason: 'stall_timeout'. Pinned by test/embed.serial.test.ts (heartbeat + stall blocks) + test/jobs-embed-stall-wiring.serial.test.ts.

  • src/core/retrieval-upgrade-planner.ts — legacy ze-switch planner, slated for deletion with the ZeroEntropy removal; CLI-unreachable (the ze-switch shim refuses every operation) and kept ONLY as a test vehicle: applyRetrievalUpgrade/resumeRetrievalUpgrade carry the multimodal-column preservation pins and env-override gate cases in test/retrieval-upgrade-planner.test.ts + test/ze-switch-env-override.test.ts. The shared primitives (runSchemaTransition, transitionDimPinnedColumn, TEXT_EMBEDDING_DIM_PINNED_TABLES, detectEnvOverride, EnvOverrideWarning, formatEnvOverrideWarning) LIVE in embedding-migration.ts; this file re-exports them until the deletion. Its resume/undo paths probe readContentChunksEmbeddingDim first and skip the schema transition when the column is already at the target width — a same-width DROP+ADD still deletes every stored vector. Note the plane caveat: apply/undo write DB-plane config (engine.setConfig) that the file-plane-canonical embed pipeline does not read.

  • src/core/embedding-migration.ts — provider-agnostic embedding migration core; owns the schema-transition + env-gate primitives. runSchemaTransition(engine, targetDim): ONE transaction rebuilds all three dim-pinned text-embedding-space columns (content_chunks.embedding, query_cache.embedding, facts.embedding) preserving each column's vector/halfvec type, HNSW gated on hnswIndexExpected (dims > 2000 skip the index; exact scans stay correct — 2048d targets work); image/multimodal columns deliberately untouched; AFTER commit it clears embedded_at in yielding 50k batches (belt-and-braces hygiene — read surfaces key on the vector itself). planEmbeddingMigration: workload via widened stale predicates (absent-column fallback counts every chunk), cost, signature_census (top-5 page signatures — DB-reality corroboration of From), synopsis_tier_pages (context-tier downgrade consent), bundle-aware reranker_warning, dim_change via the shared schemaRebuildNeeded (null ⇒ rebuild, same computation apply uses). applyEmbeddingMigration: env≠target refusal (detectEnvOverride; detectEnvPresence drives the ==target notice) → marker v2 write (same-target re-apply preserves started_at; retarget records superseded history + force_sunset_target) → full rebuild when the chunks column is off-width, else INDEPENDENT repair of stale dim-pinned columns (readDimPinnedWidths) → false-target-stamp clearing (chunk-model truth) + guarded NULL-signature-inclusive invalidation (embed_skip pages' retained vectors preserved; both via embedding-invalidation.ts) BEFORE config writes (same-dim-swap crash safety) → DB plane → file-plane callback → cache purge. verifyMigrationComplete: the ONLY basis for "nothing to migrate" — column + pinned widths, wide stale census, false-stamp census blocker (embedded chunks whose model contradicts a page already stamped with the target signature), missing_embeddings residue, chunkless-pages census, embed_skip-NULL visibility (report-only detail, never a blocker), marker state, un-merged file plane (or env-canonical), env-contradiction blocker; never trusts from_model. completeEmbeddingMigration: marker delete + completion stamp in ONE transaction (no lost-receipt crash window); accepts content-free extra (smoke-check outcome). readMigrationState (corrupt-safe), readMigrationStatus (read-only, spend-free, everything-degrades-to-null), verifySearchRoundTrip (completion smoke check: query-side embedQuery + searchVector, hit identity by page_id, NEVER throws, warn-don't-block, content-free samples), resolveRerankerExposure/resolveRerankerPlan/applyRerankerAction (D8 companion switch: bundle-resolved exposure; auto→target-provider default reranker, never a silent third provider; config write + cache purge in one tx), reconcilePageSignatures (batch-boundary stamp repair). Engine-pure; every step idempotent under crash + re-run — the NULL-embedding column is the checkpoint. Pinned by test/embedding-migration.test.ts, test/migrate-embeddings-hardening.serial.test.ts, test/e2e/migrate-embeddings-postgres.test.ts.

  • src/core/embedding-invalidation.ts — signature-truth guards shared by the migration + embed paths, engine-pure via executeRaw (identical SQL on both engines). invalidateStaleSignatureEmbeddingsGuarded(engine, {signature, sourceId?, includeNullSignature?}): same semantics as the engines' invalidateStaleSignatureEmbeddings PLUS the NOT (frontmatter ? 'embed_skip') predicate every stale selector applies — never NULL what nothing will re-embed (without it a migration would destroy vectors retained on embed_skip pages permanently and silently). ALL invalidation call sites (applyEmbeddingMigration, runEmbedCore, embedStaleChunks) route through it. countFalseStampedChunks / clearFalseStampedSignatures: the chunk-model truth cross-check — pages stamped WITH the target signature whose embedded chunks' model names another provider (bare provider-less model tails exempt, no surprise paid re-embed); count feeds plan/verify/--status honesty, clear runs in apply so the NULL-signature-inclusive invalidation re-embeds them. Pinned by test/embedding-migration.test.ts (#4305 false target stamps + #4306 embed_skip-safe invalidation).

  • src/core/ze-exposure.ts — ZeroEntropy sunset exposure detection: the one shared answer to "is this brain still depending on ZeroEntropy?", consumed by the v0_46_3 version migration (src/commands/migrations/v0_46_3.ts) and the stage-2 upgrade banner. Standalone module by design — it must outlive the ze-switch/retrieval-upgrade subsystem. detectZeExposure(engine, fileCfg?, env?) resolves exposure from the EFFECTIVE embedding model (env GBRAIN_EMBEDDING_MODEL → file embedding_model → the legacy configless runtime fallback — resolution-based, NOT vector evidence, so a configless brain is exposed even with zero vectors), the resolved reranker (through resolveSearchMode, the same plane search actually reranks with — the bundle default is Voyage, so only an explicit zeroentropyai:* search.reranker.model row exposes), and ZE-backed custom embedding_columns (file + DB planes). Tri-state status: a failed DB probe downgrades to 'unknown', never to 'clear' — callers nag on 'unknown' (fail-safe). Blast-radius counts (ZE-stamped pages, embedded chunks, est. re-embed cost at the recommended target) are LIMIT-capped at BLAST_RADIUS_CAP (100K) so a million-chunk brain can't stall apply-migrations on a full-table COUNT; blast-radius probe failures are informational and never flip status. renderZeActionRequired(exposure) renders the shared ACTION REQUIRED body both consumers wrap (migrate command with --dim 1024 + the 1280-not-a-valid-Voyage-width note, the voyage:rerank-2.5 reranker fix, the no-automated-custom-column-off-ramp honesty, and the env-override callout when GBRAIN_EMBEDDING_MODEL itself forces ZE).

  • src/commands/migrate-embeddings.tsgbrain migrate embeddings --to <provider:model> [--dim N] [--dry-run] [--yes] [--json] [--no-embed] [--status] [--reranker auto|off|keep|<model>] [--retarget] [--batch-size N] [--pace[=mode]] [--ignore-env-override] [--force-sunset-target] (alias: gbrain retrieval-upgrade, slated for removal). ONE shared orchestrator for CLI and the migrate_embeddings op: planMigrationFlow (plan + verifyMigrationComplete + env presence + un-merged file plane + brain/DB identity via redactPgUrl + concurrent-writer census + reranker plan + in-flight-other marker) and executeMigrationFlow (global gbrain-embedding-migration DbLock → retarget gate under it → all-source embed locks sorted with includeArchived → live embed probe → apply → reranker probe + switch → drain via runEmbedCore({heldLocks, …}) so the migration never lock_skips itself, with a 5-min heartbeat whose refresh-false/3-errors ABORTS as lock_lost → reconcile → completion smoke check stamped into the marker → transactional complete; locks released in finally). The skip path exits 0 ONLY on verify.complete with no pending retarget decision and no pending reranker action (a resolved switch/disable runs as a config-only completion). --status is read-only/spend-free and never refuses on env (it REPORTS planes, key PRESENCE booleans only, censuses, markers verbatim incl. corrupt, the exact resume command, and the last completion + smoke-check outcome). persistEmbeddingFileConfig writes through loadConfigFileOnly (never persists env-sourced keys) and supports env-canonical no-file deployments (env pins target ⇒ proceed with notice). Exit codes: 0 completed/verified-no-work, 1 locked/refused/failed/incomplete (message names which; lock_skipped and lock_lost get their own copy), 2 non-TTY without --yes. Pinned by test/migrate-embeddings-flow.serial.test.ts, test/migrate-embeddings-boundary.serial.test.ts, test/migrate-embeddings-hardening.serial.test.ts. Discovery surfaces all render the canonical command from renderCanonicalMigrationCommands (src/core/ai/defaults.ts): gateway deprecation line, init warnings, upgrade ACTION REQUIRED banner, doctor provider_sunset, ze-switch refusal, advisor — drift-guarded by test/canonical-migration-command.test.ts; the shutdown date lives once as ZEROENTROPY_SUNSET_DATE in defaults.ts.

  • src/commands/ze-switch.ts — pure refusal/redirect shim for the sunset ZeroEntropy switch. Every invocation refuses or redirects with exit 1 and reason: 'provider_sunset', printing the off-ramp (gbrain migrate embeddings --to voyage:voyage-4 --dim 1024 --dry-run); the --json refusal envelope is {status:'refused', reason:'provider_sunset', migrate, migrate_preview, message} (live command + cost preview, each carrying an explicit --brain suffix); --undo REDIRECTS (reads the ze_switch_previous_snapshot config row and prints the exact migrate command that returns the brain to its pre-switch provider — {status:'redirected', reason:'provider_sunset', undo_command, undo_preview, message} in --json; missing/corrupt snapshot degrades to the refusal); --help answers engine-free via CLI_ONLY_SELF_HELP + the SELF_HELP_WITHOUT_ENGINE wrapper in cli.ts (exit 0, truthful sunset copy). Its flags stay REGISTERED (quoted literals feed the generated registry row) so existing scripts reach the refusal instead of a pre-dispatch unknown-flag error — the shim never consults them. Nothing here mutates the brain. The whole command is slated for removal after the sunset. Pinned by test/ze-switch-cli.test.ts + test/cli-help-without-brain.serial.test.ts.

  • src/commands/providers.tsgbrain providers list | test [--touchpoint T] [--model ID] | env <id> | explain [--json]: provider-recipe discovery + smoke-testing over the recipe registry. list renders formatRecipeTable against the SAME env the gateway actually sees (buildGatewayConfig(cfg).env, file-plane keys folded in) so the STATUS column matches what providers test and init would report. Home of the ONE shared sunset-marker primitive (sunsetMarkerText/sunsetMarker, generic on recipe.sunset — any future provider sunset inherits it) consumed by all three human-facing renderings so they can't drift: the list status cell, explain rows (lead marker is ⚠ regardless of key readiness — never a green ready-check on a sunsetting provider), and the env block, where the pure formatEnvOutput(recipe, env) (testable without spawning the CLI) replaces the signup funnel (setup_url/setup_hint) with the deprecation notice, replacement models, and the canonical migration command from renderCanonicalMigrationCommands — key STATUS still renders for existing users. Pinned by test/providers.test.ts.

  • src/core/conversation-parser/ — 19-pattern built-in chat-format registry + opt-in LLM polish/fallback. Modules: types.ts (PatternEntry + ParseResult + DateContext + CaptureMap + TimezonePolicy), builtins.ts (19 hand-vetted patterns sourced from public format docs — iMessage/Slack, Telegram bracket + text-export, bold-paren-time, bold-time-dash, bold-name-no-time, chatgpt-export-you-chatgpt, Discord classic + export, WhatsApp ISO + US, Signal, Matrix/Element, IRC classic + weechat, Teams export, markdown-heading-turn; module-load validation runs every test_positive[] + test_negative[] sample at startup so a typo in any built-in regex makes gbrain refuse to start; DEFAULT_SPEAKER_CLEAN exported as a module-level default), parse.ts (orchestrator with pattern-priority scoring across the first 10 lines + date derivation chain explicit > frontmatter.date > effective_date > '1970-01-01' + multi-line continuation + timezone warning; also populates ParseResult.unrecognized_headings — when the WINNING pattern is heading-anchored, heading-shaped lines whose label is outside the pattern's speaker set FOLD into the previous turn's body (or drop before the first anchor) while the parse still returns regex_match, silently crediting one speaker with another's words; detection is diagnostic-only and unconditional (not behind opts.diagnostic — the extractor's decline gate depends on it), fence-aware (a fence closes only on ITS OWN marker; an unclosed fence suppresses detection below it), labels deduped/capped at 10 entries ≤48 chars, undefined when empty so healthy-page JSON stays byte-identical; gbrain conversation-parser scan surfaces the field in both human and JSON output), llm-base.ts (shared runLlmCall<T> with content-hash cache in-process + DB-persistent via migration v97 + 4-strategy JSON repair + Anthropic-key probe; polish and fallback are thin wrappers), llm-polish.ts (opt-IN; headroom guard skips when tracker within $0.10 of cap; pure applyPolish for merge/drop/edit ops), llm-fallback.ts (opt-IN; NO regex inference + NO persistence), eval.ts (scoreFixture + aggregateScores + parseFixtureJsonl for the fixture-corpus CI gate), nightly-probe.ts (DI-stubbed; mode-gated default tokenmax=ON, conservative/balanced opt-in; adversarial false-positive detection). Pattern bold-name-no-time (regex /^\*\*(?!\[)(.+?):\*\*\s*(.*)$/, ordered after the time-bearing bold patterns) parses **Speaker:** text with NO per-line timestamp (Circleback/Granola/Zoom), anchoring every message at T00:00:00Z of the frontmatter date (line order preserves sequence, same no-time convention as irc-classic); the (?!\[) lookahead rejects telegram-bracket **[18:37] Name:**; non-shadow is the colon-INSIDE-bold regex (NOT declaration order — parse.ts scores every candidate independently, order is only the tie-break). Because **Label:** text is a common prose idiom, the pattern sets optional PatternEntry.score_full_body: true so parse.ts recomputes the winner's acceptance score over the FULL body before the SCORING_MIN_ACCEPTANCE floor, keeping a bold-label notes page at no_match. Pattern chatgpt-export-you-chatgpt (regex /^\*\*(You|ChatGPT):\*\*\s*(.*)$/, declared just before bold-name-no-time so it wins the score tie on the two literal ChatGPT-export labels) parses ChatGPT's web-export → Markdown shape, where each turn's **You:** / **ChatGPT:** anchor is followed by a blank line and a multi-paragraph reply; bold-name-no-time alone cannot absorb that shape (multi_line: false scores a long reply's density near zero and falls below SCORING_MIN_ACCEPTANCE), so this pattern sets multi_line: true + score_continuations_as_body: true (excludes non-**-prefixed reply lines from the density denominator) + score_full_body: true (belt-and-suspenders full-body recompute of the winner). The speaker capture is a closed two-value enumeration (You or ChatGPT exactly, never an arbitrary (.+?) label), so it cannot reopen bold-name-no-time's BROAD-REGEX GUARD against notes pages. Because a bare **You:**/**ChatGPT:** heading (unlike bold-time-dash's bold-name+time+dash anchor) is a plausible label in ordinary prose ABOUT ChatGPT, the pattern also sets the optional PatternEntry.score_continuations_min_distinct_speakers: 2: scoreFromLines only grants the continuation-density-exclusion score when the anchor lines that fully match regex collectively capture at least that many DISTINCT speaker_group values, so a page with only a solitary **You:** heading (or several repeats of the SAME role) falls back to the ordinary flat-density score instead of getting the same acceptance immunity a genuine two-party exchange gets. It ALSO sets PatternEntry.score_continuations_max_preamble_lines: 5, because distinct-speaker count alone still lets ONE genuine **You:**/**ChatGPT:** pair embedded anywhere inside an otherwise unrelated long document (both roles present) through: scoreFromLines tracks the index of the FIRST fully-matching anchor line and only grants the immunity when that index is at or before the bound (tolerating a short title/heading preamble, not an arbitrary amount of unrelated prose before the transcript). Pattern bold-paren-time parses **Speaker** (HH:MM): text and (HH:MM:SS) (date_source: frontmatter). Fallback gates: SCORING_HEAD_TRIGGER_THRESHOLD = 0.3 triggers a full-body re-score when the head pass scores below that; SCORING_MIN_ACCEPTANCE = 0.05 blocks essay false-positives. Exported scorePatternFull(body, entry); private getNonBlankLines(body, headCap?) + scoreFromLines(lines, entry) DRY the quick_reject+regex loop. CLI surfaces at src/commands/eval-conversation-parser.ts (gbrain eval conversation-parser <fixture.jsonl> exit 0/1/2, wired into bun run verify via check:conversation-parser) and src/commands/conversation-parser.ts (scan <slug> debug, list-builtins, validate <file>). Doctor checks: conversation_format_coverage, progressive_batch_audit_health, conversation_parser_probe_health. Pinned by test/conversation-parser/{parse,llm-base,llm-fallback,llm-polish,nightly-probe}.test.ts + the 27-case baseline at test/extract-conversation-facts.test.ts (back-compat invariant). Migration v97 (conversation_parser_llm_cache_table). Fixtures at test/fixtures/conversation-formats/{imessage,imessage-time-only-12h,telegram-bracket,whatsapp-iso,whatsapp-us,signal-export,irc-classic,irc-weechat,matrix-element,teams-export,all,adversarial,bold-name-no-time,bold-time-dash,chatgpt-export-you-chatgpt}.jsonl with scripts/check-fixture-privacy.sh banning real-name leaks. Maintainer guidance: conversation parser patterns.

  • src/core/progressive-batch/ — shared ramp-up + cost-cap + verification primitive (trial 10 → ramp 100 → ramp 500 → full, with verification at each stage), with verifier+policy injection (callers describe HOW TO MEASURE SUCCESS, not WHEN TO WAIT FOR CTRL-C). Modules: types.ts (Stage, StageVerdict, AbortReason, discriminated Verifier union OutputCountVerifier | IdempotentMutationVerifier | NoopVerifier, Policy, StageReport), orchestrator.ts (runProgressiveBatch(items, verifier, policy, runner) — reads getCurrentBudgetTracker() ahead of Policy.maxCostUsd fail-closed; null both ways triggers abort_cost_cap reason='no_budget_safety_net'), audit.ts (ISO-week JSONL at ~/.gbrain/audit/progressive-batch-YYYY-Www.jsonl via the shared audit-writer primitive), stage-report.ts (ASCII formatter for the default Policy.onStageReport). Env knobs: GBRAIN_PROGRESSIVE_BATCH_DISABLED=1, GBRAIN_PROGRESSIVE_BATCH_AUTO=1 (skip Ctrl-C grace), GBRAIN_PROGRESSIVE_BATCH_STAGES=10,100,500. Sites that "jump straight to full" stay that way by default; ramp is opt-in per-site via Policy.interactiveAbortMs > 0. Pinned by test/progressive-batch/orchestrator.test.ts (35 cases, every verdict path).

  • src/commands/extract-conversation-facts.ts + src/core/cycle/conversation-facts-backfill.ts — bulk fact extraction for long-form conversation pages. Walks conversation/meeting/slack/email/imessage/imessage-daily pages, splits them into time-windowed segments (30-min gap or 30-msg cap), prepends a topical/temporal header, and uses the strict extractFactsFromTurnWithOutcome() path so provider and output failures remain retryable instead of becoming successful empty pages. Invariants: strict per-source core (runExtractConversationFactsCore({sourceId, ...}) always takes one sourceId; CLI + cycle phase each do their own multi-source iteration because PHASE_SCOPE='source' is taxonomy-only); bounded two-phase enumeration (paginated listPages({type, sourceId, limit:10}); per-page body cap MAX_PAGE_BODY_BYTES=25MB); page-global row_num accumulator (the facts unique index is (source_id, source_markdown_slug, row_num)); versioned snapshot-bound outcomes (cli:extract-conversation-facts:terminal:v2 for complete pages and a separate non-extractable:v2 source for recognized pages with no eligible segment); operation checkpoints are scheduling hints only and never suppress a replay without a matching v2 outcome; optional opts.budgetTracker? is used as-is, while an absent tracker is created with maxCostUsd; body reads cover compiled truth, timeline, and configured raw-transcript sidecars; facts.extraction_enabled kill-switch with --override-disabled; --types LIST allowlist (conversation,meeting,slack,email,imessage,imessage-daily); --background via maybeBackground; and speaker-shaped-fold decline — when the parse reports unrecognized_headings containing a speaker-shaped label (1-2 title-cased words, not in the doc-heading stoplist — the stoplist gates the DECLINE only, so a miss is warn-noise, never data loss) AND the parse produced fewer than two distinct speakers, the page is declined instead of extracted under wrong attribution: counted in pages_skipped_unrecognized_speaker (Result/CLI/cycle surfaces), warned to stderr with the folding pattern id, and deliberately NON-terminal (no durable audit row and no orphan cleanup, so a future parser/pattern fix retries the page); multi-speaker pages with folds proceed warn-only. The companion conversation_facts_backfill cycle phase is default-off, iterates every source, and enforces per-source plus brain-wide cost and wall-time caps. Migration v94 provides the partial facts index used by outcome lookups. computeConversationFactsBacklogCheck reports fresh completed, scanned-not-extractable, and unfinished counts separately, warning when more than 10 eligible pages lack a fresh v2 outcome. sources audit exposes facts_backfill_estimate: {pages, est_segments, est_cost_usd, types}. Pinned by test/extract-conversation-facts.test.ts and test/doctor-conversation-facts-backlog.test.ts. --workers N for LLM-bound fact extraction over conversation pages, with a per-page advisory lock via src/core/db-lock.ts:withRefreshingLock (lock id extract-conversation-facts:<source>:<slug>, TTL PER_PAGE_LOCK_TTL_MINUTES=2 with 20s refresh via Math.max(15s, 120s/6); LockUnavailableError triggers skip-and-continue with rate-limited log per (source, minute) + pages_lock_skipped counter + CLI exits 3 when non-zero AND no hard failures). deleteOrphanFactsForPage(engine, sourceId, slug) provides delete-orphans-first replay safety — wipes facts from a prior crashed run for this (sourceId, slug) before re-extracting, so a terminal audit row can never certify a partial insertFacts failure. assertFactsEmbeddingDimMatchesConfig(engine) is the startup preflight (throws FactsEmbeddingDimMismatchError with paste-ready ALTER hint BEFORE the first insert; cached per engine via WeakMap). Result type carries pages_lock_skipped + orphan_facts_cleaned. Checkpoint state is a shared cpMap: Map<slug, endIso> (NOT a per-page-mutated cpEntries: string[]) so atomic Map.set survives parallel workers. Minion handler extract-conversation-facts in src/commands/jobs.ts round-trips workers via job.data.workers for --background --workers 20. Cycle config key cycle.conversation_facts_backfill.workers (default 1; opt-in concurrency under brain-wide cost + walltime caps). Pinned by test/extract-conversation-facts-workers.test.ts + the existing extract-conversation-facts behavioral tests. with src/commands/doctor.ts durable outcome authority: page completion survives operation-checkpoint GC through versioned terminal audit rows (cli:extract-conversation-facts:terminal:v2), while recognized pages with no eligible segment use the separate cli:extract-conversation-facts:non-extractable:v2 source. Each outcome is bound to the exact parsed snapshot: regular pages use content_hash plus the UTC effective date; raw-conversation sidecars and legacy null-hash pages use a canonical SHA-256 over every parser-relevant input. Selection checks the token before locking, refetches under the lock, and verifies it again before writing the outcome, so an edit cannot be certified by stale work. The strict extraction path treats provider, refusal, truncation, malformed/schema-invalid output, segment-write, cleanup, and terminal-write failures as unfinished work; bulk failures increment pages_failed, affect CLI/cycle receipts and exit status, and never advance the legacy checkpoint. Checkpoints are only a scheduling hint: a slug without a matching v2 outcome is replayed delete-first. no_match, errors, cancellation, and dry runs never become durable negatives. Result, CLI, cycle, and doctor surfaces keep completed, scanned-not-extractable, unfinished, failed, and lock-skipped counts separate. See Conversation backfill durable outcomes for the operator and maintainer contract. Pinned by test/extract-conversation-facts.test.ts and test/doctor-conversation-facts-backlog.test.ts.

  • src/core/facts/conversation-types.ts — single-source conversation-type allowlist (ALLOWED_TYPES: conversation, meeting, slack, email, imessage, imessage-daily + AllowedType). Every consumer derives from this frozen leaf module — extract-conversation-facts.ts (which re-exports both names verbatim for its existing importers), the cycle backfill phase, doctor.ts + doctor/checks/search-eval.ts, sources.ts, jobs.ts — so no hand-copied list can drift. It lives under src/core/facts/ (not src/commands/) because scripts/generate-flag-registry.ts scans option-shaped literals one relative-import level deep — importing the constant straight from extract-conversation-facts.ts would transitively attribute that command's whole option surface to doctor/jobs/sources in the generated CLI_ONLY registry. Drift-guarded by test/conversation-facts-type-allowlist-drift.test.ts.

  • src/core/link-extraction.ts — shared library for the graph layer. extractEntityRefs (canonical) matches [Name](people/slug) markdown links and Obsidian [[people/slug|Name]] wikilinks; extractPageLinks, inferLinkType heuristics (attended/works_at/invested_in/founded/advises/source/mentions), parseTimelineEntries, isAutoLinkEnabled. Markdown links, bare-slug prose refs, and slash-shaped wikilinks match ANY dir-shaped path (ANY_DIR_SEGMENT), not a directory whitelist — nonexistent targets are dropped by the persist paths' page-existence checks (resolveCandidateSources, put_page's allSlugs filter, addLinksBatch INNER JOINs) and counted as skippedMissingTarget in the extract summaries; the DIR_PATTERN whitelist is only the typed fast-path for pass-2b wikilinks (non-whitelisted [[dir/...]] get an equivalent direct typed candidate in pass 2c, plus the flag-gated suffix rescue for non-exact matches). Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts. Opt-in global-basename wikilink resolution (default off): WIKILINK_GENERIC_RE catches bare [[name]] wikilinks outside DIR_PATTERN (third pass 2c in extractEntityRefs); EntityRef.needsResolution: true tags refs from this pass (the ref's slug is the wikilink TARGET, name the optional display alias). SlugResolver gains optional resolveBasenameMatches(name): Promise<string[]> (multi-match by design — emits one edge per matching page). The single shared basename matcher is buildBasenameIndex(slugs) + queryBasenameIndex(index, name) + normalizeBasename (keys raw/lower/slugified tail, stable-sorted shorter-first then lexical), used by makeResolver, the FS resolveBasenameMatchesFromSlugs, AND the doctor check so they cannot drift. makeResolver(engine, {mode, sourceId}) builds the index lazily via engine.getAllSlugs({sourceId}) — source-scoped so a bare [[name]] never resolves to a same-tail page in a different source. extractPageLinks gains opts.globalBasename (routes needsResolution refs through resolveBasenameMatches keyed on ref.slug, emits candidates tagged linkType: 'wikilink_basename' + linkSource: 'wikilink-resolved', skips self-loops) and opts.skipFrontmatter. All three surfaces (FS extract, DB extract, put_page auto-link) tag provenance with link_source='wikilink-resolved'; put_page includes it in its reconcilable-edge set so stale basename edges are removed when the wikilink or the flag goes away. Exports WIKILINK_BASENAME_LINK_TYPE + isGlobalBasenameEnabled(engine) (resolution order: env GBRAIN_LINK_RESOLUTION_GLOBAL_BASENAME → DB config link_resolution.global_basename → default false). gbrain doctor's link_resolution_opportunity check surfaces a paste-ready enable hint when ≥5 bare wikilinks would resolve AND ≥20% match. Migration v113 widens links_link_source_check to admit 'wikilink-resolved'; v114 then opens it to any kebab-case provenance (^[a-z][a-z0-9]*(-[a-z0-9]+)*$, ≤64 chars) so external derivers register their own tag (e.g. citation-graph) without a migration. LINK_EXTRACTOR_VERSION_TS also lives here (bump like CHUNKER_VERSION to invalidate prior extract-stale stamps). Pinned by test/link-extraction.test.ts, test/extract-fs.test.ts, test/doctor.test.ts, test/e2e/global-basename-pglite.test.ts.

  • src/commands/extract.tsgbrain extract links|timeline|all [--source fs|db] [--source-id <id>]: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use for live brains with no local checkout). No in-memory dedup pre-load — candidates buffered 100 at a time and flushed via addLinksBatch / addTimelineEntriesBatch; ON CONFLICT DO NOTHING enforces uniqueness at the DB layer, created counter returns real rows inserted. ExtractOpts.slugs?: string[] enables incremental extract via extractForSlugs() (single combined links+timeline pass); the cycle path threads sync's pagesAffected through. walkMarkdownFiles(brainDir) still runs to build allSlugs for link resolution. --source-id <id> scopes extraction to one source on federated brains (resolved via resolveSourceWithTier() before any SQL; failures hint gbrain sources list). gbrain extract --stale [--source-id <id>] [--catch-up] [--dry-run] [--json] branch (extractStaleFromDB) — incremental DB-source link+timeline sweep over pages whose pages.links_extracted_at watermark is stale. Embedded callers (the cycle) pass quiet: true so the helper writes nothing to stdout, including its JSON result; standalone CLI output is unchanged. In jsonMode every fs path, extractForSlugs included, reports a dropped flush as a {event:'batch_error'} stderr line rather than going silent. Stale predicate (shared by both engines + the doctor check): links_extracted_at IS NULL OR links_extracted_at < LINK_EXTRACTOR_VERSION_TS::timestamptz OR updated_at > links_extracted_at (the updated_at arm catches MCP put_page / sync --no-extract edited-since-extract). Three BrainEngine methods (parity in postgres-engine.ts + pglite-engine.ts + bootstrap probes): countStalePagesForExtraction(opts?), listStalePagesForExtraction({batchSize, afterPageId?, sourceId?, versionTs?}) (returns page CONTENT to avoid N+1 getPage; rowToStalePage in utils.ts maps the row, StalePageRow in types.ts), markPagesExtractedBatch(refs, defaultExtractedAt) (3-array unnest slug[],source_id[],ts[]; each ref may carry its own extractedAt). STALE_BATCH_SIZE default 25 (GBRAIN_EXTRACT_STALE_BATCH; small because page bodies are unbounded — the LIMIT is the only fetch-time memory bound); STALE_TIME_BUDGET_MS 30min wall-clock (--catch-up removes the cap). Non-swallowing flush: link/timeline flush throws propagate and abort the batch; stamp LAST so a crash leaves pages unstamped and they re-extract idempotently (addLinksBatch ON CONFLICT DO NOTHING + timeline dedup). extractStaleFromDB stamps with each row's READ updated_at (not now()), so a concurrent edit during the sweep keeps the page stale and it re-extracts next run rather than marked fresh-with-old-content. Source-correct stamping at DB-extract sites via stampExtracted (best-effort, never throws); extractLinksFromDB only stamps the combined watermark when subcommand === 'all' (a links-only run must not hide timeline staleness). LINK_EXTRACTOR_VERSION_TS lives in src/core/link-extraction.ts (bump like CHUNKER_VERSION to invalidate all prior stamps). Migration v112 (pages_links_extracted_at) adds nullable TIMESTAMPTZ + composite (source_id, links_extracted_at) index (CONCURRENTLY + invalid-remnant pre-drop on Postgres, plain on PGLite), NO backfill so the real backlog surfaces on first gbrain doctor. Schema parity in schema.sql + pglite-schema.ts + schema-embedded.generated.ts + REQUIRED_BOOTSTRAP_COVERAGE. src/commands/doctor.ts:checkLinksExtractionLag (the links_extraction_lag check, also in doctorReportRemote) warn-only by default (>GBRAIN_EXTRACTION_LAG_WARN_PCT, default 20%; shared EXTRACTION_LAG_WARN_PCT_DEFAULT + EXTRACTION_LAG_MIN_PAGES=100 + exported _resolveEnvNumber), hard-fails only when GBRAIN_EXTRACTION_LAG_FAIL_PCT is set; vacuous-skips <100 pages (no --source); pre-v112 brains graceful-skip via isUndefinedColumnError; strictly a SQL COUNT (safe on remote/thin-client). src/commands/sync.ts carries --no-extract (threaded through single-source + --all + syncOneSource), stamps links_extracted_at for pagesAffected at the inline-extract call site, and maybeExtractionNudge prints a one-line stderr nudge after a synced | first_sync | up_to_date sync that leaves a backlog (shouldNudgeAfterSync pure predicate; GBRAIN_SYNC_NO_EXTRACT_NUDGE suppresses). src/core/retry.ts lists 'extract.stale' in BATCH_AUDIT_SITES; src/core/doctor-categories.ts lists links_extraction_lag in BRAIN_CHECK_NAMES. Pinned by test/extract-stale.test.ts (incl. edited-after-stamp + crash-contract), test/sync-inline-extract-stamps.serial.test.ts, test/sync-nudge-status-gate.test.ts, test/doctor-links-extraction-lag.test.ts, engine-parity (Postgres↔PGLite) for the 3 methods + v112 round-trip. The stale SELECT in both engines projects a deterministic full-µs UTC string to_char(updated_at AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS.US"Z"') AS updated_at_iso (carried on StalePageRow.updated_at_iso, populated by rowToStalePage in utils.ts with an ISO-only fallback — never String(Date), which ::timestamptz misparses); extractStaleFromDB stamps that exact-precision value, not a JS Date (which truncates to milliseconds), so on Postgres links_extracted_at equals the row's updated_at to the microsecond and links_extraction_lag clears — a ms-truncated stamp stays strictly below the µs updated_at and leaves every page perpetually stale, which extract --stale could never satisfy. to_char (not raw ::text, which is DateStyle-fragile) keeps the projection deterministic. The markPagesExtractedBatch SQL is unchanged, so callers passing an explicit (e.g. backdated) extractedAt still control the stamp and the edited-since arm is exact. A deterministic PGLite case in test/extract-stale.test.ts injects a µs updated_at, runs --stale, and asserts the lag is 0 and stays 0.

  • Extract CLI help — EXTRACT_HELP in src/commands/extract.ts is the canonical detailed usage shared by --help and invalid-subcommand errors. src/cli.ts routes extract --help before engine connection so help works on unconfigured installs; the top-level TOOLS block advertises every mode-specific flag. Pinned by test/cli-help-discoverability.test.ts.

  • src/core/extract/receipt-writer.ts + src/core/extract/rollup-writer.ts + src/commands/extract-status.ts + src/commands/extract-explain.ts + src/commands/extract-benchmark.ts + src/core/schema-pack/scaffold-extractable.ts — unified extract operator surface. Every shipped extractor (deterministic facts.conversation in src/commands/extract-conversation-facts.ts + three LLM-backed cycle phases at src/core/cycle/{extract-atoms,synthesize-concepts,propose-takes,extract-facts}.ts) writes ONE receipt page per run (writeReceipt) + UPSERTs a row to extract_rollup_7d (upsertExtractRollup). Receipt slug extracts/{date}/{kind}/{source_id}/{run_id_short}/round-{N}.md; frontmatter stamps BOTH type: extract_receipt AND dream_generated: true (belt+suspenders against extraction-loop guard drift). extract_receipt joins ALL_PAGE_TYPES in src/core/types.ts; extracts/ prefix gets a 0.3x source-boost demote in src/core/search/source-boost.ts. Migration v104 adds extract_rollup_7d (kind, source_id, day, cost_usd, halt_count, eval_pass_count, eval_fail_count, round_completed_count, rollup_write_failures, updated_at) with PK (kind, source_id, day) + idx_extract_rollup_7d_day. Rollup writes best-effort with process-scoped error-dedup so transient DB failures bump rollup_write_failures instead of crashing the cycle. extract_health doctor check reads last 7 days, warns at halt-rate > 10% AND when rollup_write_failures > 0; pre-v104 brains report ok. CLI: gbrain extract status [--source-id ID] [--kind X] [--verbose] [--json] (7-day rollup, sorted halt_rate desc + cost desc, top-5 + "more rows" hint, stable schema_version: 1); gbrain extract --explain <kind> (resolution chain pack-declared vs built-in cycle phase, prompt_template + fixture_corpus paths with /(missing), last 7d rollup); gbrain extract benchmark --pack X --kind Y (loads pack fixture corpus through strict path validation — rejects absolute paths, .. traversal, null bytes, AND symlinks resolving outside pack root; ships as a stub-reporter). src/core/schema-pack/manifest-v1.ts widens extractable from z.boolean() to z.union([z.boolean(), ExtractableSpecSchema]) (carries prompt_template, fixture_corpus, eval_dimensions, benchmark_min_recall, plus reserved verifier_path — parses but refuses at runtime); extractableSpecsFromPack + getExtractableSpec + refuseVerifierPathInV042 in src/core/schema-pack/extractable.ts; gbrain schema scaffold-extractable <type> --pack <pack> declares the type extractable, generates 5 placeholder fixtures + a prompt template stub under packs/<pack>/{fixtures,prompts}/extract/, refuses to overwrite without --force. Pinned by test/extractable-spec-widening.test.ts (22), test/extract/receipt-writer.test.ts (12, canonical PGLite block R3+R4), test/extract/benchmark.test.ts (17), test/extract/status.test.ts (15), test/schema-pack/scaffold-extractable.test.ts (15, privacy guards), test/doctor-extract-health.test.ts (8).

  • src/commands/import.tsgbrain import <path> [--source-id <id> | --source <id>]: page import with the path-set checkpoint. runImport contains NO process.exit: all five preflight/argv failure sites (deferred-setup embedding sentinel, missing embedding credentials, invalid --workers, missing dir arg, unreadable target) throw the exported typed ImportAbortError (carries exitCode; the user-facing message is printed at the throw site), so in-process callers — the sync_brain MCP op, autopilot, the Minions import handler — survive a failed preflight as a normal tool/job error instead of the whole serving process dying mid-call. The CLI's import case catches it and exits e.exitCode. Pinned by test/import-abort-error.test.ts. Human-only output (the info() lines and the end-of-run Import complete summary) goes through slog() from console-prefix.ts — identical to console.log outside a wrap, but an in-process caller in its own --json mode (sync --jsonperformFullSyncrunImport, wrapped in withHumanLogsToStderr) keeps stdout pure JSON; import --json itself sends them to stderr directly. Pinned by test/sync-json-stdout-clean.test.ts (first-sync case) + test/import-json-stdout.serial.test.ts. --source-id <id> (alias --source <id>; passing both with different values aborts) routes pages to the named source (resolved via resolveSourceWithTier() at the boundary; consistent across import, extract, graph-query, sources current; graph-query takes --source <id>). Pinned by test/import-source-id.test.ts. An unscoped import that resolves to tier seed_default runs the default-write assessment through assessDefaultWriteGuardOnce(engine) (memoized per engine — in-process callers such as the sync_brain op, autopilot, and minion sync invoke runImport repeatedly on one engine) and WARNS via formatDefaultWriteWarning(a, '--source-id') when the brain's pages overwhelmingly live outside default — never aborts, since aborting would take an in-process host down mid-call; GBRAIN_ALLOW_DEFAULT_WRITE=1 skips it. Pinned by test/import-default-write-guard-once.test.ts. gbrain import CLI + runImport library entrypoint. Uses a path-set checkpoint via src/core/import-checkpoint.ts (the walk still applies sortNewestFirst() for embed-cost ordering, but checkpoint correctness does not depend on sort order). A file enters completed: Set<relativePath> only when its processFile returns success (including content-hash short-circuit no-ops); failed files never enter the set so the next run retries them automatically with no manual ~/.gbrain/import-checkpoint.json delete. This rules out three failure classes: parallel-import-with-slow-worker dropping the slow file on crash-resume (the slow file isn't in completed until its own processFile resolves), failed-file-bumps-counter-past-itself (failures don't add to completed), and sort-flip-drops-newest-N-on-cross-version-resume (order is not part of the checkpoint). Old positional checkpoints are detected and discarded with a stderr line on first resume (re-walking is cheap because content_hash short-circuits unchanged files). Checkpoint persists every 100 successful adds, not every 100 processed files. The managedBookmark opt (set by performFullSync when runImport is the full-sync engine) suppresses runImport's own sync.last_commit advance so the shared applySyncFailureGate (src/core/sync-failure-ledger.ts) owns the bookmark + failure-ledger gating on that path — one gate decides advance/block/auto-skip across both sync paths. Pinned by test/import-checkpoint.test.ts + test/import-resume.test.ts (incl. the SLUG_MISMATCH retry case). collectSyncableFiles' shared emit filter isCollectibleForWalker applies the SAME segment-level pruneDir gate as incremental sync's classifySync — load-bearing for the git ls-files fast path, which enumerates tracked files under dot-dirs/vendored trees that the FS walk never descends into; without it sync --full would import (and resurrect soft-deleted) pages incremental sync excludes. Pinned by test/import-git-fastpath-prune.test.ts. runImport opts also carry exclude (glob filter over dir-relative paths, threaded by performFullSync for sync --exclude; warns when every file is excluded — NAV-4) and slugRoot (slug/source_path base for monorepo subdir syncs; the resume checkpoint stays dir-relative per resumeFilter's contract). The --json payload carries, beside the counts, unchanged (content-hash no-ops), malformed_skipped, and failures: [{path, error}] — returned per-file failures (invalid frontmatter, oversize, symlink, slug mismatch) count under both skipped and errors, are excluded from unchanged, and produce status: "partial" with a nonzero CLI exit. Failed imports retain their completed-path checkpoint; queued imports report failure instead of successful completion. The failure ledger is written only for Git repositories, so the JSON failure list remains a non-Git caller's per-file channel. Pinned by test/import-json-stdout.serial.test.ts.

  • src/core/import-checkpoint.tsloadCheckpoint(brainDir), saveCheckpoint(brainDir, completed), resumeFilter(files, completed, brainDir), clearCheckpoint(), plus the ImportCheckpoint type. Path-set format {schema_version, brainDir, completed: string[]}. Atomic write via .tmp + rename() so a mid-write crash never leaves a partial JSON. loadCheckpoint returns null on: missing file, malformed JSON, brainDir mismatch (ran against a different brain), and the old positional format (logged to stderr before discard). resumeFilter returns {toProcess, skippedCount} — pure, no I/O, deterministic. clearCheckpoint is no-op-on-missing for clean-exit cleanup. Honors GBRAIN_HOME via gbrainPath() so withEnv({GBRAIN_HOME: tmpdir}) test isolation works without monkey-patching fs. Best-effort persistence — saveCheckpoint logs warnings on write errors but never throws.

  • src/commands/graph-query.tsgbrain graph-query <slug> [--type T] [--depth N] [--direction in|out|both] [--source <id>] [--include-foreign]: typed-edge relationship traversal (renders indented tree). The local walk is scoped to the resolved source (--source / GBRAIN_SOURCE / .gbrain-source / path match / brain default via resolveSourceId, the same scalar scope the traverse_graph op gets from sourceScopeOpts); an explicit --source that fails to resolve throws. --include-foreign (or --source __all__) drops the scope so the walk spans every source. The foreign-edge footer (N edges to foreign-source pages hidden; pass --include-foreign) prints only when the walk was actually scoped, so cross-source edges never disappear silently and the footer never describes a filter that did not apply. The local path uses engine.traversePathsDetailed and prints a stderr note when the walk hit TRAVERSE_PATH_ROW_CAP (shallowest edges kept; lower --depth or narrow with --type/--direction); the thin-client path goes through the traverse_graph op, which has no source_id param (the server scopes the walk to the caller's grant): an explicit --source there is rejected with exit 1 (same policy + wording as applyThinClientSourceScope, never silently dropped) and --include-foreign prints a stderr note that it is not forwarded. Pinned by test/graph-query.test.ts (local) + test/graph-query-thin-client-source-flag.serial.test.ts (thin client; mocks isThinClient + callRemoteTool).

  • src/commands/sources.tsgbrain sources {list,add,remove,archive,restore,archived,purge,current,status,audit}. current [--json] calls resolveSourceWithTier() and prints source_id, tier (flag | env | dotfile | local_path | brain_default | seed_default), and optional detail (decision table in skills/conventions/brain-routing.md). status [--json] — read-only per-source dashboard (last sync, staleness, page count, embedding coverage, unacked failures); thin wrapper around buildSyncStatusReport + printSyncStatusReport from src/commands/sync.ts; --json emits stable {schema_version: 1, sources, ...} on stdout; filters input to local_path IS NOT NULL AND archived IS NOT TRUE. audit <id> [--json] — read-only dry-run disk scan for size distribution + would-blocks + junk-pattern hits WITHOUT touching the DB; walks sources.local_path, reads each markdown file, runs assessContent() from src/core/content-sanity.ts, aggregates by verdict (ok | warn_oversize | hard_block_junk_pattern). The live runStatus health table gains a BACKFILL column between EMBED and FAILS (active(N) beats queued(N) beats idle, from SourceMetrics.backfill_active / backfill_queued in src/core/source-health.ts) so operators see deferred embed-backfill minion work after sync --all exits 0; jobCountsBySource in source-health.ts widens its minion_jobs SQL with two COUNT(*) FILTER (WHERE name = 'embed-backfill' AND ...) aggregates (best-effort, all-0 on pre-minions brains). remove pre-checks OAuth-client referents (guided refusal via formatClientReferentsBlock — the raw FK violation never reaches the operator), then commits the row DELETE atomically with an in-tx referents re-check (a registration racing the pre-check maps to the same guided refusal); durability-scaffolding teardown (unhardenBrainRepo: hook/cron/credential) runs only AFTER the commit, so a refused delete leaves the scaffolding intact. purge runs the same referents pre-check before its hard DELETE. Pinned by test/content-sanity.test.ts, test/import-file-content-sanity.test.ts, test/source-health.test.ts.

  • src/commands/sources-set-path.tsgbrain sources set-path <id> <path> [--force]: non-destructive local_path repair (a DB pointer update, never touches files on disk). Loud rejection on a missing source (never a silent 0-row UPDATE); prints the prior value before changing it so the change is visible/reversible; absolutizes relative paths + normalizes MSYS/Git-Bash drive spellings BEFORE the existence check and the UPDATE (storing . verbatim would plant the phantom-path class this command exists to fix). Enforces the shared assertNoOverlappingPath guard (src/core/sources-ops.ts) before the UPDATE — a repointed source nesting inside or swallowing another source's tree exits 6 with the same overlapping_path wording sources add uses; --force bypasses it. Lives in its own module (like sources-demo.ts) so sources.ts stays under its ratchet ceiling. The default_source_local_path doctor check names this as the repair.

  • src/commands/reindex-frontmatter.tsgbrain reindex-frontmatter. reindexFrontmatterCli(engine, args) takes the ALREADY-CONNECTED engine from cli.ts's dispatch; it must never build/connect its own engine — a second connect on the same PGLite data dir self-deadlocks on the data-dir lock (this process already holds it). Same rule applies to runBackfillCommand(engine, args) in src/commands/backfill.ts and any future command dispatched from cli.ts's engine-connected switch. Pinned by test/reindex-frontmatter-connect.test.ts (library path) and test/reindex-frontmatter-pglite-spawn.serial.test.ts (CLI dispatch seam, both commands).

  • src/core/source-config-sql.ts + src/core/sources-load.ts — canonical recovery for non-object sources.config values. The application reader unwraps nested JSON strings and merges recoverable array fragments left-to-right; the shared SQL expression mirrors that policy atomically for both engines, source config updates, archive/restore, and the paste-ready source_config_shape doctor repair. localFederatedSourceIds reads config through the same parser so stdio/CLI federation cannot silently disagree with sources list. sourceConfigHasRemoteUrl uses that parser for autopilot pull policy, including PGLite's JSON-string config shape. Invalid fragments degrade to {} rather than throwing. Pinned by test/sources-load.test.ts, test/job-pull-policy.test.ts, test/list-all-sources.test.ts, test/local-federated-search-scope.test.ts, test/destructive-guard.test.ts, and test/doctor-source-config-shape.test.ts.

  • src/core/source-resolver.ts — 6-tier source resolution. resolveSourceWithTier(engine, explicit, cwd) returns { source_id, tier: SourceTier, detail? } alongside resolveSourceId() (unchanged). SOURCE_TIER_NAMES = ['flag', 'env', 'dotfile', 'local_path', 'sole_non_default', 'brain_default', 'seed_default'] (7 entries; order matches priority). Tier sole_non_default slots between local_path and brain_default: when NO sources.default config is set AND exactly one registered source has local_path AND isn't 'default' AND the 'default' source holds no active pages (the emptiness guard — the tier's charter is rescuing brains whose 'default' is empty; when 'default' carries an established corpus, auto-routing would hijack every bare write into the side-source, so the resolver falls through to seed_default and prints a one-line stderr notice naming both sides so the suppressed flip is diagnosable, same GBRAIN_NO_SOLE_NON_DEFAULT_NUDGE=1 suppression knob as the routing nudge; a failed pages probe on an exotic/legacy schema keeps the auto-route), auto-route to it; archived sources excluded (try/catch for pre-v34 brains); private pickSoleNonDefaultSource(engine) shared by both resolver entry points so they cannot drift. Exported formatSoleNonDefaultNudge(sourceId): string | null builds the user-facing stderr nudge (null when GBRAIN_NO_SOLE_NON_DEFAULT_NUDGE=1). src/commands/sync.ts:1497-1519 calls resolveSourceWithTier unconditionally so the tier fires; src/commands/import.ts:96-128 mirrors with the tier-gated nudge. Consumed by gbrain sources current, import --source-id, extract --source-id, and the source_routing_health doctor check. Pinned by test/source-resolver-with-tier.test.ts (withEnv() per test-isolation lint), test/source-resolver-sole-non-default.test.ts (20 cases incl. both-ways emptiness-guard + probe-failure fallback), test/sync-sole-non-default-routing.test.ts (3 PGLite cases driving real runSync), test/pages-source-scoping-4329.test.ts (real-engine both-ways guard coverage). Also exports noGrantFederatedScope(engine, hasSourceGrant, sourceId), the widening decision transports share: returns localFederatedSourceIds(..., 'seed_default') ONLY when hasSourceGrant === false (the legacy-bearer no-grant floor), undefined for a granted token, for an OAuth client (flag undefined — a falsy gate here would widen every OAuth client), and when the resolver throws (best-effort; the scalar scope stands rather than failing the request). Called by src/commands/serve-http.ts; pinned by test/no-grant-federated-scope.test.ts (6 cases). Unscoped default-write guard family (the seed_default tier is the only one that actually lands a write in 'default'): assessDefaultWriteGuard(engine)DefaultWriteAssessment { shouldGuard, defaultPages, nonDefaultPages, nonDefaultSources, failed? } — an unindexed full-pages aggregate; shouldGuard when ≥1 non-default source holds pages AND non-default pages outnumber default's; a query failure returns the fail-open no-guard verdict stamped failed: true, which cachers/latchers must not treat as settled. assessDefaultWriteGuardOnce(engine) is the per-engine WeakMap memo (arms on the first SUCCESSFUL assessment whatever its verdict, forgets a failed attempt so the next caller retries, coalesces concurrent callers; __resetDefaultWriteGuardMemo is the test seam). assessUnscopedDefaultWrite(engine, tier, mutating){ warning, assessed } (assessed: false only when the aggregate failed), with maybeWarnUnscopedDefaultWrite as its warning-only projection; formatDefaultWriteWarning(a, sourceFlag?) / formatDefaultWriteRefusal(cmd, a) render the two surfaces; defaultWriteAllowedByEnv() is the GBRAIN_ALLOW_DEFAULT_WRITE=1 escape hatch. Consumers: sync refuses (dry-run warns), import warns via the memo, MCP stdio's createDefaultWriteAdvisory prints once. Also exports isResolverUserError(e) — the ONE predicate for this module's user-facing throws (Source "…" not found. / not found or is archived., Invalid --source value, Invalid GBRAIN_SOURCE value), consumed by dream.ts, agent.ts and code-scope.ts so a CLI command turns an unknown/archived source into a clean stderr line + exit 1 while anything else keeps propagating.

  • src/core/migrate.ts — schema-migration runner. Owns the MIGRATIONS array (source of truth for schema DDL). Migration interface carries sqlFor?: { postgres?, pglite? } (engine-specific SQL overrides sql) and transaction?: boolean (false for CREATE INDEX CONCURRENTLY, which Postgres refuses in a transaction; ignored on PGLite). Key migrations: v14 (handler branches on engine.kind for CONCURRENTLY-on-Postgres with invalid-remnant pre-drop via pg_index.indisvalid, plain CREATE INDEX on PGLite); v15 (minion_jobs.max_stalled default 1→5 + backfill non-terminal rows); v24 rls_backfill_missing_tables (sqlFor: { pglite: '' } no-op — PGLite has no RLS engine, targets subagent tables absent from pglite-schema.ts); v30 dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash)) (RLS-enabled under BYPASSRLS; synthesize reads/writes to avoid re-judging); v35 auto-RLS event trigger auto_rls_on_create_table fires on ddl_command_end for WHEN TAG IN ('CREATE TABLE','CREATE TABLE AS','SELECT INTO') running ALTER TABLE … ENABLE ROW LEVEL SECURITY on new public.* tables (no FORCE) + one-time backfill on every existing public.* base table whose comment doesn't match ^GBRAIN:RLS_EXEMPT\s+reason=\S.{3,} (per-table failure aborts the offending CREATE TABLE; no EXCEPTION wrap; PGLite no-op via sqlFor.pglite: ''; breaking change: intentionally-RLS-off public tables need the GBRAIN:RLS_EXEMPT comment before upgrade); v40 pages_emotional_weight (pages.emotional_weight REAL NOT NULL DEFAULT 0.0, column-only metadata-only); v46 mcp_request_log_params_jsonb_normalize (UPDATE ... SET params = params::text::jsonb WHERE jsonb_typeof(params) = 'string', idempotent); v60-v65 six-migration chain wiring source-scoping into oauth_clients — v60 (oauth_clients_source_id_fk: source_id TEXT NULL→'default' backfill + FK to sources(id) ON DELETE SET NULL), v61 (federated_read TEXT[] NOT NULL DEFAULT '{}'), v62 (explicit-CASE backfill so source_id IS NULL'{}'), v63 (fail-loud check every row's source_id is in its federated_read array), v64 (FK flipped to ON DELETE RESTRICT), v65 (GIN index for array-containment); v68 eval_candidates_embedding_column (eval_candidates.embedding_column TEXT NULL per-row provenance for gbrain eval replay to reproduce the same retrieval space; NULL-tolerant); v108 pages_embedding_signature (pages.embedding_signature TEXT NULL = <provider:model>:<dims> stamped via setPageEmbeddingSignature; GRANDFATHER — stale predicate is embedding_signature IS NOT NULL AND embedding_signature <> $current so NULL is NEVER stale and upgrade never re-embeds the whole corpus; no index; metadata-only); v109 sources_newest_content_at (sources.newest_content_at TIMESTAMPTZ durable newest-COMMIT HEAD committer time written by writeSyncAnchor, read by the REMOTE staleness path instead of shelling to git; mirror in pglite-schema.ts + schema.sql + bootstrap probe); v110 page_aliases ((id, source_id, alias_norm, slug, ...) with UNIQUE (source_id, alias_norm, slug) + lookup indexes on (source_id, alias_norm) and (source_id, slug); alias_norm is normalizeAlias() output so WRITE/READ key on the same form; also in src/core/pglite-schema.ts); v111 search_telemetry_rank1_columns (ADD COLUMN IF NOT EXISTS on both engines: sum_rank1_score, count_rank1, three buckets rank1_lt_solid/rank1_solid/rank1_high on search_telemetry — aggregate not per-query rows so rank-1 median drift is bounded-growth; ALTERs right after v57 which created the table); v114 links_link_source_check_kebab_regex (opens link_source from the closed allowlist to a kebab-case format gate ^[a-z][a-z0-9]*(-[a-z0-9]+)*$ + char_length<=64; Postgres branch uses NOT VALID + VALIDATE CONSTRAINT with transaction:false, PGLite plain DROP+ADD; existing built-ins all satisfy the regex so VALIDATE never fails on existing data); v116 code_edges_source_backfill_and_callee_index (idempotent: backfills NULL code_edges_symbol/code_edges_chunk source_id from each edge's from_chunk page — a NULL never matches a scoped AND source_id = … filter, so scoped code-callers/code-callees would return 0 rows on multi-source brains — plus plain CREATE INDEX on from_symbol_qualified for both edge tables, which had no index and seq-scanned per BFS node); v129 dream_verdicts_triage_v1_columns (additive ADD COLUMN IF NOT EXISTS widening dream_verdicts into a scored triage record — score, content_type, segments, entities, model, triage_version; legacy rows keep score NULL and read as cache misses, re-judged once; no backfill, no index; same SQL on both engines since dream_verdicts is migration-created on PGLite too, so the columns take the COLUMN_EXEMPTIONS route in test/schema-bootstrap-coverage.test.ts rather than bootstrap probes); v143 dream_verdicts_ttl (30-day expires_at TTL: nullable ADD COLUMN + SET DEFAULT run BEFORE the judged_at-derived backfill — statement order is load-bearing so a legacy writer racing the upgrade window picks up the default instead of inserting a NULL that would fail SET NOT NULL on every retry; on Postgres the schema blob's dream_verdicts_expires_idx forward-references the column, so forward-reference-bootstrap.ts probes + ALTERs it before the blob replays, and both engines' getDreamVerdict read predicate is NULL-tolerant — expires_at IS NULL OR expires_at > now() — so a pre-backfill row reads as a hit instead of silently re-judging the corpus). The dedup-index self-heal (timeline_dedup_index, see timeline-dedup-repair.ts) is NOT version-gated: runMigrations invokes repairTimelineDedupIndex on every pass (including the no-pending early-return path) because a merge-renumbered migration can leave the version counter past the index change while the index stays the old shape. retry-matcher.ts and timeline-dedup-repair.ts are static dependencies because runMigrations() executes from live engine initialization; the engine dynamic-import guard scans this file with both engine implementations. v95: pages_dedup_partial_index adds CREATE INDEX pages_dedup_idx ON pages (source_id, content_hash) WHERE deleted_at IS NULL. Postgres uses CREATE INDEX CONCURRENTLY with transaction: false + pre-drops any invalid remnant; PGLite uses plain CREATE INDEX. Powers findDuplicatePage hot path (O(log n) instead of O(n)). v74 mcp_spend_log uses BTREE on (client_id, created_at) + (token_name, created_at)date_trunc('day', TIMESTAMPTZ) is NOT IMMUTABLE so can't appear in index expressions; a created_at range scan covers the per-day rollup. v75 embedding_multimodal_column is column-only (no HNSW index — deferred to post-reindex per pgvector best practice).

  • src/cli.ts — no-DB fallbacks announce themselves on stderr instead of degrading silently. dream dispatch binds the caught engine-connect error and emits [dream] WARNING: could not connect to DB (...) before falling through to filesystem-only phases; the runDream(null, ...) no-DB fallback is preserved (pinned by test/cli-dream-engine-warn.test.ts, 2 subprocess cases against good + bad DATABASE_URL). doctor dispatch does the same: when connectEngine OR the DB-backed runDoctor run throws, it emits [doctor] DB-backed doctor run failed (...) — falling back to filesystem-only checks, scrubbing the error message through BOTH url-redact.ts:redactUrlsInText and redact-connection-info.ts:redactConnectionInfo first, because doctor output is exactly what users paste into issues and CI logs (pinned by the fallback case in test/doctor-minions-check.test.ts: stderr note present, credentials absent, stdout stays parseable --json).

  • skills/conventions/brain-routing.md — agent-facing convention skill documenting the canonical 6-tier source resolution chain (flag → env → dotfile → local_path → brain_default → seed_default) with paste-ready decision tables. Linked from CLAUDE.md's "Two organizational axes" section and from gbrain sources current's hint output.

  • test/operations-trust-boundary.test.ts + scripts/check-operations-filter-bypass.sh — operations trust-boundary contract coverage. Pure assertions over all 74 ops (every op has a scope annotation; every mutating op has a non-read scope; localOnly: true ops are excluded from operations.filter(op => !op.localOnly); the seven sensitive localOnly ops snapshot-pinned by name) plus targeted handler-invocation guards for the HTTP-callable classes: submit_job with name='shell' + ctx.remote=true MUST reject (shell-job RCE); search_by_image with image_path + ctx.remote=true MUST reject (image leak); and the list-op read-leak class — find_orphans/get_recent_salience/find_anomalies/find_experts invoked with ctx.remote=true over a seeded visibility: private page MUST NOT return its slug/title, with a local positive control proving the assertions non-vacuous, count self-consistency assertions pinning the no-count-oracle posture (both orphan denominators shrink with hidden rows; excluded untouched; anomaly count mirrors the visible slug list), and a soft-delete re-probe pinning the includeDeleted fail-closed path. file_upload and sync_brain omitted from handler-invocation tests because they're localOnly: true (that path would test an impossible production scenario). The shell guard greps src/ for any module importing the operations value outside the canonical filter site at src/commands/serve-http.ts (three import shapes: destructured, aliased, namespace), with an explicit 10-entry allow-list + a literal-string check that serve-http.ts still contains operations.filter(op => !op.localOnly). Wired into bun run verify. Dynamic sibling: test/remote-privacy-sweep.test.ts (next entry) — same doctrine, corpus-seeded, catches leaks in ops neither curated list has heard of yet.

  • test/remote-privacy-sweep.test.ts — registry-driven, corpus-seeded leak detection across the ENTIRE dispatch surface (catches a new remote surface that forgets the world-only filter). Seeds a hermetic PGLite corpus carrying high-entropy PRIVATE sentinels (private page title/body, a private fact, a non-world take, fence-row content — never slugs, which are legitimately echoed in "Page not found" errors and would false-positive on correct behavior) plus WORLD markers as positive controls; enumerates every non-localOnly op from the registry; dispatches each one remote-shaped through dispatchToolCall (the exact layer both MCP transports share) in BOTH caller shapes — scalar source-scoped (like stdio) and federated with an auth-array read grant; and asserts no private sentinel appears anywhere in the serialized response envelope: structured fields, rendered text, error messages, and the _meta.brain_hot_memory channel alike. Fail-closed maintenance contract: EXPECTED_OUTCOME's key set must be set-equal to the enumerated registry, so a NEW op FAILS the suite until classified (data = must prove corpus contact via a WORLD marker / ok / error — envelope still sentinel-checked / denied = publish-gated, must error naming its mcp.* gate; plus a PARAM_FACTORY entry if it can return corpus data — the failure message prints exactly what to add). A separate arm asserts every localOnly op is denied fail-closed over non-stdio transports; KNOWN_EXPOSED op-wide exemptions must stay empty absent written justification + review. Phase W covers mutating ops against fresh slugs (destructive ops last, corpus-intact re-checks between phases). Honest scope: covers the dispatch surface for transport-floor callers; does NOT cover the IPC context-pack handler (src/mcp/context-pack-handler.ts bypasses dispatchToolCall) or write-triggered restoration echoes — the write-side sweep is a filed TODO. Static siblings: test/operations-trust-boundary.test.ts (previous entry) + scripts/check-operations-filter-bypass.sh.

  • src/core/content-sanity.ts — pure assessor for the content-sanity defense. assessContent(content, opts): SanityVerdict returns one of ok | warn_oversize | hard_block_junk_pattern | soft_block_oversize with {reason, bytes, matched_pattern_name?}. Six built-in junk patterns (Cloudflare challenge dumps, CAPTCHAs, 403 dumps, bare error-page titles) compiled at module load; operator literal substrings via loadOperatorLiterals() from src/core/content-sanity-literals.ts. ContentSanityBlockError tagged class is the typed throw shape every wrapper site (gbrain import, put_page MCP op, gbrain sync, /ingest webhook) catches via the existing exception flow. The bytes-parity contract pins Buffer.byteLength(content, 'utf8') against the embedder's actual byte count so a 499K-byte page can't be soft-blocked on assessment then overflow on embed. Knob resolution chain env > file (~/.gbrain/config.json) > DB > defaults. Four knobs: content_sanity.bytes_warn (50_000), content_sanity.bytes_block (500_000), content_sanity.junk_patterns_enabled (true), content_sanity.disabled (false; GBRAIN_NO_SANITY=1 is the loud-stderr kill-switch). assessContentSanity(opts): SanityAssessment returns the three-tier disposition (shouldQuarantine / shouldFlag + reason/detail) consumed by importFromContent and gbrain quarantine scan; adds the fuzzy prose-vs-markup ratio pass (markup chars / total above max_markup_ratio; code pages exempt; gated by prose_check_enabled) on top of the byte + junk-pattern passes. Three more knobs: content_sanity.junk_disposition (quarantine default | reject; no env override — a destructive flip belongs in explicit config), content_sanity.max_markup_ratio (0.85, env GBRAIN_MAX_MARKUP_RATIO, clamped (0,1]), content_sanity.prose_check_enabled (true). Pinned by test/content-sanity.test.ts. Per-pattern opt-out without the kill-switch: content_sanity.disabled_patterns names built-in junk patterns to skip (a JSON array ["access_denied"] or a comma list access_denied,error_title; malformed JSON falls back to the comma parse so a hand-typed value still lands) — the file plane takes the array, the DB plane the string.

  • src/core/content-sanity-literals.ts — operator literal-substring loader. Reads ~/.gbrain/junk-substrings.txt, one literal per non-comment non-blank line; optional # name=<id> header pairs an identifier with the following literal so audit JSONL groups by site (linkedin_auth_wall, reddit_blocked, etc.). Fail-soft on ENOENT (missing file = empty array). Loaded on every ingest. Deliberately literal substrings (NOT regex) to defeat ReDoS. Pinned by test/content-sanity-literals.test.ts.

  • src/core/embed-skip.ts — 5-site shared predicate for the soft-block embed-skip filter. Exports shouldSkipEmbedding(frontmatter): boolean (JS predicate for callers holding the page in memory), EMBED_SKIP_SQL_FRAGMENT (parameterized SQL clause shared by Postgres + PGLite via executeRaw), and buildEmbedSkipMarker(reason: string) (writes frontmatter.embed_skip = {at: ISO_TIMESTAMP, reason} so the JSONB shape stays uniform). The 5 sites: embed.ts --stale, embed.ts --all, the embed-stale Minion helper, plus both engines' listStaleChunks + countStaleChunks. Single source of truth so the filter cannot drift. Pinned by test/embed-skip.test.ts (cross-site invariant + JSONB shape).

  • src/core/audit/content-sanity-audit.ts — ISO-week JSONL audit at ~/.gbrain/audit/content-sanity-YYYY-Www.jsonl built on the audit-writer.ts primitive. Records every hard-block, soft-block, warn-trip, and quarantine/flag event with {kind, source_id, slug, bytes, matched_pattern_name?, reason, ts}. Doctor reads the last 7 days, aggregates by (matched_pattern_name, source_id) so operators see which scraper is the problem. Honors GBRAIN_AUDIT_DIR for shared-filesystem multi-host setups. Pinned by test/audit/content-sanity-audit.test.ts.

  • src/core/quarantine.ts — the two frontmatter markers the content-quality gate writes, sibling of src/core/embed-skip.ts (same marker-as-JSONB-object pattern, same JSONB ? existence check that works on Postgres AND PGLite; no schema migration — both are frontmatter JSONB keys). quarantine (key QUARANTINE_KEY) HIDES: set ONLY for high-confidence junk, writes zero chunks, excluded from search via quarantineFilterFragment(pageAlias) / QUARANTINE_FILTER_FRAGMENT (the p-aliased constant), the single source of truth buildVisibilityClause calls so the search filter and marker key can't drift. content_flag (key CONTENT_FLAG_KEY) WARNS, does NOT hide: set for fuzzy markup-heavy / oversize, page stays searchable, marker is READ INTO search/get_page output — deliberately NO SQL filter fragment. Three distinct markers, three reasons (never overloaded): embed_skip = oversized-but-clean, quarantine = junk hidden, content_flag = odd-examine-still-here; a page can carry more than one (oversize → embed_skip + content_flag:oversized) and each clears independently. Exports buildQuarantineMarker / isQuarantined / filterOutQuarantined, buildContentFlagMarker / getContentFlag / hasContentFlag, plus the two key constants. Pinned by test/quarantine.test.ts.

  • src/commands/quarantine.tsgbrain quarantine <list|clear|scan> operator surface for the content-quality gate. list [--json] [--include-flagged] paginates listPages and reports quarantined (HIDDEN) pages, optionally also content_flag (FLAGGED, searchable) pages. clear <slug> [--force] [--no-embed] [--json] drops both markers and re-imports through the normal pipeline so the page re-chunks + re-embeds and becomes searchable; the gate re-runs on import so genuinely-junk pages re-quarantine (exit 1) unless --force sets GBRAIN_NO_SANITY=1 for that one import. scan [--limit N] [--apply] [--no-embed] [--json] re-assesses already-ingested pages so junk predating the gate gets marked (unchanged content short-circuits normal sync, so it never re-assesses otherwise); dry-run uses the SAME effective content_sanity config thresholds --apply will use, idempotent (skips already-marked pages), --apply re-imports with forceRechunk to set markers + (for quarantine) drop chunks. Dispatched in cli.ts. Pinned by test/quarantine-cli.test.ts.

  • src/core/zombie-reap.ts — idempotent installSigchldHandler() so JS-spawned children get reaped via Bun's internal waitpid(). Bun (like Node) only auto-reaps when a SIGCHLD listener is registered; without it, every child the worker spawns (shell jobs, embed batches, sub-agents) becomes a zombie on exit and holds connection slots. Called once at module load from src/cli.ts (Windows platform guard — SIGCHLD doesn't exist on Windows). Cross-file leak guard via _uninstallSigchldHandlerForTests(). Layer 1 of the three-layer zombie defense; Layer 2 is tini-as-PID-1 wrapping the worker subtree (via src/core/minions/spawn-helpers.ts); Layer 3 is the container's own tini for hard Bun crashes.

  • src/core/minions/ — Minions job queue: BullMQ-inspired, Postgres-native (queue, worker, backoff, types, protected-names, quiet-hours, stagger, handlers/shell). Private dream-inline-* queues carry an explicit lifecycle: owner job id + owner token + a renewable lease (columns on minion_jobs; queue.ts reconcilePrivateQueue terminalizes one queue through the normal cancelJobs bookkeeping and stamps every cancellation with the machine-readable private_queue_reconciled: reason family; renewPrivateQueueLease is MONOTONIC — GREATEST — so a default-horizon renewal can never shrink a creation-time lease; reconcileOrphanedPrivateQueues cancels only provably-orphaned queues: no healthy child lock, owner terminal/missing or lease expired; legacy unowned rows are left to Doctor/retriage; classifyPrivateQueueForRecovery is PUBLIC so the doctor orphan check buckets through the same verdict). Recovery runs on every lane that can strand a queue: supervisor beforeSpawn, bare gbrain jobs work startup (skipped under GBRAIN_SUPERVISED=1 — the supervisor already ran it; autopilot children never set the flag, so autopilot spawns/respawns recover too), and dream-cycle start (cycle.ts, next to the cycle lock reap — the ONLY lane on PGLite, which inlines every child). deriveWedgeSignal is queue-aware: a dream-inline queue is never 'wedged' (no shared worker can claim it) — it reports private_queue: true so jobs stats / get_job_stats / doctor all point at reconciliation instead of the impossible worker restart.

  • src/core/minions/queue.ts — MinionQueue class (submit, claim, complete, fail, stall detection, parent-child, depth/child-cap, per-job timeouts, cascade-kill, attachments, idempotency keys, child_done inbox, removeOnComplete/Fail). add() takes a 4th trusted arg (separate from opts to prevent spread leakage); protected names in PROTECTED_JOB_NAMES require {allowProtectedSubmit: true} and the check runs trim-normalized (whitespace-bypass safe). The same trusted arg carries {allowPgliteInlineWorker: true}, a narrowly scoped exception used only when gbrain jobs submit embed-backfill --follow starts and awaits a worker in that process. Before config, schema, policy, SQL, or transaction access, add() invokes the centralized embed-backfill admission gate; PGLite and unknown engine kinds refuse, while explicit Postgres preserves queued submission. add() plumbs max_stalled through with a [1, 100] clamp; omitted values let the schema DEFAULT (5) kick in. handleWallClockTimeouts(lockDurationMs) is Layer 3 kill shot for jobs where FOR UPDATE SKIP LOCKED stall detection and the timeout sweep both fail to evict (wedged worker holding a row lock via a pending transaction). The submission backpressure guard covers two options sharing one pg_advisory_xact_lock namespace keyed on (name, queue, source): maxWaiting (rate cap, counts waiting only, NULL-source-as-wildcard scope) and maxPending (single-flight, counts waiting + live-lock active rows where lock_until > now(), EXACT source scope via COALESCE(data->>'sourceId', data->>'source_id') — an expired-lock active never suppresses, keeping the waitingClaimable wedge detectors fed). All three coalesce return paths (idempotency fast-path, cap-hit, ON CONFLICT race fallback) stamp non-persisted coalesced: true metadata on the returned job. Both filter on queue in addition to name so cross-queue same-name jobs don't suppress each other. claim and renewLock issue their UPDATE via engine.executeRawDirect (not executeRaw) so the lock heartbeat runs on the direct session-mode pool that the transaction pooler won't recycle mid-hold; on PGLite this is identical to executeRaw. The two terminal dead-letter paths (handleWallClockTimeouts wall-clock kill and the stall dead-letter CTE in the stall sweep) BOTH increment attempts_made so a long job killed there reads as an honest attempt instead of attempts 0 / started N; the stall path also bumps stalled_counter, surfaced by gbrain jobs get as Attempts: M/N (started: X, stalled: S/MaxS). At submit, add() stamps a default timeout_ms via defaultTimeoutMsFor(jobName) (from handler-timeouts.ts) when the caller passed none, and claim() COALESCEs a still-NULL timeout_ms from HANDLER_DEFAULT_TIMEOUT_MS (raw-object jsonb bind) deriving timeout_at from the coalesced value — the durable invariant covering rows that predate submit-time stamping; an explicit opts.timeout_ms always wins. The lock lease follows the identical three-layer shape: add() stamps opts.lock_duration_ms (clamped [5s,1h] via the shared clampLockDurationMs; INSERT-only — an idempotency-key re-submit never mutates the first submitter's lease) else defaultLockDurationMsFor(jobName); claim() derives lock_until from COALESCE(lock_duration_ms, map, worker default) and stamps the resolved lease, SQL-clamping the row/map-resolved value to [5s,1h] (the worker-default fallback passes through unclamped — operator-configured, and tests legitimately use sub-5s leases) so a bypass-written out-of-range row can't produce a pathological lease; handleWallClockTimeouts' null-timeout fallback uses COALESCE(lock_duration_ms, worker default). handleStalled(graceMsOverride?) applies the stall-sweep reclaim grace (GBRAIN_MINION_STALL_RECLAIM_GRACE_MS, default 15s, 0 = legacy predicate, capped at 600s with a warn-once clamp so an oversized value can't disable stalled-job recovery) to all three sweep predicates — a lease that lapsed within the grace is a starved owner's head start, not a steal candidate. Guarded by test/queue-lock-retry.test.ts (claim never falls back to executeRaw), test/postgres-execute-raw-direct.test.ts (routing decision matrix), and test/minions.test.ts (attempt accounting + default-timeout stamping). MinionQueue.add() gates subagent jobs on capability, not provider: data.model is classified via classifyCapabilities() (lazy-imported from src/core/ai/capabilities.ts to keep queue's eager-load surface small) and rejected only when the model cannot run a tool loop (unusable:no_tools) or names an unknown provider; degraded verdicts pass through with a gateway cost warning. Layer 1 of the three-layer subagent capability enforcement (layers 2+3: model-config.ts:enforceSubagentCapable runtime fallback + src/commands/doctor.ts subagent_provider check). Pinned by test/agent-cli.test.ts. All three terminal reaper paths (stall dead-letter, wall-clock kill, cascade) route through the ONE private killJobs(tx, rows, cause, message) tail: it emits child_done(outcome) to each non-terminal parent's inbox and flips any waiting-children parent whose last open child just died back to waiting — so a dead-lettered child can never strand its aggregator parent. Callers lock parents FIRST via lockParentsOrdered (ascending-id row locks, matching failJob's parent-before-child order) so the reapers and failJob can never deadlock on parent/child lock acquisition. An idempotent stranded-parent sweep runs once per stall tick (~30s): any waiting-children parent with zero non-terminal children flips back to waiting (single UPDATE, NOT EXISTS on an indexed FK) — self-heals every stranding class, including parents stranded before the sweep existed. Every automatic re-run path clears started_at (failJob's delayed retry branch, the stall requeue, lease release, promoteDelayed, and the parent-unblock flips) so the per-attempt wall clock measures execution, not backoff/queue wait — a retried job can't be dead-lettered by the wall-clock sweep before executing a line. Pinned by test/queue-stall-parent-unblock.test.ts + test/queue-started-at-retry.test.ts. renewLock accepts optional {signal} forwarded to executeRawDirect so the renewal tick's timeout race CANCELS a hung UPDATE instead of orphaning a checked-out pool slot (best-effort — the token fence is the correctness authority; a late-landing renewal can only extend a lease nobody else claimed).

  • src/core/minions/embed-backfill-admission.ts — Centralized, dependency-light embed-backfill admission contract shared by the public MinionQueue.add boundary, CLI, MCP operations, automatic submitters, sync cost reporting, and doctor remediation. embedBackfillWorkerSurface recognizes only explicit postgres as worker-backed; the declared engine-kind switch is compile-time exhaustive, while untyped/cast unknown runtime values still fail closed as no_worker_surface. assertEmbedBackfillQueueAdmission refuses before any database/config access unless the explicit PGLite inline-worker trust is present, validates payload source IDs through the canonical source-id.ts contract before rendering any remedy, and embedBackfillManualDrainCommand(sourceId) is the single injection-safe exact gbrain embed --stale --source <id> recovery command.

  • src/core/embed-backfill-submit.ts — Automatic per-source embed-backfill submitter with a true discriminated result union: submitted requires jobId plus an explicit spend-bypass payload, cooldown requires a reason plus a numeric remaining duration or explicit null for active work, spend_capped requires cap/spend payloads, and no_worker_surface requires the refused engine kind. It queries cooldown/spend state only after the shared admission classification accepts a worker-backed engine, preserving Postgres behavior while guaranteeing PGLite/unknown refusal before config, schema, or SQL access; callers handle every status exhaustively.

  • src/core/minions/worker.ts — MinionWorker class (handler registry, lock renewal, graceful shutdown, timeout safety net). Aborted jobs call failJob with reason (timeout/cancel/lock-lost/shutdown); shutdownAbort (instance field) fires on SIGTERM/SIGINT and propagates to ctx.shutdownSignal (shell handler listens; non-shell handlers don't). Per-job timeout fires abort.abort(new Error('timeout')) then a 30s grace-then-evict safety net force-evicts the job from inFlight and marks it dead if the handler ignores the abort signal (the generic abort listener fires for ANY abort reason). The launchJob lock-renewal block is a thin sync wrapper around the pure runLockRenewalTick from src/core/minions/lock-renewal-tick.ts (NEVER setInterval(async () => await renewLock(...)) — that shape surfaces an unhandledRejection during PgBouncer rotation). Guarantees: (1) cancelled flag captured in the timer closure stops in-flight IIFEs writing misleading audit events after the job ended; (2) re-entrancy guard tickInFlight (skips counted as overlapSkips telemetry) + per-call Promise.race timeout with best-effort AbortSignal cancellation; (3) verify-before-evict: eviction requires a fenced-false (certain loss) or the hardEvictMs backstop — never bare local arithmetic; the per-job lease (job.lock_duration_ms ?? opts.lockDuration) drives the renewal state, and the cadence clamps to min(lease/2, 60s); (4) explicit .catch() on the stored executeJob(...).finally(...) promise closes the second unhandledRejection vector; (5) exported INFRASTRUCTURE_ABORT_REASONS = new Set(['lock-renewal-failed', 'lock-lost']) so executeJob's catch skips failJob for these (PgBouncer blips don't dead-letter healthy jobs; the stall detector reclaims); (6) inFlight generation-safety — force-evict and the handler's finally delete the inFlight entry ONLY when it still carries their own lockToken (the token is the generation; an evicted execution's late delete cannot remove a same-worker re-claim's entry); (7) eviction observability — the worker keeps a perf_hooks.monitorEventLoopDelay histogram (reset on every successful renewal, sampled ns→ms at eviction) plus raw loadavg, and both the abort and grace-evict log lines carry cause / since-last-success / tick-lateness / overlap-skips / load / event-loop-delay from the stashed abortMeta. CI guard scripts/check-worker-lock-renewal-shape.sh (in bun run verify) asserts the bug pattern stays absent AND launchJob keeps calling runLockRenewalTick. Engine-ownership invariant: start() does NOT call engine.disconnect() on shutdown — the CLI handler in src/commands/jobs.ts case 'work' owns engine lifecycle via try/finally with loud error logging. RSS watchdog uses non-file-backed pages on Linux: exported parseRssFromProcStatus(status) (pure parser; field-presence regex so RssAnon: 0 + RssShmem: 512 parses correctly) and getAccurateRss(readStatus?) (reads /proc/self/status for RssAnon + RssShmem, falls back to process.memoryUsage().rss on macOS / restricted containers / kernel <4.5); the default getRss in WorkerOpts is getAccurateRss. checkMemoryLimit tracks peak RSS, fires an 80%-of-cap soft-warn (once per crossing, carrying peak + in-flight job kinds), and on exceed sets _rssWatchdogTriggered=true (exposed via get rssWatchdogTriggered()) so jobs work's finally can process.exit(WORKER_EXIT_RSS_WATCHDOG) after disconnect (drain self-identifying instead of an opaque code-0 exit). The poll loop wraps claim in try/catch: on a retryable conn error it reconnects ONCE and continues to the next tick rather than blind-retrying (a retry after UPDATE...RETURNING committed but the socket died would double-claim). LockRenewalDeps is wired with reconnect when the engine supports it. The self-health DB-liveness probe runs EVEN under a supervisor (GBRAIN_SUPERVISED=1): the outer guard is if (this.opts.healthCheckInterval > 0) and only the STALL-detection block is wrapped in if (!isSupervisedChild) — so a supervised worker whose own pool dies self-exits unhealthy(db_dead) after dbFailExitAfter probes (the supervisor watches a different connection and can't see this worker's dead pool), while the supervisor's progress watchdog owns forward-progress. Pinned by the test/worker-lock-renewal.test.ts hermetic suite, test/audit/lock-renewal-audit.test.ts, test/scripts/check-worker-lock-renewal-shape.test.ts, test/worker-shutdown-disconnect.test.ts (asserts disconnectSpy).not.toHaveBeenCalled()), test/worker-rss.test.ts (11), test/worker-supervised-db-probe.test.ts (3). Per-job process isolation: MinionWorkerOpts.jobIsolation (inline|process) + childCliInvocation/childTiniPath swap handler(context) for runJobInChild(...) in executeJob — every reporting branch (completeJob, failJob, lease release, infra no-burn) is reused on the child outcome; when isolated the parent-side context is not built at all. Three extra no-burn child classes in the catch: ChildSpawnInfraError + ChildWorkerShutdownError (released, no attempt burned) and ChildNotClaimedError (the child proved the claim was already gone — reclaimed/cancelled before the handler ran — so nothing is recorded against it). The health probe delegates to runDbProbe (db-probe.ts) with a cancellation signal on every probe and emits a verdict (pool_starved/server_unreachable/unknown) on the db_dead unhealthy payload. getHandler(name) is the read-only registry accessor run-child uses.

  • src/core/minions/supervisor.ts — MinionSupervisor process manager. Spawns gbrain jobs work as a child, restarts on crash with exponential backoff, periodic health check. consecutiveHealthFailures counter; on 3 consecutive failures emits health_warn with reason: 'db_connection_degraded' and calls engine.reconnect() to swap in a fresh pool, then resets. Worker exit classifier emits likely_cause on worker_exited events: oom_or_external_kill (SIGKILL), graceful_shutdown (SIGTERM), runtime_error (code 1), clean_exit (code 0), unknown. Consumes detectTini() + buildSpawnInvocation() from src/core/minions/spawn-helpers.ts to wrap the worker subtree in tini-as-PID-1 when tini is on PATH (handles native-addon zombie reaping the in-process SIGCHLD reaper can't reach); exposes isTiniDetected read-only accessor. The spawn-and-respawn loop is the shared ChildWorkerSupervisor core: MinionSupervisor composes it via runSuperviseLoop()new ChildWorkerSupervisor({...}) and maps ChildSupervisorEvent back through emit() SupervisorEvent (JSONL audit consumers see byte-compatible output). PID lock, signal handlers, health check, and process.exit on the HARD crash ceiling stay in MinionSupervisor. Crossing the SOFT budget (maxCrashes) does not permanently give up: the core drops into degraded retry (capped backoff + a crash_budget_degraded health_warn) and self-heals when a respawn runs stably; permanent process.exit(MAX_CRASHES) fires only at the hard ceiling resolveHardStopMaxCrashes(maxCrashes) (default maxCrashes × 10, env GBRAIN_SUPERVISOR_HARD_STOP_CRASHES, 0 = never). Separately, gbrain jobs supervisor status + gbrain doctor detect a live supervisor through this queue lock (inspectLock + isLockHolderLive, freshness-keyed so PID reuse can't false-positive) when the $HOME-derived pidfile is absent, so a split-$HOME deployment does not read a healthy supervisor as "not running". code=0 leaves crashCount untouched (so a worker alternating real crashes + watchdog drains still trips max_crashes); cleanRestartBudget (default 10 restarts per 60s) caps the macOS/non-Linux-fallback tight-loop via health_warn { reason: 'clean_restart_budget_exceeded' } + backoff. shutdown() drains via childSupervisor.killChild('SIGTERM') + awaitChildExit(35_000). Progress watchdog: healthCheck() restarts an alive-but-wedged child via childSupervisor.restartCurrentChild(35_000) when a queue has claimable work, 0 live-lock active jobs, and stale completions across wedgeRestartChecks (default 3) consecutive checks past wedgeRestartMinutes (default 15, 0 disables) + a startupGraceMs window; bounded by wedgeRestartLoopBudget (default 3 / wedgeRestartLoopWindowMs) which switches to a one-shot wedge_restart_loop alert. The wedge query is the exported queryWedgeSignals(engine, queue, handlerNames) — name+queue-scoped, active_healthy = live-lock only (an expired-lock active row does NOT mask the wedge), due-delayed counted. Claimable names are derived at start via a throwaway registerBuiltinHandlers worker (its new quiet opt). Flags --wedge-restart-minutes / --wedge-restart-checks + env GBRAIN_WEDGE_RESTART_MINUTES / GBRAIN_WEDGE_RESTART_CHECKS. The worker argv is built by the exported pure buildWorkerArgs(opts) (appends --nice N when opts.nice_requested is set, alongside --concurrency/--queue/--max-rss); the niceness apply RESULT (nice_requested/nice_effective/nice_error, computed by the CLI in jobs.ts — the supervisor doesn't call setPriority) rides on the started/worker_spawned audit emissions. Queue-scoped singleton: the real authority is a DB lock (tryAcquireDbLock from src/core/db-lock.ts) keyed on supervisorLockId(queue) = gbrain-supervisor:<queue> — keyed on the QUEUE ALONE because the lock row lives inside the target database, so the (database) half of the mutex is physical, not part of the key (mixing in a config-derived DB identity would let two supervisors on the same physical DB via different-but-equivalent URLs compute different ids and both acquire). Two supervisors with different $HOME/--pid-file against the same (database, queue) cannot both run with conflicting --max-rss: the second exits LOCK_HELD. The pidfile-cleanup process.on('exit') listener is installed BEFORE the DB-lock acquisition so the LOCK_HELD early-exit can't strand the pidfile this process just created. The default pidfile is brain-scoped (supervisor-<brainId>.pid) so different brains under one HOME don't false-block. The lock refreshes on its own setInterval (TTL 5min, refresh 60s, max 3 failures = 180s < TTL); a refresh that THROWS past the threshold exits LOCK_LOST (code 4) rather than risk a split-brain, while a fenced refresh returning false (0 rows matched — the lock was stolen or force-cleared) is treated as CERTAIN loss, not a blip: the supervisor emits health_error { reason: 'supervisor_lock_lost' } and exits LOCK_LOST immediately (counting it toward the failure threshold would let two supervisors drain the same queue for up to two more refresh windows). shutdown() releases the lock so a clean restart re-acquires immediately. The started audit records max_rss_mb so gbrain doctor's supervisor_singleton check can surface the effective cap. Exports supervisorLockId() and the pure classifySupervisorSingleton({lockLive, lockHolderHost, lockHolderPid, localHost, localPid}) → 'no_lock'|'single'|'mismatch' (host+pid compare, bare pid meaningless cross-host) that doctor consumes. Pinned by test/supervisor.test.ts (16 cases), test/supervisor-tini.test.ts, test/supervisor-wedge.test.ts, test/supervisor-build-worker-args.test.ts, and test/supervisor-db-lock.test.ts. SupervisorOpts.jobIsolation passes --job-isolation process to the spawned worker via a CONDITIONAL buildWorkerArgs push (inline argv stays byte-identical). queryWedgeSignals/probeQueueState thread a per-probe AbortSignal; the probe timeout cancels the losing query (pool-slot release under exhaustion).

  • src/core/minions/detached-stderr.ts — durable stderr sink + spawn helper for gbrain jobs supervisor start --detach. openDetachedStderrSink() opens an append-mode log in the audit dir (${GBRAIN_AUDIT_DIR:-~/.gbrain/audit}/supervisor-stderr.log, next to the JSONL lifecycle audit), falling back to the null device, then 'ignore'; never throws (a sink failure must not block the supervisor start). The detached supervisor gets the fd as stdio[2] and its worker inherits it, so ONE durable descriptor covers both processes — not a per-call EPIPE swallow. Prevents the SIGPIPE failure of inheriting the INVOKER's stderr: a short-lived automation runner closing its capture pipe after the start payload would make the worker exit 141 on its next stderr write (and could take the supervisor down writing its own crash event), leaving active jobs and DB locks stale while producers keep enqueueing. Pinned by test/detached-stderr.test.ts.

  • src/core/minions/child-worker-supervisor.ts — shared spawn-and-respawn core reused by both MinionSupervisor (standalone gbrain jobs supervisor daemon) and src/commands/autopilot.ts (autopilot daemon), so the two consumers can't drift into parallel-loop bugs. Pure class: NO PID file, NO signal handlers, NO process.exit, NO health check. Lifecycle events fire via injected onEvent: (ChildSupervisorEvent) => void. Exit classifier: code === 0 leaves crashCount UNCHANGED (preserves flap detection across mixed exit sequences); code != 0 follows runDuration > stableRunResetMs ? 1 : ++crashCount. Clean-restart budget: sliding window of code=0 exits; when count exceeds cleanRestartBudget (default 10) inside cleanRestartWindowMs (default 60s), emits health_warn { reason: 'clean_restart_budget_exceeded' } and applies cleanRestartBudgetBackoffMs (default 1s). The exit classifier special-cases WORKER_EXIT_RSS_WATCHDOGlikely_cause='rss_watchdog', and that exit does NOT bump crashCount (routes to its own breaker); a dedicated _watchdogExitTimestamps sliding window trips a loud rss_watchdog_loop health_warn naming the cap when N watchdog exits land inside the window, INDEPENDENT of the stable-run reset (which would otherwise hide a >5-min-run watchdog drain loop). Opts watchdogLoopBudget (3), watchdogLoopWindowMs (600000), watchdogBackoffMs (30000); ChildSupervisorEvent extended. Public read-only accessors childAlive, inBackoff, crashCount; killChild(signal) gates on liveness (exitCode === null && signalCode === null), NOT .killed.killed flips true once a signal is sent, so a !this._child.killed guard would make a follow-up SIGKILL after an ignored SIGTERM a silent no-op. restartCurrentChild(graceMs) (wedge self-heal) captures the CURRENT child ref, SIGTERM→grace→SIGKILLs THAT ref (never the respawn — closes the timer-kills-fresh-worker race), and flags _intentionalRestart so the exit is likelyCause='wedge_restart', leaves crashCount UNTOUCHED (never trips max_crashes; like rss_watchdog), and respawns immediately (backoff ms:0 reason='wedge_restart'). awaitChildExit(timeoutMs) short-circuits when child.exitCode !== null || child.signalCode !== null so fast-SIGTERM responders don't cause a 35s shutdown hang. Degraded-retry: the run() loop does not fire onMaxCrashesExceeded at the soft maxCrashes; it announces health_warn { reason: 'crash_budget_degraded' } once per episode and keeps respawning with capped backoff (the 60s cap makes it a paced retry, not a hot loop), re-arming after a stable-run reset drops the count. Permanent give-up fires only at hardStopMaxCrashes (default maxCrashes × HARD_STOP_CRASH_MULTIPLIER = 10×; 0 disables). Test hooks _backoffFloorMs, _now. supervisor-audit.ts carries rss_watchdog as a non-clean cause + its own CrashSummary.by_cause bucket, and wedge_restart to CLEAN_EXIT_CAUSES (a self-heal, not a crash; denylist preserved so future causes route to legacy). Pinned by test/child-worker-supervisor.test.ts (12 cases).

  • src/core/minions/spawn-helpers.ts — pure detectTini() + buildSpawnInvocation() consumed by both supervisor.ts and autopilot.ts (one implementation for both spawn sites; tini wrapping is testable without mock.module(), rule R2 of scripts/check-test-isolation.sh). detectTini() calls execFileSync('which', ['tini']) with explicit env: process.env so Bun sees runtime PATH mutations. buildSpawnInvocation(tiniPath, cmd, args) returns {cmd, args} with tini prepended when present, or the bare invocation otherwise. Pinned by test/spawn-helpers.test.ts (5) and test/supervisor-tini.test.ts (4).

  • src/core/minions/job-isolation.ts — per-job process-isolation protocol: atomic outcome-file codec (writeChildOutcomeFile tmp+rename, decodeChildOutcomeFile with a 32MiB cap that throws UnrecoverableError — oversize results die loudly on attempt 1, decode errors report byte counts never content), encodeHandlerError/reconstructHandlerError preserving the two instanceof classes executeJob branches on (UnrecoverableError, RateLeaseUnavailableError), the child argv/env contract (CHILD_ENV, buildChildArgs), resolveChildCliInvocation (env override → compiled binary → bun-dev fallback → null for fail-fast), and killProcessGroup(pid, sig) — children are spawned detached (own group) because SIGKILL on a tini pid alone orphans the handler grandchild, and Bun rejects negative pids in process.kill (oven-sh/bun#15791) so group signaling falls back to POSIX /bin/kill. Pinned by test/job-isolation-protocol.test.ts (incl. real-process grandchild-death).

  • src/core/minions/child-job-runner.ts — parent-side runner: runJobInChild(opts) spawns the child (detached + tini when available, stdio inherit for handler logs, per-job lifecycle log lines), maps per-job abort → group SIGTERM now + group SIGKILL at +25s (CHILD_KILL_GRACE_MS, inside the 30s force-evict backstop), gives worker-shutdown children the drain window to finish AND report (a non-reporting shutdown kill throws ChildWorkerShutdownError → released, no attempt burned), classifies pre-exec spawn failure as ChildSpawnInfraError (released, no attempt burned), and bounds child pools via env (GBRAIN_POOL_SIZE default 3, GBRAIN_DIRECT_POOL_SIZE=1). Pinned by test/child-job-runner.test.ts (real children) + test/worker-job-isolation.test.ts (full parent path on PGLite).

  • src/core/minions/run-child.tsrunChildJobEntry(engine, opts, injectables): the jobs run-child core. Re-reads the job row and validates status+token (exit 14 on mismatch, handler never runs), builds the shared token-fenced context against the CHILD's engine, runs the handler, writes ONE atomic outcome file (handler failure = reported outcome = exit 0; only write-failure exits 15). Installs a SIGTERM→shutdownSignal-ONLY handler (inline signal-separation parity: ctx.signal stays live so cooperative handlers finish + report inside the drain window; the parent's group SIGKILL at drain end is the backstop) and a parent-liveness watchdog polling process.kill(parentPid, 0) (ppid checks are dead code under tini) that aborts BOTH signals on parent death with a hard exit after grace. No worker machinery (parent owns liveness). Pinned by test/run-child-entry.test.ts.

  • src/core/minions/job-context.ts — shared buildJobContext(engine, queue, job, lockToken, signal, shutdownSignal) (extracted verbatim from executeJob) so inline mode and the run-child child wire identical token-fenced DB callbacks.

  • src/core/minions/db-probe.ts — hermetic DB-liveness probe with pool-starvation disambiguation: runDbProbe(deps) probes the read pool (signal-cancelled at timeoutMs), on failure probes the DIRECT lane (DIRECT_PROBE_TIMEOUT_MS, only when dual-pool is genuinely active) and returns a verdict — pool_starved ("server IS reachable; fault is in the transaction-pooler path — client pool exhaustion or a pooler-layer fault", deliberately an honest disjunction), server_unreachable, or unknown. Gauge counts render as a labeled tracked SUBSET; no waiter/available arithmetic. Pinned by test/db-probe.test.ts.

  • src/core/minions/niceness.ts — OS scheduling-priority (niceness) primitives for the --nice flag. parseNiceValue(raw) whole-string parses + range-validates to POSIX [-20, 19] (rejects "3.5"/"10abc" that parseInt would silently truncate). applyNiceness(nice, setPriority?, getPriority?) calls os.setPriority(0, n) and ALWAYS re-reads os.getPriority(0) afterwards — in both the success and the catch paths — so a denied renice (EPERM) or an RLIMIT_NICE clamp records the real effective value (e.g. 0), not null; returns {applied, requested, effective, error?}. getEffectiveNiceness(pid, getPriority?) reads an arbitrary pid's niceness (null on dead/unreadable). formatNice(n)+10/0/-5. Applied only at the CLI layer (jobs.ts) so worker.ts/supervisor.ts stay embeddable. Pinned by test/niceness.test.ts.

  • src/core/minions/worker-registry.ts — live worker registry backing niceness observability. Each running gbrain jobs work self-registers worker-<pid>.json under gbrainPath('workers') (brain-isolated via GBRAIN_HOME; entries tagged with currentBrainId() so multiple DBs under one home don't cross-report). registerWorker(info) is best-effort (never blocks the worker) and returns a cleanup fn the caller wires to BOTH the shutdown finally AND process.on('exit') (the unhealthy process.exit(1) bypasses the awaited finally). readWorkers(getNice?) enumerates the dir, drops confirmed-dead pids (classifyLiveness: ESRCH = dead/prune, EPERM = alive/keep), applies a pid-reuse start-time guard (zone-free ps -o etime= parsed by parseEtimeToMs, start = now - elapsed; rejects a pid that started >5s after the entry was written), and re-measures each live worker's niceness now. Reports the worker's REAL pid, sidestepping the tini-wrapper-pid problem. Pinned by test/worker-registry.serial.test.ts.

  • src/core/minions/supervisor-pid.tsreadSupervisorPid(pidFile) → {pid, running}: the shared existsSync → readFileSync → parseInt → process.kill(pid,0) PID-file + liveness reader shared by jobs.ts (supervisor status), jobs.ts (stats), and doctor.ts. EPERM from the liveness probe counts as running. Pinned by test/supervisor-pid.test.ts.

  • src/core/minions/handler-timeouts.ts — per-handler-type defaults for BOTH per-job time knobs, co-located so they can't drift apart unseen (they are different quantities: the budget bounds total runtime, the lease bounds dead-worker reclaim — never derive one from the other). HANDLER_DEFAULT_TIMEOUT_MS: 30 min for subagent, subagent_aggregator, embed-backfill, autopilot-cycle, autopilot-global-maintenance; 10 min for chronicle_extract + facts-absorb; 60 min for contextual_reindex_per_chunk. HANDLER_DEFAULT_LOCK_DURATION_MS: 300 s for the long LLM/loop handlers, 120 s for the single-LLM-call handlers, shell deliberately absent (fast dead-worker reclaim; verify-before-evict protects it anyway). defaultTimeoutMsFor / defaultLockDurationMsFor return the mapped default or null (short handlers keep the tight null-default wall-clock / the 30 s worker lease). clampLockDurationMs + LOCK_DURATION_MS_MIN/MAX ([5s, 1h]) is the ONE clamp shared by queue.add, the CLI --lock-duration-ms flag (--dry-run echoes the clamped value), and the MCP submit_job param; the same bound is mirrored in SQL at claim time and by the minion_jobs.lock_duration_ms range CHECK, so every layer agrees. Layers (explicit value always wins): MinionQueue.add() stamps at submit; MinionQueue.claim() COALESCEs NULL columns from the maps (durable invariant); migration v128 one-shot backfilled timeout_ms with authoring-time snapshot values — do NOT sync v128 when editing the maps (lock_duration_ms has no backfill: NULL = worker default). Pinned by test/minions.test.ts + test/migrations-v128.test.ts + test/migrations-v130.test.ts.

  • src/core/minions/admission.ts — submit-side queue admission control (the drain-side pool-starvation half is the job-isolation work; claim fairness is a filed TODO, deliberately out of scope). Three primitives resolved per name via resolveAdmissionPolicy (config minions.* > per-name defaults tables, 60s in-process cache, fail-open with a once-per-process stderr warn; env kill-switch GBRAIN_MINIONS_ADMISSION=0 disables all three): PARAM-COALESCING (PARAM_COALESCE_DEFAULT — on for subagent; computeParamHash = sha256 of stable-stringified payload excluding only __param_hash itself — __owner_client_id is deliberately INCLUDED so owner lanes never cross; parentless + waiting-only + age-bounded to ttl/2), WAITING-TTL (WAITING_TTL_DEFAULT_HOURS — 48h for subagent; swept by MinionQueue.handleWaitingTTL through cancelJobs(ids, {reason, rootStatuses:['waiting']}) so descendants cancel, child_done lands, aggregator parents resolve, and the reason stamps ROOT ids only; ≤500/tick oldest-first; warn-before-act is runWaitingTtlTick — first tick counts affected + stamps TTL_NOTICE_SHOWN_KEY with an ISO timestamp, sweeping starts only after ttlNoticeGraceMs() (1h default, env GBRAIN_MINIONS_TTL_NOTICE_GRACE_MS) elapses; gbrain upgrade prints the same one-shot notice and starts the same clock; legacy 'true' flag values sweep immediately), and NAME-GLOBAL QUOTA (QUOTA_MAX_WAITING_DEFAULT EMPTY by operator decision — activates only via minions.quota_max_waiting.<name>; counts the name across ALL queues so per-run dream-inline-* fanout queues can't dodge it, EXACT under concurrency via a minion_quota:<name> advisory xact lock taken only when a quota is configured; throws typed QueueQuotaExceededError, checked everywhere via isQueueQuotaExceededError — dream submitters record a phase skip, synthesize rolls back the current transcript's fresh chunks, agent fanout cancels the whole tree, submit_agent maps to a structured rate_limited OperationError). TTL_REASON_PREFIX is the single source for the sweep's error_text prefix and the stats/doctor LIKE patterns; safeConfigSegment gates untrusted job names out of copy-pasteable config hints. Alerting rides getStats (drained_completed/failed/dead/cancelled keyed on finished_at + waiting_now + oldest_waiting_minutes), the jobs stats DIVERGENT-QUEUE / waiting-TTL screams (GBRAIN_QUEUE_DIVERGENCE_RATIO=2, GBRAIN_QUEUE_DIVERGENCE_MIN_WAITING=50; divergence compares intake vs COMPLETED so TTL-cancel storms can't masquerade as throughput) + --json, and doctor checkQueueHealth. Pinned by test/minions-admission.test.ts + test/jobs-stats-divergence.serial.test.ts.

  • src/core/minions/types.tsMinionJobInput + MinionJobStatus + handler context types. MinionJobInput.max_stalled is optional; omitted values let the schema DEFAULT (5) kick in, provided values are clamped to [1, 100].

  • src/core/minions/errors.ts — dependency-free UnrecoverableError, re-exported through types.ts for compatibility. Delegated policy imports this leaf directly, avoiding initialization cycles through durable authority. Fresh-process import tests preserve the shared error identity and isolated-worker reconstruction.

  • src/core/minions/protected-names.ts — side-effect-free constant module exporting PROTECTED_JOB_NAMES + isProtectedJobName(). Kept pure so queue core can import without loading handler modules. PROTECTED_JOB_NAMES includes synthesize, patterns, consolidate. These phases internally submit subagent children with allowProtectedSubmit=true and can spend Anthropic credits. Only trusted local callers (CLI, autopilot, doctor --remediate) can submit them; MCP requests are rejected by submit_job's protected-name guard.

  • src/core/minions/handlers/shell.tsshell job handler. Spawns /bin/sh -c cmd (absolute path, PATH-override-safe) or argv[0] argv[1..] (no shell). Env allowlist PATH, HOME, USER, LANG, TZ, NODE_ENV + caller env: overrides + inherit:-resolved keys. UTF-8-safe stdout/stderr tail via string_decoder.StringDecoder. Abort (either ctx.signal or ctx.shutdownSignal) fires SIGTERM → 5s grace → SIGKILL on child. Requires GBRAIN_ALLOW_SHELL_JOBS=1 on worker (gated by registerBuiltinHandlers). ShellJobParams.inherit?: string[] is a free-form list of snake_case config-key names; the worker resolves each via loadConfig() and injects the value under the derived env key (database_urlGBRAIN_DATABASE_URL; else uppercased). Names persist in minion_jobs.data (and the shell-audit JSONL); values never do. The canonical validator validateShellJobParams (sibling shell-validate.ts) runs PRE-ENQUEUE in both submit surfaces — gbrain jobs submit shell (jobs.ts:271) AND the submit_job op for name='shell' (operations.ts:2085); the handler-entry re-validation here is defense-in-depth (so validation can never run only AFTER queue.add() has persisted the row). The validator does NOT police which config keys the agent inherits — same-uid trust model treats the agent as a peer of the worker.

  • src/core/minions/handlers/shell-inherit.ts — three helpers. INHERIT_NAME_RE (/^[a-z][a-z0-9_]*$/) is the snake_case shape guard used by the validator; rejects __proto__, leading-underscore, uppercase, and path-traversal shapes so audit logs stay readable and prototype-pollution lookups can't smuggle through. deriveEnvKey(name) maps config-key → child-env-key (name.toUpperCase() with one override: database_urlGBRAIN_DATABASE_URL because plain DATABASE_URL is ambiguous). resolveInheritValue(cfg, name) is the value lookup; uses Object.hasOwn to defeat prototype-pollution lookups, returns undefined for missing / non-string / empty-string values. No closed enum — agent and worker share a uid, so refusing arbitrary config keys defends nothing in that trust model.

  • src/core/minions/handlers/shell-validate.tsvalidateShellJobParams(data, opts?) shared pre-enqueue validator. Throws UnrecoverableError with paste-ready operator hints on every failure. Three rules: (1) cmd/argv/cwd/env shape, (2) inherit array shape + snake_case regex per element (prototype-pollution defense), (3) fail-fast on missing config value with gbrain config set <key> hint. Optional redact_secrets?: boolean for output-side scrubbing. Deliberately does NOT police WHICH secrets the agent passes — single-uid trust model. Test seam: opts.config drives the validator hermetically without mocking. Re-called at shell.ts handler entry for defense-in-depth (catches rows that bypassed the pre-enqueue validator).

  • src/core/minions/handlers/shell-redact.ts — opt-in output-side scrubbing for shell-job stdout/stderr. Pure redactSecretsInText(text, secrets): string-mode replaceAll so regex metacharacters in values stay literal. When the caller passes redact_secrets: true (or --redact-secrets), the handler builds a Map of inherit-name → resolved-value and post-processes both tails before throw/return so persisted result.stdout_tail / result.stderr_tail / error_text carry <REDACTED:name>. Only inherit:-resolved values are scrubbed; caller-supplied env: values pass through. Heuristic — defeats echo "$GBRAIN_DATABASE_URL", not adversarial encode-then-print. Default false.

  • src/core/config.ts:ensureGitignore — idempotent retroactive writer of ~/.gbrain/.gitignore (single line *). Called from saveConfig() so every config-writing path lays it down, AND from runPostUpgrade() so existing users pick it up on gbrain upgrade. Never clobbers a user-customized .gitignore (checks file exists + content non-empty before writing). Scope: blocks casual git add ~/.gbrain from inside an enclosing worktree, but does NOT cover already-tracked files, screenshots, backups (Time Machine / iCloud / Dropbox), or git add -f. The doctor check home_dir_in_worktree surfaces what .gitignore can't.

  • src/core/minions/handlers/shell-audit.ts — per-submission JSONL audit trail at ~/.gbrain/audit/shell-jobs-YYYY-Www.jsonl (ISO-week rotation; override via GBRAIN_AUDIT_DIR). Best-effort: mkdirSync(recursive) + appendFileSync; failures logged to stderr, submission not blocked. Logs cmd (first 80 chars) or argv (JSON array). Never logs env values.

  • src/core/minions/handlers/supervisor-audit.ts — supervisor lifecycle JSONL audit at ~/.gbrain/audit/supervisor-YYYY-Www.jsonl (ISO-week rotation; shares computeIsoWeekName() with shell-audit.ts). writeSupervisorEvent(emission, supervisorPid) appends one line per event (started, worker_spawned, worker_exited, backoff, health_warn, health_error, max_crashes_exceeded, shutting_down, stopped, worker_spawn_failed). readSupervisorEvents({sinceMs}) is the readback for gbrain doctor. Exports isCrashExit(event), summarizeCrashes(events), CrashSummary type, and CLEAN_EXIT_CAUSES denylist ('clean_exit' | 'graceful_shutdown'). Single shared point — both gbrain doctor (supervisor check) and gbrain jobs supervisor status import from here so the two surfaces can't drift. isCrashExit classifies a single worker_exited against the denylist: clean/graceful are NON-crashes; everything else (incl. any future likely_cause from child-worker-supervisor.ts) is a crash; audit lines lacking likely_cause fall back to code !== 0. summarizeCrashes returns {total, by_cause: {runtime_error, oom_or_external_kill, unknown, legacy}, clean_exits} — the legacy bucket catches both old fallback entries AND unrecognized future causes (fail-loud, not silent underreport); denylist-over-allowlist is deliberate. Pinned by test/supervisor-audit.test.ts (14 cases) and 4 source-grep wiring assertions in test/doctor.test.ts.

  • src/core/minions/backpressure-audit.ts — sibling of shell-audit.ts for maxWaiting AND maxPending coalesce events. JSONL at ~/.gbrain/audit/backpressure-YYYY-Www.jsonl. One line per coalesce with (queue, name, waiting_count/max_waiting OR pending_count/max_pending, returned_job_id, ts). readRecentCoalesceCounts feeds the jobs stats Backpressure line (reads current + previous ISO-week files so a 24h window survives week boundaries, filtered per queue). Makes the backpressure guards' coalesces visible instead of silent drops. Pinned by test/backpressure-audit-read.test.ts.

  • src/core/minions/handlers/subagent.ts — LLM-loop handler. Two-phase tool persistence (pending → complete/failed), replay reconciliation for mid-dispatch crashes, dual-signal abort (ctx.signal + ctx.shutdownSignal), Anthropic prompt caching on system + tool defs. makeSubagentHandler({engine, client?, ...}) factory; MessagesClient is an injectable interface the real SDK implements structurally. Per-turn output cap resolves via resolveMaxOutputTokens (data.max_tokensagent.max_output_tokens config → 8192 default); a stop_reason: 'max_tokens' final turn surfaces as SubagentStopReason 'max_tokens' (not a silent end_turn), and a max_tokens stop mid-tool-round injects a truncation note into the tool-result turn so the model re-issues the dropped call. Throws RateLeaseUnavailableError (renewable) when rate-lease capacity is full. Both loop paths (the direct Anthropic SDK turn loop and the gateway toolLoop's acquireTurnPermit) heartbeat the held lease at ttl/3 for the duration of each provider call — single-flight renewals so a stalled renewal never stacks behind a starved pool; a FALSE renewal means the lease row was pruned/stolen (the slot is already re-admitted), so the in-flight call aborts and converts to a lease-full requeue instead of running above maxConcurrent. Anthropic 400 prompt is too long responses (status 400 + body matches /prompt is too long|prompt_too_long|context.*length/i) classify as UnrecoverableError so the job goes straight to dead on first attempt instead of stalling three times. Catches both initial-prompt overflow and turn-N tool-loop accumulation that synthesize.ts's chunker can't bound ahead of time. terminal-state short-circuit on resume. When a stored message thread already ends in stop_reason: 'end_turn', the handler returns { ok: true } immediately instead of issuing another messages.create call (re-prompting past end_turn would get a 400 and dead-letter an already-successful job). Pinned by test/subagent-handler.test.ts. Oneshot dispatch: data.mode === 'oneshot' on a FRESH job (zero persisted messages) routes to runSubagentOneshot (subagent-oneshot.ts) before either loop; a validation fallback re-enters the loops in the SAME job, stamped synth_mode_used: 'agentic_fallback' + fallback_reason. Write accounting (finalizeWriteAccounting in subagent-persistence.ts): every job's result carries pages_attempted/written/failed derived from the tool-execution ledger (settled rows only); data.require_writes jobs (dream synthesize + patterns fan-outs) throw UnrecoverableError → dead when attempted>0 with zero successes. resolveMaxOutputTokens takes the model: thinking-by-default models (Claude 5 by name or recipe-declared thinking_by_default such as DeepSeek v4, via the gateway's shared isThinkingModel) default to 32000 when neither per-job nor config caps are set. Gateway-path onToolCallStart has a uniq_subagent_tools_use_id backstop: a provider repeating a tool id across turns persists under a #m<idx>o<ordinal>-suffixed debug id (second violation reconciles to the existing row's gbrain_tool_use_id) instead of dead-lettering the job. Persistence helpers live in subagent-persistence.ts (pure peel; __testing unchanged). Tool-execution rows carry NO job-wide unique on the raw provider tool_use_id (migration v131 drops uniq_subagent_tools_use_id: providers like claude-cli legitimately re-mint the same short id every turn, and a collision would dead-letter the job); row identity is the stable (job_id, message_idx, ordinal) unique, and readers/settlement resolve an execution by (message_idx, tool_use_id). Settle writes (complete/failed) target exactly ONE row — the call's own ordinal first, then a legacy ordinal=NULL row, never a row that already settled complete (a broader status disjunct could only capture a same-id sibling's row); pending inserts guard legacy NULL-ordinal rows via NOT EXISTS and use the stable-id ON CONFLICT as the zombie-worker backstop. Replay/reconcile resolve an execution by persisted ordinal first (validated against the raw id), then by (message_idx, tool_use_id) only when the id is unique within the turn, then the same-tool legacy positional fallback. Mixed-version window: a still-running pre-v131 gbrain jobs work daemon errors on tool persistence after the migration until it restarts (its two-phase writes targeted the dropped constraint)

  • src/core/minions/handlers/subagent-oneshot.ts — oneshot synthesis runner. ONE rate-leased tool-less gateway.chat call (static ONESHOT_SYSTEM JSON contract rides the prompt-cache prefix; sub-budget min(5 min, timeout/4) → fallback_reason: 'oneshot_timeout'), then all-or-nothing validation BEFORE any write: JSON contract (parseOneshotResponse, ≤12 pages), slug grammar + allowed_slug_prefixes + reflections/originals task shape + the exact oneshot_slug_suffix (the idempotency boundary, enforced structurally), exact-match wikilink rule against existing ∪ in-batch slugs (extractWikilinkTargets; cold-brain relaxation: <5 pages AND no manifest accepts syntactic presence). Writes go through the SAME brain_put_page ToolDef the loop uses (fences/side-effects/provenance identical) with deferEmbeds, bracketed by standard ledger rows under invocation-scoped ids oneshot-<inv8>-p<i>; a post-batch autoLinkWrittenPage pass materializes in-batch forward wikilink edges. Ledger-first crash recovery: any prior oneshot rows → finalize from the ledger, never re-call the nondeterministic model. Transcript rows persist ONLY after success (a failed attempt is never replayable as a completed result). Every failure shape returns {kind:'fallback', reason} and the same job falls through to the agentic loop. Pinned by test/minions/subagent-oneshot.test.ts + the oneshot describes in test/subagent-handler.test.ts and test/e2e/dream-synthesize-pglite.test.ts. ONESHOT_SYSTEM spells out JSON string escaping (quotes, backslashes, line breaks). A parse failure whose reported output usage reaches the requested cap falls back as length even when the provider normalized the stop to end/other; below-cap malformed output stays unparseable, and valid JSON at the cap proceeds normally.

  • src/core/minions/handlers/subagent-persistence.ts — subagent transcript + tool-execution persistence (pure peel from subagent.ts): loadPriorMessages/Tools/ToolsV2, persistMessage, persistToolExecPending/Complete/Failed (all $N::text::jsonb discipline), plus finalizeWriteAccounting: derives pages_attempted/written/failed from the job's put_page ledger rows (settled only — pending counts toward neither), merges them into every SubagentResult, and throws UnrecoverableError on require_writes jobs whose every attempted write failed; scopeToolUseIdPrefix narrows the scan to the oneshot invocation family.

  • src/core/minions/handlers/subagent-aggregator.tssubagent_aggregator handler. Claims AFTER all children resolve (queue guarantees every terminal child posts a child_done inbox message with outcome). Reads inbox via ctx.readInbox(), builds a deterministic mixed-outcome markdown summary. No LLM call.

  • src/core/minions/handlers/subagent-audit.ts — JSONL audit + heartbeat writer at ~/.gbrain/audit/subagent-jobs-YYYY-Www.jsonl. Events: submission (one per submit) + heartbeat (per turn boundary: llm_call_started | llm_call_completed | tool_called | tool_result | tool_failed, plus the oneshot markers oneshot_fallback | oneshot_timeout; oneshot-path events carry mode: 'oneshot' and fallback events a reason — both rendered by gbrain agent logs). Never logs prompts or tool inputs. readSubagentAuditForJob(jobId, {sinceIso}) is the readback for gbrain agent logs.

  • src/core/minions/rate-leases.ts — lease-based concurrency cap for outbound providers (default key anthropic:messages, max via GBRAIN_ANTHROPIC_MAX_INFLIGHT). Owner-tagged rows with expires_at auto-prune on acquire; pg_advisory_xact_lock guards check-then-insert; CASCADE on owning job deletion. renewLeaseWithBackoff retries 3x (250/500/1000ms). Canonical home of RateLeaseUnavailableError (thrown when acquire finds no slot; the worker and the inline dream drain both recognize it and requeue WITHOUT burning an attempt — lease-full is a scheduling condition, not a failure; subagent.ts re-exports it for compatibility) and leaseFullBackoffMs() (the shared 1–3s jittered lease-full requeue backoff worker.ts and inline-drain.ts both use, so the two curves cannot silently desync).

  • src/core/minions/handlers/contextual-reindex-per-chunk.ts — per-page contextual re-embed handler. Resolves models.contextual_synopsis once and isolates cross-worker leases by the full resolved model id. GBRAIN_CONTEXTUAL_SYNOPSIS_RPM controls the cap; GBRAIN_CONTEXTUAL_HAIKU_RPM is the compatibility alias.

  • src/core/minions/handlers/embed-backfill.tsembed-backfill job handler (the deferred lane behind sync's cost gate). Cap ladder for embed.backfill_max_usd: a present-but-invalid value (0, negative, garbage) is misconfigured and FAILS CLOSED to the $10 default, which is never dropped; only the IMPLICIT default cap is lifted (uncapped-with-warning to stderr) when the embedding model is unpriceable via isModelPriceable + pricing.overrides; the off tokens (off/unlimited/none) remove the ceiling. Single-flights via the same per-source lock key as CLI embed --stale (embed-backfill-lock.ts). Carries the progress-keyed stall watchdog arm: progress = banked embedded + chunksProcessed; a stall aborts the drain and throws stall_timeout so the queue fails the job and the resumable cursor re-runs.

  • src/core/minions/wait-for-completion.ts — poll-until-terminal helper for CLI callers. TimeoutError does NOT cancel the job; AbortSignal exits without throwing. Default pollMs: 1000 on Postgres, 250 on PGLite inline.

  • src/core/minions/transcript.ts — renders subagent_messages + subagent_tool_executions to markdown. Tool rows splice under their owning assistant tool_use by (message_idx, tool_use_id) — raw provider tool ids may repeat across turns, so tool_use_id alone is not an identity; echoed tool_result blocks check ownership against the nearest preceding assistant turn's key. UTF-8-safe truncation; unknown block types fall through to fenced JSON.

  • src/core/minions/plugin-loader.tsGBRAIN_PLUGIN_PATH discovery. Absolute paths only, left-wins collision, gbrain.plugin.json with plugin_version: "gbrain-plugin-v1", plugins ship DEFS only (no new tools), allowed_tools: validated at load time against the derived registry.

  • src/core/minions/tools/brain-allowlist.ts — derives the subagent tool registry from src/core/operations.ts (13-name allow-list, size pinned by test/brain-allowlist.serial.test.ts). Attachment tools file_list and file_url are excluded. Registry construction and execution reject every localOnly operation; selectAllowedTools rejects malformed bindings and preserves explicit empty lists as no tools. Includes add_timeline_entry (the canonical timeline write), fenced server-side by the same enforceSubagentSlugFence policy as put_page. By default put_page schema is namespace-wrapped per subagent (^wiki/agents/<subagentId>/.+). When BuildBrainToolsOpts.allowedSlugPrefixes is set, the put_page schema describes the prefix list to the model and the OperationContext is threaded with allowedSlugPrefixes — trusted local jobs receive these from the submitter; remote-owned jobs receive the grant intersection plus delegatedAuth. Their per-call operation contexts carry current source grants and retain remote visibility policy, without requiring direct read/write scopes for independently bound delegated tools. BuildBrainToolsOpts.deferEmbeds (server-side-only, set by the oneshot runner for its programmatic writes; never hydrated from any wire payload) threads OperationContext.deferEmbeds so put_page defers chunk embeddings for the phase-end backfill. Allow-list includes get_recent_salience + find_anomalies but deliberately NOT get_recent_transcripts (all subagent calls run ctx.remote === true and the trust gate rejects remote callers, so it would always reject; the cycle synthesize phase calls discoverTranscripts directly instead). paramsToInputSchema() consumes paramDefToSchema from src/mcp/tool-defs.ts; required-aggregation at the tool-def level stays here (the shared helper is per-param). execute() runs the same validateParams the MCP dispatchers use before the op handler, so a call that omits a required parameter (or sends the wrong type / an unknown enum value) fails with <tool>: Missing required parameter: <name> the model can act on, instead of a handler crash.

  • src/mcp/tool-defs.tsbuildToolDefs(ops, opts?) helper; the stdio MCP server, the OAuth HTTP tools/list, and the subagent tool registry all consume it, byte-for-byte equivalence pinned by test/mcp-tool-defs.test.ts. opts.strictParams: true (when mcp.strict_params resolves 'reject') additionally declares the _meta/dry_run passthrough keys in properties and closes each schema with additionalProperties: false (schema-validating clients must not strip _meta.session_id); both emission states pinned. Exports the recursive paramDefToSchema(p: ParamDef) — single source of truth for ParamDef→JSON Schema mapping shared by buildToolDefs and src/core/minions/tools/brain-allowlist.ts (subagent registry). Recursive on items so nested array-of-arrays preserves inner shape on the wire. Key ordering (type, description, enum, default, items) is intentional so JSON.stringify output stays byte-stable. test/mcp-tool-defs.test.ts has a findArrayWithoutItems walker that fails on any type: 'array' lacking items.type.

  • src/core/verbs.ts — MEMORY_VERBS v1: the four frozen protocol verbs (remember, entity, synthesize, forget) as first-class Operations, plus MEMORY_VERBS_VERSION (single source of truth, =1), VERB_NAMES, the hand-authored RESPONSE_SCHEMAS registry (Operation carries input params only; response shapes live here and conformance validates LIVE responses against them), and ERROR_SCHEMA. The fifth verb is the extended recall op in operations.ts. RUNTIME LEAF invariant: operations.ts spreads verbOperations into its array at module-eval time, so this file must never statically import operations.ts VALUES (type-only imports fine; handlers use dynamic import) — violating it causes a TDZ crash on whichever module evaluates second. Every verb error carries a populated suggestion + protocol_version (via verbError in operations.ts). The forget verb deliberately has NO cliHints (CLI_ONLY forget dispatches first and would shadow it). Frozen contract: docs/protocol/MEMORY_VERBS_v1.md; pinned by test/memory-verbs-conformance.test.ts.

  • src/core/verbs/entity-card.tsbuildEntityCard(engine, sourceId, name, {remote}): the zero-LLM sub-100ms card behind the entity verb. Resolution reuses the Retrieval Reflex precision arms (alias > exact slug > exact title > slug-suffix; exact-slug candidates include the RAW input because slugify flattens slashes in namespaced slugs). Within exact-title collisions, canonical entity types (person, company, organization, entity) outrank note/conversation containers, then GREATEST(updated_at, last_retrieved_at) breaks same-shape ties; a lone non-entity exact-title page remains a valid fallback. Per-arm degradation: a pre-page_aliases brain still resolves via arm 2 (aka returns empty, never throws). Card assembly is a parallel Promise.all of depth-1 indexed reads (page row, alias reverse lookup, getLinks+getBacklinks mentions-excluded cap 10, getBacklinkCounts, getTimeline(5), listFactsByEntity world-only-when-remote); deliberately NOT the recursive-CTE traversePaths — the card is a latency contract (CI gate: test/entity-card-perf.slow.test.ts, p99 < 100ms × GBRAIN_PERF_BUDGET_MULTIPLIER + 50× getPage-p50 ratio guard on a 20K corpus). summary runs through the exported safeSynopsis (the get_page fence boundary). Miss → keyword near-miss suggestions with create_safety hints.

  • src/core/verbs/usage-log.ts — observability sidecar: one JSONL line per verb call at ~/.gbrain/integrations/memory-verbs/usage.jsonl (gbrainPath — GBRAIN_HOME honored; brainId() = the resolved gbrain home). LOCAL ONLY, never uploaded, stats-only (lock-free 10MB rotation may drop lines; O_APPEND line-atomic, best-effort on Windows). logVerbUsage is fire-and-forget (never awaited, never throws); written from the DISPATCH layer so param-validation failures count. readVerbUsage/earliestVerbUsageTs feed gbrain protocol stats (incl. measured TTHW vs the init-stamped protocol_installed_at) and the doctor memory_verbs_usage check.

  • src/core/verbs/conformance.ts + src/core/verbs/conformance-fixtures.ts — the conformance runner core (transport-agnostic: minimal ConformanceClient = list_tools + call_tool) and the embedded fixture set. Deterministic by construction: shape/enum/behavior/round-trip checks only, never ranking quality. Validation is NON-STRICT on extra fields (additive-forever means unknown fields are always legal). Entity-card cases seed via put_page when the target exposes it and skip honestly on verbs-only targets; synthesize is cost-gated behind --synthesize. validateAgainstSchema is a minimal JSON-Schema-subset validator (type unions, required, properties, enum, const, items). Fixtures mirror to test/fixtures/memory-verbs/cases.json (BrainBench seeds; drift-guarded by the conformance test). The negative self-test (test/memory-verbs-conformance.test.ts) proves the runner FAILS a lying server.

  • src/core/facts/write-single.tswriteSingleFact(fact, ctx): the zero-LLM single-fact seam behind remember. runFactsPipeline is extraction-first (LLM-gated) and cannot back a pre-formed fact; this reuses the pipeline's post-extraction stages directly: resolve → embedding-cosine dedup (same 0.95 threshold) → fence-first write with the same legacy DB-only fallbacks (thin-client, unparented, stub-guard). Supersession: deterministic rule — same entity_slug + same kind + similarity ≥ threshold + DIFFERENT text ⇒ the new fact supersedes (fence path: append new + forgetFactInFence(old) + superseded_by link; DB path: engine insertFact supersedeId). Provenance lands on NewFact.source verbatim (no FactsBackstopCtx). No embedding provider ⇒ degraded_dedup: true (near-duplicates may insert — documented). isNullLikeEntity(entity) (exported) normalizes null-like entity tokens ("null", "undefined", "none", "n/a", "nil", "-", empty/whitespace) to ABSENT before resolution — applied here rather than only at the verb boundary so EVERY writeSingleFact caller gets the guard and the resolved?.slug ?? entityRef fallback can never adopt "null" as a slug.

  • src/core/facts/forget.tsforgetFactInFence(engine, id, {reason?, sourceId?, worldOnly?}) checks source and visibility before inspecting state, then commits a durable withdrawal through facts/withdrawal.ts. A writable source fence is atomically struck with a date and reason; otherwise the DB-body mirror is best-effort. Both paths remain withdrawn on stale reimport while the withdrawal database is preserved. The file resolver and per-page lock match fence writes. The body-only mirror deliberately leaves a hash mismatch so the next import re-chunks. Forget is retraction, not deletion of original prose, files or backups. Pinned by test/forget-reconcile-durability.test.ts, test/privacy-strip-and-forget.test.ts, and test/e2e/delegated-grants-withdrawal.test.ts.

  • src/core/facts/withdrawal.ts + withdrawal-schema.ts — durable withdrawal key is (source_id, visibility, SHA256(normalized claim)); entity/page names are excluded so rename and index recreation do not restore it. Normalization folds case and whitespace, not meaning. The source row lock serializes withdrawal with the facts trigger; the trigger prevents insert/update from reactivating a matching claim. Import overlays valid matching fence rows as struck before hashing/chunking; malformed fences retain their existing diagnostics and the DB trigger still protects derived facts. writeSingleFact refuses an identical withdrawn claim before embedding, using the protocol error envelope. Ordinary TTL/supersession expiry creates no withdrawal. Migration backfill only trusts explicit forgotten: markers; old unmarked DB-only intent cannot be inferred. Markdown-only copies do not preserve DB-only withdrawals.

  • src/core/minions/delegated-policy.ts — immutable submitted grant ceiling intersected with the current OAuth grant. Tools, explicit operation authority, source-only reads, write prefixes, concurrency and finite budget may narrow but never widen during execution; changed source/brain, revoked clients, archived sources or unusable intersections fail closed. Explicit job namespaces resolve to wiki/agents/<job-id>/*. Removing submit_agent stops the next execution boundary; explicit operation allowlists also constrain delegated tools, while legacy null operation lists preserve separately bound agent-only tools. Surface controls transport discovery. Empty tools never mean all tools. Legacy running jobs adopt the ceiling frozen in their persisted payload by grant rescope; host/current brain aliases mean the serving brain.

  • src/core/minions/delegated-admission.ts + delegated-tools.ts — queue admission locks the client row through validation, idempotency/coalescing, counting all nonterminal owned jobs across queues, and insertion. Returning an identical existing job adds no slot. Every delegated tool execution, including replay and one-shot writes, rechecks the current intersection and rebuilds the operation context with its current source/path bounds and authenticated read-source ceiling, including explicit per-call source overrides. Remote-owned writes cannot overwrite private existing pages (including dedup redirects), append their timeline, or run trusted-workspace auto-link/timeline hooks. One-shot remote-owned writes keep links as literal text without probing hidden target existence or counts; trusted local behavior is preserved. Trusted owner fields are separate from user parameters. Replay cannot replace identity or original bounds; valid owned replays and terminal-job retries consume a slot under the same client lock. Legacy terminal jobs missing a stored ceiling require fresh authenticated submission.

  • src/core/minions/delegated-spend.ts + src/core/ai/invocation-guard.ts + guarded-generation.ts — async-local admission around each owned provider attempt, including nested paid tool work; unrelated local calls retain their transport behavior. Finite caps require canonical pricing and a declared input ceiling plus enforced output maximum before IO; unknown pricing/bounds refuse. Hidden SDK retries are disabled for owned attempts. Actual reported usage is settled with cache categories normalized across SDK/raw responses; absent usage remains explicit unresolved liability. The current grant is checked again under the reservation's client lock.

  • src/core/minions/budget-meter.ts — atomic reserve/settle accounting. Per-client advisory lock plus client row lock protects current cap and outstanding holds. NULL cap means unlimited; a caller's existing finite ceiling still applies. Pending and expired unresolved holds count across UTC dates; TTL marks overdue rather than releasing liability. Unknown-bound holds prevent subsequent finite admission until reconciled. Settlement is idempotent and writes reservation state plus spend log in one transaction. Tests cover races, changed caps, unknown usage, late settlement and both engines.

  • src/core/facts/extract.ts — the LLM facts extractor (extractFactsFromTurnWithOutcome and friends; the output cap, truncation retry, and salvage behavior are described under the trajectory entry). Deterministic junk gate: isJunkFact(text, kind?) tests JUNK_FACT_PATTERNS — assistant plan/offer narration, meta-narration about the conversation itself, provider billing/rate-limit strings captured verbatim — and is kind-aware: a candidate the extractor classified commitment is exempt from the plan-narration arm ONLY ("I'll send the deck by Friday" is the surface shape of a genuine commitment, the one kind the loop engine exists to capture); meta-narration and provider-error strings are junk for every kind. isJunkFilterEnabled(engine) is the config kill switch (fail-open on a config read error); getFactsExtractionPromptAppendix supplies the config-driven prompt appendix. Entity gate at the candidate loop (the ONE seam where LLM-emitted entity tokens become entity_slug): isUnknownSpeakerLabel (anonymous diarizer labels — Speaker A, SPEAKER_00, spk_0, other|unknown|guest) and isNullLikeEntity (placeholder STRINGS "null"/"None"/"n/a" where the prompt asked for JSON null; shared with write-single.ts) both null the attribution and KEEP the fact, so no consumer (runFactsPipeline, extract-conversation-facts) can mint entity_slug='null' through the resolver's fallback-slugify floor. The provider-error arm is start-anchored (^\W*, optional error/status token or a <step> stopped|failed because … prefix, then (spend|rate) (limit|cap) … (hit|exceeded|reached)), so a genuine fact that merely mentions a spend limit or rate limit mid-sentence survives; the verbatim provider strings still match. Two config keys: facts.extraction_prompt_appendix (non-empty text is appended to the extraction prompt, so an operator can steer what counts as a fact without a release) and facts.extraction_junk_filter (false disables the deterministic junk gate; default on).

  • src/mcp/surface.ts — MCP tool-surface modes: 'verbs' (exactly the ops marked verb: true) | 'starter' (the ~27-op daily-driver set) | 'full' (default — identity; existing installs unchanged). STARTER_OPS is composed PROGRAMMATICALLY: a spread of VERB_NAMES (never a hand-count) + the fallback daily slice (BRAIN_TOOL_ALLOWLIST + the agent lane submit_agent/get_agent_job) + whoami + request_tools + capture (a DIRECT literal, deliberately NOT via the allowlist); re-derived from production usage via scripts/derive-starter-ops.ts (paste-in proposal, never an auto-edit). ALWAYS_INCLUDED_STARTER_OPS (verbs + whoami + request_tools + the agent lane + capture) is the exported always-set consumed by BOTH derive-starter-ops.ts and the advisor starter-fit collector, so neither re-types the composition. parseSurfaceFlag (strict, loud reject), resolveSurface (flag > config mcp_surface > full), filterOpsForSurface, allowedOpNames. Enforcement is TWO-layer and fail-closed: the advertised list AND dispatchToolCall's allowedOps set (a hidden op returns unknown_tool even when called by name) — applied on stdio (server.ts) and BOTH HTTP paths (serve-http.ts after !localOnly, and the second transport http-transport.ts). Per-client ceiling machinery (OAuth transport only): effectiveSurfaceForClient is the pure composition clamp(min(server ceiling, client row surface ?? mcp.default_surface_dcr ?? ceiling)); the per-REQUEST application lives in serve-http.ts's resolveEffectiveSurface, which short-circuits a verbs ceiling (min() cannot go lower, so the config read is skipped) and, on a default-surface read failure, applies the LAST successfully read default (held per process, fail-closed) so a transient config outage cannot silently widen a NULL-surface client to the ceiling; resolveClientRowSurface ignores unknown row values with a bounded warn-once (the column's value space is documented OPEN for future tier names); resolveDefaultClientSurface reads the DCR default dual-plane (DB > file > null) and never throws; clampSurface/readForceSurfaceEnv fold in the GBRAIN_MCP_FORCE_SURFACE kill switch — NARROW-ONLY by construction (min(); can never widen past the ceiling). minSurface/surfaceWiderThan own the verbs < starter < full rank math. Pinned by test/mcp-surface.test.ts (membership, monotonicity verbs ⊆ starter ⊆ full, ceiling + force-narrow cases).

  • src/commands/protocol.tsgbrain protocol [--json] | conformance [--target <http-url|stdio-cmd>] [--token] [--synthesize] | stats [--days N]. --json emits input schemas from the LIVE Operation defs + RESPONSE_SCHEMAS (doc/code can't drift). Conformance default target self-spawns gbrain's own stdio server (dev .ts entry vs compiled binary both handled); CI certifies stdio with --synthesize (no key ⇒ asserts the clean unavailable error). Stats aggregates the usage sidecar + the measured TTHW; output states "local JSONL only — never uploaded". CLI_ONLY + SELF_HELP wired in cli.ts; no pre-bound engine.

  • src/core/minions/attachments.ts — Attachment validation (path traversal, null byte, oversize, base64, duplicate detection).

  • src/commands/agent.tsgbrain agent run|logs|register dispatcher. run submits subagent (or N children + 1 aggregator) under {allowProtectedSubmit: true}; single-entry --fanout-manifest short-circuits; children get on_child_fail: 'continue' + max_stalled: 3; --follow is the default on TTY (streams logs + polls waitForCompletion in parallel; Ctrl-C detaches, does not cancel). logs delegates to agent-logs.ts; register lazy-imports agent-register.ts. Subcommand-aware help is answered BEFORE any engine or queue work and STOPS at the -- terminator (agent run -- --help submits the LITERAL prompt); on a brainless machine (null engine) help still prints while real run/logs invocations refuse with an init hint.

  • src/commands/agent-logs.tsgbrain agent logs <job> [--follow] [--since]. Merges JSONL heartbeat audit + subagent_messages into a chronological timeline. parseSince accepts ISO-8601 or relative (5m, 1h, 2d). Transcript tail renders only for terminal jobs.

  • src/commands/agent-register.tsgbrain agent register <name> --harness claude-code|codex|opencode|openclaw: mints a scoped OAuth client + access token and prints the exact harness wiring in one step (CLI-only, never an MCP op). Composes existing parts — registerScopedClient (auth.ts), exchangeClientCredentials, the mcp-registration argv builders, renderCodexHttpServerBlock, openclawThinClientBlock — no new auth machinery. Order is load-bearing: cli.ts pre-connect guards (thin-client refusal + PGLite live-serve probe) → pure parse (exit 2) → preset resolve → source validation (existence + not-archived, engine lane ANY(\$1::text[])) → column pre-flight OUTSIDE any tx (25P02 forbids in-tx degrade) → ONE engine.transaction (name-scoped pg_advisory_xact_lock → duplicate-name pre-check → ensureWorkspaceSource create-or-reuse-only-when-truly-empty (refuses page-, fact-, or file-bearing and archived sources) → registerScopedClient with ttl + surface) → COMMIT → post-commit fail-open surface audit → token exchange on the OUTER engine (the tx sql is dead) → serve probe (probeServeHealth + the unconditional SCOPES_MIN_SERVE_VERSION floor line): a probe-PROVEN pre-scopes serve REFUSES registration (serve_too_old) unless --allow-old-serve — such a serve verifies scoped tokens as full access; an unreachable serve stays a warning, and the reissue lane warns instead of refusing (the secret is already rotated by that point) → render + print (human, or ONE JSON doc carrying probe_note + serve_warning from the probe; secrets redacted unless --show-token). Presets: daily-driver (write one source; federated reads = SNAPSHOT of non-archived sources at registration, EXCLUDING other agents' *-workspace scratch sources — a workspace named explicitly in --federated-read is still granted, and the print counts the exclusions) and coding-agent (write-isolated derived <name>-workspace DB-only source; requires --federated-read); both default the client to the starter surface — override with --surface at registration or widen per client via gbrain auth rescope-client. Always writes token_ttl (default 30 days — the server default is 1 hour and would kill a pasted config). --url|--port required (every block embeds the brain URL). A scope blocklist keeps operator-grade scopes on auth register-client. --reissue <client-id> rotates the secret under the same advisory lock and reprints the block (headed by the stored client_name); rotation is not revocation — outstanding tokens live until expiry. Failure after COMMIT prints the client_id + the exact revoke command (never a false "nothing was created"). openclaw renders the honest thin-client CLI block (no native remote-MCP client upstream yet). Pinned by test/agent-register.test.ts.

  • src/commands/jobs.tsgbrain jobs CLI subcommands + gbrain jobs work daemon. Help is real and guarded: JOBS_HELP (full block incl. watch/stats/smoke flags + a footer naming exactly the five subcommands with dedicated help) and JOBS_SUBCOMMAND_HELP (work/supervisor/submit/watch/prune) print from a guard at the TOP of runJobs, BEFORE the thin-client refusal and the subcommand switch — jobs work --help can never start a daemon; only --help/-h are help tokens (bare help can be a job name); cli.ts routes it engine-free via SELF_HELP_WITHOUT_ENGINE + CLI_ONLY_SELF_HELP. formatJobDetail prints the effective wall-clock budget (Timeout:/Deadline: — 1x deadline kill when claimed, 2x wall-clock backstop, or which default applies) with a Date|string-defensive renderer; timeout_at is in JOB_DATE_FIELDS for thin-client rehydration. jobs stats prints a Backpressure (24h) line (per-name coalesce counts from the backpressure audit, current+previous ISO-week files, queue-filtered, best-effort) plus a suppressed-by hint naming the in-flight live-lock job when waiting=0 past the shared GBRAIN_WEDGED_QUEUE_WARN_MINUTES threshold — the read-only visibility for maxPending single-flight suppression. Pinned by test/jobs-subcommand-help.serial.test.ts, test/jobs-format-detail.test.ts, test/jobs-stats-backpressure.serial.test.ts. case 'work' wraps worker.start() in try/finally and owns engine lifecycle — calls engine.disconnect() on shutdown with loud error logging (the worker must not disconnect an engine it doesn't own; pool slots free immediately on shutdown rather than waiting for TCP keepalive). jobs submit surfaces the MinionJobInput retry/backoff/timeout/idempotency surface as flags: --max-stalled, --backoff-type fixed|exponential, --backoff-delay, --backoff-jitter, --timeout-ms, --idempotency-key, --max-waiting (maxPending is deliberately internal-only — no flag; see TODOS). jobs smoke --sigkill-rescue is the SIGKILL-rescue guard. registerBuiltinHandlers always registers subagent + subagent_aggregator (no env flag — ANTHROPIC_API_KEY is the cost gate, trust is via PROTECTED_JOB_NAMES) and loads GBRAIN_PLUGIN_PATH plugins at startup with a loud per-plugin line; shell handler still gated by GBRAIN_ALLOW_SHELL_JOBS=1 (RCE surface). The autopilot-cycle handler forwards job.data.phases to runCycle, validated against ALL_PHASES from src/core/cycle.ts (invalid names filtered; empty/missing falls back to the default cycle); when source_id is set it binds brainDir to that source's local_path (null for a pure-DB source, never the global repo) and checks isSourceInCooldown before runCycle, returning a no-op skipped (not a failure) for a source still in its failure cooldown. The sibling autopilot-global-maintenance handler runs MAINTENANCE_PHASES (mixed ∪ global) once (no sourceId, pull:false) and stamps autopilot.last_global_at on success. resolveJobPull gives both cycle and standalone sync jobs one positive-polarity pull contract while preserving queued payloads that still carry the inverse legacy noPull key; explicit pull wins. The sync handler resolves sourceId at entry from sources.local_path (mirrors cycle.ts:480) so multi-source brains read the per-source last_commit anchor; concurrency routes through autoConcurrency() in src/core/sync-concurrency.ts (PGLite stays serial); noEmbed default is true. gbrain jobs supervisor status consumes summarizeCrashes() from src/core/minions/handlers/supervisor-audit.ts for parity with gbrain doctor: JSON adds crashes_by_cause: {runtime_error, oom_or_external_kill, unknown, legacy} + clean_exits_24h; human output includes per-cause + clean-exits lines. Pinned by test/job-pull-policy.test.ts and 4 source-grep wiring assertions in test/doctor.test.ts requiring crashes_by_cause + clean_exits_24h= in both doctor.ts and jobs.ts. gbrain jobs watch decouples its two output axes: --json picks FORMAT (human default, never gated on isTTY), --follow picks LOOP (default isTTY && !json). Non-TTY with no flags prints ONE human snapshot then exits (clean for subagent/pipe/cron); --follow opts into a continuous stream (human plain per tick, or JSONL with --json); a TTY with no flags keeps the live ANSI dashboard. Resolution is the pure resolveWatchMode(opts, isTTY): {json, follow, useAnsiDashboard} in src/commands/jobs-watch.ts; the dispatch wires --follow. Pinned by test/jobs-watch-mode.test.ts (format×loop matrix incl. the TTY+--json-one-shot case) + test/e2e/non-tty-output.serial.test.ts (the cmd </dev/null non-empty-stdout contract). registers 11 Minion handlers: reindex, repair-jsonb, orphans, integrity, purge, synthesize (PROTECTED), patterns (PROTECTED), consolidate (PROTECTED), extract_facts, resolve_symbol_edges, recompute_emotional_weight. Phase wrappers delegate to runCycle({phases:[name]}) so src/core/cycle.ts stays the single source of truth for phase semantics. The standalone sync handler passes noExtract: true to match runPhaseSync's contract (doctor's remediation plan emitting [sync, extract] would otherwise double-extract). The extract handler routes {stale: true} jobs (submitted by performSync's size-gate defer branch) through extractStaleFromDB scoped to data.sourceId, chaining a continuation job (no maxWaiting — same NULL-sourceId coalesce hazard; timeout derived from STALE_TIME_BUDGET_MS) when the sweep's budget leaves staleRemaining > 0 with forward progress. parseJobIsolationFlag(args, env?) (--job-isolation, space/= forms, GBRAIN_JOB_ISOLATION fallback, default inline); case 'work' resolves + fail-fast validates the child CLI invocation and warns when --max-rss is combined with isolation (watchdog covers the worker only); case 'run-child' is the INTERNAL child entrypoint (quiet registerBuiltinHandlers incl. plugin discovery, CLI owns disconnect+exit); startup warning when a Supabase-shaped engine runs single-pool (kill-switch collapse made loud); the db_dead fatal text is verdict-aware (pool starved vs server unreachable). Generic jobs submit normalizes the job name once before trust, queueing, handler lookup, and audit; after the admission check and dry-run return, its explicit ensureSchema() preflight still runs before shell/handler validation so stale brains retain the canonical init/migration error precedence. On PGLite, jobs submit embed-backfill refuses with no_worker_surface and a paste-ready inline embed command before any queue access; --dry-run evaluates the same read-only feasibility gate and never says “Would submit” for an impossible job. --follow is the sole CLI exception because it starts and awaits the inline worker; padded names follow the same normalized path and cannot strand a row. The MCP submit_job operation applies the same gate (including dry-run), translates the expected refusal to branchable OperationError('no_worker_surface'), and never emits internal_error for this capability miss. The sync handler forwards job.data.github_item ({repo, number, kind}) into performSync for github-kind single-item webhook refreshes.

  • src/commands/features.tsgbrain features --json --auto-fix: usage scan + feature adoption salesman.

  • src/commands/autopilot.tsgbrain autopilot --install: self-maintaining brain daemon (sync+extract+embed). Freshness sync jobs always send an explicit positive-polarity pull value derived from the source's parsed remote_url, so local-only sources skip pull and PGLite JSON-string configs behave like Postgres objects. Consumes detectTini() from src/core/minions/spawn-helpers.ts, resolved once at startup. Composes a ChildWorkerSupervisor instance for spawn-and-respawn with --max-rss 2048 and maxCrashes: 5. onMaxCrashesExceeded routes through autopilot's own shutdown('max_crashes') so the autopilot lockfile gets cleaned up. shutdown() drains via childSupervisor.killChild('SIGTERM') + awaitChildExit(35_000). Pinned by test/autopilot-fanout-wiring.test.ts and test/autopilot-supervisor-wiring.test.ts (6 static-shape guards: composes ChildWorkerSupervisor not legacy names, --max-rss 2048 in argv, maxCrashes: 5 literal, shutdown-via-callback, no workerProc reference). tick body invokes runNightlyQualityProbe when cfg.autopilot.nightly_quality_probe.enabled === true (default OFF — opt-in to protect API spend). NO scheduler-side rate-limit check — runNightlyQualityProbe's internal shouldRunNightly (reading the audit JSONL) is the single source of truth. Probe call wrapped in try/catch that logs via logError and does NOT bump consecutiveErrors (probe failure is informational, never crashes the loop). Default max_usd cap = 5. Pinned by test/autopilot-nightly-probe-wiring.test.ts. per-source extract_atoms auto-drain. Postgres-only block after the freshness fan-out: gated on autopilot.auto_drain.enabled (default true) AND !packDeclaresPhase(engine,'extract_atoms') (the silent-backlog condition) AND per-source countExtractAtomsBacklog > threshold (default 25) AND a daily cap floor(max_usd_per_day / ~\$0.30). Enumerates loadAllSources. Submits the PROTECTED extract-atoms-drain job ({allowProtectedSubmit:true}) with a UTC-day time-sloted idempotency key autopilot-extract-atoms-drain:<src.id>:<utcDay> (a static key would block the source after the first job completed). src/core/minions/protected-names.ts lists extract-atoms-drain; src/commands/jobs.ts registers the handler (thin wrapper over runExtractAtomsDrainForSource, LockUnavailableError{deferred:true}; a provider_failure result throws formatDrainProviderFailure(result) — batches/remaining plus the drain's sanitized last_error — so the dead-lettered job's error_text names the cause, e.g. a missing provider key); src/core/config.ts carries the autopilot.auto_drain.* config keys + the autopilot. key prefix. Pinned by test/extract-atoms-drain-handler.test.ts, test/autopilot-auto-drain-wiring.test.ts. federated-brain co-existence + launchd hygiene. (1) LOCK_PATH resolves via gbrainPath('autopilot.lock') so it honors GBRAIN_HOME (two brains can run autopilot simultaneously without lock-stealing); lock file stores PID, startup checks kill -0 <pid> before refusing to start (a stale lock from a crashed process does not block). (2) exported classifyReconnectError(err) returns 'recoverable' | 'unrecoverable'; unrecoverable causes process.exit(0) so launchd backs off instead of looping config.database_url undefined. (3) exported pure generateLaunchdPlist(wrapperPath, home) sets ThrottleInterval=300 so launchd respects the exit-0 backoff. Pinned by test/autopilot-lock-path.test.ts + test/autopilot-reconnect-classifier.test.ts. targeted-submit loop instead of blanket autopilot-cycle dispatch. Each tick: cheap engine.getHealth() (single SQL count) + computeRecommendations(), then route by shape — score >= 95 AND no plan AND <60min since last full → sleep; score >= 95 AND >=60min → submit autopilot-cycle (60-min floor exercises phase-coupling invariants on healthy brains); plan <= 3 steps AND est <5min → submit individual handlers; plan large OR score < 70 → submit full autopilot-cycle. The gbrain-cycle lock ensures targeted submissions and the full cycle can't run concurrently. maxWaiting: 1 per submit closes the queue-fan-out vector. daemon env lane: writeWrapperScript additively sources <gbrainDir>/env (honors GBRAIN_HOME) after the shell profiles with a set -a wrap so dotenv-style KEY=value lines export too, and creates a fully-commented 0600 template on install (never overwritten, never chmod'd, never removed by uninstall; template excludes GBRAIN_HOME — the wrapper bakes it AFTER sourcing). Exported pure chatBootWarning(chatAvailable, gbrainDir) derives both remediation paths from the passed dir and prints via console.log at daemon boot (stdout is the autopilot.log sink on all four targets; stderr lands in the unsurfaced autopilot.err) using the bare global-model isAvailable('chat') probe on purpose (mirrors the bare-gated phases; facts extraction is model-aware and doctor owns it). Install is reload-safe so the warning's re-run---install remediation is true: launchd unloads before load (bare load errors on a loaded agent), systemd try-restarts after enable --now, cron/container print a residual-process notice instead of auto-killing. Pinned by test/autopilot-install.test.ts (16 cases incl. real-bash wrapper execution).

  • src/mcp/instructions.ts — canonical MCP initialize-time agent operating contract. It is compile-time source text (no runtime filesystem dependency; static imports of pure core leaves are fine — the rule bans filesystem/dynamic loading) and is imported by the stdio SDK server, OAuth HTTP SDK server, and legacy raw HTTP initialize path so all transports return the same instructions without drift. The contract covers skill routing, untrusted-content posture, source scope, and the put_page whole-page replacement/read-before-write rule. Two append-only extensions compose on top of it, in a fixed order: buildMcpInstructions({writeback?}) adds the opt-in ambient-writeback section (from src/core/facts/writeback-instructions.ts) when memory.auto_writeback is enabled — absent/off is BYTE-IDENTICAL to GBRAIN_MCP_INSTRUCTIONS, and three exact-equality transport pins depend on that; resolveMcpInstructions(config, env?, { writeback? }) takes that composed base and appends the operator-set deployment identity LAST under a Deployment identity: banner (no transport can weaken the contract): GBRAIN_MCP_INSTRUCTIONS env wins when non-blank, else mcp.instructions from the FILE plane (~/.gbrain/config.json; gbrain config set|unset mcp.instructions route there via FILE_PLANE_DOTTED_KEYS in src/commands/config.ts); an empty or whitespace-only env value is UNSET, not an override, and blank identity + writeback off returns the byte-identical canonical contract. Every transport calls resolveMcpInstructions with its own writeback opts: stdio once at boot via the fail-closed resolver (restart-to-flip, the strict-params posture; read failure = OFF bundle, never a wrong visibility posture; a running gbrain serve also needs a restart to pick up a new identity); OAuth per request (write-scope-gated, and the section renders only when THIS token's actual visible set — surface filter + bound-client fence — can call remember; extract_facts advertised under the same predicates); legacy bearer per initialize (surface-clamped: a surface without remember gets no section). All three route availability through ambientOptsFrom(wb, {remember, extractFacts}). Pinned over a real SDK StdioClientTransport by test/e2e/serve-stdio-roundtrip.test.ts, over raw HTTP by test/http-transport.test.ts, the identity plane (incl. the three-way composition order) by test/mcp-server-identity.test.ts + test/config-set-mcp-instructions.test.ts, and the enabled-state builder/parity matrix by test/mcp-instructions-writeback.test.ts.

  • src/mcp/server.ts — MCP stdio server (generated from operations). Its SDK Server is initialized with the shared contract from src/mcp/instructions.ts, so a real client's getInstructions() is populated during the initialize handshake. createDefaultWriteAdvisory(engine, {enabled, write?}) is the stdio lane's once-per-process unscoped-default-write advisory: on a mutating call whose source resolved to tier seed_default it runs assessUnscopedDefaultWrite (src/core/source-resolver.ts) and prints the warning to stderr; it latches on the first SUCCESSFUL assessment whatever the verdict (the aggregate is expensive and its inputs are process-stable), never latches on a failed one (fail-open for that write; the next mutating seed_default call retries), coalesces concurrent calls onto one in-flight assessment, and never blocks a write (a throwing writer still counts as assessed). Non-mutating calls and non-seed tiers return early without latching. Tool-call handler delegates to dispatchToolCall from src/mcp/dispatch.ts so stdio + HTTP transports share one validation, context-build, and error-format path. Stdin 'end' / 'close' shutdown hooks are skipped when process.env.MCP_STDIO === '1' — gateway-piped stdio MCP wrappers (OpenClaw's bundle-mcp) pipe the handshake then close their stdin half, which would otherwise kill the server before the first tool call; signal handlers (SIGTERM/SIGINT/SIGHUP) + the parent-process watchdog still cover legitimate disconnects. src/commands/serve.ts exposes ServeOptions.mcpStdio?: boolean as a test seam so the guard is exercisable without process.env mutation. The resolve-IPC listener (retrieval-reflex resolve / turn_context / context_pack + the serve-delegated sync and sweep kinds) binds through src/mcp/resolve-ipc-binding.ts (next entry) on BOTH serve transports — stdio here and gbrain serve --http; the shutdown chain awaits shutdownDelegatedSync() before engine.disconnect(). Pinned by test/serve-stdio-lifecycle.test.ts and test/e2e/serve-stdio-roundtrip.test.ts.

  • src/mcp/source-preflight.tsassertStdioSourceBindable(engine, env?): stdio-lane boot preflight called first thing in startMcpServer, before any transport attaches. A well-formed GBRAIN_SOURCE that names no ACTIVE source row (SELECT id FROM sources WHERE id = \$1 AND archived = false, the same predicate as assertSourceExists in src/core/source-resolver.ts) throws a message naming the value and the fix (gbrain sources list), so serve exits 1 instead of binding every call to a phantom scope where reads return [] and writes die on the sources FK while health stays green. Exemptions mirror resolveMcpStdioSourceScope: unset env, __all__, and malformed values are left to the resolver; an engine error fail-opens (guards config, not connectivity); a degraded engine (isEngineDegraded) is never touched so the boot does not spend the proxy's reconnect attempt. HTTP is out of scope (tokens carry their own source grant). Pinned by test/mcp-stdio-source-preflight.test.ts (helper cases + a real startMcpServer integration case with a tmp GBRAIN_HOME).

  • src/mcp/resolve-ipc-binding.ts — shared resolve-IPC listener wiring for BOTH serve transports. bindResolveIpcForServe(engine, defaultSource) binds the retrieval-reflex resolve, turn_context, and context_pack kinds (socket + secret keyed off hash12(database_url) under ~/.gbrain/run via resolveSocketPathForConfig; the bound-source posture rejects requests naming any other source) plus the serve-delegated sync (sync_start/sync_status/sync_abortsrc/core/serve-sync-runner.ts) and maintenance-sweep (sweep_start/sweep_statussrc/core/serve-sweep-runner.ts) kinds — each delegation family in its OWN try/catch, so a runner failure logs [serve-sync]/[serve-sweep] handlers unavailable and the core kinds still start; both families share the GBRAIN_SERVE_SYNC_IPC=0 kill switch. Engine-uniform: Postgres brains listen too (the hook lane is engine-free by design, so IPC through a serve is its only DB path there). Best-effort by contract — a bind failure never blocks the serve, and a socket already owned by a live listener (another serve) returns the null binding (that serve is the IPC provider). Reflex-channel event logging fires at DELIVERY (onDelivered/onTurnContextDelivered), never inside the resolver. Pinned by test/resolve-ipc-binding.test.ts.

  • src/mcp/dispatch.ts — shared tool-call dispatch consumed by both stdio (server.ts) and HTTP transports. Exports dispatchToolCall(engine, name, params, opts), buildOperationContext(engine, params, opts), and re-exports normalizeOptionalParams/validateParams from src/mcp/validate-params.ts (the validation module — call order normalize → validate → findUnknownParams is load-bearing). Single source of truth for (ctx, params) handler arg order and the OperationContext shape. Defaults remote: true (untrusted); local CLI callers pass remote: false. Deny layers, in order: opts.allowedOps surface enforcement (hidden op → unknownToolEnvelope, byte-identical to a nonexistent op; did-you-mean candidates drawn ONLY from the caller's visible surface minus localOnly minus gated names), unknown op, the localOnly transport backstop (op.localOnly && transport !== 'stdio' → same envelope; stdio IS the local surface), param validation, strict/warn unknown-arg handling (reject mode returns invalid_params with suggestions; warn mode attaches _meta.warnings + a model-visible notice block), then enforceBoundClientOpAllowList (the bound-client fence) inside the handler try. _meta assembly per docs/protocol/MCP_META_CHANNELS.md: handler-emitted keys via ctx.emitResponseMeta (retrieval, warnings) attach first and independently of the metaHook (brain_hot_memory, built by src/core/facts/meta-hook.ts — its 30s cache is keyed by ENGINE IDENTITY + source + visibility tier + session + allow-list hash, so a process hosting multiple engines can never serve one brain's cached facts into another's envelope; each entry's deadline is additionally CLAMPED to the earliest retained valid_until, so a fact expiring inside the window never outlives its read-time TTL by cache staleness), each producer isolated so one failure never drops another's key; empty retrieval results get a SECOND model-visible content block via buildEmptyRetrievalBlock. Exports isListLevelDenialEnvelope(parsed) — the honest-catalog metric classifier: true for op-level denials the tools/list filter should have prevented (detail: 'config_key=...' publish-gate backstop, detail: 'fence=op' fence op-level deny), false for argument-level denials; serve-http logs matches as status='denied_after_list'. Exports requestLogStatusForResult(result) — the ONE request-log status decision (success / success_with_warnings / denied_after_list / error); BOTH HTTP transports (serve-http.ts and the legacy-bearer http-transport.ts) route every tools/call row through it so the honest-catalog metric sees all HTTP traffic (pinned by test/denied-after-list.test.ts + test/http-transport.test.ts). Also exports summarizeMcpParams(opName, params) — privacy-preserving redactor for mcp_request_log and the admin SSE feed, returns {redacted, kind, declared_keys, unknown_key_count, approx_bytes}. Intersects submitted top-level keys against the operation's declared params allow-list (declared keys preserved sorted; unknown keys counted but never named, closing the attacker-controlled-key-name leak). Byte counts bucketed up to nearest 1KB so an attacker can't binary-search secret-content sizes by probing. Raw payload visibility is opt-in via gbrain serve --http --log-full-params (loud stderr warning). Logging paths route through this helper, not JSON.stringify(params).

  • src/mcp/validate-params.ts — param normalization + validation for dispatch (direct unit surface). validateParams (required/type/enum — enum membership is a TYPE error in BOTH strict modes; the caller's raw value is never echoed into the message), normalizeOptionalParams (null/'' optional-param idioms become truly absent, copy-on-write), findUnknownParams (unknown top-level keys on the NORMALIZED object; UNKNOWN_PARAM_ALLOWLIST = _meta + dry_run; per-op did-you-mean from the op's OWN declared params only), buildUnknownParamWarnBlock (the model-visible warn notice), resolveStrictParamsMode(engine, config) (dual-plane mcp.strict_params, DB > file > 'warn'; a failed DB read applies the LAST successfully read DB mode, held per process — fail-closed, so a transient config outage on a reject-mode server cannot re-open the warn grace period; test seam resetStrictParamsModeCache()). Privacy: raw unknown key names reach the CALLER only, never mcp_request_log. Pinned by test/validate-params.test.ts.

  • src/mcp/publish-gates.ts — publish-gate resolution for the honest tools/list. readPublishGate(engine, config, key): dual-plane (DB > file > false), never throws — a FAILED read resolves false (hide-on-doubt matches the default-off consent posture). disabledOpsForPublishGates(engine, config): the op-name set tools/list subtracts; one getConfig read per distinct gate key per call, deliberately NOT memoized so gbrain config set mcp.publish_skills true takes effect on the next list without a restart. Call-time gates inside the handlers stay as the fail-closed backstop (their denials carry detail: 'config_key=<key>' — the machine-readable denial grammar). Pinned by test/publish-gates.test.ts.

  • src/mcp/tool-catalog.tsrenderToolCatalogMarkdown(): the docs/TOOL_CATALOG.md renderer. Config-independent + deterministic (no engine/config reads, no timestamps): non-localOnly ops grouped one section per Operation.area, per-op first-sentence description (from buildToolDefs's non-strict shape), scope, STARTER_OPS membership, publish-gate key. Generated by scripts/generate-tool-catalog.ts; freshness-guarded by scripts/check-tool-catalog-fresh.sh in bun run verify (the METRIC_GLOSSARY pattern). Pinned by test/tool-catalog.test.ts.

  • src/core/surface-audit.tswriteSurfaceChangeAudit(engine, audit): the surface-mutation audit trail. Every surface mutation (rescope CLI, POST /admin/api/rescope-client, request_tools persist) writes one typed mcp_request_log row — operation='surface_change', params a RAW object {actor, client_id, old, new, via} via executeRawJsonb (never JSON.stringify into ::jsonb). Zero new DDL; rides idx_mcp_log_time_agent + the retention TODO. Best-effort: a failed audit write warns to stderr, never fails the committed mutation (callers needing fail-closed semantics inspect the returned boolean). CLI-actor rows are written even though stdio ops don't otherwise log (documented exception — audit outranks the transport-logging convention). The usage reader excludes these rows from op-call stats.

  • src/core/mcp-usage.tsreadClientOpUsage(engine, {days}): the ONE shared reader over mcp_request_log, consumed by gbrain auth clients --usage, the advisor mcp-client-fit collector, and scripts/derive-starter-ops.ts. Encodes the row-hygiene rules once (normalizeLoggedOperation): JSON-RPC method rows (tools/list, initialize, …) and surface_change audit rows drop; the legacy tools/call:<name> prefix strips to the op name, and the hygiene check re-runs on the stripped name (tools/call:tools/list is still not an op call). Only status IN ('success','success_with_warnings') rows count as usage — denied or erroring traffic cannot "use" its way into starter derivation or advisor fit findings. Windows on created_at (rides idx_mcp_log_time_agent); plain SQL through engine.executeRaw, both engines. Behavioral automation classification: likely_automation = >90% of calls are context_pack/delta boundary verbs (the hook lane is stdio and never logs, so there is no name convention to key on). Sees HTTP clients ONLY — stdio never writes the log. Pinned by test/mcp-usage.test.ts.

  • docs/protocol/MCP_META_CHANNELS.md — normative _meta conventions for MCP tool responses: one producer per top-level key, additive-forever within a key, producer isolation, and the registered-keys table (brain_hot_memory, retrieval, warnings). Anything the model must SEE rides a content block (mainstream harnesses don't feed _meta to the model); _meta serves structured consumers. Add a key by registering it in the table — one producer, additive-forever.

  • src/mcp/rate-limit.ts — Bounded-LRU token-bucket limiter. buildDefaultLimiters() returns the two-bucket pipeline: pre-auth IP (30/60s, fires BEFORE the DB lookup so brute-force load against access_tokens is capped) + post-auth token-id (60/60s). Tracks lastTouchedMs separately from lastRefillMs so an exhausted key can't be reset by hammering past the TTL. LRU cap bounds memory under attacker-controlled key growth. refund(key) returns one consumed token (capped at the limit) — the request_tools persist path refunds when a race-lost 0-row UPDATE (concurrent operator pin) means no write actually happened; dry-run previews never draw a token at all (the limiter meters actual writes only).

  • src/commands/serve-http.ts — Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. The legacy POST /ingest queue refuses operation-snapshot clients; they use approved MCP writes so execution stays inside the shared grant contract. Started via gbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--bind HOST] [--log-full-params]. Combines MCP SDK's mcpAuthRouter (authorize/token/register/revoke), a custom client_credentials handler running BEFORE the router (SDK's token endpoint throws UnsupportedGrantTypeError for CC; custom handler falls through for auth_code / refresh_token), requireBearerAuth middleware for /mcp with scope enforcement + localOnly rejection before op dispatch, and express-rate-limit at 50 req / 15 min on /token. Serves the built admin SPA from admin/dist/ with SPA fallback. /admin/events SSE broadcasts every MCP request. cookie-parser wired (Express 5 has no built-in). Startup logging prints port, engine, issuer URL (honors --public-url), client count, DCR status, and the admin bootstrap token line — but the generated token's raw value only prints when stderr is an interactive TTY (shouldSuppressBootstrapPrint): a non-TTY (containerized/piped) start hides it so the secret never lands in centralized log storage, env-sourced tokens ($GBRAIN_ADMIN_BOOTSTRAP_TOKEN) are always hidden, --print-admin-token forces the raw value on a trusted terminal, and --suppress-bootstrap-token hides everything. The /mcp request handler's OperationContext literal sets remote: true explicitly (without it submit_job's protected-name guard would see a falsy undefined and a read+write-scoped OAuth token could submit shell jobs). summarizeMcpParams from src/mcp/dispatch.ts feeds both mcp_request_log writes and the SSE feed by default (raw via --log-full-params). Cookie Secure flag set behind HTTPS or a public-URL proxy; magic-link nonce store LRU-bounded; DCR disable routes through the GBrainOAuthProvider dcrDisabled constructor option (not a router monkey-patch); transport.handleRequest wrapped in try/catch to return a JSON-RPC 500 envelope; OperationError + unexpected exceptions unified through buildError / serializeError so /mcp always returns the same envelope. /health is liveness-only via probeLiveness(engine, engineName, version, timeoutMs) racing engine.executeRaw('SELECT 1', undefined, { signal }) against the exported HEALTH_TIMEOUT_MS = 3000; when the timeout wins, an AbortController cancels the Postgres query (PGLite can only discard its eventual result) before the same tagged ProbeHealthResult 503 envelope is returned (single timer-cleanup site); body shape {status, version, engine} only. Full stats live at admin-only /admin/api/full-stats (gated by requireAdmin, calls probeHealth(engine, ...)) — keeps getStats()'s 6× count(*) off the public route so a saturated pool doesn't trigger orchestrator restart cascades. OAuth/admin/audit SQL outside the abortable liveness probe routes through sqlQueryForEngine(engine) from src/core/sql-query.ts so it works against PGLite; the four mcp_request_log.params INSERT sites (success / auth_failed / scope_denied / server-error) go through executeRawJsonb(engine, ...) so the column stores real objects (params->>'op' returns search, not the quoted string). --bind HOST defaults 127.0.0.1 (self-hosters pass --bind 0.0.0.0); a stderr WARN fires when --public-url is set without --bind; the banner prints a Bind: line. AuthInfo.sourceId + AuthInfo.allowedSources + AuthInfo.takesHoldersAllowList are the typed source of truth, populated by oauth-provider.ts:verifyAccessToken (source scope from the oauth_clients row; takes-holders from access_tokens.permissions.takes_holders for legacy bearer tokens). The /mcp dispatch site reads authInfo.takesHoldersAllowList ?? ['world'] — absent grants (OAuth-client tokens, pre-v29 brains) fail closed to world-only takes visibility, while an explicit [] grant is preserved as deny-all; pinned end-to-end by test/e2e/serve-http-takes-holders.test.ts. The HTTP MCP tools/list handler at :837-849 uses paramDefToSchema(v) from src/mcp/tool-defs.ts so array params keep items (strict-mode OAuth clients otherwise reject the whole tool list). POST /ingest enforces the slug-prefix write fence at the ROUTE, not the op layer: the route hands its payload to the ingest_capture minion handler, which deliberately bypasses put_page, so no OperationContext exists and enforceClientSlugFence never runs — a slug-bound client must therefore supply X-Gbrain-Slug and it must satisfy slugUnderBoundPrefixes, else 403 (without the check a bound client could overwrite any page inside its granted source). The write source is resolved server-side as authInfo.sourceId ?? 'default' and travels on the job as job.data.sourceId; the caller-supplied X-Gbrain-Source-Id header routes nothing and only names the emitter (webhook-<clientId>), which is what the event's source_id and the 202's back-compat source_id field carry. The 202 additionally reports the routed destination as write_source_id, and the resolved write source joins the queue idempotency key so a rescoped client lands a new capture instead of deduping against its old job. confidential revoke: a pre-router /revoke handler validates the RFC 7009 body, verifies hash-only secrets for both client_secret_post and client_secret_basic, rejects mixed authentication, preserves the SDK path for public clients, and separates opaque client-auth failures from retryable/backend failures. OAuth metadata advertises both confidential methods. Pinned by test/e2e/serve-http-oauth.test.ts. three admin routes: /admin/api/calibration/profile, /admin/api/calibration/charts/:type (image/svg+xml; type in {brier-trend, domain-bars, pattern-statements, abandoned-threads}), /admin/api/calibration/pattern/:id (drill-down). The manual mint route /admin/api/register-client accepts optional source + federatedRead bindings mirroring the CLI's --source/--federated-read (validated via normalizeSourceInput/normalizeFederatedReadInput from src/core/source-id.ts; omitting both preserves the default binding source_id='default' / federated_read=[source_id], invalid values return a structured 400 invalid_source), so an admin SPA or provisioning proxy can mint a source-confined client over HTTP. The route mirrors the CLI lane end to end: structured 400s for unknown_source / archived_source (one batched ANY(\$1::text[]) existence check), invalid_token_ttl (shared TOKEN_TTL_MIN/MAX_SECONDS bounds, integer-validated BEFORE the tx), and brain_too_old (column pre-flight OUTSIDE the tx — a pre-scoped-clients brain refuses up front instead of aborting mid-transaction); the duplicate-name pre-check + INSERT run in ONE transaction under the SAME name-scoped advisory lock the CLI takes (registerClientNameLockKey), returning 409 duplicate_name with the existing client_id; the INSERT composes registerScopedClient (the CLI's registration core) so the two paths cannot drift; any post-commit failure includes the created client_id so the operator can revoke (never a false "nothing was created"). Pinned by test/register-client-source-normalize.test.ts. The /mcp tools/list is the honest catalog: per-request filters (token scope incl. the agentCallable carve-out, bound-client fence via opAllowedForBoundClient, publish gates via disabledOpsForPublishGates) over the surface-filtered op set, schemas via buildToolDefs (strict emission when mcp.strict_params resolves reject); the tools/list mcp_request_log row records the listed size as params.tool_count. Call-time op-level denials the list should have prevented — the inline scope deny, publish-gate backstop (config_key=...), fence op-level deny (fence=op, classified via isListLevelDenialEnvelope) — log status='denied_after_list' instead of 'error' (the trend-to-zero honest-catalog metric, see docs/operations/mcp-surface-runbook.md) — every tools/call row's status resolves through requestLogStatusForResult (dispatch.ts) on BOTH transports. The admin health-indicators error rate counts status NOT IN ('success','success_with_warnings') and excludes operation='surface_change' audit rows from numerator AND denominator (audit rows record operator/self actions, not traffic). Per-request surface resolution lives in resolveEffectiveSurface: a verbs ceiling skips the default-surface config read, and a failed read applies the last successfully read default per process (fail-closed — a transient config outage cannot widen a NULL-surface client). OAuth protected-resource metadata (RFC 9728) names /mcp as the resource (resourceServerUrl), so the SDK mounts the PRM at /.well-known/oauth-protected-resource/mcp and the 401 resource_metadata URL is derived from the same value; the bare root path is kept as an alias by rewriting req.url onto the SDK's handler (same document, same CORS).

  • src/core/sql-query.ts — engine-aware tagged-template SQL adapter for OAuth/admin/auth infrastructure. sqlQueryForEngine(engine) returns a SqlQuery ((strings, ...values) => Promise<rows[]>) that walks the template, builds $N positional SQL, asserts every value is a SqlValue (string | number | bigint | boolean | Date | null), and routes through engine.executeRaw(sql, params) (Postgres via postgres.js unsafe(sql, params), PGLite via db.query(sql, params)). Deliberately narrower than postgres.js's sql tag: no nested fragments, sql.json(), sql.unsafe(), sql.begin(), or array binding — the narrow scalar-only surface is the feature (keeps it from drifting into a partial postgres.js clone). JSONB writes go through executeRawJsonb(engine, sql, scalarParams, jsonbParams) which composes positional $N::jsonb casts and passes JS objects through; an object reaches the wire with the correct type oid, so executeRawJsonb is safe (verified by test/sql-query.test.ts on PGLite, test/e2e/auth-permissions.test.ts:67 on Postgres). Positional binding is NOT universally immune, though: binding a JSON.stringify(x) string to a bare $N::jsonb via unsafe() double-encodes it into a jsonb string scalar on real Postgres (PGLite hides it). Fixes: pass a raw object (executeRawJsonb / sql.json), or cast through $N::text::jsonb. scripts/check-jsonb-pattern.sh (template grep) doesn't fire on executeRawJsonb(...) because it passes objects; the positional $N::jsonb + JSON.stringify form is caught by scripts/check-jsonb-params.mjs. Consumed by src/commands/auth.ts, src/commands/serve-http.ts, src/core/oauth-provider.ts, src/commands/files.ts, src/mcp/http-transport.ts so all five work uniformly against PGLite and Postgres.

  • src/commands/serve.tsgbrain serve stdio MCP entrypoint with idempotent shutdown across every parent-disconnect signal. Stdio EOF, SIGTERM, SIGINT, SIGHUP, and parent-process death (every reparent case — PID 1, launchd subreaper, systemd, tmux, or a parent shell with PR_SET_CHILD_SUBREAPER) all funnel into one cleanup(reason) that first awaits shutdownDelegatedSync() (idempotent shared promise; a running delegated sync is aborted and settles its checkpoint against the live engine, with the cleanup deadline extended by GBRAIN_SERVE_SYNC_SETTLE_MS only while a job runs) and then releases the engine and the PGLite write-lock dir within 5 seconds (otherwise the lock is held indefinitely after Claude Desktop / Cursor / launchd-managed gateways disconnect, forcing a 5-minute stale-lock wait on next start). Watchdog reparent check is getParentPid() !== initialParentPid (the === 1 check missed the subreaper case under launchd/systemd). Bun's process.ppid cache is stale across reparenting (oven-sh/bun#30305) so getParentPid() runs spawnSync('ps', ['-o', 'ppid=', '-p', PID]) per tick. Startup probe verifies ps is on PATH; if not (stripped containers, busybox), the watchdog skips installing AND emits a loud [gbrain serve] watchdog disabled: ps unavailable ... stderr line so operators see the degraded mode. The idle maintenance sweep skips ticks while a delegated sync runs and drains deferred embeds afterwards (maybeDrainDeferredEmbeds). Both stdin activity listeners (the default-on idle sweep and the opt-in --stdio-idle-timeout) attach only AFTER the MCP SDK transport's own stdin listener — the sweep on its first interval tick, the idle timeout through the activator installStdioLifecycle returns, which runServe invokes once startMcpServer resolves (and which re-arms the countdown so boot time does not eat into the idle window) — so neither can flip stdin into flowing mode and consume a fast client's initialize frame before the SDK sees it. Pinned by test/serve-stdio-lifecycle.test.ts (22 cases).

  • src/core/grants/ — canonical OAuth client grant model shared by CLI, admin HTTP, provisioning, and worker admission. model.ts validates active sources, independent direct/delegated fences, explicit reviewed tools, namespaces, concurrency, spend, and TTL; host/current brain aliases normalize to the serving brain (null), while other IDs need independently verified host identity. profiles.ts snapshots current eligible operation names for six explicit profiles; new defaults are renewable 1 hour (static 30 days), concurrency 1, and unlimited delegated spend. allowed_operations = NULL preserves legacy scope-based behavior; a new profile missing its snapshot fails closed. service.ts locks the client, checks grant_revision, and commits the grant, secret-free audit, and original ceilings for pending/active legacy jobs atomically; stale edits fail, repeated repair is a no-op, and repair preserves populated limits. Issued scopes intersect live scopes on access and refresh; source/path/operation/delegation changes are live, newly added scopes require new issuance, and TTL changes do not rewrite existing expiry. migration.ts splits legacy fences and removes only invalid agent grants with repair reasons; it loads registry validation lazily only after locking existing delegated clients, so empty engine imports and cold migrations do not register unrelated background sinks. This justified engine-live import is line-marked; constraints still install when no clients exist. schema.ts supplies migration147/bootstrap columns and invariants for both engines. SQL JSON passes through text::jsonb for driver parity. Tests: test/client-grants.test.ts, OAuth compatibility suites, and test/e2e/client-grants.test.ts (Postgres CAS, job ceilings, and actual HTTP preview/edit).

  • src/commands/serve-http-grants.ts + admin/src/components/ClientGrant.tsx — typed admin grant request validation and shared profile/advanced permission editor. Register/rescope/TTL routes in serve-http.ts use canonical grant validation and audit; legacy response shapes remain available. New UI creation defaults to memory-writer and requires a preview, while edits carry the displayed revision and require a fresh preview after any field change or conflict. Detail/catalog endpoints expose non-secret grants and the shared harness registry. Operation authority snapshots, source sets, delegated tools/namespaces, spend, concurrency, and future TTL are independently reviewable; profile regrant is explicit. admin/src/pages/Agents.tsx downloads the private HarnessCredentials handoff and links current harness guides instead of maintaining native config snippets. Rebuild committed admin/dist and src/admin-embedded.ts after SPA edits.

  • src/core/oauth-provider.tsGBrainOAuthProvider implementing the MCP SDK's OAuthServerProvider + OAuthRegisteredClientsStore. Backed by raw SQL (works on both PGLite and Postgres — OAuth is infrastructure, not a BrainEngine concern). Full OAuth 2.1: authorize + exchangeAuthorizationCode with PKCE, client_credentials, refresh_token with rotation, revokeToken, registerClient (DCR validates redirect_uri is https:// or loopback per RFC 6749 §3.1.2.1). All tokens + client secrets SHA-256 hashed before storage. Owner-approved authorization codes have a 10-minute TTL. Grant transactions lock the active client row before consuming codes or refresh tokens and creating replacements; revocation uses the same lock. Provider-side S256 PKCE validation is mandatory (skipLocalPkceValidation=true prevents the SDK from discarding the verifier first). Existing active-client grants remain usable. pgArray() escapes commas/quotes/braces so a comma-bearing redirect_uri can't smuggle a second array element. Legacy access_tokens fallback in verifyAccessToken honors the original-schema scopes TEXT[] column via normalizeTokenScopes (NULL = grandfathered read+write+admin, so every pre-scopes token is byte-identical; an array is filtered to known scopes and honored as-is, []/all-unknown preserved as deny — a dedicated column is structurally immune to the permissions-object-replacement wipe class), and threads BOTH stored grants off the token's permissions JSONB: source_id via parseLegacyTokenScope and takes_holders via parseTakesHoldersAllowList (both in src/core/legacy-token-scope.ts, shared with the legacy HTTP transport so the two transports cannot drift; [] takes-holders preserved as explicit deny-all, missing/non-array → undefined → the /mcp dispatch site's fail-closed ['world']; OAuth-client tokens carry no takes-holders grant pending per-client storage — TODOS.md). sweepExpiredTokens() runs on startup in try/catch and returns the count via RETURNING 1 + array length. RFC hardening: client_id folded atomically into the DELETE WHERE for both auth-code exchange and refresh rotation (wrong-client paths don't burn the row); refresh-scope-subset enforced against the original grant on the row (RFC 6749 §6, so revoking a scope shrinks existing refresh tokens); client_id bound on revokeToken (RFC 7009 §2.1); /token redirect_uri validated against the /authorize value (RFC 6749 §4.1.3, empty-string treated as missing not wildcard); verifyAccessToken/getClient catch only isUndefinedColumnError from src/core/utils.ts (only SQLSTATE 42703 falls through to legacy; lock timeouts/network blips throw); dcrDisabled constructor option lets serve-http.ts disable /register without monkey-patching the router. Module-private coerceTimestamp() normalizes postgres-driver-as-string BIGINT columns to JS numbers at 5 read sites (getClient for RFC 7591 §3.2.1 numeric timestamps, exchangeRefreshToken + verifyAccessToken for the SDK's typeof === 'number' check); throws on NaN/Infinity (fail loud at boundary), returns undefined for SQL NULL (callers treat NULL as expired). Not promoted to utils.ts — generic BIGINT precision-loss risk. registerClient honors token_endpoint_auth_method: "none" (RFC 7591 §3.2.1): public PKCE clients store client_secret_hash = NULL and the response omits client_secret; confidential clients (client_secret_post / client_secret_basic) keep their one-time-reveal shape; getClient normalizes NULL client_secret_hash to JS undefined so the SDK's clientAuth path accepts public clients. verifyAccessToken JOINs oauth_clients.source_id (write scope, scalar) + oauth_clients.federated_read (read scope, TEXT[]) + oauth_clients.bound_slug_prefixes (write fence, TEXT[] — consumed by enforceClientSlugFence in operations.ts) onto the returned AuthInfo; legacy brains degrade via isUndefinedColumnError fallback, dropping the newest projection first. rescopeClient(clientId, {sourceId?, federatedRead?, boundSlugPrefixes?}) is the trusted-operator rescope (CLI gbrain auth rescope-client, admin POST /admin/api/rescope-client); boundSlugPrefixes is tri-state — undefined leaves the binding untouched, null clears it, a non-empty array replaces it (explicit empty array rejected as ambiguous deny-all) — so roster churn updates the write fence in place without rotating secrets. with src/commands/serve-http.ts: custom /token middleware that runs BEFORE the MCP SDK's clientAuth. The SDK does plaintext compare against the request's client_secret; gbrain stores SHA-256 hashes only, so every confidential-client /token request would fail. The middleware detects confidential auth via Authorization: Basic header OR client_secret_post form body (both shapes per RFC 6749 §2.3.1), verifies via verifyClient(client_id, presented_secret) (SHA-256 hash compare), and falls through to the SDK for public PKCE clients (which the SDK's clientAuth still accepts via NULL-client_secret_hash normalization). Pinned by test/oauth-confidential-client.test.ts (both client_secret_basic and client_secret_post).

  • admin/ — React 19 + Vite + TypeScript admin SPA embedded in the binary via admin/dist/ served by serve-http.ts. 7 screens: Login (bootstrap token → session cookie), Dashboard (metrics + SSE feed + token health), Agents (sortable table + sparklines + Register), Register (modal with scope checkboxes + grant type selector), Credentials reveal (Copy + Download JSON + one-time-only warning), Request Log (filterable paginated), Agent Detail drawer (Details / Activity / Config Export tabs + Revoke). Design tokens: #0a0a0f bg, Inter for UI, JetBrains Mono for data, 4-32px spacing scale, rounded pill badges. HTTP-only SameSite=Strict cookie auth. 65KB gzip. Build: cd admin && bun install && bun run build; output at admin/dist/ is committed for self-contained binaries.

  • src/commands/auth.ts — token management. gbrain auth create/list/revoke/test for legacy bearer tokens (create --scopes read,write narrows a token via the scopes TEXT[] column with mint-time validation — a typo'd scope refuses loudly, never silently denies or widens; list shows id + scopes columns with grandfathered rows rendered honestly; revoke --id <uuid> revokes exactly one row since names are not unique, and bulk revoke-by-name says when it hit several; permissions set-takes-holders MERGES into the permissions JSONB via COALESCE(permissions,'{}'::jsonb) || \$2::jsonb — a whole-object replace would silently wipe the source_id federation grant), plus gbrain auth register-client and gbrain auth revoke-client <client_id> for OAuth 2.1 client lifecycle. revoke-client runs an atomic DELETE...RETURNING on oauth_clients; FK ON DELETE CASCADE on oauth_tokens.client_id and oauth_codes.client_id purges every active token + auth code in one transaction; process.exit(1) on no-such-client (idempotent). Legacy tokens stored as SHA-256 hashes in access_tokens; OAuth clients in oauth_clients; legacy tokens with no scopes grant grandfather to read+write+admin on the OAuth HTTP server (no migration); scoped tokens are honored at exactly their grant. Every SQL site routes through sqlQueryForEngine(engine) from src/core/sql-query.ts (and executeRawJsonb for the takes-holders permissions JSONB column) so gbrain auth works against PGLite; the takes-holders write goes through executeRawJsonb(engine, sql, [name, hash], [{takes_holders:[...]}]) which round-trips with jsonb_typeof = 'object'. register-client accepts --source <id> (write authority, scalar), --federated-read <S1,S2,...> (read scope, array), and --token-ttl <seconds> (per-client access-token TTL persisted to oauth_clients.token_ttl, bounds TOKEN_TTL_MIN_SECONDS=60 to TOKEN_TTL_MAX_SECONDS=7,776,000/90d) and prints the resolved Write source + Federated reads; clients without a source_id backfill to 'default' via migration v60. The registration core is the exported registerScopedClient(sql, engine, name, parsed, opts) — exit-free, print-free, injected-handle (engine-bound callers like agent register pass the dispatcher's engine; a second withConfiguredSql engine self-deadlocks PGLite's single-writer lock), returns a RegisteredClient data object and throws on failure; the thin CLI wrapper owns exit/print. Its printer formatRegisterClientOutput is a BYTE-PINNED contract (connect.ts's defaultRegisterOAuthClient regex-scrapes Client ID:/Client Secret: from it in production) — pinned by test/auth-register-client-output-pin.test.ts. preflightOauthClientColumns(sql) probes information_schema.columns for the optional columns (token_ttl, surface, federated_read, source_id, deleted_at) so statement shapes are decided BEFORE any transaction — Postgres/PGLite abort the whole tx on any statement error (25P02) and SqlQuery has no savepoint seam, so "catch 42703 and continue" inside a tx is impossible; skipped optional writes surface as RegisteredClient.skipped with an apply-migrations hint. gbrain auth clients [--usage] [--days N] [--json] lists clients with scopes, per-client surface, write source (source_id), and federated reads (federated_read) — one projection-widened SELECT with a degrade ladder for pre-migration brains (drops the newest columns first, never errors); --usage joins per-client op-call counts via src/core/mcp-usage.ts. The bare gbrain auth create <name> form (no --takes-holders) mints a token via the exported pure parseAuthCreateArgs(rest). Pinned by test/auth-create-args.test.ts + test/auth-register-client-args.test.ts.

  • src/core/mcp-client.ts — the thin-client transport (trust boundary). callRemoteTool(config, toolName, args, opts) with CallRemoteToolOptions {timeoutMs, signal}; buildAbortController composes an external signal with the timeout. All transport errors normalize to RemoteMcpError via the toRemoteMcpError funnel: stable RemoteMcpErrorReason union, RemoteMcpErrorDetail.kind ('timeout' | 'aborted' | 'unreachable') sub-tag, RemoteMcpErrorDetail.code carrying server-supplied error codes (e.g. missing_scope). extractToolErrorCode parses the operation {error: string, message} shape as well as legacy envelopes. Only the locked SDK’s typed StreamableHTTPError with numeric code 401 triggers one refresh and reconstructed client; application error text never causes authentication replay. The outer abort signal reaches discovery, minting, initialization, fetch and tool calls, with per-request limits retained and clients closed after failed initialization or abort. unpackToolResult<T>(res) parses tool-call JSON content. _clearMcpClientTokenCache() test escape. The CLI routing seam that consumes this lives in src/cli.ts (runThinClientRouted); see docs/architecture/thin-client.md.

  • src/commands/connect.ts + src/core/connect-probe.tsgbrain connect <mcp-url> [--token <bearer>] one-command coding-agent onboarding from a bearer token. Turns an MCP URL + token into a paste-ready claude mcp add ... -H "Authorization: Bearer ..." block (default) or, with --install, runs it directly and smoke-tests the token. Direct HTTP MCP — Claude Code talks straight to a remote gbrain serve --http, no local install needed. Token resolution: --token > $GBRAIN_REMOTE_TOKEN > placeholder (print) / error (install). The generated block tells the agent to call get_brain_identity + list_skills (the LEARN_INSTRUCTION export, which names capture — a starter-surface MCP op — alongside put_page for full-control writes) with a core-tools fallback for hosts without skill publishing. URL normalization appends /mcp to a bare host but REJECTS a scheme-less host; the pure registration helpers (normalizeMcpUrl, isLinkLocalOrMetadata, buildClaudeMcpAddArgv with its optional scope param — claude's default is local, so the harness lane passes user explicitly — buildCodexMcpAddArgv, validateToken, redactToken, shellQuote/cmdString, issuerFromMcpUrl, the OAUTH_SECRET_NOTE secret-hygiene constant, and openclawThinClientBlock — the honest openclaw wiring print: a scoped gbrain init --mcp-only thin-client block, deliberately NOT a stdio mcpServers config since that grants full local DB access) live in src/core/mcp-registration.ts (core must not import from commands; connect.ts re-exports them — OAUTH_SECRET_NOTE included — so its surface and tests are unchanged) and are unit-tested. Flags: --token, --name <id> (default gbrain, validated against NAME_RE), --agent claude-code|codex|opencode|perplexity|generic, --install, --yes (required for --install in non-TTY), --force, --json (token redacted unless --show-token), --timeout-ms. connect is in CLI_ONLY + CLI_ONLY_SELF_HELP; dispatched in cli.ts:handleCliOnly with no local DB connect. AGENT_SPECS drives per-agent rendering + --install: claude-codebuildClaudeMcpAddArgv (literal -H "Authorization: Bearer <tok>"); codexbuildCodexMcpAddArgv = codex mcp add <name> --url <url> --bearer-token-env-var GBRAIN_REMOTE_TOKEN (on the CONNECT lane Codex reads the token from the env var at runtime, never written to config — the harness lane in src/core/bootstrap/harness.ts is the deliberate exception, writing a managed block with the inline http_headers = { Authorization = "Bearer <t>" } credential (codex-cli >=0.149 rejects inline bearer_token for streamable_http at config load) because framework-spawned codex inherits no shell profile; --install runs it and prints an export GBRAIN_REMOTE_TOKEN hint when missing); opencodebuildOpencodeMcpAddArgv = opencode mcp add <name> --url <url> --header "Authorization=Bearer {env:GBRAIN_REMOTE_TOKEN}" (the interpolation is stored LITERALLY and resolved by opencode at read time — token never in argv/config/--json; --install writes the entry directly through ConnectDeps.writeOpencodeRemoteEntryopencode-json.ts in env token mode, no binary required; --force maps to the writer's allowReplaceOtherSource so an OURS entry at an old url — a rotated serve — is replaceable, mirroring the exec lanes' --force semantics, while foreign same-name entries still refuse with url-appropriate copy: pick --name); perplexity + generic are installable:false and reject --install. --oauth (supportsOAuth:true = perplexity/generic only) emits an OAuth 2.1 client-credentials connector block (Issuer URL via issuerFromMcpUrl = mcp-url minus /mcp, Client ID, Client Secret) — least-privilege scopes + short-lived rotating tokens vs a long-lived full-access secret. Creds from --client-id/--client-secret (BYO) or --register (deps.registerOAuthClient shells gbrain auth register-client <name> --grant-types client_credentials --scopes <DEFAULT_SCOPES="read write"> --token-endpoint-auth-method client_secret_post and parses Client ID:/Client Secret:); --oauth rejected for claude-code/codex and incompatible with --install. buildJson is a generic shape (agent, command/command_argv null for perplexity/generic, header, env_var, oauth fields with redaction); the codex command carries only the env-var name, never the token. cmdString(binary, argv) POSIX-single-quotes args. ConnectDeps = {isTTY, promptYesNo, hasBinary(bin), runBinary(bin, argv), probe, env(name), registerOAuthClient, writeOpencodeRemoteEntry} — binary-generic so claude and codex share the exec path while opencode rides the writer member; env injectable for tests. Security: rendered command single-quotes the token so shell metacharacters can't run code when pasted; token validated before it lands in an HTTP header; link-local / cloud-metadata addresses (incl. IPv4-mapped IPv6 ::ffff:169.254.x.x and AWS IMDSv2-over-IPv6 fd00:ec2::254) refused as a token-exfil guard while localhost/RFC1918/LAN stay allowed; token redacted from all error output. src/core/connect-probe.ts is the raw-bearer MCP smoke probe backing --install: connects the official MCP SDK Client over StreamableHTTPClientTransport with a STATIC Authorization header (no OAuth/discovery — distinct from mcp-client.ts:callRemoteTool which is OAuth-only and remote-mcp-probe.ts:smokeTestMcp which only sends initialize), runs the full initialize handshake via client.connect(), then calls get_brain_identity (read-scope, non-localOnly) to prove a tool call round-trips. Never throws — every failure maps to { ok: false, reason: 'auth' | 'unreachable' | 'timeout' | 'tool_error' | 'unknown', message } so a wrong/expired token fails at setup, not on the agent's first request. DEFAULT_PROBE_TIMEOUT_MS = 15_000 shared with connect.ts. serve-http.ts adds exported pure skillPublishStatus(publishSkills) for the startup banner Skills: published / not published line + a one-line gbrain config set mcp.publish_skills true stderr nudge when publishing is OFF. Docs: docs/mcp/CODEX.md, docs/mcp/PERPLEXITY.md, docs/mcp/CLAUDE_CODE.md, docs/tutorials/connect-coding-agent.md. Pinned by test/connect.test.ts (pure-helper + render, all five agents incl. the opencode writer-install lane) + test/e2e/connect-bearer.test.ts (raw-bearer probe + full OAuth chain register→connect→discovery→/token mint→get_brain_identity, client registered in beforeAll before serve takes the PGLite single-writer lock; drives real claude + codex binaries through connect --install with sandboxed HOME/CODEX_HOME, asserts registration + token never in Codex config, skips when a binary is absent) + test/e2e/serve-stdio-roundtrip.test.ts (spawns real gbrain serve stdio against a fresh init --pglite brain, drives the SDK client through initializetools/listtools/call, asserts the advertised core-tool set including capture, a starter-surface MCP op) + test/serve-skills-publish-nudge.test.ts.

  • src/commands/upgrade.ts — self-update CLI. runPostUpgrade() enumerates migrations from the TS registry (src/commands/migrations/index.ts) and tail-calls runApplyMigrations(['--yes', '--non-interactive']) so the mechanical side of every outstanding migration runs unconditionally.

  • src/commands/migrations/ — TS migration registry (compiled into the binary; no runtime walk of skills/migrations/*.md). index.ts lists migrations in semver order. v0_11_0.ts = Minions adoption orchestrator (8 phases). v0_12_0.ts = Knowledge Graph auto-wire orchestrator (5 phases: schema → config check → backfill links → backfill timeline → verify); phaseASchema has a 600s timeout for duplicate-heavy brains. v0_12_2.ts = JSONB double-encode repair orchestrator (4 phases: schema → repair-jsonb → verify → record). v0_14_0.ts = shell-jobs + autopilot cooperative (pending-host-work ping for skills/migrations/v0.14.0.md). All orchestrators are idempotent and resumable from partial status. The RUNNER owns all ledger writes — orchestrators return OrchestratorResult and apply-migrations.ts persists a canonical {version, status, phases} shape (orchestrators never call appendCompletedMigration). statusForVersion prefers complete over partial (never regresses); 3 consecutive partials → wedged → --force-retry <version> writes a 'retry' reset marker. Schema-only migrations v14 (pages_updated_at_index) + v15 (minion_jobs_max_stalled_default_5 with UPDATE backfill) live in the MIGRATIONS array in src/core/migrate.ts. in-process.ts exports runMigrateOnlyCore({timeoutMs?}) — single source of truth for "bring schema to head" (configureGatewaycreateEngineconnectinitSchemadisconnect, idempotent, 600s MIGRATE_ONLY_TIMEOUT_MS guard, throws MigrateOnlyError on no-config / timeout); the orchestrators' 9 schema phases AND init.ts:initMigrateOnly both delegate to it so schema bring-up can't drift (in-process, so no spawn can die with getaddrinfo ENOTFOUND on Windows + bun + Supabase pooler). runGbrainSubprocess is the diagnostic wrapper for the remaining non-schema spawns (extract/repair/stats): captures child stderr (64MB buffer) into the thrown error. v0_13_1.ts:phaseCGrandfather is a CHUNKED bulk SQL pass keyed on pages.id (globally unique PK, NOT slug — slug uniqueness is (source_id, slug)), filters deleted_at IS NULL (no tombstones), chunked in CHUNK_SIZE batches (DELETE_BATCH_SIZE convention) for bounded lock-hold; the rollback log carries {id, slug, source_id, pre_frontmatter} so rollback is unambiguous across sources; idempotent + resumable (each UPDATE flips its rows out of GRANDFATHER_WHERE). Pinned by test/migration-in-process.serial.test.ts and test/migrations-v0_13_1-grandfather.test.ts.

  • src/commands/migrations/v0_46_3.ts — ZeroEntropy sunset notice migration, detect-and-notify ONLY. Detects the HOST brain's exposure via src/core/ze-exposure.ts (read-only); when exposed — or when exposure is UNKNOWN, fail-safe — prints the ACTION REQUIRED banner and appends one idempotent entry to ~/.gbrain/migrations/pending-host-work.jsonl pointing the host agent at skills/migrations/v0.46.3.0.md. Performs NO config writes, NO pinning, and never invokes migrate embeddings (the migration costs money and needs a target key — that decision belongs to the user/agent via the playbook). UNKNOWN returns complete with an exposure_unknown detail rather than partial (three consecutive partials would wedge the whole migration chain behind --force-retry); the stage-2 upgrade banner and gbrain doctor carry the ongoing nag instead. Host-scoped (apply-migrations runs once per host with a global completed.jsonl); mounted/team brains are covered by the per-brain stage-2 banner (ze_sunset_notice_v2_shown in each brain's own DB config) + doctor gates.

  • src/commands/repair-jsonb.tsgbrain repair-jsonb [--dry-run] [--json]: rewrites jsonb_typeof='string' rows in place across 8 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter, subagent_messages.content_blocks, subagent_tool_executions.input, subagent_tool_executions.output). The subagent targets are jsonPayloadOnly: those columns can legitimately hold jsonb string scalars (persistToolExec binds a tool's pre-serialized string payload as-is), so their damage predicate additionally requires container-shaped content (^\s*[\[{]) that pg_input_is_valid(..., 'jsonb') (PG16+ floor, same as the IS JSON predicate updateSourceConfig relies on) actually parses — a plain-text value that merely starts with [ or { is never flagged or corrupted. Targets whose table doesn't exist on the brain are skipped via to_regclass (brains without the subagent tables), and one target's failure is recorded on stderr while the run continues — earlier repairs are already committed and the v0_12_2 migration orchestrator JSON-parses stdout. runDoctor's jsonb_integrity check counts damage with the same predicate over the same target list. Repairs double-encoded rows on Postgres; PGLite no-ops. Idempotent. Pinned by test/repair-jsonb.test.ts + test/doctor.test.ts.

  • src/commands/orphans.tsgbrain orphans and MCP find_orphans share islanded (no live inbound or outbound links) and legacy inbound-only modes. Read policy scopes candidates and excludes private candidates, endpoints and independent origin pages before counting. Cross-source public links still establish reachability by the documented orphan definition. Reporting exclusions (shouldExcludeFromOrphanReporting) stay centralized; the scoped live-page denominator excludes the same private/reporting rows. --source remains explicit; no new remote privacy parameter.

  • src/commands/salience.tsgbrain salience [--days N] [--limit N] [--kind PREFIX] [--json]: pages ranked by emotional + activity salience over a recency window. Mirrors orphans.ts shape (pure data fn + JSON formatter + human formatter). Calls engine.getRecentSalience(opts). Score formula: (emotional_weight × 5) + ln(1 + active_take_count) + 1/(1 + days_since_update).

  • src/commands/anomalies.tsgbrain anomalies [--since YYYY-MM-DD] [--lookback-days N] [--sigma N] [--json]: cohort-level activity outliers. Calls engine.findAnomalies(opts). Two cohort kinds: tag, type.

  • src/commands/whoknows.tsgbrain whoknows <topic> and MCP find_experts: expertise and relationship routing. findExperts() threads source, page-privacy and holder policy through hybrid retrieval and enrichment, then requires an authorized effective-date read after optional salience work for every caller, including trusted local callers. Candidates absent from that exact (source_id, slug) map are omitted; required admission failures propagate. Optional salience failures retain the existing neutral factor. rankCandidates() uses score = log(1 + raw_match) × max(0.1, exp(-days/180)) × (0.5 + 0.5 × salience), with salience normalized before ranking and alphabetical slug tie-breaking. Hybrid salience/recency boosts are disabled here to avoid applying them twice. Expert types are filtered in SQL and resolved from the pack by ops/insights.ts; a failed pack load supplies an empty type list. CLI supports --explain, --limit, --json and thin-client routing. Math is pinned by test/whoknows.test.ts; exact final admission, mutation races and SQL-failure behavior run on both engines in test/e2e/read-enrichment-privacy.test.ts.

  • src/commands/eval-whoknows.tsgbrain eval whoknows <fixture.jsonl> [--json] [--skip-replay]: two-layer eval gate. Layer 1 quality (hand-labeled fixture, top-3 hit rate ≥ 0.8). Layer 2 regression (eval_candidates replay set-Jaccard@3 ≥ 0.4). Sparseness fallback: < 20 replay-eligible rows → Layer 2 auto-skips with stderr warning. Stable JSON envelope with schema_version: 1; exit 0/1/2 for pass/fail/usage. WhoknowsFn callable abstraction makes the gates impl-agnostic; runEvalWhoknows(engine: BrainEngine | null, args) picks the impl at entry — thin-client mode (isThinClient(cfg)) routes per-query through callRemoteTool(cfg, 'find_experts', {topic, limit}), local mode calls findExperts(engine, ...) directly. cli.ts adds a thin-client bypass before connectEngine (dispatch shape under src/commands/eval.ts); the regression gate auto-skips in thin-client mode (no DB access to eval_candidates). Public exports jaccardAtK, topKHit, readFixture, WhoknowsFn, threshold constants pinned by test/eval-whoknows.test.ts (25 cases incl. null-engine signature contract).

  • test/fixtures/whoknows-eval.jsonl — 10-row synthetic placeholder demonstrating the eval-fixture schema ({query, expected_top_3_slugs, notes?} JSONL). End users replace with their own real queries; placeholder uses obviously-example slugs (wiki/people/example-alice). Drives test/e2e/whoknows.test.ts (seeds a matching synthetic brain, asserts the >=80% gate) and the whoknows_health doctor check.

  • src/core/skillopt/ + src/commands/skillopt.ts + skills/skill-optimizer/ — self-evolving skill optimization grounded in the SkillOpt paper (arXiv 2605.23904). gbrain skillopt <skill> treats SKILL.md as trainable parameters of a frozen agent: validation-gated (median-of-3 + epsilon=0.05), budget-capped (preflight estimator), per-skill DB-locked (tryAcquireDbLock('skillopt:<name>', 60min)), atomic-versioned (history-intent-first 5-step commit), body-only mutations (frontmatter forbidden). Rollouts use gateway.toolLoop directly with no-op persistence callbacks (zero subagent_messages pollution) + a read-only tool allowlist derived from BRAIN_TOOL_ALLOWLIST minus put_page/submit_job/file_upload. Two reflect calls per step; rejected-edit buffer LRU-bounded to 100; bundled-skill gate; bootstrap workflow (sentinel + --bootstrap-reviewed); D_sel floor (>=5 with --split override); audit JSONL via audit-writer.ts. Added to ALL_PHASES after patterns (default OFF; opt-in via gbrain config set cycle.skillopt.enabled true); cycle phase wrapper at src/core/skillopt/cycle-phase.ts walks stale skills with per-skill ($0.50) + brain-wide ($2.00) caps. Added to PROTECTED_JOB_NAMES. Surface: dream-cycle phase wrapper; --all batch mode (src/core/skillopt/batch.ts:runBatchAll); --target-models fleet (runFleet parallel per-model receipts under skillopt/fleet/<slug>/); MCP op run_skillopt (admin scope + per-skill skillopt.allowed_skills allowlist, NOT localOnly, validates skill_name kebab-only + confines caller-supplied benchmark/held-out paths to skillsDir for remote callers); Minion skillopt handler + --background with allowProtectedSubmit: true; write-flavored optimization via src/core/skillopt/write-capture.ts:buildWriteCaptureRegistry (virtual put_page/submit_job/file_upload captured in-memory; --write-capture flag); held-out real-user test set via src/core/skillopt/held-out.ts (capture infra at ~/.gbrain/skillopt-captures/<skill>/<run>.jsonl, --held-out <path> flag, runHeldOutGate candidate >= baseline). Hermetic via DI seams (opts.chatFn for optimizer + judge; opts.toolLoopFn for rollouts; no mock.module). --bootstrap-from-skillrunBootstrapFromSkill in src/core/skillopt/bootstrap-benchmark.ts: reads SKILL.md directly (no routing-eval.jsonl), makes ONE LLM call emitting a full starter benchmark (tasks + rule judges) as JSONL, parsed line-by-line with skip-bad-line salvage and a min-2-valid-checks-per-task drop; provider/transport errors PROPAGATE (not collapsed to bootstrap_empty). --bootstrap-tasks N (default 15, capped 50); maxTokens scales min(8000, max(4000, N*220)). The stderr REVIEW line prints the literal gbrain skillopt <name> --bootstrap-reviewed --split 1:1:1 — load-bearing because the default 4:1:5 split makes a 15-task starter's D_sel = floor(15/10) = 1, below the >=5 floor, so a 15-task benchmark needs --split 1:1:1. Both bootstrap generators share assertBenchmarkAbsent + readSkillBodyOrThrow; --bootstrap-from-skill is mutually exclusive with --bootstrap-from-routing/--benchmark/--all/--target-models/--resume. Generated rule judges are explicitly WEAK DRAFTS to be strengthened during the review gate. The held-out gate is wired: --held-out <path> is parsed and threaded through every caller (CLI main + --background held_out_path + batch/fleet heldOutPath + the run_skillopt held_out_path param), running at CHECKPOINT ACCEPTANCE so no-mutate/fleet paths can't promote a held-out-failing candidate. assertBundledMutationHeldOut in bundled-skill-gate.ts: bundled + --allow-mutate-bundled requires a NON-EMPTY held-out (MIN_HELD_OUT_SIZE = D_SEL_MIN_SIZE = 5, derived so they can't desync) or hard-refuses (exit 2), for ALL callers (they funnel through runSkillOpt); held-out must be task_id-DISJOINT from the benchmark (overlap rejected — can't catch overfitting). receipt.baseline_sel_score populated + a real final-test eval (test_score + baseline_test_score) scoring best + baseline on split.test; shared scoreSkillOnTasks primitive (validate-gate.ts) backs baseline/final-test/held-out scoring. --no-mutate writes proposed.md via writeProposed in version-store.ts. maxRuntimeMin ENFORCED (wall-clock deadline between steps → skillopt_runtime_exceeded → outcome aborted). Three eval-internal ablation opts on SkillOptOpts (NOT on CLI): reflectMode ('both'/'failure-only'), disableValidationGate (greedy-accept), optimizerMode ('reflect'/'one-shot-rewrite'), recorded in RunReceipt + audit run_start for replayability; ROLLOUT_SUCCESS_THRESHOLD = 0.5 named constant for the partition; one-shot fence-strip is anchored (^```...```$) so an embedded code sample isn't truncated. Claude Haiku 4.5's dateless canonical id claude-haiku-4-5 is in src/core/anthropic-pricing.ts (a BudgetTracker-capped run on Haiku would otherwise throw no_pricing on the FIRST chat() of every rollout); runValidationGate (validate-gate.ts) scans settled results for isMustAbortError(error) (from worker-pool.ts; BUDGET_EXHAUSTED is in MUST_ABORT_ERROR_TAGS) and re-throws so the caller aborts loudly instead of recording a hollow selScore:0 — ordinary non-abort rollout errors still fail-open to score:0 (judge-hiccup posture preserved). Pinned by 152 tests across 18 files (foundation + adversarial + v2 surface + E2E PGLite serial), test/skillopt/bootstrap-from-skill.test.ts (20 cases), test/skillopt/rollout.test.ts, test/skillopt/validate-gate-abort.test.ts (3 cases), held-out ENFORCE + one-shot-rewrite unit cases, and e2e (held-out block/allow, bundled no-mutate, runtime deadline, receipt honesty, held-out disjointness, no-DB-pollution). Drives the Track B SkillOpt benchmark suite in the sibling gbrain-evals repo.

  • src/core/brainstorm/{domain-bank,orchestrator,judges}.ts + src/commands/{brainstorm,lsd,eval-brainstorm}.ts + src/core/last-retrieved.ts — bisociation-grounded idea generation pair: gbrain brainstorm <question> (defensible, cite-heavy, 4 close × 6 far, judge threshold 4.0/5, save by default) and gbrain lsd <question> (Lateral Synaptic Drift — inverted judge rejecting ideas with resistance >4.5 "too obvious", stale-page bias via pages.last_retrieved_at, 2 close × 12 far, axiomatic inversions required, ephemeral by default). The "domain bank" is prefix-stratified sampling from the user's own brain (SELECT DISTINCT substring(slug from '^[^/]+/[^/]+') cached 1h-TTL in config per source) tiebroken by JOIN page_links connection_count, with corpus-sampling fallback when fewer prefixes than M exist. Distance normalized to [0,1] via 1 - clamp(cosine_distance, 0, 2) / 2. judges.ts exports runJudge(config, ideas) + two configs (BRAINSTORM_JUDGE_CONFIG weighted originality/resistance/thesis_density/concrete_grounding/cognitive_load 0.25/0.20/0.20/0.20/0.15 vs LSD_JUDGE_CONFIG cognitive_load 0.50 + inversion rule). Calibration cold-start fallback: when calibration_profiles.active_bias_tags is empty, judge runs without anti-bias context AND stderr-warns. Op-layer write-back in src/core/operations.ts search/query/get_page handlers fires bumpLastRetrievedAt(engine, pageIds) (fire-and-forget, 5-min throttled via SQL clause, default-on with search.track_retrieval config escape hatch); internal callers (sync, migrations, dream cycle) bypass the op layer so the LSD stale signal stays clean. The fire-and-forget IIFE is tracked in a module-scoped Set<Promise<unknown>>; awaitPendingLastRetrievedWrites(timeoutMs?: number): Promise<{outcome, pending}> resolves once all tracked promises settle, bounded by a 5s Promise.race timeout that stderr-warns the pending count. src/cli.ts awaits the drain unconditionally for every op in the op-dispatch finally block BEFORE engine.disconnect(), then a fallback process.exit(0) fires ONLY when outcome === 'timeout' AND shouldForceExitAfterMain(argv) (excludes serve so daemons stay alive) — so the IIFE cannot race disconnect and leave PGLite's WASM keeping Bun's event loop alive. pages.last_retrieved_at TIMESTAMPTZ NULL has a full (NOT partial) B-tree index covering both NULL and range branches; full forward-reference bootstrap probe on both engines. Frontmatter mode: lsd makes the dream-cycle synthesize phase skip LSD output via isLsdOutput() in src/core/cycle/transcript-discovery.ts short-circuiting isDreamOutput(). gbrain eval brainstorm <fixture.jsonl> is a three-axis conjunctive gate (distance + usefulness + grounding — distance alone is gameable). gbrain doctor has a brainstorm_health check (migration applied, search.track_retrieval setting, calibration cold-start status). judges.ts computes the judge token budget via computeJudgeMaxTokens(ideaCount, modelId) (named constants TOKEN_BUDGET_PER_IDEA, TOKEN_BUDGET_ENVELOPE, LEGACY_MIN_MAX_TOKENS, MAX_OUTPUT_TOKENS_CEIL; ANTHROPIC_OUTPUT_CAPS map: Opus 4.7 32K, Sonnet 4.6 / Haiku 4.5 64K, legacy Claude 3.5 8K) so a large multi-call judge doesn't truncate mid-JSON; with no modelOverride the cap routes through the gateway's actual configured chat model via getChatModel(). --save for both commands persists through the canonical ingestion path: persistSavedIdea(engine, {slug, content, provenanceVia}) calls importFromContent({noEmbed:true, sourcePath}) (chunked + tagged + content_hash so search finds it, no embedding cost at save) THEN renders the saved row to disk via the shared writePageThrough helper (file rendered FROM the row so the two sinks can't diverge and gbrain sync doesn't churn it). formatSaveOutcome(outcome, ctx) returns an honest per-branch message (both-sinks, DB-only when no sync.repo_path/repo-not-a-dir, DB-saved-but-file-errored, total-failure → loud save FAILED … NOT persisted on stderr + nonzero exit) — --save never prints "Saved" when the DB write failed. buildIdeaSlug(question, label, nonce?) adds a random nonce suffix (injectable for tests) so two same-day runs sharing the first 60 slug chars don't clobber. --json callers stay DB-only. buildBrainstormFrontmatterObject(result) in orchestrator.ts returns the object form for serializeMarkdown. Pinned by test/last-retrieved.test.ts, test/e2e/pglite-cli-exit.serial.test.ts (IRON-RULE: real bun src/cli.ts subprocess against a hermetic PGLite tempdir asserts search/get/query exit 0 in <15s + daemon-survival), test/fix-wave-structural.test.ts (asserts the drain await is textually BEFORE engine.disconnect), test/brainstorm/{distance,lsd-mode-skip,eval-brainstorm,judges-maxtokens,save}.test.ts. Open Collider source: github.com/CL-ML/open-collider.

  • src/core/write-through.ts — shared atomic disk write-through for the canonical ingestion path. resolvePageWriteTarget(engine, slug, sourceId) is the exported single source of truth for the ONE file a (source, slug) pair lives in on disk: a source with its own local_path writes at that tree's ROOT (never nested under .sources/); a source without one nests under the host repo (sync.repo_path, default at root, non-default under .sources/<id>/) unless that path is another source's own working tree (leak guard → source_repo_belongs_to_other_source); the page's recorded source_path (or a contained file:// source_uri) is preferred over a slug-derived name so writes land in the file of record instead of minting a twin; and isWriteTargetContained rejects hostile rows escaping the tree (path_escapes_source_root). The ok-result also carries sourcePathToBind — the target expressed in the file scanner's source_path convention (GIT-ROOT-relative when the scan root sits inside a git repo, so a subdirectory-scoped local_path binds the same form delete-reconcile keys on; scan-root-relative otherwise). The resolver is shared by writePageThrough, the facts fence writer (src/core/facts/fence-write.ts), and fence forget (src/core/facts/forget.ts) — the fence appends to the page's file, so all writers MUST compute the identical path or the fence lands in a file sync never reads back and the next extract_facts reconcile deletes the fence-owned DB rows. writePageThrough(engine, slug, {sourceId?, frontmatterOverrides?, logger?}) resolves the target through it, re-reads the just-written DB row (getPage), renders it via serializePageToMarkdown, and writes the .md so the brain has a committable artifact that round-trips through gbrain sync. Rendering FROM the row means file and row cannot diverge. ATOMIC: writes to a unique temp sibling (<file>.tmp.<pid>.<rand>) + renameSync, cleaning up temp on any failure, so a crash or concurrent gbrain sync/autopilot walking the live git tree never reads a half-written .md (matches the .tmp + rename convention in import-checkpoint.ts / op-checkpoint.ts). After a successful rename, a row whose source_path is still NULL is immediately bound to sourcePathToBind (pages born via put/capture/reverse-write otherwise stay source_path=NULL forever because mtime-watermark incremental sync never rescans an untouched file); NULL-guarded so a scanner-recorded path is never rewritten, best-effort so a failed bind never fails the write. Never throws — returns WriteThroughResult { written, path?, skipped?: 'disabled_by_config' | 'no_repo_configured' | 'repo_not_found' | 'source_repo_belongs_to_other_source' | 'page_not_found_after_write' | 'path_escapes_source_root' | 'case_insensitive_collision', error? } so the caller decides messaging + exit codes. isWriteThroughDisabled (shared with the fence lane, ~30s per-engine cache) honors the sync.write_through opt-out. Trust gating (subagent sandbox, dry-run) stays at the CALLER. On a durability-hardened repo (isDurabilityHardened — the gbrain post-commit hook is installed, i.e. the user ran gbrain sources harden), a successful write is best-effort COMMITTED via commitWriteThroughFile (path-limited git commit -- <file>, never sweeps unrelated edits; the hook then background-pushes) so write-through content reaches git instead of accumulating uncommitted forever; result carries committed?: boolean. Unhardened repos keep write-only behavior. Consumers: put_page op and gbrain brainstorm/lsd --save via persistSavedIdea. Pinned by test/write-through.test.ts + test/write-through-commit.serial.test.ts. The put_page op is that deciding caller for direct writes: a skip outside the deliberate DB-only set (no_repo_configured/disabled_by_config/subagent_sandbox/dry_run) throws storage_error, deleting a just-created row first so "created" is never answered for a page with no file backing.

  • src/core/model-id.tssplitProviderModelId(input: string | null | undefined): {provider: string | null, model: string} shared parser for the pricing side. Splits on : first, then /. Defensive contract: null/undefined/empty/whitespace returns {provider: null, model: ''}. Five sites consume it (src/core/anthropic-pricing.ts:estimateMaxCostUsd, src/core/budget/budget-tracker.ts:lookupPricing, src/core/eval-contradictions/cost-tracker.ts:pricingFor, src/core/minions/batch-projection.ts at two call sites, src/core/model-config.ts:isAnthropicProvider) so the pricing + classification surface has no parallel re-implementations of provider:model splitting — slash-form ids (anthropic/claude-sonnet-4-6) classify correctly instead of falling through to "unknown model". Distinct from the gateway-side parseModelId in src/core/ai/model-resolver.ts, which throws on bare names because routing needs an explicit provider; this one returns {provider: null, model: 'bare'} because pricing lookups happen against bare model ids. Pinned by test/model-id.test.ts.

  • src/commands/recall.ts — the gbrain recall CLI (facts-first read; local engine or thin-client). Entity→text fallback: a bare positional is entity-first, but when the entity arm matches ZERO facts (and no --grep/--supersessions/--session-id narrows the read) the query retries as the SQL-level fact-text grep with a one-line stderr note — so keyless/casual usage (gbrain recall coffee) finds the fact either way. One gating predicate + one note formatter (entityTextFallbackApplies/noteEntityTextFallback) shared by the local path and the thin-client mirror so the two can't drift; explicit --grep callers keep exact semantics (their filter already ran, no fallback surprise). --since composes with the positional entity and with --session-id through engine.listFactsSince({entitySlug, sessionId}) (one query, cutoff before LIMIT, event time unless --since-last-run), mirroring the recall op's composition in src/core/ops/facts.ts; pinned by test/facts-recall-since-composition.test.ts.

  • src/commands/transcripts.ts — the transcripts command family, all local-only (ctx.remote=false by construction). recent: raw .txt corpus reads via listRecentTranscripts (same library as the gated get_recent_transcripts op). ingest <path-or-glob>: the cross-harness session importer — resolves ONE source id (6-tier chain), threads activePack once, streams progress (phase transcripts.ingest), and calls runTranscriptsIngest; embedding is OFF by default (embed backfill is the catch-up lane; the embed flag opts in); the max-bytes flag (validated size string, e.g. 4gb) overrides the per-format file/store byte caps for oversized stores while omission preserves each adapter's native default; the since-last watermark is an op-checkpoint (op transcripts-ingest, fingerprint = source + pathspec + format + adapter version + any explicit byte cap via ingestCheckpointFingerprintInput — a checkpoint written under one cap is never silently reused under another, so a capped run's skipped tail can't read as scanned; the pathspec binds the user-stated paths resolved + sorted, and the no-arg/all lane substitutes hostname + sorted harness roots so DB-backed checkpoints shared across a brain's machines never let one machine inherit another's watermark and skip sessions it never scanned) advanced ONLY after a clean, untruncated, non-dry scan; no-arg = confined discovery table, the all flag imports the discovered set. status: found-vs-imported gap table (disk scan vs ONE conversation-pages frontmatter query — executeRaw, both engines — in src/core/transcripts/discover.ts) — the correctness surface that catches late-arriving sessions no watermark can. The facts flag hands EVERY touched slug (including hash-skipped) to src/core/transcripts/ingest-facts.ts. Registered in CLI_ONLY + CLI_ONLY_SELF_HELP + SELF_HELP_WITHOUT_ENGINE (engine-free help). expandTilde(raw) (bare ~, leading ~/ or ~\\ → home dir; ~user forms and a mid-string ~ stay literal) is the one tilde expansion expandPaths applies to every path spec.

  • src/core/transcripts/ (directory) — the transcript-adapter seam. types.ts: the TranscriptAdapter contract — parse(path): AsyncGenerator<ParsedSession, FileDiagnostics> (one FILE may hold many sessions; the generator RETURN value carries bytes/skipped-lines/zero-session diagnostics so an empty file explains itself), format-specific byte caps (JSONL 50MB hard cap — the 10MB default in claude-code-jsonl.ts belongs to the hook-lane tail reader, not imports; monolithic export JSON rejects-not-truncates at 200MB), and the ONE buildTranscriptSlug helper (per-provider dirs matching the conversation-archive layout; id suffixes are sha256 prefixes of the session id — hash12 in slugs, hash16 for the dedup identity — never a cleaned prefix of the source id, which would let same-prefix session ids collide). detect.ts: adapter registry + head-sample detection (explicit format wins; symlinks lstat-rejected) + injectable harnessRoots (the confined discovery surface). Adapters, each with a DATED SPEC_TARGET + scrubbed fixture + bytes>0 && sessions==0 drift alarm: claude-code.ts (thin wrapper over the shipped claude-code-jsonl.ts, which also exposes the full-file parseClaudeSessionFile with real per-message timestamps — hook-lane parseTranscript output is pinned byte-identical; owns isClaudeCodeSubagentFile<session>/subagents/agent-*.jsonl logs are all-sidechain, zero-turn files carrying the PARENT session id, so discover.ts and the CLI's expandPaths both skip them and they never read as gap-table backlog or drift; strict parent-dir + basename shape so a project slug containing "subagents" is never hidden), codex.ts (turn selection is STRUCTURAL: user turns from event_msg user_message, assistant from response_item output_text; role user/developer response_items are injected preambles and never leak), openclaw.ts (session header + message lines; .checkpoint.*.jsonl snapshots rejected), hermes.ts (COPY-THEN-READ of state.db + wal/shm sidecars — readonly WAL opens need -shm write access and lock against a live writer; schema from the installed hermes-agent source; PROVISIONAL), grok.ts (Grok Build session dirs — structural turn selection, sidecar-aware discovery; see its own entry), chatgpt-export.ts (the mapping-tree current_node walk; branches dropped by design, orphaned parents terminate quietly, latest-leaf fallback; extracted conversations.json only) + claude-export.ts (flat chat_messages); both export adapters load through the shared export-json.ts (monolithic JSON over the cap REJECTS with a split hint; a zip or wrong-shape file gets the unzip-first hint) and give id-less conversations content-derived fallback ids so two id-less exports cannot dedup-skip or overwrite each other. render.ts: session → part pages — imessage-slack line format with the regex IMPORTED from conversation-parser/builtins.ts (round-trip pinned), REAL timestamps (missing ones carry forward; zero-timestamp sessions refused — provenance is never fabricated), anchor-shaped BODY lines backslash-escaped (hostile message content cannot forge speakers), quoted facts/takes fence-marker tokens backslash-escaped at the same per-message seam (gbrain\:facts:begin — a session that read another page cannot plant a live or unbalanced fence on the transcript page; every fence consumer is a substring matcher, so the escape lands inside the token), fail-closed redaction (secret-scan + harvest-private-patterns.txt user patterns, slack-channel default excluded because it eats issue refs; imperatives COUNTED into hash-covered transcript_import frontmatter, never content_flag), ~300KB message-boundary splitting with 2-message overlap (under the embed-skip threshold), part 1 keeps the base slug, frontmatter.id unique per part. ingest.ts: the engine-facing core — SESSION atomicity (failed sessions skip; integrity failures abort the run), stale-part reconciliation (deletes part > of leftovers), session metadata banked in the base page's raw_data as the REDACTED copy and healed by content-compare even on hash-skipped re-runs (a run that died before putRawData, or a private pattern added after first import, repairs on the next pass), cleanScan/maxSessionTs for the watermark. ingest-facts.ts: ONE runExtractConversationFactsCore invocation (batch slugs selector) inside ONE withBudgetTracker, isFactsExtractionEnabled pre-checked. Pinned by test/transcript-adapters.test.ts, test/transcript-render.test.ts, test/e2e/transcripts-ingest-pglite.test.ts, test/e2e/transcripts-writeback-fidelity.test.ts (raw files through the adapters into the gold-extractor facts path).

  • src/core/connectors/ (directory) — the LIVE chat-history connector layer (front-end to the transcripts pipeline). types.ts: the ChatHistoryProvider contract (mirror of TranscriptAdapter — leaf module per provider; spoolFormat names the adapter that parses its output; dated HostSpecTarget). credentials.ts: file-plane store at ~/.gbrain/connectors/<provider>.json (0600, dir 0700, atomic tmp+rename), env-above-file resolveCredential returning {cred, source}; credentials NEVER touch the DB / sources.config / config planes / op payloads. classify.ts: the pure §2A/§2B response classifier — distinguishes a Cloudflare-fingerprint 403 (forbidden_fingerprint, dead-end) from an auth-expired 403 (auth_required, re-auth) and flags a parses-but-no-items body as drift. client.ts: ConnectorClient modeled on GitHubClient (plain fetch against a hardcoded ORIGIN + off-origin guard so the cookie/bearer never leaks, browser UA, refresh-once-on-401, Retry-After backoff; fetchImpl/sleep/now injectable — NOT fetchWithSSRFGuard, which breaks SNI on Cloudflare hosts). providers/chatgpt.ts + providers/claude.ts (registry in registry.ts; Perplexity deliberately absent — no adapter): offset/updated pagination with a per-pass 500-page cap and a stopBefore watermark break, epoch/ISO normalization, ChatGPT's is_archived second pass, org discovery + content-block text assembly for Claude, and a refreshAccessToken (ChatGPT re-mints from the cookie via /api/auth/session; Claude's cookie is terminal). spool.ts: writes fetched conversations in native-export shape (0600, batched, pruned in the orchestrator finally). sync.ts: runConnectorSync — resolve credential → probe → read the config-scalar watermark connectors.<p>.watermark_iso (NOT op_checkpoint, whose 7-day GC would wipe it → full re-fetch) → list to watermark − windowDays → fetch → spool → runTranscriptsIngest → advance the watermark ONLY on a fully clean run → logIngest receipt → stamp last_sync_at → engine-branched embed kickoff (maybeKickoffEmbed: Postgres submitEmbedBackfill, PGLite runEmbedCore inline — NEVER submitEmbedBackfill, which refuses no_worker_surface). config-keys.ts: the connectors. key builders + the pure isConnectorSyncStale(last, now, floor) dispatch gate. oauth-pkce.ts: dependency-free S256 PKCE loopback (Bun.serve, timing-safe state) — best-effort --try-oauth only. Surfaces: the connectors_status/connector_sync ops (src/core/ops/connectors.ts, both localOnly), the gbrain connectors command family (src/commands/connectors/ peeled dir), the connector-sync minion handler (src/core/minions/handlers/connector-sync.ts, single-flight lock), the autopilot maybeDispatchConnectorSyncs gate, and the connectors doctor check. Pinned by test/connectors-*.test.ts + test/e2e/connectors-sync-pglite.test.ts (full pipeline against a Bun.serve fixture backend, keyless) + test/e2e/connector-sync-handler-pglite.test.ts + test/e2e/doctor-connectors-pglite.test.ts.

  • src/commands/integrity.tsgbrain integrity check|auto|review|extract: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip). scanIntegrity() is the shared library function called from gbrain doctor (sampled at limit=500) and cmdCheck (full scan). Batch-load fast path on Postgres uses a single SQL query (avoids the PgBouncer per-row round-trip timeout), gated by engine.kind === 'postgres' at the call site so PGLite never enters batch; fallback catch logs at GBRAIN_DEBUG=1. Batch projection is SELECT ... ORDER BY source_id, slug (NOT SELECT DISTINCT ON (slug), which would collapse same-slug-different-source pages into one scan) so multi-source brains scan each (source, slug) row independently. Sequential and auto-repair loops use listAllPageRefs() to enumerate (slug, source_id) pairs and thread sourceId to getPage; batch + sequential paths report the same page count on multi-source brains.

  • src/core/timeline-dedup-repair.ts — schema-drift self-heal for idx_timeline_dedup. A master-merge migration renumbering can leave a brain's version counter stamped past the index change while the index keeps an old shape — and every addTimelineEntry batch then fails its ON CONFLICT inference, silently breaking timeline writes brain-wide. The version counter can't detect this, so the repair is keyed off the actual index SHAPE. Canonical shape is (page_id, date, md5(summary), source) — md5-keyed (migration v138) because a raw long/incompressible summary would overflow the btree v4 row cap (~2704 bytes) and abort every timeline insert for that page; both engines' insert sites infer ON CONFLICT on the md5 tuple, so EXPECTED_COLUMNS MUST carry the md5 form (a raw-shape expectation would make the repair revert v138 on every migrate pass). checkTimelineDedupIndex(engine) returns {tablePresent, indexPresent, columns, needsRepair} (read-only; powers the timeline_dedup_index doctor check; the indexdef column parser is paren-depth-aware so md5(summary) — or any future expression column — parses as ONE column instead of flagging a correct index as drifted forever) and repairTimelineDedupIndex(engine) dedupes-then-rebuilds to the canonical shape (raw-summary grouping ⟺ md5 grouping modulo negligible collisions). runMigrations invokes the repair on every pass (including the no-pending early-return path); idempotent no-op when the index already matches. gbrain apply-migrations --force-schema triggers it on demand. Pinned by test/timeline-dedup-repair.test.ts.

  • src/core/progress.ts — Shared bulk-action progress reporter. Writes to stderr. Modes: auto (TTY \r-rewriting; non-TTY plain lines), human, json (JSONL), quiet. Rate-gated by minIntervalMs and minItems. startHeartbeat(reporter, note) for single long queries. child() composes phase paths. Singleton SIGINT/SIGTERM coordinator emits abort events for every live phase. EPIPE defense on both sync throws and stream 'error' events. Zero dependencies. emitHumanLine is prefix-aware — inside a withSourcePrefix(id, ...) scope from src/core/console-prefix.ts it prepends [id] (and TTY-rewrite mode \r\x1b[2K carries the prefix inside the clear-to-EOL escape); emitJson is intentionally NOT prefixed so NDJSON consumers don't choke on a [id] {...} shape.

  • src/core/console-prefix.tsAsyncLocalStorage<string>-backed per-source line-prefix helper. Exports withSourcePrefix(id, fn) (runs fn with id as active prefix; nested wraps replace then restore), getSourcePrefix() (read-only accessor; test seam), slog(...) / serr(...) (prefix-aware console.log/console.error), and withHumanLogsToStderr(fn) (runs fn with every slog line, prefixed or not, routed to stderr; runSync wraps its body in it under --json so stdout carries only JSON lines — the envelope plus any JSON status lines — while serr and direct console.log(JSON.stringify(..)) sites are unaffected; scoped via its own AsyncLocalStorage, never a process-wide console rebind). Embedded-newline-safe: a multi-line string under prefix [foo] emits [foo] line1\n[foo] line2. Outside a wrap, slog/serr fall through to bare console.log/console.error so single-source callers see identical output (back-compat invariant). Use src.id (slug-validated by sources add) NOT src.name (free-form) to defeat log-injection through newline/control-character names. Coverage: src/commands/sync.ts performSync + callees, src/commands/embed.ts runEmbedCore + helpers, src/core/progress.ts emitHumanLine, src/commands/import.ts runImport's human-only lines (info() + the end-of-run Import complete summary — so the first-sync full import under sync --json lands on stderr too).

  • src/core/cli-options.ts — Global CLI flag parser. parseGlobalFlags(argv) returns {cliOpts, rest} with --quiet / --progress-json / --progress-interval=<ms> / --brain <id> stripped. --brain is the brain-axis (which database) selector: exact-match only (--brain-* per-command flags pass through), value validated against the mount-id regex at parse time, missing/malformed value THROWS — never a silent host fallback. connectEngine in src/cli.ts feeds it (plus the ambient GBRAIN_BRAIN_ID / .gbrain-mount / mount-path tiers) through resolveBrainIdBrainRegistry.getBrain, which throws UnknownBrainError for an unregistered id; mounts get no auto-migrations and keep the host-config AI gateway. getCliOptions() / setCliOptions() expose a module-level singleton so commands reach resolved flags without parameter threading. cliOptsToProgressOptions() maps to reporter options. childGlobalFlags() returns the flag suffix to append to execSync('gbrain ...') calls in migration orchestrators (propagates --brain=<id> so children stay on the parent's brain). OperationContext.cliOpts extends shared-op dispatch for MCP callers. CliOptions carries explain: boolean. parseGlobalFlags recognizes --explain anywhere in argv (stripped before command dispatch). src/cli.ts formatResult for search + query cases routes to formatResultsExplain from src/core/search/explain-formatter.ts when CliOptions.explain is set; falls through to the existing JSON / human formatters otherwise. maybeBackground(opName, fingerprintArgs, runDirect) helper. Same semantics in TTY and cron (no --no-tty-detect flag, no surprise behavior change between contexts): when --background is passed, submits the op as a Minion job via op_checkpoints for resumability and returns the job_id. --background --follow execs gbrain jobs follow <id> so the user sees the same stderr stream they'd get from a direct call. PGLite degrades to inline execution with a clear stderr note ("PGLite worker pool not yet supported; running inline"). Returns a tagged union the caller dispatches on.

  • src/cli.ts strict flag validation + src/core/cli-flag-registry.generated.ts + scripts/generate-flag-registry.ts — pre-dispatch, pre-engine unknown-flag rejection for every command: a flag no handler consults fails loud (unknown flag --x for 'gbrain <cmd>', exit 1; --json invocations also get a structured {status:'error', reason:'invalid_flag'} on stdout) instead of being silently ignored while the un-asked-for real operation runs. validateCommandFlags(command, subArgs) runs after the --help short-circuit and before any dispatch or engine connect, in two lanes mirroring dispatch order (CLI_ONLY first — think/salience/anomalies are both ops AND CLI_ONLY members whose handlers parse flags the op contract doesn't declare): CLI_ONLY commands validate against the generated CLI_FLAG_REGISTRY (per-command legal sets derived from each command's source — case block + imported modules + one level of relative imports + EXTRA_FLAGS; deliberately over-inclusive, help-text mentions count; regenerate via bun run build:flag-registry); op commands validate via findUnknownOpFlag, which mirrors parseOpArgs's traversal (non-boolean flags consume their value token; --key=value inline form recognized) plus the CLI-local flags consumed outside the op contract (json, explain, help, source, dry-run); the CLI_ONLY token scan is the exported findUnknownFlag(args, legal). In parseOpArgs, json/dry_run are CLI-local booleans that never consume a value token, so a trailing --dry-run is a real rehearsal switch feeding makeContext's ctx.dryRun. Uppercase flag spellings are treated as unknown (every handler is case-sensitive-lowercase, so they'd be silently ignored downstream — the exact class the validator kills). Exempt by contract: call (arbitrary --param interface), config (arbitrary set values), jobs submit (handler-defined payload params); everything after a literal -- is passthrough and never validated. A command missing from the registry fails OPEN at runtime (never bricks a command); test/cli-flag-validation.test.ts pins registry freshness, per-command drift, and consumption evidence — a safety flag (--dry-run, --yes, --force) may only be advertised if the command's source actually reads it. The generator segments handleCliOnly into per-command text blocks with segmentDispatchBlocks: both case 'X': labels and every if (command === 'X' …) head — plain, compound (&& args[0] === 'sub': the no-DB eval <sub> bypasses such as eval longmemeval, the <cmd> --help pre-engine branches, agent register) and multi-line — are markers, and ownership follows the command === 'X' head, never the condition's tail, so a compound block's flags land on its own command's row (the eval row is a union across eval subcommands, the registry's shape for every multi-subcommand command); a bare command === 'X' inside a non-if expression is deliberately not a marker. Only import('./commands/*.ts') inside a block counts as a command module (core helpers a block reaches for directly are not scanned — a flag the block consumes through one is already a literal in the block's own text), and isValueOnlyImport skips a destructured import whose bindings are all SCREAMING_CASE constants (a borrowed message string, not a handler). Pinned by test/generate-flag-registry.test.ts (acceptance AND rejection: --frobnicate is still refused) and test/eval-longmemeval-cli-smoke.test.ts (the documented gbrain eval longmemeval … --retrieval-only --by-type --no-trajectory --keyword-only invocation exits 0 as a subprocess).

  • src/core/source-id.ts — single canonical source_id validation, dependency-free by design (imported by both engines, cycle, source-resolver, sources-ops). SOURCE_ID_RE (strict: 1-32 lowercase alnum, interior hyphens only, no edge hyphens); isValidSourceId (boolean — for tiers that silently fall back: dotfile, brain_default) vs assertValidSourceId (throws — for tiers that must reject loudly: explicit --source, GBRAIN_SOURCE, cycleLockIdFor); ALL_SOURCES = '__all__' sentinel (deliberately NOT a valid id so it can never collide with a real source or leak into lock ids/path joins; sourceScopeOpts translates it to an unscoped read for trusted local callers and keeps it unsatisfiable for remote callers, fail-closed). normalizeSourceInput/normalizeFederatedReadInput normalize the /admin/api/register-client HTTP body, mirroring the CLI's --source/--federated-read flags: omitted source'default'; omitted federatedReadundefined so registerClientManual applies its own [sourceId] default; present-but-invalid values throw so the route returns a structured 400 (invalid_source) instead of failing at INSERT time. Pinned by test/register-client-source-normalize.test.ts.

  • src/core/entity-identity.ts + src/core/ops/entity-identity.ts — cross-source entity identity (cross-source federation). THE IDENTITY KEY IS (source_id, slug): the same real-world entity routinely exists as a different page per mounted source, and nothing else in gbrain asserts "these pages are the same entity". The entity_identities table (migration v137) records that assertion explicitly — member pages grouped under an opaque entity_id handle (validateEntityId: slug-flavored, lowercase, max 128 chars), one optional canonical member per group, a page belongs to at most one identity (UNIQUE (source_id, page_id); re-linking MOVES it). Posture is MANUAL-ONLY by design: rows come exclusively from the entity_identity_link op — no auto-matching or name-similarity heuristics, because a wrong identity merge silently corrupts retrieval. Three ops: entity_identity_link / entity_identity_unlink (write, localOnly — merges change what retrieval unions across sources, too sharp for a remote surface) + entity_identity_list (read; restricts MEMBER visibility to a federated grant or remote scalar scope, while a trusted local caller sees the full cross-source group — cross-source IS the feature). Retrieval union is OFF by default behind the entity_identity.union config key; when on, get_links/get_backlinks merge edges from identity co-members (dedup'd), never widening past the caller's source grant. Same SQL text on both engines (parity by construction); pinned by the entity-identity test + the DATABASE_URL-gated engine-parity suite. Topology context: docs/architecture/brains-and-sources.md.

  • src/core/db-lock.ts — generic tryAcquireDbLock(engine, lockId, ttlMinutes) over the gbrain_cycle_locks table. Parameterized lock id so scopes nest cleanly: gbrain-cycle for the broad cycle (held by cycle.ts) and gbrain-sync (SYNC_LOCK_ID) for performSync's narrower writer window. UPSERT-with-TTL semantics survive PgBouncer transaction pooling (unlike session-scoped pg_try_advisory_lock); crashed holders auto-release once their TTL expires. Every handle is FENCED to its exact acquisition: DbLockHandle.acquiredAt captures the row's acquired_at as epoch-seconds text (extract(epoch from acquired_at)::text — GUC-independent, unlike timestamptz::text which varies with per-session TimeZone/DateStyle across pools) and refresh()/release()/the cleanup hook all match (id, holder_pid, fence), so a recycled PID or a superseded holder can never refresh or delete a successor's row. refresh() returns a boolean: true = still owned; false = the fenced UPDATE matched 0 rows (stolen or force-cleared — certain loss, the caller must stop relying on mutual exclusion); transient DB errors still THROW (not evidence of a steal; the TTL is the backstop). Exports LockStolenError (thrown/used as an AbortSignal reason by consumers like cycle.ts's refresher and the supervisor). It also does automatic same-host dead-pid takeover: when the upsert finds a held, NOT-TTL-expired lock whose holder is on this host and provably dead, it reclaims via a guarded DELETE WHERE id=\$1 AND holder_pid=\$2 + one normal-upsert retry returning the standard handle (refresh/release intact). The liveness check is the exported classifyHolderLiveness(pid, host, ageMs, opts?) / isHolderDeadLocally(...) (injectable process.kill seam; HOLDER_TAKEOVER_GRACE_MS = 60_000 PID-reuse guard; EPERM classified as alive so a live process you don't own is never stolen). TTL-expired locks stay the upsert's job; cross-host stays TTL-only. runBreakLock (src/commands/sync.ts) consumes the same predicate. Background reaper: reapDeadHolderLocks(engine) is the periodic sweep the contention path lacked — it deletes locks whose holder is isHolderDeadLocally, scoped to the gbrain-sync:* / gbrain-cycle/gbrain-cycle:* namespaces ONLY (election/supervisor/reindex locks keep TTL-only behavior, untouched), via deleteLockRowExact(engine, id, pid, acquiredAt) — a snapshot-matched delete (date_trunc('milliseconds', acquired_at) = \$3, so the ms a JS Date keeps survives) that's TOCTOU-safe against a reused PID taking the lock between SELECT and DELETE. cycle.ts runs it at cycle start (before the sync phase); gbrain doctor --fix runs it for no-autopilot brains. selectLockRows(engine, opts?) + a shared row→LockSnapshot mapper are the single canonical reader backing inspectLock + listStaleLocks + the reaper. isLockHolderLive(snap, ttlMinutes) is the observability liveness predicate — freshness-keyed (ttl_expired plus the heartbeat steal-grace), never process.kill, so gbrain jobs supervisor status / gbrain doctor can report a live supervisor via its queue lock without a PID-reuse false-positive. Pinned by test/db-lock-auto-takeover.test.ts + test/db-lock-reap.test.ts + test/db-lock-fencing.test.ts. DbLockHandle.refresh(opts?) accepts {signal} (Postgres forwards to executeRawDirect; PGLite ignores); withRefreshingLock's heartbeat aborts the per-tick signal on timeout and guards re-entrancy (15s min cadence vs 30s timeout could stack ticks).

  • src/core/sync-concurrency.ts — single source of truth for the parallel-sync policy. Exports autoConcurrency(engine, fileCount, override?) (PGLite always serial; explicit override clamped to >=1; auto path returns DEFAULT_PARALLEL_WORKERS=4 when fileCount > AUTO_CONCURRENCY_FILE_THRESHOLD=100), shouldRunParallel(workers, fileCount, explicit) (explicit --workers bypasses the >50-file floor), and parseWorkers(s) (rejects '0', '-3', 'foo', '1.5', trailing chars). Used by performSync, performFullSync, runImport, and the Minion sync handler so the sites can't drift. DEFAULT_PARALLEL_SOURCES = 4 is a SEPARATE constant for the per-source fan-out under gbrain sync --all — kept distinct from DEFAULT_PARALLEL_WORKERS because total live Postgres connections per wave ≈ DEFAULT_PARALLEL_SOURCES×DEFAULT_PARALLEL_WORKERS×2(perfilepool)\text{DEFAULT\_PARALLEL\_SOURCES} \times \text{DEFAULT\_PARALLEL\_WORKERS} \times 2 (\text{per}-\text{file} \text{pool}) = 32 at both defaults (each per-file worker opens its own PostgresEngine with poolSize = min(2, resolvePoolSize(2))); sync.ts warns when parallel×workers×2>16\text{parallel} \times \text{workers} \times 2 > 16. resolveWorkersWithClamp(engine, override, commandName, fileCount) wraps autoConcurrency with a per-command stderr clamp warning on PGLite (per-(command, requested) dedup via module-scoped warned-once set with _resetWorkersClampWarningsForTest() seam) and is the canonical surface for every bulk-command --workers N flag (extract-conversation-facts, extract, edges-backfill, reindex-multimodal, reindex, reindex-code); embed.ts deliberately bypasses it and keeps GBRAIN_EMBED_CONCURRENCY || 20. resolveMaxConnections() (reads GBRAIN_MAX_CONNECTIONS, undefined when unset) + clampWorkersForConnectionBudget(workers, perWorkerPool, maxConnections, parentPool) back the opt-in single-sync connection-footprint clamp so a big sync stays under a low pooler cap (parent_pool + workers×perWorkerPool ≤ budget); gbrain doctor's pool_budget check (computePoolBudgetCheck / checkPoolBudget in src/commands/doctor.ts) warns when the budget leaves no room for a worker, pointing at GBRAIN_POOL_SIZE=2. Pinned by test/pglite-workers-clamp.test.ts.

  • src/core/worker-pool.ts — Canonical sliding-pool + bounded-semaphore primitive (used by the src/commands/embed.ts sliding-pool sites and src/commands/eval-cross-modal.ts's runWithLimit semaphore). Two exports: runSlidingPool<T>({items, workers, onItem, signal?, onError?, failureLabel?, onProgress?}) + runWithLimit<TIn, TOut>({items, limit, fn, signal?}). Atomicity invariant: const idx = nextIdx++ is one synchronous JS statement (no await between read and write — guaranteed by the single-threaded event loop), documented in the module header AND enforced by scripts/check-worker-pool-atomicity.sh (wired into bun run verify), which rejects importing worker_threads in any consuming file and inserting await between the nextIdx read and write. MUST_ABORT_ERROR_TAGS set is seeded with BUDGET_EXHAUSTED from src/core/budget/budget-tracker.ts; tagged errors (matched via err.tag === 'BUDGET_EXHAUSTED' to avoid cross-module import) bypass onError and hard-abort the pool via AbortController.abort() to in-flight onItem, then the pool JOINS every worker (each finishes its current item) before rethrowing — the budget cap is a structural ceiling under concurrency and no worker is left running detached past the caller's catch. failures[] shape is {idx, label, error} records (NOT full items; callers supply failureLabel(item) => string) for bounded memory under huge brains. Pinned by test/worker-pool.test.ts + test/scripts/check-worker-pool-atomicity.test.ts. Drives every --workers N bulk command.

  • src/core/embedding-dim-check.ts — facts.embedding dim drift surface. readFactsEmbeddingDim(engine): Promise<FactsColumnDimResult> covers both vector(N) and halfvec(N) shapes (migration v40 falls back to vector on pgvector < 0.7); regex ordering is halfvec-before-vector (substring "vec" appears in "halfvec"; naive /vector/i would shadow). buildFactsAlterRecipe(dims, configured, type) emits the paste-ready DROP INDEX IF EXISTS idx_facts_embedding_hnsw; ALTER TABLE facts ALTER COLUMN embedding TYPE halfvec(N) USING embedding::halfvec(N); CREATE INDEX idx_facts_embedding_hnsw ON facts USING hnsw (embedding halfvec_cosine_ops) WHERE ... flow (NOT bare REINDEX, which doesn't rewrite the index after a column-type change). assertFactsEmbeddingDimMatchesConfig(engine) is the preflight — throws FactsEmbeddingDimMismatchError (tagged tag: 'FACTS_EMBEDDING_DIM_MISMATCH' for parity with the worker-pool MUST_ABORT semantics) when configured dim ≠ column width; cached per-engine via WeakMap; PGLite engines silently skip. Doctor check facts_embedding_width_consistency (registered after embedding_width_consistency) reuses the same helpers with an identical ALTER recipe. decorateEmbeddingDimError(err, slug): maps pgvector's bare "expected N dimensions, not M" write rejection to a named OperationError('embedding_plane_split', …) carrying the consequence (page NOT written, transaction rolled back) + recovery command; applied at importFromContent's transaction boundary so gbrain put fails loudly instead of echoing an unexplained provider string; every other error passes through untouched. Pinned by test/embedding-dim-check-facts.test.ts + test/put-page-embedding-plane-split.test.ts.

  • src/core/sort-newest-first.ts — single source of truth for the descending-lex sort that gbrain import and gbrain sync both apply. Mutates in place (Array.prototype.sort semantics), returns the same array reference for fluent chaining. Empty/single-element inputs short-circuit. Future ordering changes flip one line here instead of touching two CLI commands. Pinned by test/sort-newest-first.test.ts (descending order, mixed prefixes, empty, single-element, in-place-mutation contract).

  • src/core/cycle.ts — brain maintenance cycle primitive (23 phases; ALL_PHASES is the ordered source of truth). runCycle(engine: BrainEngine | null, opts: CycleOpts): Promise<CycleReport> composes phases in semantic order along the core spine lint → backlinks → sync → synthesize → extract → patterns → recompute_emotional_weight → embed → orphans, with the extraction/graph/consolidation phases (extract_facts, extract_atoms, resolve_symbol_edges, …) slotted between per the ordering comments on ALL_PHASES. synthesize runs after sync (cross-references see a fresh brain) and before extract (auto-link materializes its writes); patterns runs after extract so it reads a fresh graph (subagent put_page sets ctx.remote=true and skips auto-link/timeline by default, so extract is the canonical materialization); recompute_emotional_weight sees the union of syncPagesAffected + synthesizeWrittenSlugs incrementally, or all pages when neither anchor is set (full backfill via gbrain dream --phase recompute_emotional_weight). CycleReport.schema_version: "1" is stable; totals is additive (pages_emotional_weight_recomputed, transcripts_processed, synth_pages_written, patterns_written). Three callers: gbrain dream CLI, gbrain autopilot daemon inline path, the Minions autopilot-cycle handler. Coordination via gbrain_cycle_locks DB table + ~/.gbrain/cycle.lock file lock with PID-liveness for PGLite; the two handles compose into ONE lock whose refresh() treats the fenced DB row as the authoritative multi-writer identity and the file half as best-effort freshness — the file lock is refreshed ONLY while the fenced DB refresh reports ownership, so a losing holder can never clobber the successor's file lock on the very tick it detects the steal. A dedicated serialized refresher — startCycleLockRefresher(lock, controller, lockId) (exported; unref'd setInterval at max(15s, TTL/6), env-only override GBRAIN_CYCLE_LOCK_REFRESH_MS, in-flight guard so a slow refresh never overlaps the next tick) — heartbeats the lock through long phases (synthesis/patterns/consolidation waits routinely outlive the TTL with no other heartbeat). A fenced miss aborts the controller with LockStolenError; that steal signal combines with the worker's external signal via the exported anyAbortSignal(signals) (duck-type-tolerant — stubs without addEventListener are observed by poll; returns {signal, dispose} and dispose detaches the caller-signal listener + clears the poll timer so daemons don't leak), and the run stops at the next phase boundary with a structured partial report carrying reason: 'lock_stolen' (for the 5 long phases — synthesize / extract_atoms / patterns / synthesize_concepts / consolidate — the steal races the phase promise and stops the WAIT immediately; their in-flight work runs to its own bounded timeout because their opts can't carry a signal yet). Transient refresh errors log and retry next tick (the TTL stays the backstop). yieldBetweenPhases runs between phases; yieldDuringPhase is in-phase keepalive. Engine nullable; lock-skip on read-only phase selections. CycleOpts.signal?: AbortSignal propagates the worker's abort signal with checkAborted() between every phase. CycleOpts.deadlineAtMs (the enclosing minion job's ABSOLUTE wall-clock deadline, threaded from MinionJobContext.deadlineAtMs by the autopilot-cycle handler; null for direct gbrain dream callers) flows into patterns, propose_takes, AND synthesize so time-spending phases derive their deadlines from the REAL remaining job budget instead of duplicated literals that collide with the job's own kill timer. Pinned by test/cycle-lock-steal.serial.test.ts (mid-run steal → partial report, no further phases, successor row intact) + test/cycle-any-abort-signal.test.ts + test/db-lock-fencing.test.ts. runPhaseSync returns pagesAffected via SyncPhaseResult (threaded to runPhaseExtract as the 4th arg) and takes willRunExtractPhase: boolean setting noExtract: phases.includes('extract') so gbrain dream --phase sync doesn't silently lose extraction. The extract phase calls runExtractCore and extractStaleFromDB with quiet: true (and jsonMode: false), so the helpers' human summaries never land on stdout ahead of the dream --json CycleReport while batch-loss diagnostics stay human-readable on stderr (quiet is its own knob on ExtractOpts — using jsonMode: true as a stand-in flipped the stderr channel to JSON events in a plain gbrain dream); totals.pages_extracted counts pages (pages_processed from the targeted pass + stale_pages_drained), not links created. resolveSourceForDir(engine, brainDir) threads sourceId to performSync() so sync reads the per-source sources.last_commit anchor (not the drift-prone global config.sync.last_commit). CycleOpts.brainDir is string | null; when null (checkout-less postgres/Supabase brain) the 6 filesystem phases (lint/backlinks/sync/synthesize/extract/patterns) skip with details.reason: 'no_brain_dir' and the DB-only phases run; resolveSourceForDir is null-tolerant. cycleSourceId = opts.sourceId ?? resolveSourceForDir(engine, brainDir) is the canonical per-source scope for extract_facts/extract_atoms/calibration — and for synthesize (threaded as SynthesizePhaseOpts.sourceId so synthesized pages land in the cycle's resolved source, not 'default') — so gbrain dream --source repo-a reconciles repo-a's facts even with no checkout (instead of scoping to 'default' while stamping repo-a fresh). deriveStatus counts edges_resolved/edges_ambiguous as work so an edges-only cycle reports ok not clean, and scores ONLY attempted phases — the implicit source-cycle exclusion skip-records are bookkeeping and never dilute failure aggregation; the jobs.ts autopilot-cycle + phase-wrapper handlers pass null (not '.') when no repo is configured. The cycle is SPLIT for autopilot fan-out along the PHASE_SCOPE taxonomy in src/core/cycle/phase-scope.ts (see its entry): cycle.ts derives SOURCE_PHASES / MIXED_PHASES / GLOBAL_PHASES / MAINTENANCE_PHASES (mixed ∪ global, original cycle order) plus SOURCE_FRESHNESS_PHASES (deterministic, non-LLM: lint/backlinks/sync/extract/extract_facts/recompute_emotional_weight — defined in phase-scope.ts) and SOURCE_BACKGROUND_PHASES (LLM-backed/unbounded source work = SOURCE minus FRESHNESS). resolveCyclePhases(requested, sourceId) at the shared runCycle boundary: default/no-source → requested ?? ALL_PHASES (the canonical default cycle remains full); a named non-default source with NO explicit phases → SOURCE_FRESHNESS_PHASES only (the freshness keeper's implicit dream --source X path must stamp freshness without first draining LLM-backed maintenance); explicit phase lists are honored VERBATIM — dream --source X --phase synthesize (and --input <file>, which implies synthesize) is deliberate operator intent, and --phase orphans --source X still narrows orphans via forceGlobalOrphans. The N-way duplication guard lives at the QUEUE boundary instead: the autopilot-cycle handler intersects queued per-source payloads with SOURCE_FRESHNESS_PHASES (queued payloads may carry mixed+background phases; an all-rejected or empty list is an explicit no-op skip with reason all_phases_rejected_by_normalization, never an implicit run, and rejected phases surface on the job result as phases_rejected_by_normalization). On the implicit path, excluded phases surface as skipped with details.reason: 'excluded_from_implicit_source_cycle' + phase_scope. Per-source autopilot-cycle jobs enqueue phases: SOURCE_FRESHNESS_PHASES and stamp last_source_cycle_at; the single autopilot-global-maintenance job runs MAINTENANCE_PHASES (no sourceId) and stamps the brain-level autopilot.last_global_at config key (LAST_GLOBAL_AT_KEY). SOURCE_BACKGROUND phases have NO automatic lane on multi-source brains (the legacy full-cycle job fires only when the brain has no sources rows) — run them explicitly (gbrain dream --source X --phase extract_atoms) until the background lane lands (see TODOS). The freshness stamp gate is opts.sourceId && phases.length > 0 && engine && !dryRun && !aborted && status ∈ {ok, clean, partial}; last_full_cycle_at is still written alongside last_source_cycle_at on a per-source success for doctor/legacy readers (not a gate for the brain-wide phases). Pinned by test/dream-postgres.serial.test.ts + test/jobs-autopilot-cycle-braindir.serial.test.ts + test/autopilot-global-maintenance.test.ts + test/cycle-enabled-phase-completeness.test.ts. runPhaseLint + runPhaseBacklinks carry the export keyword so behavioral tests can drive them directly (internal helpers exposed for test-only consumption; downstream code should NOT depend on them). Pinned by test/cycle-legacy-phases.test.ts (11 cases across both phases: clean run → status='ok', partial fix → status='warn' with dryRun in details, dry-run path doesn't write, throw-from-lib → status='fail' with the wrapper's try/catch envelope populated). with src/core/cycle/extract-atoms.ts + src/core/cycle/synthesize-concepts.ts + src/commands/extract.ts + src/commands/doctor.ts + src/core/op-checkpoint.ts: six daily-driver ops behaviors. (1) Batch idempotency: atomsExistingForHashes(engine, sourceId, hashes[]) (exported from src/core/cycle/extract-atoms.ts) answers in one batched SQL roundtrip (never a per-hash loop) returning already-extracted content_hash16 values; fail-open (SQL error → empty set, extraction proceeds); powered by migration v104 pages_atom_source_hash_idx (partial expression index on frontmatter->>'source_hash' for atom rows where deleted_at IS NULL; Postgres CREATE INDEX CONCURRENTLY with invalid-remnant pre-drop, PGLite plain). (2) Cycle lock TTL + heartbeat: LOCK_TTL_MINUTES = 5; buildYieldDuringPhase(lock, outer) (exported, with LockHandle) calls lock.refresh() + any external hook on every fire, throttled to 30s via maybeYield, firing both in the main loop AND immediately after every await chat(...); synthesize_concepts uses the same throttled hook. A crashed cycle releases its lock within one short TTL, while the dedicated startCycleLockRefresher interval (see the coordination sentence above) keeps a healthy long-running cycle alive even across a single multi-minute await chat(...) — the timer fires during awaits, so no single slow call can silently expire the lock. (3) Progress wiring: progress?: ProgressReporter opt on ExtractAtomsOpts and SynthesizeConceptsOpts; cycle.ts passes its phase-level reporter down (NOT a child reporter, which would collide on cycle.extract_atoms.extract_atoms.work); phases only call tick()/heartbeat(), cycle.ts owns start()/finish(). (4) by-mention resume: mentionsFingerprint({source, type, since, gazetteerHash}) in src/core/op-checkpoint.ts — the gazetteer hash is load-bearing (adding entity pages mid-pause shifts the hash → new fingerprint → fresh scan against the new gazetteer, never silent skip); the gazetteer itself is entity-page titles PLUS live-verified page_aliases entries joined to entity-typed pages (ambiguous aliases and alias-vs-title collisions within a source are skipped), so body mentions of a documented alias link too; gbrain extract links --by-mention resumes via op_checkpoints with flushAndCheckpoint ordering (links flush to DB FIRST, page keys commit to checkpoint SECOND, persist THIRD, so a crash mid-batch leaves the page un-checkpointed and resume re-scans it); persist every 1000 items OR 30s; clean exit clears the checkpoint; --dry-run skips both load and write. (5) sync_consolidation doctor check (multi-source brains see a paste-ready gbrain sync --all --parallel 4 --workers 4 --skip-failed; single-source "not applicable"; SQL errors return warn via the check's own try/catch). (6) Test-isolation: test/cycle-last-full-cycle-at.test.ts + test/schema-cli.test.ts use per-test GBRAIN_HOME=tempdir. Pinned by test/cycle/extract-atoms-batch.test.ts, test/cycle/cycle-lock-ttl.test.ts (pins LOCK_TTL_MINUTES === 5), test/op-checkpoint-mentions-fingerprint.test.ts, test/cycle/extract-atoms-progress.test.ts, test/cycle/synthesize-concepts-progress.test.ts, test/cycle/yield-during-phase-refresh.test.ts, test/cycle/yield-during-phase-throttle.test.ts, test/extract-by-mention-resume.test.ts, test/doctor-sync-consolidation.test.ts. Companion sync --all recipe block in skills/cron-scheduler/SKILL.md. synthesize_concepts writes concept pages through importFromContent (the same parse→chunk→embed pipeline put_page uses, with put_page's isAvailable('embedding')noEmbed gate) so concepts/ pages carry content_chunks + embeddings and are reachable by retrieval (where source-boost.ts weights them 1.3×). purge phase (soft-delete TTLs) also GCs stale op_checkpoints rows older than 7 days. Non-fatal on pre-v67 brains (DROP-target-table check before DELETE). The cycle threads its abort signal into the embed phase (runPhaseEmbed(engine, dryRun, signal)) so a timed-out cycle's long embed phase honors cancellation and releases gbrain_cycle_locks right away instead of after a full backlog run.

  • src/core/cycle/phase-scope.ts — the phase-scope taxonomy: PHASE_SCOPE: Record<CyclePhase, 'source'|'mixed'|'global'> maps each of the 23 phases (source: safe to parallelize per source; global: must serialize across the brain; mixed: brain-wide read + page write, so it stays in the default/global-maintenance lane until decomposed), plus SOURCE_FRESHNESS_PHASES — the deterministic, non-LLM subset (lint, backlinks, sync, extract, extract_facts, recompute_emotional_weight) that alone defines source freshness, so LLM-backed enrichment can never hold a freshness stamp hostage. cycle.ts re-exports both and derives the scheduling lists from them (see the cycle.ts entry). Consumed by runCycle's resolveCyclePhases boundary, the autopilot fanout's per-source enqueue (src/commands/autopilot-fanout.ts), the jobs global-maintenance handler, and doctor's routing-federation phase-scope surface (src/commands/doctor/checks/routing-federation.ts). Pinned by test/autopilot-global-maintenance.test.ts (SOURCE ∪ MIXED ∪ GLOBAL == ALL_PHASES with no overlap; FRESHNESS ∪ BACKGROUND == SOURCE; resolveCyclePhases boundary semantics).

  • src/core/cycle/synthesize.ts — Synthesize phase: conversation-transcript-to-brain pipeline, a two-stage cascade where cheap scored triage gates frontier synthesis. Reads dream.synthesize.session_corpus_dir, runs runTriagePass (exported; bounded pool dream.triage.concurrency default 4, wall-clock cache-MISS budget dream.triage.max_ms default 5 min — cache hits are free and deferred files report deferred: true, never cached, so the next pass continues) whose judge judgeSignificance emits {score 0-1 ordinal salience, content_type, segments ≤8 verbatim quotes, entities ≤12, reasons} (non-overlapping bands LOW 0-0.29 / MEDIUM 0.30-0.69 / HIGH 0.70-1.0; head 50%/middle 20%/tail 30% sample within dream.triage.max_chars default 24K via safeSplitIndex; out-of-[0,1] scores are unparseable, never clamped) cached in dream_verdicts with the judging model + TRIAGE_VERSION — cache validity requires BOTH to match (switching models.dream.triage re-judges; max_chars/max_tokens deliberately excluded from validity — dream retriage --force re-judges under new sampling knobs). Degenerate verdicts (truncated/refusal/unparseable) are never cached. THE gate is passesTriageGate (triage-rescue.ts): score >= dream.triage.threshold (default 0.5) OR the verified-segment rescue, applied at report construction inside runTriagePass at READ time — retuning the threshold or rescue knobs re-gates with zero re-judging; the stored worth_processing boolean derives from the fixed DEFAULT_TRIAGE_THRESHOLD constant (back-compat only, never the live dial). Passing files fan out one subagent per chunk with max_turns from dream.synthesize.max_turns (default 16) and a bounded advisory buildTriageMapBlock (exported; score/type/entities + chunk-filtered segments, '' for legacy/degraded verdicts so the prompt is unchanged for those) spliced into buildSynthesisPrompt, with allowed_slug_prefixes (sourced from skills/_brain-filing-rules.json dream_synthesize_paths.globs; when dream.synthesize.output_root is set, loadAllowedSlugPrefixes(outputRoot) remaps the wiki/-rooted globs to the configured namespace, and the same root drives the prompt slug templates; default 'wiki', validated against the slug grammar via the exported loadOutputRoot). The phase is source-scoped: cycle.ts threads cycleSourceId as opts.sourceId → each child's SubagentHandlerData.source_id → the subagent tool registry's OperationContext.sourceId, so put_page writes, collected refs, the summary page, and reverse-writes all target the cycle's resolved source ('default' when unscoped; reverse-writes for the cycle's own source land at brainDir/<slug>.md, foreign sources under brainDir/.sources/<id>/). Orchestrator collects slugs from subagent_tool_executions (NOT pages.updated_at) and reverse-renders DB → markdown via serializeMarkdown. Cooldown via dream.synthesize.last_completion_ts, written ONLY on success. Idempotency keys dream:synth-v2:<enc source>:filename:<enc basename>:<hash16>[:c<i>of<n>] (byte-stable, pinned by test/e2e/dream-synthesize-chunking.test.ts; grammar parsed by exported parseSynthV2Key). Fan-out self-heals idempotency-coalesced rows stranded waiting in a FOREIGN dead dream-inline-* queue (cancel releases the key slot → re-add into the live run's queue) instead of burning the 35-min wait on a row no worker will ever claim. Opt-in per-source daily submission cap dream.synthesize.max_submissions_per_source_per_day (default 0 = off; skips whole files — never partial chunk sets; bypassed for explicit --input/--date/--from/--to targets; count-query failure fails OPEN with a stderr warn). --dry-run runs triage, skips synthesis; deferral-aware headlines append "(N not yet triaged — time budget...)" so a time-boxed cold pass never reads as mass rejection. details.triage (threshold/judged/cache_hits/unreliable/degraded/deferred/below_threshold + rescue_band/rescue_checked/rescue_fired + tokens_in/tokens_out/cost_usd) + details.synthesis (jobs/avg_turns/max_turns_config + quote_verify/spend/children_zero_pages) carry the phase telemetry. Subagent never gets fs-write access. renderPageToMarkdown (exported) stamps dream_generated: true + dream_cycle_date into every reverse-write's frontmatter; writeSummaryPage does the same on the summary index — this marker is the explicit identity surface isDreamOutput checks in transcript-discovery.ts. stampDreamProvenance additionally persists the same marker into the pages.frontmatter JSONB row (merge via executeRawJsonb, raw object bound to $N::jsonb) for every child-written page BEFORE reverse-rendering, so generated pages are DB-queryable and a later put_page write-through (which re-renders from the DB row) can't erase the stamp. judgeSignificance, JudgeClient, runTriagePass, buildTriageMapBlock, parseSynthV2Key, loadSynthConfig, TRIAGE_VERSION, and DEFAULT_TRIAGE_THRESHOLD are exported; the triage model resolves via an explicit pre-read of models.dream.triage (preferred; through exported resolveAlias) falling back to the standard resolveModel chain (models.dream.synthesize_verdict → deprecated dream.synthesize.verdict_model → tier utility). splitTranscriptByBudget(content, contentHash, maxChars) splits oversized transcripts at paragraph boundaries (## Topic:---\n ladder) using a deterministic offset seeded from the first 32 bits of contentHash so retries chunk identically; per-chunk char budget = MODEL_CONTEXT_TOKENS[resolvedModel] × 0.9 × 3.5 chars/token (non-Anthropic ids fall back to a 180K-token safe default + once-per-process stderr warn); operator overrides dream.synthesize.max_prompt_tokens (floor 100K, wins) and dream.synthesize.max_chunks_per_transcript (default 24); per-chunk subagent job/wait timeouts are dream.synthesize.subagent_timeout_ms / dream.synthesize.subagent_wait_timeout_ms (defaults 30/35 min). Legacy dream:synth: keys are never produced — loadSuccessfulSynthesisKeys(engine, sourceId, keyPrefix) reads the completed rows of BOTH key families once per phase, so existing brains skip with already_synthesized_legacy_single_chunk/_chunked instead of re-spending the synthesis model, and a transcript whose synth-v2 children already completed skips with already_synthesized_v2_single_chunk/_chunked (findSynthV2Completion; a cancelled row never counts) BEFORE its link manifest is built — otherwise queue.add's idempotency fast path coalesces onto the completed children and they re-enter writtenRefs, so their old pages get quote-repaired, restamped, reverse-written and re-embedded every night. collectChildPutPageSlugs raw-fetches every (job_id, slug) pair (not SELECT DISTINCT) and rewrites bare-hash6 slugs to <hash6>-c<idx> for chunked children (orchestrator-side, zero Sonnet trust). Cap-hit skips write nothing new to dream_verdicts (the pass's cached triage verdict remains — a free cache hit on retry) and record no synthesis job, so raising the cap on next run re-attempts cleanly. Bounds INITIAL prompt size only; tool-loop turn-N accumulation is caught by terminal-error classification in subagent.ts. Verdict routing is gateway-routed: makeJudgeClient(verdictModel) (exported) mirrors tryBuildGatewayClient in src/core/think/index.ts — a construction-time provider/key probe returns null on a clear miss (unknown provider id via resolveRecipe AIConfigError, or Anthropic provider with no key via hasAnthropicKey()). The verdict loop wraps judgeSignificance in try/catch for AIConfigError so mid-run provider failures surface as per-transcript worth=false, reasons=['gateway error: ...'] instead of crashing the phase. Canonical config key models.dream.synthesize_verdict (per PER_TASK_KEYS in src/core/model-config.ts); JudgeClient signature preserved verbatim for test-seam stability; CI guard scripts/check-gateway-routed-no-direct-anthropic.sh prevents reintroducing new Anthropic() here or in think/index.ts. At the queue.add boundary a conditional anthropic: prefix is applied ONLY when the resolved model has no colon AND starts with claude- (because resolveModel returns bare ids from TIER_DEFAULTS/DEFAULT_ALIASES and the subagent validator requires provider:model form) — avoids changing the shared constants which would ripple across every resolveModel caller. Pinned by test/cycle/synthesize-gateway-adapter.test.ts, test/e2e/dream-synthesize-pglite.test.ts (gateway-adapter mid-run AIConfigError catch), test/cycle/regression-pr-wave-r1-r2-r4.test.ts. Drain-loop lock renewal runs through exported runDrainRenewalTick (per-call AbortSignal + timeout + re-entrancy guard) — a hung renewLock cannot stack one checked-out slot per interval firing. Pinned by test/cycle-drain-renewal.test.ts. Execution mode: dream.synthesize.mode (default oneshot) threads to every child as data.mode alongside oneshot_slug_suffix (the structural suffix contract) and require_writes: true; details.synthesis adds mode/oneshot_jobs/fallback_jobs/agentic_jobs/fallback_reasons plus the drain telemetry (inline_concurrency_config/effective, drain_ms, queue-wait + runtime p50/p95, dead_jobs, degraded). Pre-retrieval LINK CANDIDATES manifest (link-manifest.ts, dream.synthesize.link_manifest default on) is built once per transcript from the cached triage entities/segment notes and spliced into buildSynthesisPrompt together with an ALLOWED WRITE PATHS block rendered from the trusted allow-list (the oneshot path never sees a tool schema, so the fence lives in the prompt; rule 2 points at the candidates first). Phase outcome gate: ALL children dead/cancelled → phase fails with SYNTH_ALL_CHILDREN_DEAD (fan-out details preserved on the failure); ANY non-completed child → the cooldown stamp is skipped so released idempotency keys retry next run; 'timeout' (parent stopped waiting) degrades but never triggers the all-dead error. Phase-end deferred-embed closure: whenever the phase wrote pages (regardless of mode — an agentic revert still sweeps debt left by earlier oneshot runs), a 120s-bounded embedStalePages (embed-stale.ts) embeds the NULL-embedding chunks of exactly the pages this phase wrote — never a source-wide sweep; the pre-existing stale backlog stays with the budget-tracked embed-backfill machinery — closing the gap on invocation shapes that never reach the global embed phase (--phase synthesize, autopilot per-source NON_GLOBAL_PHASES). Best-effort (never fails a phase that wrote its pages) and stamps the current embedding signature on fully-re-embedded pages so they stay inside the model-drift invalidation contract. The inline drain lives in src/core/cycle/inline-drain.ts (re-exported here for patterns.ts + __testing). Budget clamping: clamped to the remaining parent-job budget when opts.deadlineAtMs is threaded (via patterns.ts's clampSubagentBudgets template): the clamp re-runs against the live clock PER SUBMIT (the phase is a fan-out with claim-time-anchored child kill switches, so one phase-start clamp only bounds the first child); under the minimum child budget the phase skips honestly (insufficient_cycle_budget) or defers the remaining transcripts (details.budget_deferred_transcripts), the inline drain stops CLAIMING when the budget can't fit another child and unclaimed children are cancelled + deferred (they'd otherwise strand forever in the run's private queue), each completion wait is additionally bounded by the remaining parent budget (floored at 1s so already-terminal children still resolve), and a deferring run neither writes the 12h cooldown timestamp nor counts deferred transcripts as processed — deferral genuinely retries next cycle. THE gate is passesTriageGate (triage-rescue.ts) applied at report construction inside runTriagePass: threshold pass OR the verified-segment rescue for band scores [dream.triage.rescue_floor (0.30), threshold) with content_type in dream.triage.rescue_content_types and ≥ dream.triage.rescue_min_segments (default 2; 0 = off) of the judge's segments verifying as normalized transcript substrings (≥40 chars, deduped) — reports carry rescued/verified_segments, details.triage adds rescue_band/rescue_checked/rescue_fired + judge tokens_in/tokens_out/cost_usd (null when unpriced), and dream-retriage's reconcile/audit read the SAME predicate. TRIAGE_VERSION is 2 (peak-not-average rubric; segment selection prefers concrete facts/decisions). After slug collection and BEFORE the provenance stamp/reverse-write/embed sweep, the quote verify/repair pass (synthesize-verify.ts, dream.synthesize.quote_verify default on) runs on newly-created pages; the phase is wrapped in withChatPhase('phase:synthesize') (children keep their own job tag — minion_jobs stays the child-spend authority) and details.synthesis adds quote_verify, children_zero_pages (completed children with zero put_page writes — the rule-D disposition), and spend (cost_basis: 'in+out+cache_read'; children summed from minion_jobs tokens_input/output/cache_read priced at the configured synth model, triage from the pass usage; total_usd null unless both price) buildSynthesisPrompt is mode-aware (mode: 'agentic' | 'oneshot', passed from config.mode at the sole submit site): agentic children keep the search-tool and final-summary guidance; tool-less oneshot children receive neither, so the prompt never contradicts ONESHOT_SYSTEM's JSON-only rule (the oneshot prompt is stored as data.prompt and reused verbatim by the in-job agentic fallback, whose tools stay in the schema). Pinned by test/cycle-dream-output-root.test.ts.

  • src/core/cycle/cycle-date.ts — the dream-cycle calendar-date policy: resolveCycleDate resolves explicit --date > cycle.timezone config > host IANA timezone > UTC, so a run after local midnight buckets into the day the user actually lived instead of rewriting yesterday's UTC-day summary. isValidTimeZone gates configured values (config.ts also validates at gbrain config set time); an invalid configured timezone falls back loudly instead of killing the cycle. utcDate is retained for source-date fallbacks and legacy pages.

  • src/core/cycle/extract-atoms.ts — the extract_atoms lens phase: mines eligible pages into atoms/<date>/<stem>-<hash> pages. Eligibility is COALESCE(frontmatter->>'atoms_scan_hash','') <> substring(content_hash from 1 for 16); the completion marker (ATOMS_SCAN_HASH_KEY in utils.ts) is EXCLUDED from contentHash()'s input so stamping it can't re-arm the very page it marks (and import-file.ts strips it from untrusted remote===true frontmatter so a remote writer can't suppress mining). Atom identity: page-derived atoms fold the source-page slug into the slug hash (8 chars, NUL-separated) so two same-date pages emitting the same title get DISTINCT slugs; the source CONTENT hash is deliberately NOT folded (a reworded source still upserts rather than duping); transcript atoms keep the legacy title-only 6-char hash. resolvePageAtomSlug adopts a legacy-slug atom with a compatible binding in place on re-extraction (upgrade idempotency — no migration, no duplicate), while assertAtomImportBinding fail-closed refuses to reuse a slug bound to a DIFFERENT source locator. Both share isCompatibleAtomBinding(frontmatter, sourcePageSlug) — the ONE definition of compatible: bound to THIS source page, or carrying no source_slug/source_path at all (pre-binding-era adoption, not a clobber); a source_path-bound legacy transcript atom or a different source_slug is a different origin, and a non-atom page squatting on the slug is refused. source_quote is verified at extraction time against the exact truncateUtf8 prompt prefix the model saw (locateQuote advances by full code points): located quotes persist original characters + [start,end) offsets + source_quote_verified; unlocatable quotes are dropped with the atom kept. Edges write before the completion flip; per-item drain failures stay typed (bounded, sanitized, reconcilable). Transcript items carry the same failure-count/tombstone machinery as pages, in the extract_atoms_transcript_state table (migration extract_atoms_transcript_state_table; keyed (source_id, file_path, content_hash) with the 16-char hash prefix, so an edit is a new row and re-eligibilizes): recordItemFailureCount counts a transcript's malformed output toward MAX_DETERMINISTIC_FAILURES, a zero-yield transcript tombstones immediately (the page atoms_scan_hash semantics), and tombstonedTranscriptsForHashes is the batch read-side gate mirroring atomsExistingForHashes (fail-open on error). Reported as tombstoned_transcripts (paths), separate from the page-slug tombstoned_for_failures.

  • src/core/cycle/extract-atoms-cost-gate.ts — pure cost-gate decision for the extract_atoms BudgetTracker. resolveExtractAtomsCostGate(extractModel, embedModel, overrides?, {explicitBudget?}) returns {enforceCap, unpricedModel?, unpricedKind?, zeroPricedEmbedModel?, pricingOverrides?}: the cap is enforceable only when EVERY model billed under the phase's tracker is priceable — the extraction chat model AND the embedding model the atom import (importFromContent, inside the same withBudgetTracker scope) calls — with operator pricing.overrides consulted first, matching BudgetTracker.reserve(). resolveEmbedModelForCostGate() mirrors the write site's isAvailable('embedding') gate (null when embedding is unavailable, so the import runs noEmbed and nothing embeds). Only a DEFAULT cap may be dropped that way: when the operator SET cycle.extract_atoms.budget_usd (explicitBudget), an unpriced EMBED route keeps the cap and is priced at $0 via the returned pricingOverrides (the caller's map plus the $0 row); an unpriced CHAT model drops the cap either way. The phase passes the decision into maxCostUsd and hands the gate's overrides (or its own) to the tracker; it warns once either way (unpriced: kind + model + the pricing.overrides remedy, running uncapped rather than latching budget_exhausted on the first item; zero-priced: the embed model + the rate remedy). Pinned by test/extract-atoms-embed-cost-gate.test.ts (pure cases + a PGLite round-trip with a $0 chat model and an unpriced embed route).

  • src/core/cycle/phases/consolidate.ts — the consolidate phase: clusters facts per page and promotes the best claim into a take. Take identity for the upsert lookup is (page_id, claim) restricted to rows this phase authors (kind='fact' AND holder='self') — since_date is NOT identity (it derives from MIN(valid_from) of the cluster and moves whenever extract_facts re-inserts, which degraded the lookup into duplicate INSERTs); deliberately NOT filtered on active (a superseded take still owns its claim — skipping it would resurrect a retired claim via the INSERT path); ORDER BY id keeps the pick deterministic across pre-existing duplicate rows. A resolved take is immutable — its id is reused for consolidation but the row is left untouched.

  • src/core/cycle/inline-drain.ts — the dream cycle's private-queue drain. runSubagentsInline(engine, queue, queueName, yieldDuringPhase?, handler?, lockMs?, concurrency?) runs N independent drainLoops (dream.synthesize.inline_concurrency, clamp [1,8], PGLite forced serial at the callsite) sharing the queue via the same SKIP-LOCKED claim fencing workers use — everything job-scoped (lockToken, abort, timeout timer, keepalive, renewTimer, outcome recording) stays loop-local. Pool hygiene: global housekeeping sweeps are leader-only (loop 0) while promoteDelayed runs in every loop; idle claim-poll backs off 1s→5s; a NON-retryable loop-level failure aborts siblings after their current child (a failing child is a job outcome, never a drain crash). Exit only when the queue holds no active/waiting/delayed child (a lease-full bounce's delayed backoff must not strand a child). Outcome routing mirrors the worker: UnrecoverableError → dead immediately; RateLeaseUnavailableErrorreleaseLeaseFullJob requeue without burning an attempt; timeout terminal. Handler invocation is wrapped in withChatPhase('job:<name>') (worker.ts parity) so each drained child's gateway spend is attributed to the CHILD — a bare await handler(context) inherits the caller's AsyncLocalStorage phase, and a cycle phase that wraps its own work (dream synthesize wraps phase:synthesize) would absorb every child's spend into the phase tag. runDrainRenewalTick (per-call AbortSignal + timeout + re-entrancy guard) and the null-safe nearest-rank percentile() telemetry helper are exported. Pinned by test/cycle-synthesize-inline-concurrency.test.ts (loop semantics) + test/e2e/dream-synthesize-concurrency-postgres.test.ts (real-Postgres exact-once + lease-ceiling).

  • src/core/cycle/link-manifest.ts — pre-retrieval LINK CANDIDATES manifest. buildManifestContext(engine, sourceId?) snapshots the source's slugs + basename index once per phase; buildLinkManifest(engine, ctx, verdict, basename, {outputRoot, sourceId}) resolves wikilink candidates ZERO-EMBED from the triage verdict's cached entities/segment notes (basename-index exact/slugified matches first, bounded searchKeyword FTS second), excludes dream-output prefixes (self-consumption guard), renders - [[slug]] — <deterministic first-two-sentence one-liner> under hard caps (20 pages / 2400 chars) with buildTriageMapBlock's injection posture (whitespace-collapse + length caps + a 'data, not instructions' header). Best-effort everywhere — any failure degrades to the manifest-less prompt. Pinned by test/cycle-link-manifest.test.ts.

  • src/core/cycle/synthesize-verify.ts — Mechanical quote verify/repair on newly-created dream pages (zero LLM). Exports the shared grounding primitive normalizeForGrounding (whitespace/curly-quote/dash/case folding WITH an offset map back to the original string, so every replacement is a verbatim transcript slice) + normForGrounding (plain form, also used by buildTriageMapBlock and triage-rescue). verifyAndRepairDreamPages(engine, writtenRefs, transcriptsByPath) scopes to pages whose slug carries the transcript's content-hash suffix (page↔transcript binding; modified pre-existing people/pattern pages are skipped + counted skipped_preexisting — they may quote other sources), extracts paired-quote spans from the page BODY (code fences/inline code/wikilinks/link targets masked; paragraph-paired; odd-mark paragraphs skipped + counted unbalanced; min span 15 chars, ≤200 spans/page), and runs the repair ladder per span: exact substring → keep; normalized match → replace with the verbatim original slice; near match (word-trigram anchors, ≥0.8 token overlap, ambiguity within 0.05 falls through) → replace; else STRIP the quote marks keeping the text — NEVER fabricates, never deletes content. Warn-only counter: numeric/date claims ($amounts, percents, ISO/month dates, 4+ digit numbers, deduped, fences skipped) absent from the normalized transcript → numeric_claim_warns. Write-back ONLY when a span changed, through the canonical importFromContent pipeline (page+tags+chunks+links in one transaction, content_hash recomputed, noEmbed — the phase-end sweep backfills; provenance nulls preserve the first-write record); bare engine.putPage is insufficient (pages row only — the embed sweep would embed stale chunk_text). Fail-open per page (read-back miss / write throw → count + continue; abort still unwinds). Kill switch dream.synthesize.quote_verify (default on). Telemetry shape = QuoteVerifyStats in details.synthesis.quote_verify. Pinned by test/cycle-synthesize-verify.test.ts + the write-path mini-eval harness test/cycle-write-path-mini-eval.test.ts.

  • src/core/cycle/triage-rescue.ts — THE dream triage gate (passesTriageGate) + the verified-segment rescue (applyTriageRescue). Rescue fires only for scores in [floor, threshold) (floor inclusive; at/above threshold is the plain gate's job) with content_type in the buried-signal allowlist (default mixed/reflection/idea/strategy/people — never routine/technical) and ≥ minSegments of the judge's own segments verifying as normalized transcript substrings (normForGrounding, ≥40 chars, deduped by normalized quote — repetition is not more evidence). $0 — no LLM calls; fabricated segments cannot fire it; works on cached verdicts (gate-time only). Fail-closed on every malformed shape (null score, missing/short/non-string segments), never throws. minSegments: 0 is the kill switch (gate degenerates to the plain threshold). ONE-GATE RULE: runTriagePass (report construction — worth/rescued/telemetry/dry-run), the synthesize fan-out, and dream retriage (reconcile-queue cancels + --audit-rejects sampling) all read this predicate — a second hand-rolled score >= threshold check is how an operator sweep cancels exactly the jobs the rescue admitted. Config: dream.triage.rescue_floor / rescue_min_segments / rescue_content_types. Pinned by test/cycle-triage-rescue.test.ts + the rescue suites in test/cycle-synthesize-triage.test.ts / test/dream-retriage.test.ts.

  • scripts/check-gateway-routed-no-direct-anthropic.sh — CI guard that fails the build if src/core/cycle/synthesize.ts or src/core/think/index.ts reintroduces a runtime new Anthropic() constructor call or a value-shaped import Anthropic from '@anthropic-ai/sdk' import. Type-only imports (import type Anthropic from '@anthropic-ai/sdk') stay allowed for adapter types; comment lines (// or * prefixes) are excluded so JSDoc doesn't false-fire. Mirrors scripts/check-jsonb-pattern.sh. Wired into bun run verify. Extend GUARDED_FILES when migrating another file off direct SDK construction.

  • src/core/cycle/patterns.ts — Patterns phase: cross-session theme detection over reflections within dream.patterns.lookback_days (default 30). Names a pattern only when ≥dream.patterns.min_evidence (default 3) reflections support it. Reflection excerpts use the shared UTF-16-safe truncator; a raw .slice(0, 600) can split an emoji pair and make Postgres reject the subagent job's JSONB payload. Single Sonnet subagent; same allow-list path as synthesize (imports loadAllowedSlugPrefixes + loadOutputRoot from synthesize.ts — the reflections lookup, prompt slug templates, and allow-list all honor dream.synthesize.output_root, default 'wiki'). Subagent job/wait timeouts are config keys dream.patterns.subagent_timeout_ms / dream.patterns.subagent_wait_timeout_ms (defaults 30/35 min, mirroring the dream.synthesize.* pair). The phase status reflects the child outcome: non-completed outcome with zero writes → fail (error code PATTERNS_CHILD_<OUTCOME>); non-completed with partial writes → warn. Runs AFTER extract so the graph is fresh. The fan-out sets require_writes: true so an all-writes-failed child dead-letters instead of reporting completed. Budget clamping: clamped to the remaining parent-job budget via clampSubagentBudgets when the cycle threads deadlineAtMs — the clamp template synthesize.ts reuses; MIN_PATTERNS_SUBAGENT_BUDGET_MS gates an honest skip, and CYCLE_DEADLINE_RESERVE_MS is re-exported here from its base-phase.ts home

  • src/core/cycle/extract-facts.ts — extract_facts cycle phase. Fence is canonical: per-page wipe (deleteFactsForPage) + reinsert from parseFactsFence + extractFactsFromFenceText + engine.insertFacts. The per-page wipe passes excludeSourcePrefixes: ['cli:'] so conversation facts (written by extract-conversation-facts, on pages with NO ## Facts fence to recreate them from) survive the reconcile instead of being deleted-with-nothing-to-reinsert. The destructive phase never inherits a failed sync's full-brain walk: slugs: [] (a real incremental no-op) is distinguished from slugs: undefined (full-walk intent) by presence, not length. runPhaseExtractFacts (cycle.ts) surfaces a warn (net_fact_deletion) when the reconcile deletes at least NET_DELETION_WARN_FLOOR (50) more facts than it reinserts — the exact signature of a conversation-facts wipe. Empty-fence guard refuses when legacy rows (row_num IS NULL AND entity_slug IS NOT NULL) pend backfill (status: warn, hint: gbrain apply-migrations --yes). A phantom-redirect pre-pass runs AFTER the legacy-row guard, BEFORE the main reconcile loop: when opts.brainDir is set, runPhantomRedirectPass(engine, brainDir, sourceId, dryRun) walks unprefixed-slug pages capped by GBRAIN_PHANTOM_REDIRECT_LIMIT (default 50). The pass returns touched_canonicals — canonical slugs whose disk fence merged with phantom rows; runExtractFacts UNIONs them into the main reconcile slug set so canonical's DB facts derive from the merged fence in the same cycle (handles phantom-had-only-on-disk-fence). ExtractFactsResult carries six phantom fields: phantomsScanned, phantomsRedirected, phantomsAmbiguous, phantomsSkippedDrift, phantomsLockBusy, phantomsMorePending. Three bubble to CycleReport.totals (phantoms_redirected, phantoms_ambiguous, phantoms_skipped_drift).

  • src/core/facts-fence.ts — the ## Facts fence primitives: parse (parseFactsFence), render (renderFactsTable), and strip (stripFactsFence({keepVisibility}) — the remote-read privacy boundary get_page/fetch_page/the chunker apply), plus the remote write-back merge, plus the ONE fence-placement rule: replaceOrInsertFactsFence(body, fenceBlock) replaces an existing fence in place or inserts a fresh ## Facts section ABOVE the timeline sentinel (timelineSentinelOffset, #4756 — below it splitBody() files the fence into page.timeline where extract_facts refuses to reconcile it), EOF only when the page has no sentinel; every writer that materializes a fence (upsertFactRow, the phantom-redirect canonical append, the importer's hidden-row merge) routes through it, so no writer can re-grow its own EOF append. restoreHiddenFactRows(incoming, existing) is the row-level, visibility-aware merge import-file.ts applies per fence-bearing column (compiled_truth AND timeline) when a remote caller writes back a page it read with hidden rows stripped — without it, a remote get→edit→put round-trip would silently drop every non-world fact row. Rules: only non-world rows of the existing fence are restoration candidates (a caller's deletion of a VISIBLE row stays honored — world rows are never restored); a hidden row whose rowNum is absent from the incoming fence is restored at its stable rowNum (cross-page #F<N> refs survive); a rowNum collision (a caller-authored addition landing on a hidden number the caller never saw) keeps the hidden row's number and renumbers the CALLER's row onto fresh appended numbers (upsertFactRow's append-only contract); same rowNum + same claim keeps the incoming version (idempotent full-content write-throughs); either side parsing with warnings returns null — re-rendering a fence that didn't fully parse would drop the caller's unparsed rows — and factsGapWarning surfaces exactly that residual loss. Pure and side-effect-free. Pinned by test/facts-fence.test.ts + the remote write-back describes in test/privacy-strip-and-forget.test.ts (row-level merge, world-only deletion honored, timeline-embedded fence round-trip).

  • src/core/fence-shared.ts — shared pipe-table primitives for the ## Takes (takes-fence.ts) and ## Facts (facts-fence.ts) fences: parseRowCells, isSeparatorRow, stripStrikethrough, parseStringCell, escapeFenceCell. parseRowCells is escape-aware: \| stays inside its cell and decodes back to a literal | (exact inverse of escapeFenceCell), while any other backslash passes through verbatim so existing fence text (e.g. Windows paths) is byte-stable across a render → parse cycle. This matters because the fence is canonical and reconcile is wipe-and-reinsert — a pipe inside a cell shifting the column layout would corrupt every row behind it on the next reconcile. Pinned by the escape cases in test/facts-fence.test.ts + the full render → parse → reconcile round-trip in test/e2e/facts-fence-reconcile-postgres.test.ts.

  • src/core/entities/resolve.ts — Free-form entity name → canonical slug resolution. resolveEntitySlug(engine, source_id, raw): exact slug → alias-exact (an unambiguous page_aliases hit via resolveAliases, verified against LIVE pages since page_aliases has no FK — a stale alias row can never point at a deleted page; fail-open on pre-v110 brains missing the table; ResolutionSource reports alias_exact) → unambiguous bare-name prefix expansion across people/<token>-% + companies/<token>-% → high-specificity fuzzy match for multi-token input (pg_trgm @ 0.7 threshold) → deterministic slugify holding fallback. Bare-name collisions never use popularity as confidence; shared-token company names below the threshold remain unresolved. Two helpers for the phantom-redirect pass: resolvePhantomCanonical(engine, sourceId, phantomSlug) SKIPS the exact-slug step (a phantom slug 'alice' would exact-match itself and no-op the redirect); returns the canonical only when non-null AND contains /. findPrefixCandidates(engine, sourceId, token) is a standalone SQL query returning ALL candidates across PREFIX_EXPANSION_DIRS (hardcoded ['people', 'companies']) via slug LIKE ANY($N::text[]) over patterns dir/token + dir/token-%, cap of 10 ordered by connection_count DESC, slug ASC. Pinned by test/entity-resolve.test.ts (explicit, unique, ambiguous-person, and shared-token-company cases) plus test/phantom-redirect.test.ts (resolvePhantomCanonical 3 cases + findPrefixCandidates 6 cases incl. multi-dir ambiguity and the people/aliceberg-doesn't-match-alice false-positive guard).

  • src/core/cycle/phantom-redirect.ts — Phantom-redirect orchestrator. Exports runPhantomRedirectPass(engine, brainDir, sourceId, dryRun): Promise<PhantomPassResult> (per-cycle wrapper acquiring the gbrain-sync writer lock once for the whole pass, 30s bounded retry, walks up to GBRAIN_PHANTOM_REDIRECT_LIMIT unprefixed phantoms) + tryRedirectPhantom(engine, page, sourceId, brainDir, dryRun): Promise<RedirectResult> + stripFenceAndFrontmatterAndLeadingH1 (pure body-shape gate helper — strips facts fence incl. preceding ## Facts heading and the leading H1; zero residue = phantom). Handler order: body-shape gate → resolvePhantomCanonical (bypasses exact-self-match) → findPrefixCandidates ambiguity check → fenceDbDrift bi-directional check → dry-run early exit → materialize canonical via serializeMarkdown if DB-only → append phantom fence rows to canonical's disk fence with (claim, valid_from) dedup-guard + row_num continuation → engine.refreshPageBody with SHA-256 content_hash recomputed via the import-file shape → engine.migrateFactsToCanonical (lossless) → engine.rewriteLinks (DB FK rewrite; wiki-link text rewrite is a documented follow-up) → engine.softDeletePage + engine.deleteFactsForPage(phantom) + fs.unlinkSync(phantomPath). RedirectResult.canonical populated on 'redirected' (incl. dry-run preview) so the caller builds touched_canonicals. Idempotent on re-run: phantom soft-deleted → predicate fails (deleted_at IS NULL); migrate UPDATE matches no rows; dedup-guard prevents double-append.

  • src/core/facts/phantom-audit.ts — JSONL audit at ${resolveAuditDir()}/phantoms-YYYY-Www.jsonl. Pattern copy of src/core/audit-slug-fallback.ts (ISO-week rotation, honors GBRAIN_AUDIT_DIR). Exports logPhantomEvent(record) + readRecentPhantomEvents(days) + computePhantomAuditFilename(now?). Records every outcome: redirected | ambiguous | drift | no_canonical | not_phantom_has_residue | pass_skipped_lock_busy. Best-effort writes — stderr warn on failure, never throws. Separate file from stub-guard-audit.ts (distinct consumer + lifecycle: stub-guard logs PREVENTIVE blocks; phantom-audit logs CLEANUP decisions, to be read by a future phantoms_pending doctor check).

  • src/core/cycle/emotional-weight.ts — Pure function computeEmotionalWeight({tags, takes}, {highEmotionTags?, userHolder?}). Deterministic 0..1 score: tag-emotion boost (max 0.5, case-insensitive match against HIGH_EMOTION_TAGS seed list), take density (0.1/take, capped at 0.3), take avg weight (0..0.1), user-holder ratio (0..0.1 over active takes; default holder 'garry'). Total clamped to [0..1]. Anglocentric / personal-life-biased seed list intentional; override via config emotional_weight.high_tags (JSON array). userHolder overridable via emotional_weight.user_holder.

  • src/core/cycle/anomaly.ts — Pure stats helpers for find_anomalies. meanStddev returns sample stddev (n-1 denominator) and (0,0) for empty input. computeAnomaliesFromBuckets(baseline, today, sigma, limit) takes densified daily-count buckets + today's counts per cohort, returns AnomalyResult[]. Zero-stddev fallback: cohort fires when count > mean + 1, with sigma_observed = count - mean as a finite sort proxy (no NaN). Brand-new cohorts (no baseline) have mean=0, stddev=0 so the fallback fires at count >= 2. Sorted by sigma_observed desc, top limit (default 20). page_slugs capped at 50 per cohort.

  • src/core/cycle/recompute-emotional-weight.ts — Cycle phase orchestrator. Two SQL round-trips: engine.batchLoadEmotionalInputs(slugs?)computeEmotionalWeight (per-row pure function) → engine.setEmotionalWeightBatch(rows). Reads config emotional_weight.high_tags (JSON array, falls back to default seed list on parse error) and emotional_weight.user_holder. Empty affectedSlugs short-circuits with zero-work success. dry-run reports the would-write count without touching the DB. Engine throw bubbles into status: 'fail' with code RECOMPUTE_EMOTIONAL_WEIGHT_FAIL so the cycle continues.

  • src/core/transcripts.tslistRecentTranscripts(engine, opts) library reused by both the gbrain transcripts recent CLI and the get_recent_transcripts MCP op. Reads dream.synthesize.session_corpus_dir + dream.synthesize.meeting_transcripts_dir config (same as discoverTranscripts); walks .txt files within days; applies the isDreamOutput guard from transcript-discovery.ts (skips dream-generated files); returns {path, date, mtime, length, summary}[] sorted newest-first. Summary mode (default true) = first non-empty line + ~250 trailing chars; full mode caps at 100KB/file. Missing/non-existent corpus dirs return [], not error. Trust gate lives in the op handler, not here: the op throws permission_denied for ctx.remote === true; this is a trusted library function used by both the gated op and the local CLI.

  • src/core/operations-descriptions.ts — Constants module for tool descriptions. Pinned via test/operations-descriptions.test.ts. Houses GET_RECENT_SALIENCE_DESCRIPTION, FIND_ANOMALIES_DESCRIPTION, GET_RECENT_TRANSCRIPTS_DESCRIPTION plus LIST_PAGES_DESCRIPTION, QUERY_DESCRIPTION, SEARCH_DESCRIPTION. Stable surface for the Tier-2 LLM routing eval — keeping them here keeps the test from binding to whatever is in operations.ts at test-run time.

  • src/core/cycle/transcript-discovery.ts — Pure filesystem walk for synthesize. discoverTranscripts(opts) filters .txt files by date range, min_chars, and word-boundary regex excludePatterns (medical matches "medical advice" but NOT "comedical"; power users may pass full regex). readSingleTranscript(path) is the gbrain dream --input <file> ad-hoc path. Self-consumption guard: DREAM_OUTPUT_MARKER_RE (anchored at frontmatter open ---\n, optional BOM + CRLF tolerance, scans first 2000 chars for dream_generated: true with case-insensitive value and word boundary on true) drives isDreamOutput(content, bypass=false). Both functions skip matching files and emit a [dream] skipped <basename>: dream_generated marker stderr log (no silent skips). An excludePatterns hit is never logged per file (the matching files are sensitive by definition); instead both functions emit ONE [dream] excluded N transcript(s) matching exclude_patterns (<label>: n, ...) stderr line at the end of discovery, naming only the count and the configured pattern labels that fired, never a path, basename, or content. DEFAULT_EXCLUDE_PATTERNS is unchanged (medical, therapy). bypassGuard?: boolean on DiscoverOpts and readSingleTranscript's opts disables the guard for the explicit --unsafe-bypass-dream-guard escape hatch only — never auto-applied for --input.

  • src/commands/dream.tsgbrain dream CLI; thin alias over runCycle. Flags: --dry-run, --json, --phase <name>, --pull, --dir <path>, --input <file> (ad-hoc transcript, implies --phase synthesize), --date YYYY-MM-DD, --from <d> --to <d> (backfill range), --unsafe-bypass-dream-guard (plumbed through runCycle.synthBypassDreamGuardSynthesizePhaseOpts.bypassDreamGuarddiscoverTranscripts({bypassGuard}) / readSingleTranscript({bypassGuard}); loud stderr warning at synthesize-phase entry; never auto-applied for --input). Conflict detection: --input + --date exits 2. ISO date validation. --dry-run runs the scored triage pass but skips synthesis (NOT zero LLM calls). Exit 1 on status=failed. resolveBrainDir returns string | null (order: --dir → resolved --source's local_path → global sync.repo_path → null); a checkout-less postgres/Supabase brain runs DB-only phases (incl. resolve_symbol_edges) and skips the 6 filesystem phases with details.reason: 'no_brain_dir'; runDream owns the only hard error (no checkout AND no engine). When --source resolves but has no on-disk checkout, returns null (DB-only) rather than borrowing another source's global sync.repo_path (would mix scopes). Pinned by test/dream-postgres.serial.test.ts. --drain [--window <seconds>] for --phase extract_atoms: runDrain() bypasses the pack-gate and runs the single-hold bounded drain from src/core/cycle/extract-atoms-drain.ts under the same cycleLockIdFor(sourceId) the routine cycle uses (concurrent autopilot tick defers with cycle_already_running), reporting {extracted, skipped, remaining}. Exits EXIT_DRAIN_INCOMPLETE=3 while remaining > 0; a null backlog count (count query FAILED) is also exit 3, never a drained success; LockUnavailableErrorcycle_already_running skip (also exit 3). The extract_atoms_backlog doctor check (computeExtractAtomsBacklogCheck) surfaces the silent pack-gated backlog with the exact --drain command; pack-gated cycle skips carry a greppable pack_gated:true marker. dream retriage dispatches on args[0] === 'retriage' BEFORE parseArgs (its flag set never collides with cycle flags; dream retriage --help prints subcommand help engine-free per the same IRON RULE).

  • src/core/cycle/transcript-discovery.ts — Pure filesystem walk for synthesize. discoverTranscripts(opts) filters .txt files by date range, min_chars, and word-boundary regex excludePatterns (medical matches "medical advice" but NOT "comedical"; power users may pass full regex). readSingleTranscript(path) is the gbrain dream --input <file> ad-hoc path. Self-consumption guard: DREAM_OUTPUT_MARKER_RE (anchored at frontmatter open ---\n, optional BOM + CRLF tolerance, scans first 2000 chars for dream_generated: true with case-insensitive value and word boundary on true) drives isDreamOutput(content, bypass=false). Both functions skip matching files and emit a [dream] skipped <basename>: dream_generated marker stderr log (no silent skips). bypassGuard?: boolean on DiscoverOpts and readSingleTranscript's opts disables the guard for the explicit --unsafe-bypass-dream-guard escape hatch only — never auto-applied for --input.

  • src/commands/dream.tsgbrain dream CLI; thin alias over runCycle. Flags: --dry-run, --json, --phase <name>, --pull, --dir <path>, --input <file> (ad-hoc transcript, implies --phase synthesize), --date YYYY-MM-DD, --from <d> --to <d> (backfill range), --unsafe-bypass-dream-guard (plumbed through runCycle.synthBypassDreamGuardSynthesizePhaseOpts.bypassDreamGuarddiscoverTranscripts({bypassGuard}) / readSingleTranscript({bypassGuard}); loud stderr warning at synthesize-phase entry; never auto-applied for --input). Conflict detection: --input + --date exits 2. ISO date validation. --dry-run runs the scored triage pass but skips synthesis (NOT zero LLM calls). Exit 1 on status=failed. Source scope enters the shared 6-tier resolver whenever --source <id> is given OR GBRAIN_SOURCE is set (a null explicit falls through to tier 2: validated + assertSourceExists, so an invalid/unknown env value is a clean exit 1; the __all__ sentinel is excluded and falls through to the unscoped routing, which already spans every source); an env scope naming the brain's default-like source keeps the full implicit cycle (#4700 semantics), any other keeps the --source freshness boundary. resolveBrainDir returns string | null (order: --dir → resolved source's local_path → global sync.repo_path → null); a checkout-less postgres/Supabase brain runs DB-only phases (incl. resolve_symbol_edges) and skips the 6 filesystem phases with details.reason: 'no_brain_dir'; runDream owns the only hard error (no checkout AND no engine). When --source resolves but has no on-disk checkout, returns null (DB-only) rather than borrowing another source's global sync.repo_path (would mix scopes). Pinned by test/dream-postgres.serial.test.ts. --drain [--window <seconds>] for --phase extract_atoms: runDrain() bypasses the pack-gate and runs the single-hold bounded drain from src/core/cycle/extract-atoms-drain.ts under the same cycleLockIdFor(sourceId) the routine cycle uses (concurrent autopilot tick defers with cycle_already_running), reporting {extracted, skipped, remaining}. Exits EXIT_DRAIN_INCOMPLETE=3 while remaining > 0; a null backlog count (count query FAILED) is also exit 3, never a drained success; LockUnavailableErrorcycle_already_running skip (also exit 3). The extract_atoms_backlog doctor check (computeExtractAtomsBacklogCheck) surfaces the silent pack-gated backlog with the exact --drain command; pack-gated cycle skips carry a greppable pack_gated:true marker. dream retriage dispatches on args[0] === 'retriage' BEFORE parseArgs (its flag set never collides with cycle flags; dream retriage --help prints subcommand help engine-free per the same IRON RULE).

  • src/commands/dream-retriage.tsgbrain dream retriage: re-scores the corpus via the shared runTriagePass (with maxMs: 0 — operator sweeps run to completion; --limit slices the discovered list caller-side) and reconciles the queued private-queue backlog: synth-v2 rows verdict-gated, plus any row stranded in a provably-dead dream-inline-* queue regardless of key family (patterns children, legacy grammars) — doctor's orphaned_private_queue check points here, so the repair selects everything the check can flag. Liveness uses ownership correlation (mirrors the doctor check): a live gbrain_cycle_locks row suppresses conversion only for queues born at/after its acquired_at; an older crashed cycle's queue stays repairable while a new cycle runs (unknown birth/acquisition stays fail-safe possibly-live). Spend-gated: upfront estimate via canonicalLookup on the resolved triage model, confirmation above SPEND_CONFIRM_USD ($5, in dream-retriage-constants.ts; unpriced models gate on UNPRICED_CONFIRM_FILES=500) unless --yes (--json non-interactive requires --yes above the gate); --max-usd soft-stops via the pass's shouldStop seam (estimate-based). --reconcile-queue (opt-in — cancels queued work): selects waiting/delayed/paused dream:synth-v2:% jobs across ALL queues, parses keys with parseSynthV2Key, then per row, reading THE shared gate (passesTriageGate from triage-rescue.ts — threshold pass OR the verified-segment rescue; a hand-rolled score >= threshold here would cancel exactly the jobs the rescue admitted): matched below the gate → cancel; matched above the gate but waiting in a stale dream-inline-* queue → cancel as converted_for_resubmit (the backlog conversion — cancelled rows release their idempotency slot so the next cycle re-adds into a live drain); matched above the gate in a live queue → keep; matched-but-unscored → keep (never cancel on missing data); unmatched → keep unless --cancel-unmatched; key-source vs data.source_id disagreement → skip + source_mismatch; status re-checked immediately before each cancel (rows turned active are skipped; residual race matches cancelJob's best-effort contract). Legacy dream:synth: keys are excluded at the SQL LIKE filter — never candidates. --source scopes cancels (other_source counted); --threshold/--since/--force/--dry-run (zero judge calls, zero cancels — cached scores only, uncached files report needs_triage). --audit-rejects <n> re-judges N stride-sampled files that the gate actually REJECTED (below threshold AND not rescued — auditing a rescued file as a rejection would misreport the disagreement rate) with the SYNTHESIS model and reports that rate — the operator calibration loop. Reports carry rescued/verified_segments so a rescued file reads as accepted everywhere the command renders. Exit 0 success (even when nothing cancelled), 1 no engine/corpus, 2 usage or declined spend gate. Pinned by test/dream-retriage.test.ts.

  • src/commands/friction.ts + src/core/friction.tsgbrain friction {log,render,list,summary,diff} reporter. Append-only JSONL under $GBRAIN_HOME/.gbrain/friction/<run-id>.jsonl. Schema is a flat extension of StructuredAgentError; every claw-test run opens with a phase-marker/start meta record carrying agent + scenario + harness_schema (agent-name resolution depends on it). Render groups by severity → phase, defaults to --redact for md output (strips $HOME/$CWD to placeholders so reports paste safely in PRs). diff --base <run-or-agent> --compare <run-or-agent> is the cross-agent instrument: exact run-id wins, else agent name resolves to that agent's latest run; identity is (kind, phase, normalized 80-char message prefix — digit runs collapsed so durations/counts don't split identities) over kind ∈ {friction, delight} as MULTISETS (per-severity counts + totals are the compared attributes: count_changed = volume, severity_changed = distribution shape via exact integer proportion test, so a delight→friction flip or a 2×error+1×nit → 1×error+2×nit redistribution always surfaces; markers/interrupted feed the compatibility banner, which warns on scenario/version mismatch); output labels sections "unique to " — an instrument, never a blame-attributor. Run-id resolves from --run-id > $GBRAIN_FRICTION_RUN_ID > standalone.jsonl. Skills the claw-test exercises carry a _friction-protocol.md callout so agents know when to log friction.

  • src/commands/claw-test.ts + src/core/claw-test/gbrain claw-test [--scenario <name>] [--live --agent <name>]. End-to-end "fresh user" friction harness. Two modes: scripted (CI gate, agent-free) and live (real agent subprocess, $1–2 in tokens). Sets GBRAIN_HOME=<tempdir> for hermeticity and captures gbrain's --progress-json events from each child's stderr to verify expected phases ran (import.files, extract.links_fs, doctor.db_checks). Scripted phases: setup → install_brain (gbrain init --pglite) → import (--no-embed) → query → extract → verify (gbrain doctor --json; top-level status is healthy|warnings|unhealthy) → render. Live mode STAGES the scenario before the agent turn (fresh-install: brain pages + AGENTS.md stub + init; upgrade: seed-first via seed-pglite.ts, NO init — the migration is the scenario under test), prepends a per-run gbrain PATH shim so the BRIEF's bare gbrain runs this checkout, hands BRIEF.md to the agent runner, then verifies a scenario-declared success ORACLE (oracle: {query, min_results, files_exist} in scenario.json; upgrade uses a non-mutating schema-version probe via readPgliteSchemaVersion — doctor would auto-migrate and pass a do-nothing agent). Child-side friction merges into the parent's friction file before tempdir cleanup. Four runners ship (src/core/claw-test/runners/{openclaw,hermes,grok,opencode}.ts; shared detectBinary/filterAllowlistEnv live in agent-runner.ts): openclaw invokes openclaw agent --local --agent <name> --message <brief>; hermes invokes hermes -z <brief> ($HERMES_BIN > which hermes; HERMES_HOME passthrough is the env-allowlist delta; shared BASE_ENV_ALLOWLIST + validateBinPathEnv live in agent-runner.ts); grok (xAI Grok Build, observed shapes in docs/mcp/GROK-CLI-PIN.md) invokes grok -p <brief> --output-format plain ($GROK_BIN > which grok; delta GROK_HOME + XAI_API_KEY; writes a version preamble into the transcript and warns loudly when the operator's ~/.claude.json registers gbrain — grok reads vendor MCP configs for trusted folders); opencode (SST, observed shapes in docs/mcp/OPENCODE-CLI-PIN.md) invokes opencode run <brief> --format default ($OPENCODE_BIN > which opencode; delta = the EXPLICIT multi-provider keys XAI/Google/Gemini/OpenRouter — BASE carries only Anthropic+OpenAI — plus XDG dirs, OPENCODE_CONFIG(_DIR), and OPENCODE_DISABLE_AUTOUPDATE; OPENCODE_CONFIG_CONTENT deliberately absent, it is a config-shadow channel; no --auto — MCP tools fire without it; bare-semver version preamble is the SST-vs-claimant discriminator; warns loudly when the user-global opencode config carries mcp.gbrain). Live-lane posture: the OPERATOR's configured agent + hermetic brain; the fully hermetic lanes are test/e2e/install-real-hermes.serial.test.ts, test/e2e/install-real-grok.serial.test.ts, and test/e2e/install-real-opencode.serial.test.ts (split-gated a step past grok: opencode's anonymous free tier drives MCP tool calls keyless, so even the nonce SMOKE — with a STRUCTURAL gbrain_* tool_use assert via parseOpencodeJsonl — runs in the keyless tier; the paid anthropic leg self-validates its pinned model id against the authed opencode models list before any spend) (grok door is split-gated: keyless compat tier needs only the binary — mcp doctor is grok's honest discriminator, proving the seven-verb surface keyless; paid SMOKE additionally needs XAI_API_KEY and asserts a per-run nonce fact, never the committed one). Transcript capture (transcript-capture.ts) uses fs.createWriteStream with 'drain'-event backpressure (so a 256KB burst cannot stall the child). Env knobs (harness escape hatches, all optional): GBRAIN_BIN_OVERRIDE (child gbrain binary; validated absolute/no-dotdot/no-metacharacter because it's interpolated into the PATH shim — under the bun runtime the harness otherwise synthesizes a launcher so children never exec bun itself), GBRAIN_CLAW_PHASE_TIMEOUT_MS (per-phase child wall clock, default 5 min), GBRAIN_CLAW_AGENT_TIMEOUT_MS (live agent turn wall clock, default 10 min).

  • skills/_friction-protocol.md — shared cross-cutting convention skill (like _brain-filing-rules.md). Tells agents when to call gbrain friction log and how to choose a severity. Routes to friction CLI from any skill the claw-test exercises.

  • scripts/check-progress-to-stdout.sh — CI guard against regressing to \r-on-stdout progress. Wired into bun run test via scripts/check-progress-to-stdout.sh && bun test in package.json.

  • docs/progress-events.md — Canonical JSON event schema reference. Additive only.

  • src/core/markdown.ts — Frontmatter parsing + body splitter. coerceFrontmatterString(v) coerces a non-string title/slug/type to a deterministic string at parse time so a YAML-typed value never reaches .toLowerCase() and throws (title: 2024-06-01 parses as a Date, title: 1458 as a number, and a throw would block the sync bookmark from advancing); a Date becomes its UTC ISO date (2024-06-01, machine-independent and matching the on-disk token, unlike String(date)), null/undefined become '', everything else uses String(). splitBody requires an explicit timeline sentinel (<!-- timeline -->, --- timeline ---, or --- immediately before ## Timeline/## History). Plain --- in body text is a markdown horizontal rule, not a separator. inferType auto-types /wiki/analysis/ → analysis, /wiki/guides/ → guide, /wiki/hardware/ → hardware, /wiki/architecture/ → architecture, /writing/ → writing (plus existing people/companies/deals/etc heuristics). resolveSourceLocalFilePath maps Git-root-relative pages.source_path values into a source whose local_path scopes a repo subdirectory; it finds the Git root with filesystem checks only, strips the exact root-relative scope, rejects unsafe/non-markdown paths, and when given a slug it tolerates historical basename-relative source_path rows by checking the slug directory only after the direct path is absent. It leaves write containment to the caller.

  • scripts/check-jsonb-pattern.sh — CI grep guard. Fails the build if any source contains (a) the ${JSON.stringify(x)}::jsonb interpolation pattern (postgres.js v3 double-encodes it), or (b) max_stalled INTEGER NOT NULL DEFAULT 1 in any schema source file (must be DEFAULT 5 to preserve SIGKILL-rescue). It also invokes scripts/check-jsonb-params.mjs and propagates its exit code. Wired into bun test.

  • scripts/check-getpage-scoped-write.mjs — CI scanner for the unscoped-check/scoped-write source-isolation bug class: flags any non-test src file containing BOTH a getPage( call with no second argument (or the X ? {sourceId} : undefined any-source-when-unset ternary) AND a write-path call (putPage(/importFromContent(/importFromFile(). Fix pattern: getPage(slug, { sourceId: x ?? 'default' }) (mirror the write's schema default); opt-out marker gbrain-allow-unscoped-getpage: <reason> for documented read-only first-match sites (span, preceding lines, or trailing same-line comment). Grandfathered allowlist is EMPTY. Comment/string-aware balanced-paren span walker (same skeleton as check-jsonb-params.mjs); argv-overridable roots; wired as check:getpage-scope in verify CHECKS + guards-manifest + test/fixtures/guards/check-getpage-scoped-write.mjs/{bad,good}/ + test/check-getpage-scoped-write.test.ts.

  • scripts/check-jsonb-params.mjs — AST-lite CI guard for the POSITIONAL jsonb double-encode form the template grep above misses: an executeRaw/executeRawDirect/.unsafe() call whose balanced arg span binds JSON.stringify(x) into a bare $N::jsonb cast. Walks each call's balanced span respecting strings/templates/comments, handles generic-typed calls (executeRaw<T>(), and allows the sanctioned forms ($N::text::jsonb, $N::text[], executeRawJsonb, sql.json, an inline jsonb-guard-ok comment). PGLite's native db.query is deliberately not scanned (it parses text→jsonb, so the bug can't occur there). Heuristic by design (whole-span correlation; can't see a JSON.stringify assigned to a variable before the call) — the real backstop is the DATABASE_URL-gated e2e parity tests. Scan roots overridable via argv for its self-test (test/check-jsonb-params.test.ts).

  • scripts/check-source-id-projection.sh — CI grep guard for the multi-source bug class. Greps src/core/postgres-engine.ts + src/core/pglite-engine.ts for SELECT.*FROM pages projections matching the rowToPage feeder shape (id + slug + type + title) and fails if source_id is missing. Page.source_id is required at the type level; a projection dropping the column produces Page rows with source_id: undefined while TypeScript's : string lies about it. Wired into bun run verify.

  • scripts/guards-manifest.tsv + scripts/guard-self-test.sh — THE single registry of scripts/check-* CI guards (52 guards) and its self-test harness. Every guard is classified scanner (greps/parses repo sources — must eventually carry fixtures), buildfresh, or repostate (exempt-with-reason, not fixture-tested). guard-self-test.sh (bun run check:guard-self-test, wired into bun run verify) runs each selftest=yes scanner against known-bad (must exit non-zero) and known-good (must pass) fixture trees under test/fixtures/guards/<guard>/{bad,good}/ via the GBRAIN_GUARD_ROOT env seam, and fails the build when a new scripts/check-* script is missing from the manifest — so a guard whose pattern rots into a permanently-green no-op fails CI instead of masquerading as coverage. The manifest registers and classifies guards but does not itself schedule them — run-verify-parallel.sh's CHECKS array remains the execution list, and a registered guard is not automatically wired into verify. New guard = new manifest row (+ fixtures if scanner) + a CHECKS entry if it should gate pushes.

  • scripts/merge-lcov.ts + scripts/coverage-diff-gate.ts + scripts/coverage-baseline-gate.ts + scripts/update-coverage-baseline.ts + scripts/render-coverage-summary.ts + scripts/coverage-gate-exemptions.txt + scripts/coverage-baseline.json — the coverage measurement + gating cluster; the operating guide is docs/TESTING.md "Coverage lanes and gates". merge-lcov.ts walks artifact dirs for lcov.info + lane-manifest.json, sums DA hits per file:line, normalizes paths repo-relative, and emits a merged lcov + summary JSON (src-only totals/per-dir/per-file, the lineHits extension the diff gate consumes, and never-loaded src files as count + sorted list — deliberately never a percentage, since physical lines ≠ executable lines); --manifest-expect pins the lane set, and a missing/incomplete lane or a shard lane with lcovCount != 1 (the xargs-batching tripwire) marks the summary degraded: true — still exit 0 (degraded is data, and both gates go report-only on it). coverage-diff-gate.ts gates added/changed gate-scoped lines (non-test, non-generated src/**.ts) at ≥80% covered plus zero changed-but-never-loaded files; report-only unless COVERAGE_GATE_ENFORCE=1; a [coverage-exempt: reason] commit trailer passes with a loud warning; coverage-gate-exemptions.txt rows (exact path or trailing-/ prefix; SHRINK-ONLY — additions need a graduation review) are excluded from the gate but still reported ([e2e-exempt] / [subprocess-undercount]); exit contract: 0 = pass or report-only, 1 = fail while enforcing, 2 = infrastructure error (never conflated with a coverage verdict). coverage-baseline-gate.ts reads the baseline via git show origin/master:scripts/coverage-baseline.json (never the working tree, so a PR can't weaken its own bar) and compares corpus-matched sections only (--corpus prCorpus|fullCorpus), failing on >0.5pp global or >1.0pp per-dir drops; provisional: true in the baseline (the current state — both corpus sections unseeded) keeps it report-only regardless of enforcement; update-coverage-baseline.ts writes the working-tree baseline (per-file detail limited to the committed watchlist) and --promote flips provisional: false. render-coverage-summary.ts renders the summary JSON as markdown on stdout for $GITHUB_STEP_SUMMARY, including the behavioral-vs-structural counts from scripts/structural-suites.tsv. Wiring: 14 PR-corpus lanes in test.yml (10 matrix shards + serial + the three dedicated slow jobs) upload coverage-* artifacts and the advisory coverage-report job merges + renders + runs both gates report-only (deliberately absent from test-status/cache-write until graduation); schedule-only coverage-full-{unit,serial,slow,e2e} + coverage-full-report in e2e.yml produce the self-contained nightly fullCorpus number (full e2e glob included) and the coverage-full-merged trend artifact. Collection is COVERAGE_DIR-opt-in in test-shard.sh/run-serial-tests.sh/run-e2e.sh — unique coverage dir per bun process (a reused dir overwrites lcov.info), lane manifest written only on a green run, run-e2e.sh requires an ABSOLUTE COVERAGE_DIR and honors E2E_FILE_TIMEOUT_SECS (both deliberately non-GBRAIN_-prefixed to survive the hermetic env scrub). Bun/JSC emits line records only (function coverage is informational) and no subprocess coverage, so src/cli.ts undercounts. Pinned by test/scripts/merge-lcov.test.ts, test/scripts/coverage-diff-gate.test.ts, test/scripts/render-coverage-summary.test.ts.

  • scripts/check-module-size.sh + scripts/module-size-limits.tsv — the module-size ratchet (bun run check:module-size, wired into bun run verify). The TSV commits a per-file wc -l ceiling (path max_lines policy note); four rules, all violations reported before a single exit 1: a file above its ceiling fails (raise a ceiling only as a conscious TSV edit); a ceiling more than 50 lines above the measured size fails (stale slack after a shrink — lower it so the ratchet holds); a TSV row whose path no longer exists fails (remove the row); an unlisted src/**/*.ts (excluding *.generated.ts/*.test.ts) above the 1500-line new-file cap fails (split it or add a row). Policy region-exempt (only src/core/migrate.ts) counts lines OUTSIDE the append-only export const MIGRATIONS = []; region, so the migrations array grows freely while the surrounding runner logic stays ratcheted. Self-test seams: GBRAIN_GUARD_ROOT, GBRAIN_MODULE_SIZE_SLACK, GBRAIN_MODULE_SIZE_NEWFILE_CAP.

  • scripts/classify-tests.ts + scripts/structural-suites.tsv — suite-level behavioral-vs-structural test classification (the intent axis described in docs/TESTING.md "File taxonomy"). Content-based detectors — repo-anchored readFileSync/Bun.file readers, exec-scan grep windows over src|scripts|docs, and the doctorSource()/doctorFileSource() helpers — mark a suite STRUCTURAL when its assertions read repo source/doc text rather than executing product code; tmpdir-anchored reads don't count, and files with detectors but no attributable suite land in an unknown bucket emitted as comment rows (surfaced, never silently dropped). Modes: bare = rewrite the TSV; --check = byte-for-byte regenerate-and-diff freshness (wired as bun run check:structural-manifest in bun run verify via scripts/check-structural-manifest.sh); --summary = counts only. Fix misclassifications in the detector list, never by hand-editing the TSV. render-coverage-summary.ts consumes the TSV for the behavioral-vs-structural line in the CI coverage report.

  • scripts/build-pglite-snapshot.tsbun run build:pglite-snapshot: bakes a post-initSchema() PGLite data dir into test/fixtures/pglite-snapshot.tar + a version file (schema hash line, then dims=/model= lines recording the embedding shape it was baked with). Idempotent (hash short-circuit ~40ms when fresh; rebuilds stale) and concurrency-safe (atomic mkdir lock at test/fixtures/.pglite-snapshot.lock with staleness-verified takeover — a live lock is never stolen; tar written first, version file last, so a crash can't leave a fresh-looking torn fixture; waiter bounded by GBRAIN_SNAPSHOT_LOCK_TIMEOUT_MS, default 120000; an exhausted waiter facing a still-live lock proceeds unlocked as a last resort — the loader's hash/shape gate validates the version file, not the tar bytes). Called through the shared ensure_pglite_snapshot helper in scripts/lib/test-env.sh (also home of detect_cpus + detect_available_mem_mb; sourced by run-unit-parallel.sh, test-shard.sh, run-slow-tests.sh, run-serial-tests.sh, run-verify-parallel.sh, and run-e2e.sh — default-on, opt out GBRAIN_NO_SNAPSHOT=1, no-op when a parent already exported the path, non-fatal on build failure with a one-line "active" echo so a silent cold-init fallback stays visible) and directly by scripts/ci-local.sh; every caller exports GBRAIN_PGLITE_SNAPSHOT. The loader side is tryLoadSnapshot + computeSnapshotSchemaHash (exported from src/core/pglite-engine.ts): the coverage-immune hash reads raw file bytes for the schema/migration entry modules and imported helpers, including grant SQL/policy/repair dependencies and withdrawal triggers (keep its dependency list and CI cache keys aligned when adding a helper); any hash or embedding-shape mismatch warns once and falls through to normal cold init — the snapshot is an optimization, never authoritative. The loader memoizes per process: the schema hash computes once (source bytes are static for the process lifetime) and the version file + ~42MB tar are read once per (path, process) instead of once per engine construction (a full suite constructs 600+ engines); a terminally-unusable path (missing/stale/torn) memoizes as null and is never retried, the tar blob loads lazily only after the FIRST caller passes the shape gate, and the dims/model shape gate itself is deliberately NOT memoized (tests reconfigure the gateway mid-process — a mismatched engine must still fall back to cold init). Accepted limitation: a snapshot rewritten mid-process is not observed; the only writer runs before test fan-out. Test seams __snapshotMemoStatsForTests/__resetSnapshotMemoForTests. Pinned by test/snapshot-shape-guard.test.ts.

  • docker-compose.ci.yml + scripts/ci-local.sh — Local CI gate. bun run ci:local spins up four pgvector/pgvector:pg16 services (postgres-1..4) + oven/bun:1 with named volumes (gbrain-ci-pg-data-{1..4}, gbrain-ci-node-modules, gbrain-ci-bun-cache), runs gitleaks on host, smoke-tests scripts/run-e2e.sh argv handling, runs guards + typecheck, then the Tier 1 default: 4-shard parallel unit + E2E (xargs -P4, one Postgres per shard; unit phase keeps DATABASE_URL unset). --no-shard falls back to the legacy unsharded sequential flow (debug aid); --diff runs the diff-aware selector unsharded. Also runs a pgbouncer service (edoburu/pgbouncer, POOL_MODE: transaction, AUTH_TYPE: plain — pg16 stores SCRAM verifiers, so the userlist must hold the plaintext password; IGNORE_STARTUP_PARAMETERS whitelists gbrain's statement_timeout/idle_in_transaction_session_timeout startup params the way the Supabase pooler does) fronting postgres-1 on host port GBRAIN_CI_PGBOUNCER_PORT (default 6543); every E2E invocation exports GBRAIN_PGBOUNCER_URL (pooled; dedicated gbrain_pgbouncer database so it never races the gbrain_test TRUNCATE fixtures) + GBRAIN_PGBOUNCER_DIRECT_URL, consumed by test/e2e/pgbouncer-teardown.test.ts — which reproduces the transaction-mode teardown failure in the local gate. --no-pull skips upstream pulls; --clean nukes named volumes. Postgres host port defaults to 5434; override with GBRAIN_CI_PG_PORT=NNNN. Stronger gate than PR CI's 2-file Tier 1 set.

  • scripts/select-e2e.ts + scripts/e2e-test-map.ts — Diff-aware E2E test selector. Reads three git sources (committed origin/master...HEAD, working-tree HEAD, and git ls-files --others --exclude-standard for untracked NOT-gitignored files), classifies as EMPTY / DOC_ONLY / SRC. Fail-closed: EMPTY → all files; DOC_ONLY (every path matches the README/CLAUDE/AGENTS/CHANGELOG/TODOS allowlist) → empty stdout; SRC → escape-hatch paths (schema, package.json, skills/) trigger all, else the hand-tuned E2E_TEST_MAP glob narrows, and an unmapped src/ change still emits ALL files (never silently nothing). Pure-function exports selectTests, classify, matchGlob. bun run ci:select-e2e prints the current selection on stdout. test/select-e2e.test.ts covers all 4 branches plus 3 guards (skills/, untracked files, unmapped src/) — 24 cases.

  • scripts/run-e2e.sh — Sequential E2E runner. Accepts an optional argv-driven file list (used by ci:local:diff) and a --dry-run-list flag that prints the resolved file list and exits (used by ci-local.sh's startup smoke-test). Falls back to test/e2e/*.test.ts plus test/phantom-redirect-engine-parity.test.ts when invoked with no args (the phantom-redirect Postgres arm is only reachable through a DATABASE_URL-bearing lane; the unit wrappers strip the URL, so this lane must carry it). This wrapper is the database-URL opt-in boundary: it exports GBRAIN_TEST_ALLOW_DATABASE_URL=1 so the bunfig preload guard (test/helpers/database-url-guard-preload.ts) lets the run start, unsets GBRAIN_DATABASE_URL (the e2e suite runs on DATABASE_URL only — an ambient GBRAIN_DATABASE_URL would pass the opt-in yet reach CLI-subprocess paths with no name floor), and its GBRAIN_* env scrub preserves GBRAIN_E2E_ALLOW_DB so the name-floor escape hatch the guard's own error message names stays usable. It also preserves GBRAIN_CI_DISABLE_TEST_ENV_FILE=1, keeping CI runs from loading checkout-local .env.testing credentials after the shell-to-Bun handoff. It also exports GBRAIN_TEST_KEEP_PROVIDER_KEYS=1 so the unit-lane provider-key strip preload (test/helpers/provider-keys-preload.ts) leaves the real keys that live embed/parity e2e tests skip-gate on. Each file runs under a gtimeout/timeout wedge backstop (default signal: SIGTERM — the bun test child installs no JS-level handler for it, so kernel-default termination applies) — 180s default, with a per-file override for known-slow files (skills.test.ts gets 420s: the real ingest-skill run replays every migration, so its floor grows as master adds migrations); the cap is a wedge backstop, not a per-test budget, and bare bun (no outer cap) is the fallback when neither timeout binary is installed. It activates the PGLite schema snapshot like the other runners: sources scripts/lib/test-env.sh + ensure_pglite_snapshot after the --dry-run-list early exit (list mode stays instant; non-fatal on build failure), re-exports GBRAIN_PGLITE_SNAPSHOT as an ABSOLUTE path (e2e tests spawn CLI subprocesses with varying cwd — a relative path silently misses the tar there), and its env keep-list preserves the var; e2e files that assert the path TO post-initSchema() state carry per-file delete process.env.GBRAIN_PGLITE_SNAPSHOT opt-outs with one-line reasons.

  • scripts/llms-config.ts + scripts/build-llms.ts — Generator for llms.txt (llmstxt.org-spec web index) + llms-full.txt (inlined single-fetch bundle). Curated config drives both. Run bun run build:llms after adding a new doc. LLMS_REPO_BASE env lets forks regenerate with their own URL base. FULL_SIZE_BUDGET (600KB) caps the inline bundle; generator WARNs if exceeded. Committed output has no runtime consumer; committed for GitHub browsing and fork-safe fetching.

  • AGENTS.md — Local-clone entry point for non-Claude agents (Codex, Cursor, OpenClaw, Aider). Mirrors CLAUDE.md intent via relative links. Claude Code keeps using CLAUDE.md.

  • docs/UPGRADING_DOWNSTREAM_AGENTS.md — Patches for downstream agent skill forks to apply when upgrading. Each release appends a new section; includes diffs for brain-ops, meeting-ingestion, signal-detector, enrich.

  • src/core/schema-embedded.generated.ts — AUTO-GENERATED from schema.sql (run bun run build:schema; the .generated.ts suffix exempts it from the module-size ratchet's new-file cap, per the guard's generated-file carve-out)

  • src/schema.sql — Full Postgres + pgvector DDL (source of truth, generates schema-embedded.generated.ts)

  • src/core/search/expansion.ts — Multi-query expansion via Haiku. Exports sanitizeQueryForPrompt + sanitizeExpansionOutput (prompt-injection defense-in-depth). Sanitized query is only used for the LLM channel; the original query still drives search. expandQuery returns the original first plus 2-3 variants; hybridSearch re-enforces queries[0] = the caller's query and dedupes repeats, then fuses each variant's vector list as a role variant arm through src/core/search/fusion-lists.ts (see that entry), where the variant arms share search.expansion_variant_budget as total RRF weight — null (every bundle's default) is legacy equal weight, the configuration under which the LongMemEval receipt shows expansion halving strict recall_all@5 (93.19% plain hybrid vs 54.89% with expansion, paired +3 / -183; docs/architecture/RETRIEVAL.md "Multi-query expansion"). The harness records expansion_variants per row and --expansion-replay serves them back so a budget sweep changes only the budget.

  • recipes/ — Integration recipe files (YAML frontmatter + markdown setup instructions)

  • docs/guides/ — Individual SKILLPACK guides (broken out from monolith)

  • docs/integrations/ — "Getting Data In" guides and integration docs

  • docs/architecture/infra-layer.md — Shared infrastructure documentation

  • docs/ethos/THIN_HARNESS_FAT_SKILLS.md — Architecture philosophy essay

  • docs/ethos/MARKDOWN_SKILLS_AS_RECIPES.md — "Homebrew for Personal AI" essay

  • docs/guides/repo-architecture.md — Two-repo pattern (agent vs brain)

  • docs/guides/sub-agent-routing.md — Model routing table for sub-agents

  • docs/guides/skill-development.md — 5-step skill development cycle + MECE

  • docs/guides/idea-capture.md — Originality distribution, depth test, cross-linking

  • docs/guides/quiet-hours.md — Notification hold + timezone-aware delivery

  • docs/guides/diligence-ingestion.md — Data room to brain pages pipeline

  • docs/designs/HOMEBREW_FOR_PERSONAL_AI.md — 10-star vision for integration system

  • docs/mcp/ — Per-client setup guides (Claude Desktop, Code, Cowork, Perplexity)

  • BrainBench retrieval benchmark (P@5/R@5 corpus + harness): lives in the separate gbrain-evals repo. Not installed alongside gbrain. Distinct from the in-repo cross-harness memory conformance suite (gbrain eval brainbenchsrc/eval/brainbench/, corpus at evals/brainbench/, methodology in docs/eval/BRAINBENCH.md).

  • skills/_brain-filing-rules.md — Cross-cutting brain filing rules (referenced by all brain-writing skills)

  • skills/RESOLVER.md — Skill routing table (based on the agent-fork AGENTS.md pattern) with skills/manifest.json: schema-author wired into the dispatcher with the full functional-area trigger list (compressed routing pattern per the dispatcher convention).

  • skills/conventions/ — Cross-cutting rules (quality, brain-first, model-routing, test-before-bulk, cross-modal)

  • skills/_output-rules.md — Output quality standards (deterministic links, no slop, exact phrasing)

  • skills/signal-detector/SKILL.md — Always-on idea+entity capture on every message

  • skills/brain-ops/SKILL.md — Brain-first lookup, read-enrich-write loop, source attribution

  • skills/idea-ingest/SKILL.md — Links/articles/tweets with author people page mandatory

  • skills/media-ingest/SKILL.md — Video/audio/PDF/book with entity extraction

  • skills/meeting-ingestion/SKILL.md — Transcripts with attendee enrichment chaining

  • skills/citation-fixer/SKILL.md — Citation format auditing and fixing

  • skills/repo-architecture/SKILL.md — Filing rules by primary subject

  • skills/skill-creator/SKILL.md — Create conforming skills with MECE check

  • skills/daily-task-manager/SKILL.md — Task lifecycle with priority levels

  • skills/daily-task-prep/SKILL.md — Morning prep with calendar context

  • skills/cross-modal-review/SKILL.md — Quality gate via second model

  • skills/cron-scheduler/SKILL.md — Schedule staggering, quiet hours, idempotency

  • skills/reports/SKILL.md — Timestamped reports with keyword routing

  • skills/testing/SKILL.md — Skill validation framework

  • skills/soul-audit/SKILL.md — 6-phase interview for SOUL.md, USER.md, ACCESS_POLICY.md, HEARTBEAT.md

  • skills/webhook-transforms/SKILL.md — External events to brain signals

  • skills/data-research/SKILL.md — Structured data research: email-to-tracker pipeline with parameterized YAML recipes

  • skills/minion-orchestrator/SKILL.md — Unified background-work skill. Two lanes: shell jobs via gbrain jobs submit shell --params '{"cmd":"..."}' (operator/CLI only; MCP throws permission_denied for protected names) and LLM subagents via gbrain agent run (user-facing entrypoint). Shared Preconditions block, parent-child DAGs with depth/cap/timeouts, child_done inbox for fan-in, PGLite --follow inline path for dev. Triggers narrowed to "gbrain jobs submit" + "submit a gbrain job" so stats/prune/retry questions fall through to gbrain --help.

  • templates/ — SOUL.md, USER.md, ACCESS_POLICY.md, HEARTBEAT.md templates

  • skills/migrations/ — Version migration files with feature_pitch YAML frontmatter

  • src/commands/publish.ts — Deterministic brain page publisher (code+skill pair, zero LLM calls)

  • src/commands/backlinks.ts — Back-link checker and fixer (enforces Iron Law). Repair-inserted references land as UNDATED rows (- Referenced in [Title](path)) in a dedicated ## Referenced by section placed ABOVE the timeline region — retroactively stamping the repair date onto a timeline would forge events that never happened on that date; only live capture writes dated timeline entries. insertBacklinkEntry(content, bodyStart, entry) never touches bytes before bodyStart, locates the timeline boundary via findTimelineSplitIndex (existing timeline sentinels take precedence over bare ## Timeline/## History headings), appends to an existing ## Referenced by section or creates one above the timeline, and is CRLF-tolerant; insertTimelineEntry is an alias for existing import sites. buildBacklinkEntry(sourceTitle, sourcePath) takes no date; dir-shaped sources get an extension-less brain-slug link so the next check pass credits the fresh row instead of re-flagging (idempotency).

  • src/commands/lint.ts — Page quality linter (catches LLM artifacts, placeholder dates). runLintCore(opts) walks the tree ONCE; --fix applies fixes during that same scan and reports per-page results through the onPageIssues(relPath, issues, fixedCount) callback (issues = what remains after this run's fix attempt for the page; fixedCount = fixes actually applied), so the summary's auto-fixed count is the true count — a second scan run after fixing would see already-fixed files and report zero. onScanStart fires once with the collected page count for progress wiring. Pinned by test/lint-fix-single-pass.test.ts. On a durability-hardened brain (isDurabilityHardened on the target dir, or the page's directory for a single-file target) every non-dry-run --fix repair is committed path-limited via commitWriteThroughFile right after the writeFileSync — the same contract as put_page write-through, so the cycle, the lint/lint-fix minion handlers and the CLI never leave hardened repairs as uncommitted drift for the sync phase to flag (pinned by test/cycle-lint-durability.test.ts). Lint rules huge-page (flags pages exceeding content_sanity.bytes_warn) and scraper-junk (flags pages matching any junk pattern). Both reuse assessContent() from src/core/content-sanity.ts so lint, doctor, and ingest share one assessor. lint.ts lifts DB config when ~/.gbrain/ is reachable; falls back to file/env on CI. Pinned by test/lint-content-sanity.test.ts. with src/commands/sources.ts: gbrain lint has a markup-heavy rule (flags pages whose prose-vs-markup ratio exceeds content_sanity.max_markup_ratio, reusing assessContentSanity so lint/gate/scan share one assessor); pinned by test/lint-content-sanity.test.ts. gbrain sources audit <id> is disposition-aware: its dry-run disk scan reports would-quarantine / would-reject / would-flag counts driven by the effective content_sanity.junk_disposition + markup config, so an operator previews the gate's verdict before sync. The content-sanity-audit JSONL (src/core/audit/content-sanity-audit.ts) records the quarantine/flag dispositions.

  • src/commands/report.ts — Structured report saver (audit trail for maintenance/enrichment)

  • src/core/destructive-guard.ts — three-layer protection against accidental data loss. assessDestructiveImpact(engine, sourceId) counts pages/chunks/embeddings/files/facts for a source (a fact-only source — a revoked agent's workspace, since facts are the primary agent write lane — is data at stake, not empty; pre-facts brains degrade to 0), plus oauthClientCount — OAuth clients whose source_id references it. checkDestructiveConfirmation(impact, opts) is the fail-closed gate (--confirm-destructive required when data is present; --yes alone is rejected). FK-RESTRICT lifecycle: clientsReferencingSource(engine, sourceId) lists ALL physical OAuth-client rows referencing a source via oauth_clients.source_id — the FK is PHYSICAL (ON DELETE RESTRICT ignores deleted_at), so soft-deleted (revoked-but-retained) rows BLOCK a hard delete too and come back tagged deleted; pre-migration brains without deleted_at fall back to untagged referents (42703-retry idiom) and brains without the table have none by construction. formatClientReferentsBlock renders the shared refusal sources remove/sources purge print (naming each client — [revoked, retained] for soft-deleted rows — + the revoke command) so the raw Postgres FK violation never reaches the operator. softDeleteSource / restoreSource / listArchivedSources / purgeExpiredSources drive the source-level archive lifecycle via sources.archived BOOLEAN, archived_at TIMESTAMPTZ, archive_expires_at TIMESTAMPTZ; purgeExpiredSources SKIPS client-referenced sources via a physical NOT EXISTS (soft-deleted clients count) so recurring maintenance keeps sweeping the rest instead of aborting. Page-level analog: BrainEngine.softDeletePage / restorePage / purgeDeletedPages plus pages.deleted_at TIMESTAMPTZ and a partial purge index. The MCP delete_page op rewires to softDeletePage; ops restore_page (scope: write) and purge_deleted_pages (scope: admin, localOnly: true) round out the surface. get_page/delete_page/restore_page (src/core/ops/pages.ts) accept an explicit per-call source_id that is honored or rejected loudly, never silently dropped: get_page resolves it through the caller's read grant ('__all__' spans every source for trusted local callers, the granted sources for remote callers), while delete_page/restore_page reject '__all__', target exactly one source, and — for every caller other than trusted local CLI (ctx.remote === false) — accept only the caller's write authority (ctx.auth.sourceId, falling back to ctx.sourceId for legacy tokens and unauthenticated transports; the federated read grant allowedSources confers no delete/restore access), echoing the targeted source_id in the response. Pinned by test/pages-source-scoping-4329.test.ts. Search visibility (buildVisibilityClause in src/core/search/sql-ranking.ts) hides soft-deleted pages and archived sources from searchKeyword / searchKeywordChunks / searchVector in both engines. The autopilot cycle's purge phase calls purgeExpiredSources + engine.purgeDeletedPages(72) so the 72h TTL is real.

  • src/commands/pages.tsgbrain purge-deleted [--older-than HOURS|Nd] [--dry-run] [--json] operator escape hatch. Mirror of gbrain sources purge for the page-level lifecycle. Hard-deletes pages whose deleted_at is older than the cutoff; cascades to content_chunks/page_links/chunk_relations.

  • src/core/op-checkpoint.ts — DB-backed checkpoint primitive for long-running ops. Migration v67 introduces op_checkpoints (op TEXT, fingerprint TEXT, completed_keys JSONB, updated_at TIMESTAMPTZ, PK(op, fingerprint)). Per-op fingerprint helpers (embedFingerprint, extractFingerprint, reindexFingerprint, integrityFingerprint, purgeFingerprint) compute sha8(canonical-JSON(relevant-params)) so re-running with the same params resumes from completed_keys and re-running with different params (e.g. --limit 100 vs --limit 200) starts fresh. Cross-worker safe on Postgres (DB row, no file-lock race); PGLite degrades gracefully. The 7-day TTL GC runs in the cycle's purge phase. All writes (recordCompleted, clearOpCheckpoint) route through engine.executeRawDirect + withRetry(BULK_RETRY_OPTS) so they survive Supavisor pool exhaustion, and recordCompleted returns boolean (banked vs failed-after-retries) — the 9 non-sync consumers keep its REPLACE-into-completed_keys semantics. Resumable sync uses the additive appendCompleted(key, deltaKeys) / appendCompletedOnce (the latter no-retry for the SIGTERM path) which INSERT a delta into the op_checkpoint_paths child table (migration v115: (op, fingerprint, path) PK, FK to op_checkpoints ON DELETE CASCADE) via a single writable-CTE unnest(\$3::text[]) write — O(delta), never an O(N²) full-set rewrite. loadOpCheckpoint returns the UNION ALL of legacy completed_keys + child-table paths (deduped in JS), so an in-flight upgrade loses nothing. The legacy arm is gated on jsonb_typeof(completed_keys) = 'array' so a non-array (scalar) parent row can't make jsonb_array_elements_text throw "cannot extract elements from a scalar" and take down the whole union (which would discard the valid child rows and lose all banked progress for the key); a third union arm flags the corruption so the loader logs it once and keeps the child rows. Migration v119 adds the op_checkpoints_completed_keys_array CHECK (jsonb_typeof(completed_keys) = 'array') — a DB-enforced, always-on guard that makes the scalar-corruption class structurally impossible going forward; the migration repairs any pre-existing scalar to '[]' under LOCK TABLE ... IN SHARE ROW EXCLUSIVE MODE and src/core/schema-embedded.generated.ts + src/core/pglite-schema.ts ship the same CHECK on fresh installs (a loader hit implies schema drift, a disabled constraint, or an out-of-band writer). recordCompleted binds its array through $3::text::jsonb (NOT a bare $3::jsonb) so postgres.js .unsafe() doesn't double-encode JSON.stringify(sorted) into the scalar string that CHECK rejects (PGLite parses it silently, so only real Postgres surfaces the bug). A DATABASE_URL-gated test/e2e/op-checkpoint-jsonb-parity.test.ts (its own CI job) asserts the array shape on real Postgres. syncFingerprint({sourceId, lastCommit}) keys the sync rows. Pinned by test/op-checkpoint.test.ts (incl. delta-append, union read, cascade clear, durable-write boolean, and the scalar-parent guard). import-checkpoint.ts stays a separate file-backed checkpoint — both systems coexist without conflict (unifying them would mean async-propagating the four sync call sites in src/commands/import.ts; deferred).

  • src/core/brain-score-recommendations.ts — pure data layer consumed by both gbrain doctor --remediation-plan / --remediate and gbrain features. computeRecommendations(checks, opts) returns Remediation[] with stable id, content-hash idempotency_key, severity, est_seconds, est_usd_cost, depends_on (references stable ids, not check names — so plan order is reproducible). classifyChecks(report) triages every doctor check three-state into remediable | human_only | blocked (human_only covers RLS warnings and other human-judgment gates; blocked covers dependency chains where a parent check failed). maxReachableScore(checks) computes the ceiling for empty/under-configured brains (no entity pages → graph_coverage caps at 70; no embedding key → embedding_coverage caps at 60). Cost estimates pull from anthropic-pricing.ts (synthesize/patterns/consolidate) and embedding-pricing.ts (embed jobs). Pinned by test/brain-score-recommendations.test.ts (~27 cases incl. determinism, content-hash idempotency, DB-backed checkpoint provenance, three-state triage).

  • src/core/abort-check.ts — one canonical place for cooperative-abort checks across gbrain's long loops. isAborted(signal?) → boolean (for loops that break and return partial progress). throwIfAborted(signal?, label?) throws an AbortError (name === 'AbortError') at phase boundaries, preferring the signal's reason ('wall-clock'/'lock-lost'/'shutdown') so the unwind self-describes. anySignal(internal, external?) composes two signals into one that fires when EITHER does (platform AbortSignal.any with a manual-relay fallback), returning the internal unchanged when there's no external so non-aborting callers pay nothing. Threading these checks through runPhaseEmbed → runEmbedCore → embedAll(Stale)/embedPage lets the embed phase bail and release gbrain_cycle_locks immediately (an embed phase that ignored its abort signal would hold the lock and make later autopilot cycles skip with cycle_already_running). Coverage spans every long cycle-reachable phase: extract (incremental extractForSlugs + the full-walk extractLinksFromDir/extractTimelineFromDir, all via runSlidingPool's signal), extract_facts (per-page loop + the per-page embed signal + runPhantomRedirectPass's 30s lock-retry), consolidate's bucket loop, and lint (which is synchronous, so it awaits a periodic yield to let the signal land). runCycle adds a terminal abort check before stamping last_full_cycle_at so a cancelled cycle never reports a completed full run, plus a per-phase duration_ms warning that names any phase overrunning the worker's 30s force-evict deadline. Pinned by test/abort-check.test.ts + test/cycle-abort.test.ts.

  • openclaw.plugin.json — ClawHub bundle plugin manifest

  • .codex-plugin/plugin.json + .codex-plugin/mcp.json + .agents/plugins/marketplace.json — the Codex plugin lane: manifest (skills → the committed plugin/ tree; mcpServers → the NON-root mcp.json), the MCP declaration (serve --surface starter --source-guard, code-derived env_vars passthrough), and the codex-native marketplace. Version lockstep with package.json + the claude/openclaw manifests pinned by test/codex-plugin-manifest.test.ts.

  • .claude-plugin/plugin.json + .claude-plugin/marketplace.json — the Claude Code plugin lane: inline MCP declaration (command/args/cwd via ${CLAUDE_PLUGIN_ROOT}; no env block — Claude passes the parent env through); the Claude marketplace carries the full plugin PLUS the persona variant entries (gbrain-coding/gbrain-dailyplugin-variants/), while the codex marketplace intentionally stays single-entry until codex's multi-entry handling gets its observation run — variant names are pinned to skills/plugin-lanes.json#personas by test/codex-plugin-manifest.test.ts.

  • .agents/gbrain-launcher — shared plugin MCP launcher (sh, Unix-only) for the Codex, Claude Code, and OpenClaw (openclaw.plugin.json mcpServers.gbrain) lanes: GBRAIN_BIN → ~/.bun/bin/gbrain → PATH resolution, one stderr resolution line, GBRAIN_SURFACE substitute-or-append override for serve argv, actionable exit-127 recovery copy, no auto-install by design. Behavioral branches all pinned in the manifest test.

  • skills/plugin-lanes.json — curation record for the plugin lanes: lane set = (openclaw bundle ∖ base_exclusions) ∪ additions, a reason per entry; starter_gaps is the generated snapshot of per-skill beyond-starter MCP ops (refresh via --write-gaps). The openclaw lane's own curation is untouched — plugin users ARE the brain host (the downstream-vs-host inversion).

  • scripts/generate-plugin-tree.ts + scripts/check-plugin-tree.sh + plugin/ — generator, byte-diff drift gate (in the skills commit gate), and the committed curated skill tree both plugin lanes ship (67 skills + shared conventions/_*.md deps + a generated README carrying the CLI-primary starter note).

  • test/e2e/codex-plugin-install-real.serial.test.ts + test/e2e/claude-plugin-install-real.serial.test.ts — the plugin doors: clean-tree staging (git archive HEAD), real marketplace add/plugin add, snapshot + exec-bit + non-root-mcp.json pins, tools/list starter-surface oracle, cold-home fast-fail, --source-guard block/allow, coexistence + removal probes, auth-gated SMOKE turns. CI: the plugin-doors job in heavy-tests.yml (pinned binary provisioning + expected-pass-count refuse-green).

  • src/commands/capture.ts + src/commands/serve-http.ts + src/core/{operations,import-file,types,utils,facts/absorb-log,brainstorm/{orchestrator,error-classify},scope,postgres-engine,pglite-engine}.ts — ingestion hardening. Capture frontmatter merge via mergeCaptureFrontmatter (uses shared data-only data-frontmatter, preserving metadata beyond parseMarkdown); /ingest null-guard + outer try/catch envelope with !res.headersSent guard; dedup via separate normalize-for-hash (normalizeForHash strips BOM/CRLF/whitespace/NFKC) + body-after-frontmatter-strip on the DB hash (excludes captured_at + ingested_at so capture-cli timestamp variations don't invalidate the chunk cache); friendly pages_source_id_fk rewrite via maybeRewriteSourceFkError on BOTH local + thin-client callRemoteTool catch blocks; facts:absorb 'No database connection' suppression via typed instanceof GBrainError && e.problem check + first-occurrence stack-trace info log (module-scoped _hasLoggedDisconnectedFactsAbsorb flag, test seam _resetFactsAbsorbDisconnectedFlagForTests); CLI help discoverability (capture in CLI_ONLY_SELF_HELP + pre-engine-bind --help short-circuit in handleCliOnly + a BRAIN section in printHelp); binary-file guard via detectBinaryNullByte(buf) first-8KB NUL scan on --file (Buffer-read, no encoding) and --stdin (readStdinBuffer accumulator); provenance write-through — put_page accepts 3 optional params (source_kind, source_uri, ingested_via; ingested_at server-stamped) + trust gate (when ctx.remote !== false IGNORE client params, server stamps mcp:put_page, fail-closed) + COALESCE-preserve UPDATE semantics (omitting params on a later put_page preserves prior values; first-write-wins); /admin/api/register-client scopes normalization via normalizeScopesInput(raw: unknown) in src/core/scope.ts (accepts string/string[]/missing; rejects ['read write'] space-in-element shape, non-string elements, empty array, unknown scopes; deduped + sorted); brainstorm timeout surfacing via an orchestrator-level try/catch at runBrainstorm entry (single-point wrap covers every internal SQL site, classifies SQLSTATE 57014 via postgres.js .code / .sqlState / message fallback into StructuredAgentError code brainstorm_timeout with a hint covering all 3 PG cancel sub-causes); read-path surfaces all 4 provenance columns via getPage projection + rowToPage 3-state optional read + Page interface; canonical source resolver routes capture through resolveSourceWithTier(engine, parsed.source, cwd); thin-client --source rejection (server-side OAuth client registration owns source scope); the source_kind taxonomy is closed (capture-cli | put_page | mcp:put_page | webhook | file-watcher | inbox-folder | cron-scheduler), --source maps to source_id only. Tests: test/capture-build-content.test.ts, test/capture-runcapture.test.ts, test/put-page-provenance.test.ts, test/scope-normalize.test.ts, test/cli-help-discoverability.test.ts, test/brainstorm-timeout.test.ts; extended test/facts-absorb-log.test.ts, test/import-file.test.ts, test/e2e/engine-parity.test.ts, test/e2e/serve-http-ingest-webhook.test.ts. Report at docs/v0.38-smoke-test-report.md. Follow-ups in TODOS.md: SQL-shape rewrite of listPrefixSampledPages for PgBouncer, magic-byte allowlist for binary detection, --source-kind override flag, ingest_capture handler migration, provenance-history table, facts:absorb root-cause trace.

BrainBench — in a sibling repo

The retrieval-quality BrainBench — the public benchmark for personal-knowledge agent stacks (P@5/R@5/MRR/nDCG corpus + harness) — lives in github.com/garrytan/gbrain-evals. It depends on gbrain as a consumer; gbrain never pulls in the ~5MB eval corpus or the pdf-parse dev dep at install time. The name "BrainBench" primarily refers to the in-repo cross-harness memory conformance suite (gbrain eval brainbench — see the src/eval/brainbench/ and evals/brainbench/ entries above and docs/eval/BRAINBENCH.md); this section covers the separate retrieval benchmark.

gbrain's public API surface (the exports map in package.json) is what gbrain-evals consumes: gbrain/engine, gbrain/types, gbrain/operations, gbrain/pglite-engine, gbrain/link-extraction, gbrain/import-file, gbrain/transcription, gbrain/embedding, gbrain/config, gbrain/markdown, gbrain/backoff, gbrain/search/hybrid, gbrain/search/expansion, gbrain/extract. Removing any of these is a breaking change for the gbrain-evals consumer.

Hindsight calibration (key files cluster)

Calibration teaches gbrain how the user tends to be wrong and uses that knowledge at every advice surface: a six-migration schema (v67-v72), three cycle phases, eight expansions, one admin tab. Convention skill at skills/conventions/calibration.md has the agent- facing rules.

Migration v80: the takes_resolution_consistency CHECK accepts quality='unresolvable' AND outcome=NULL as the 4th valid resolution state. The column-level CHECK on resolved_quality (takes_resolved_quality_values) enumerates all 4 states. Take.resolved_quality, TakeResolution.quality, and takes-fence.ts:TakeQuality are 4-state. TakesScorecard carries unresolvable_count

  • unresolvable_rate; resolved stays 3-state (correct+incorrect+partial) so scorecards stay comparable over time. finalizeScorecard: unresolvable_rate = unresolvable_count / (resolved + unresolvable_count), NULL when both 0. Spec doc at docs/architecture/calibration-quality-gate-spec.md. Pinned by R1-R5 in test/takes-resolution.test.ts and test/migrate.test.ts's v80 structural + PGLite round-trip suite (CHECK admits unresolvable+NULL, still rejects partial+true and unresolvable+true|false, pre-v80 NULL/NULL rows survive).
  • src/core/cycle/base-phase.ts — abstract BaseCyclePhase class. Enforces sourceScopeOpts(ctx) threading at the type level; rules out the source-isolation leak class structurally for every new phase. Inherits source-scope, budget meter, error envelope, progress reporter. propose_takes / grade_takes / calibration_profile all extend it. The ONE home of CYCLE_DEADLINE_RESERVE_MS (60s carved out of the enclosing job's remaining wall-clock — wait-poll + worker force-evict grace + cleanup headroom; patterns.ts re-exports it) and of BasePhaseOpts.deadlineAtMs (the enclosing minion job's absolute deadline, threaded via runCycle; null/unset for direct gbrain dream callers — phases then fall back to their derived defaults).
  • src/core/cycle/propose-takes.ts — LLM scans markdown prose, proposes gradeable claims to the take_proposals queue. Candidate discovery excludes extract_receipt pages before the LLM call so the phase never re-ingests its own operational receipts; nullable legacy page types remain eligible. Idempotency cache on (source_id, page_slug, content_hash, prompt_version) composite unique index. Fence-dedup: existing canonical takes passed to the extractor as context. Ships a stub prompt; tuned prompt arrives via the synthetic corpus build. Per-page extractor failures log a warning and continue, but a whole-run condition (per classifyGlobalLlmError in src/core/ai/errors.ts) breaks the page loop with a single combined warning line, sets aborted_global_error, and records a halt in the rollup — auth/billing on the first hit, bare rate_limit only after RATE_LIMIT_HALT_STREAK (3) consecutive hits (a successful call resets the streak). Status: fail when the halt happened with ZERO successful extractor calls (the whole LLM lane is down), otherwise any warnings fold into warn + (N warning(s)) summary suffix, so swallowed failures can't read as a clean ok. llm_calls_succeeded/llm_calls_failed/halted land in details. The phase wall-clock deadline is DERIVED, never a literal (a literal equal to the autopilot-cycle handler anchor, with non-co-started clocks, would make the clean deadline_hit partial-completion path structurally unreachable): explicit opts.deadlineMs wins (test seam), else resolveProposeTakesDeadlineMs(deadlineAtMs, now) = PHASE_DEADLINE_FRACTION_OF_JOB (0.8, headroom for grade_takes + calibration_profile which run after it with no deadline of their own) × (remaining job budget − CYCLE_DEADLINE_RESERVE_MS), clamped to the fallback (0.8 × the autopilot-cycle entry in HANDLER_DEFAULT_TIMEOUT_MS — a missing anchor throws at module LOAD, failing the whole cycle visibly). A fractioned value under MIN_PROPOSE_TAKES_BUDGET_MS (2 min) resolves to null and the phase returns an honest skipped with reason: 'insufficient_cycle_budget' (after the cheap provider probe, before any rollup/DB write — records neither a halt nor a completed round; next cycle retries with a fresh budget). Pinned by test/propose-takes.test.ts + test/propose-takes-per-claim.test.ts + test/cycle-phase-deadline-drift.test.ts. The kind vocabulary is TAKE_KIND_VALUES imported from src/core/takes-fence.ts (the fence enum — no hand-copied set); an unknown kind maps through the legacy-kind table, else take.
  • src/core/cycle/grade-takes.ts — walks unresolved takes older than 6 months, retrieves evidence, asks judge model, caches verdict. Auto-resolve DISABLED by default. Conservative thresholds: >=0.95 single OR >=0.85 ensemble 3/3 unanimous. aggregateEnsemble reuses the cross-modal substrate; fires on the borderline 0.6-0.95 band. Writes to take_grade_cache. Same global-error posture as propose-takes: a judge failure that classifies as a whole-run condition breaks the take loop with aborted_global_error (auth/billing first hit; rate_limit after 3 consecutive takes; fail status when zero judge calls succeeded, else warn + warning count; judge_calls_succeeded/judge_calls_failed/halted in details). Rejected ensemble judges are classified the same way — Promise.allSettled never flattens a revoked key or exhausted spend limit into a silent null verdict. Per-take auto-apply failures stay per-take.
  • src/core/cycle/calibration-profile.ts — aggregates resolved takes into 2-4 narrative pattern statements + active bias tags. Voice-gated via gateVoice(). Cold-brain skip when <5 resolved. Writes to calibration_profiles with audit columns (voice_gate_passed, voice_gate_attempts, grade_completion).
  • src/core/calibration/voice-gate.ts — single gateVoice() function, mode parameter (pattern_statement | nudge | forecast_blurb | dashboard_caption | morning_pulse). 2 regens then template fallback from src/core/calibration/templates.ts. Haiku judge with mode-specific rubrics; all rubrics structurally forbid clinical/preachy voice.
  • src/core/calibration/cross-brain.ts — 4-rule contract for cross-brain calibration reads. Local-first → mount-fallback (only with canReadMountsForCtx(ctx) true) → cross-brain attribution via source_brain_id + from_mount → subagent prohibition closes the OAuth-token-to-cross-brain-leak surface. All 4 rules pinned in test/cross-brain-calibration.test.ts.
  • src/core/calibration/nudge.ts — real-time pattern surfacing. evaluateAndFireNudge(opts): threshold check (conviction > 0.7, holder match, slug-derived domain hint matches active bias tag) → cooldown probe (14d via take_nudge_log) → fire + log. STDERR-only output; multi-channel deferred.
  • src/core/calibration/take-forecast.ts — Brier-trend at write time. Pure math over existing TakesScorecard; no LLM. Returns predicted_brier, bucket_n, overall_brier. Insufficient-data branch at MIN_BUCKET_N = 5. batchForecast memoizes per (holder, domain) tuple.
  • src/core/calibration/gstack-coupling.ts — outcome-driven learnings coupling. writeIncorrectResolution(opts) shells out to the gstack-learnings-log binary. Config gate cycle.grade_takes.write_gstack_learnings (default false for external users). Namespace prefix gbrain:calibration:v0.36.1.0: so --undo-wave can scrub.
  • src/core/calibration/svg-renderer.ts — server-rendered SVG for the admin SPA Calibration tab. Pure functions: data → SVG string. Inlines design tokens; XSS-safe via escapeXml(). Four renderers: renderBrierTrend, renderDomainBars, renderAbandonedThreadsCard, renderPatternStatementsCard. SPA renders via <TrustedSVG> wrapper behind requireAdmin.
  • src/core/calibration/undo-wave.tsundoWave reverses calibration's mutations: unsets takes.resolved_* for wave-applied resolutions (cross-checks resolved_by so manual writes persist), deletes calibration_profiles, purges nudge logs, marks grade-cache rows applied=false. --dry-run shows counts without writing. Idempotent on wave_version match.
  • src/core/calibration/think-ab.ts — A/B harness. runAbTrial calls thinkRunner twice (baseline + with-calibration), records preference to think_ab_results. buildAbReport aggregates over a 30-day window; flags calibration_net_negative when n>=20 + win rate < 45% on decisive trials.
  • src/core/calibration/recall-footer.ts — formatter for the morning-pulse calibration block. Cold-brain branch when <5 resolved. Opt-in via the wiring layer.
  • src/core/eval-contradictions/calibration-join.ts — cross-reference. tagFindingWithCalibration(finding, profile) returns bias-tag context for contradictions matching active patterns. Returns null when profile missing (output unchanged when no profile exists).
  • src/core/think/prompt.ts — anti-bias prompt shaping. withCalibration option on buildThinkSystemPrompt adds anti-bias rules. buildCalibrationBlock() emits the <calibration> XML. buildThinkUserMessage has TWO shapes: default (question first), and with-calibration (retrieval → calibration → question) when opt-in. Wired into runThink via opts.withCalibration + opts.calibrationHolder.
  • src/commands/calibration.ts — CLI: gbrain calibration (read + print), --regenerate, --undo-wave <ver>, ab-report. MCP op get_calibration_profile (scope: read) backs the same data path. Source-scoped via sourceScopeOpts(ctx).
  • src/core/owner-holder.ts — single source of truth for "the brain owner" holder string. DEFAULT_OWNER_HOLDER = 'self' (matches the consolidate facts→takes writer + docs/takes-vs-facts.md); resolveOwnerHolder({override, configValue}) returns override > emotional_weight.user_holder config > 'self'. Consumed by the calibration_profile cycle phase, gbrain calibration CLI, the get_calibration_profile op, think's calibration block, emotional-weight's DEFAULT_USER_HOLDER, and doctor's calibration_freshness. Pure; unit-tested in test/owner-holder.test.ts. Does NOT unify owner-identity fragmentation (self/brain/people-<owner>) — tracked separately.
  • src/commands/takes.ts — the full gbrain takes subcommand dispatcher (list/search/embed/add/update/supersede/resolve/propose/scorecard/calibration/extract/revisit). takes list --limit N --offset N validates integers at the CLI; the engine clamps limit (default 100, cap 500) and floors offset at 0. takes revisit <slug> opens $EDITOR on the source page with a <!-- gbrain:revisit --> cursor marker.
  • src/core/take-proposals.ts — owns the take_proposals queue's row contract. normalizeTakeProposalRow coerces Postgres driver BigInt/string values (id, weight, promoted_row_num) to numbers at the query boundary so both engines return the same numeric row shape.
  • admin/src/pages/Calibration.tsx — Calibration tab. Single-column layout. <TrustedSVG> wrapper handles dangerouslySetInnerHTML for the server-rendered SVG.
  • admin/src/index.css--text-muted: #777 (WCAG AA contrast bump to ~5.5 on the #0a0a0f bg).
  • test/fixtures/calibration/extract-takes-corpus/ — synthetic prompt-tuning corpus. Ships 5 representative pages; full 50-page + 10-page holdout generated by gbrain calibration build-corpus. All anonymized per CLAUDE.md placeholder list.
  • scripts/check-synthetic-corpus-privacy.sh — CI guard in bun run verify. Greps for explicit dollar amounts + verifies non-essay fixtures reference at least one placeholder name.
  • test/regressions/v0.36.1.0-iron-rule.test.ts — R1-R5 IRON-RULE inventory; pins all 5 rules in one place.
  • DESIGN.md — repo-root design system. Formalizes the de facto admin tokens. Calibration target for future /plan-design-review and /design-review.

Schema packs: mutation surface (key files cluster)

The schema-pack mutation surface: six foundation modules + a mutate skeleton + stats/sync data plane + CLI verbs + MCP ops + a first-class agent skill.

Key files:

  • src/core/atomic-write.ts — the shared atomic file writer for brain-repo markdown writers: unique tmp sibling (.tmp.<pid>.<rand>) → write loop until every byte lands (writeSync may legally short-write under disk pressure; a silent short write could atomically install truncated content) → fsync → close → optional verify(onDiskBytes) callback (throw = abort, tmp removed, target untouched) → mode-preserving atomic rename → best-effort parent-directory fsync (rename durability). Consumed by src/commands/backlinks.ts (which verifies with parseMarkdown({validate:true}) before the rename). Rename prevents torn writes, NOT lost updates — read-modify-write callers pair it with withPageLock (backlinks does). Unifying the per-module copies (skillopt/apply-edits, write-through, lint) is a filed TODO.
  • src/core/schema-pack/pack-lock.ts — Atomic O_CREAT|O_EXCL per-pack lock. DELIBERATELY NOT the existsSync + writeFileSync TOCTOU shape from src/core/page-lock.ts. Default 60s TTL, refresh every 10s while withPackLock(fn) runs, --force semantics = "steal stale lock" NOT "skip locking." Lock path per-pack so two packs never block each other.
  • src/core/schema-pack/write-vocabulary.ts — write-time pack-vocabulary enforcement shared by the write surfaces (capture op, add_link op, gbrain capture CLI). loadActivePackForWriteVocabulary(ctx) pairs the engine's DB-plane schema_pack key with FILE-ONLY config (the loadActivePackForLocalEngine posture) while threading remote (fail-closed: anything not strictly false is remote) + sourceId; NEVER throws — null means "no resolvable pack", which callers MUST treat as "no vocabulary to enforce" (the write proceeds exactly as before). packDeclaresPageType/packDeclaresLinkType + message/suggestion formatters reject an EXPLICIT undeclared name with the pack's declared vocabulary in the error so agents self-correct without a gbrain schema explain round-trip. The DEFAULT note path is never checked here — callers validate EXPLICIT names only, so bare gbrain capture works under a pack that doesn't declare note. loadActivePackForWriteVocabulary also swallows a rejecting getConfig; previewNames bounds the vocabulary preview in the error (the first 12 names + (N total)). Pinned by test/write-vocabulary.test.ts.
  • src/core/schema-pack/mutate-audit.ts — ISO-week JSONL at ~/.gbrain/audit/schema-mutations-YYYY-Www.jsonl. Privacy-redacted: type names → sha8, prefixes → first slug segment only, matches candidate-audit.ts privacy posture. Logs BOTH success AND failure events so the schema_pack_writability doctor check has signal. summarizeMutations() is the cross-surface parity primitive.
  • src/core/schema-pack/registry.tsresolvePack walks the extends chain (depth cap via EXTENDS_DEPTH_WARN / EXTENDS_DEPTH_HARD_CAP), RETAINS each ancestor manifest, materializes borrow_from, and composes all of it into resolved.manifest through mergeInheritedManifest. Every downstream consumer reads resolved.manifest, so doing the merge here is what makes inheritance visible without per-consumer wiring. borrow_from is selective (only the named types / link_types, and only from the target's OWN declarations), non-transitive, and fail-closed — a missing target throws UnknownPackError via loadByName, matching the extends path; an omitted category borrows none of it. The alias graph + closure hash are computed on the MERGED manifest, so a cross-pack alias cycle surfaces as AliasCycleError at resolve. manifest_sha8 / packIdentity stay the CHILD's own bytes — a parent edit does not move the child's identity, so the invalidation path is what keeps a child honest. invalidatePackCache(name?) walks the extends-chain reverse-graph (editing a parent pack must not leave children stale). tryCachedPack(name) TTL-gated fast path: inside STAT_TTL_MS (default 1000ms, env GBRAIN_PACK_STAT_TTL_MS) returns cached without statting; outside the window it stats every TRACKED file — the extends chain PLUS every borrowed pack — and cascade-invalidates on mtime change (cross-process detection), so editing a borrowed pack invalidates its borrowers. Pinned by test/schema-pack-registry.test.ts + test/schema-pack-merge.test.ts.
  • src/core/schema-pack/merge.ts — the pure child-wins composition helper behind resolvePack. mergeInheritedManifest(ancestorsBaseFirst, child, borrowed) returns the fully-composed manifest; precedence is child → borrowed → nearest parent … → base. SIX ingest/query-shaping fields inherit: page_types, link_types, frontmatter_links, enrichable_types, filing_rules, takes_kinds. phases + calibration_domains are DELIBERATELY child-only — they gate real cycle execution (cycle.ts packDeclaresPhase), so inheriting them would silently run phases a pack never declared; mapping_rules, migration_from, extends, borrow_from, and the identity fields are child-only too (all ride the ...child spread). mergePageTypes carries the ordering contract inferTypeFromPack depends on (first-path_prefix-match-wins, array order): the BASE (root, extends: null) pack is the ordered foundation/tail; an override of a base type keeps the base POSITION (Map.set updates the value, keeps insertion order) so base's curated priority survives; a genuinely-new type from ANY non-base layer — child, borrowed, or a middle pack — is PREPENDED nearest-first, so a more-derived prefix wins regardless of chain depth. mergeByKey keeps the first occurrence per key walking highest-precedence-first (the order-insensitive keyed fields); frontmatter_links keys on page_type\x00link_type — a NUL, not a space, because both are unconstrained strings and a space-join would collide {"a b","c"} with {"a","b c"}. mergeUnion backs takes_kinds: UNION not replace, because the Zod default makes an omitted field indistinguishable from an explicit one — so a child can ADD kinds but CANNOT narrow below base ∪ parent. Pure + deterministic: no disk, no engine. Pinned by test/schema-pack-merge.test.ts.
  • src/core/schema-pack/best-effort.tsloadActivePackBestEffort(ctx) returns ResolvedPack | null. Single source of truth for the pack-aware wiring sites. null means EMPTY FILTER (NOT hardcoded defaults, which would silently violate the pack).
  • src/core/schema-pack/type-usage.ts — stored-type classifier behind the alias-footgun visibility surfaces: classifyStoredType(type, pack) → canonical | alias_of (with the canonical type + path_prefixes[0] filing directory) | undeclared, over a STRUCTURAL pack shape so import-file's thin activePack and the full manifest both satisfy it. sanitizeTypeForDisplay strips control chars + caps length (type strings come from frontmatter and get echoed into terminals); renderTypeWarningSummary renders the once-per-type-per-run lines. Consumers: importFromContent (advisory ImportResult.type_warning at the typeExplicit site — the type is still stored literally, zero filing change), sync/import summary aggregation (+ SyncResult.type_warnings so worker-driven syncs surface counts in job results), the stored_type_is_alias/stored_type_undeclared data-plane lint rules, all gated by config schema.type_warnings (default on; lint rules always active). Pinned by test/schema-type-usage.test.ts.
  • src/core/schema-pack/lint-rules.ts — 14 pure rule functions. withMutation's pre-write validation gate composes the 10 file-plane rules; the 4 DB-aware rules (extractable_empty_corpus, mutation_count_anomaly, stored_type_is_alias, stored_type_undeclared) need an engine (CLI --with-db; the stored-type pair accepts LintOpts.sourceId scoping, not yet threaded from the CLI). Single source of truth consumed by CLI lint + MCP schema_lint + the pre-write validation gate. File-plane rule link_regex_catastrophic_backtrack — advisory ReDoS pre-screen flagging the classic nested-quantifier shapes ((a+)+, (a*)*, (a+)*, (\w+)+) in a link_type's inference.regex via NESTED_QUANTIFIER_RE. WARNING not error: a hard reject would disable the whole pack on upgrade (pages fall back to legacy typing). The runtime input-length cap in redos-guard.ts is the actual safety net; this rule tells the pack author to fix the pattern.
  • src/core/schema-pack/redos-guard.ts + src/core/schema-pack/link-inference.ts — ReDoS hardening for pack inference regexes. redos-guard.ts provides MAX_REGEX_INPUT_CHARS (default 64_000, env GBRAIN_MAX_REGEX_INPUT_CHARS) — a hard input-length cap, the real runtime safety net (catastrophic backtracking needs a long input; a link-extraction context is normally a sentence or short paragraph). Over the cap, runRegexBounded throws the tagged RegexInputTooLargeError and the regex is skipped (degrade-to-mentions) without entering the node:vm. link-inference.ts:inferLinkTypeFromPack no-budget branch (test contexts) routes through runRegexBounded so the input-length cap + per-regex vm timeout (PER_REGEX_TIMEOUT_MS = 50) apply on every path. Defensive hardening + diagnostics; the deterministic ~3100-file sync-wedge root cause remains open. Pinned by test/redos-hardening.test.ts + test/schema-pack-lint-rules.test.ts.
  • src/core/schema-pack/query-cache-invalidator.tsinvalidateQueryCache(engine, sourceId?) DELETEs query_cache rows so cached search results bound to old page types don't survive a schema mutation.
  • src/core/schema-pack/mutate.ts — 8-step withMutation skeleton (bundled-guard → lock → read → mutator → validate → atomic write → audit → invalidate) backs the 11 single-mutation primitives: addTypeToPack, removeTypeFromPack (with reference check), updateTypeOnPack, addAliasToType, removeAliasFromType, addPrefixToType, removePrefixFromType, addLinkTypeToPack, removeLinkTypeFromPack, setExtractableOnType, setExpertRoutingOnType. Each primitive's business-rule validation + transform is factored into a build*Mutator(...) pure (manifest) => manifest function shared with applyMutationsAtomic (the schema_apply_mutations batch entry point) so single-call and batched mutations can never validate differently. applyMutationsAtomic locks + reads the pack file ONCE, applies + lint-validates every mutation in the batch against an in-memory manifest, and calls writePackManifest at MOST ONCE — only after the whole batch checks out — so a batch that fails partway leaves the pack file byte-identical to its pre-batch state. Atomic single write via .tmp + fsync + rename — the pack file on disk is NEVER partial, for either a single mutation or a batch. Inline minimal JSON→YAML emitter so YAML packs stay YAML (does NOT preserve comments — pin pack.json if you care about layout).
  • src/core/schema-pack/stats.tsrunStatsCore(engine, opts) returns per-source + aggregate page counts + coverage % + dead_prefixes (declared prefixes with zero matching pages — agent drilldown signal). Multi-source aware (sourceIds[] federated, sourceId single, or whole-brain). PGLite + Postgres parity via executeRaw. Empty brain → coverage:1.0 (vacuous truth).
  • src/core/schema-pack/sync.tsrunSyncCore(engine, opts) chunked UPDATE in 1000-row batches per declared prefix. Concurrent writers never block on a single row >100ms. Write-side scoping via ctx.sourceId directly (NOT sourceScopeOpts, which inherits OAuth read federation). Idempotent on --apply re-run.
  • src/commands/schema.ts — 14 CLI verbs in the dispatch table: add-type, remove-type, update-type, add-alias, remove-alias, add-prefix, remove-prefix, add-link-type, remove-link-type, set-extractable, set-expert-routing, stats, sync, reload. withConnectedEngine routes loadConfig() through the canonical toEngineConfig() helper and passes the complete result (database_url and database_path) to factory construction and connect, so PGLite schema commands open the configured brain. Lifecycle-grouped help text (Inspection / Activation / Authoring / Discovery+repair). Pinned by test/schema-cli-database-path.serial.test.ts.
  • skills/schema-author/SKILL.md — Agent dispatcher for "evolve the schema pack." Triggers: 15+ phrasings incl. "add a page type", "my brain has untyped pages", "propose new types from my corpus", "backfill page types". Explicit Non-goals callout to brain-taxonomist (files one page) and eiirp (schema-check during iteration) so agents pick the right surface. 7-phase workflow: brain → assess → propose → apply → sync → verify → commit. Lists every gbrain schema CLI verb + every MCP op the skill uses. brain_first: exempt frontmatter. Required conformance sections: Contract, Anti-Patterns, Output Format.
  • skills/conventions/schema-evolution.md — Canonical convention: "when to add a type vs alias vs prefix." Decision tree: <20 pages → don't pack-codify; 20-100 → alias or narrow prefix on existing type; 100+ → first-class type. Don'ts section + "when to remove a type" + "when to commit the pack" all answered in one place.

Pack-aware wiring is partial. Three follow-ups are filed in TODOS.md: enrichment-service.ts union widening ('person' | 'company'string), facts/eligibility.ts pack-aware ELIGIBLE_TYPES wiring, and 3 doctor checks (schema_pack_coverage, schema_pack_writability, schema_pack_mutation_audit).

  • src/core/vector-index.ts + src/commands/doctor.ts:embedding_column_registry — shared pgvector HNSW eligibility policy. hnswIndexExpected(columnType, dims) derives the answer from the canonical vector/halfvec dimension caps already used by migration index generation. Doctor reports an HNSW-less active embedding column as a healthy exact-scan configuration when its declared width exceeds the applicable pgvector cap, and only emits the index repair recipe when an index is actually supported. Pinned at both cap boundaries by test/vector-index-lifecycle.test.ts.

Agent bootstrap cluster (the paste-in desktop-agent install)

Normative docs: docs/designs/AGENT_BOOTSTRAP_DESIGN.md (scope) + docs/designs/AGENT_BOOTSTRAP_PLAN.md (implementation, review-finding IDs inlined). User-facing contract: docs/guides/bootstrap.md. Runbook the paste block fetches: BOOTSTRAP_FOR_AGENTS.md (root; carries a version stamp CI pins to VERSION).

  • src/commands/bootstrap.ts — the gbrain bootstrap {status,interview,render,repo,hooks,verify,uninstall,attach,cloud-setup-script} dispatcher (plus the machine-level harness subcommand — see the core/bootstrap/harness.ts entry below). Engine-free everywhere except verify (which opens/closes its own engine — safe because verify runs with no live serve, before host registration). cloud-setup-script is a pure printer (prints templates/bootstrap/cloud-setup-script.sh for the cloud environment's setup step) and is dispatched entirely BEFORE workspace resolution, so it works from any cwd, including HOME.Everyothersubcommandresolvesworkspace(default:cwd)throughresolveWorkspace,whichrefuseswithBootstrapError(HOMEWORKSPACE)whentheresolvedpath(realpathcompared,soasymlinkedHOME. Every other subcommand resolves `--workspace` (default: cwd) through `resolveWorkspace`, which refuses with `BootstrapError('HOME_WORKSPACE')` when the resolved path (realpath-compared, so a symlinked HOME or cwd still matches) IS process.env.HOME (falling back to os.homedir() only when HOME is unset) — an unqualified run from a freshly-SSH'd shell must not silently stage a home-directory-scale git add -A over ~/.ssh/ and friends; the refusal is still appended to <home>/bootstrap/install.jsonl (keyed by the rejected candidate path) before returning. status, uninstall, and harness are exempt from this guard (HOME_WORKSPACE_GUARD_EXEMPT): status only reads and prints a report, uninstall removes exactly receipt.created_paths (each containment-checked) — never a git add/commit/push, never a workspace-wide scan — and harness operates only on home and never even receives a ws argument (runHarness, machine-level wiring), so the value resolveWorkspace returns for it is used only for LogCtx.ws bookkeeping. Exempting status/uninstall keeps the recovery path reachable for the guard's own victims: someone who bootstrapped into $HOME needs status to see what's there and uninstall to remove it, both run with --workspace pointing AT $HOME; the refusal message for the still-guarded subcommands names that recovery path. Known gap: a process already launched with HOME pointed away from the real account home is not caught (Bun's os.homedir() mirrors process-start HOME rather than doing an independent uid lookup, so it collapses to the same wrong value). Mutating subcommands run under the workspace bootstrap lock; render is gated on interview complete && confirmed and hard-refuses when the workspace origin is a PUBLIC remote (identity files must never land in a public repo — the same template-door gate status enforces; unverifiable visibility warns and proceeds, treating the origin as public); the provider key routes to the 0600 config sink and never touches interview state; every MUTATING subcommand appends a line to <home>/bootstrap/install.jsonl (the read-only status and cloud-setup-script do not log). hooks on Claude Code writes the committed carrier in a cloud sandbox (writeCommittedClaudeHooks) and the gitignored local file otherwise; uninstall tears down the durability wiring and removes this workspace's per-root push/debounce state, and its opencode teardown sweeps BOTH merged global filenames (under the opencode config-dir bootstrap lock) plus the project opencode.json, expectation-keyed on this workspace's source id (skipOtherSource) — a gbrain entry from a different workspace is skipped with a note, never silently deleted. Consent answers resolve FAIL-CLOSED: consentAnswer treats a hand-edited/unusable interview value (non-string, empty) as declined — loudly, with a re-record note — never falling through to a permissive bank default; hooks on Codex prints a corrective note when a persisted project MCP_SCOPE answer is found (raw state read, not the resolver — codex mcp add has no scope flag, registrations are always user-global) with safe clear instructions. GBRAIN_BOOTSTRAP_ABORT_AFTER is the deterministic kill-mid-phase test seam.
  • src/core/bootstrap/format.tsagent.json manifest (format_version 1, provisional; initialized sentinel distinguishes a template clone from a bootstrapped workspace) + the machine-local install receipt (<home>/bootstrap/receipt.json) that proves THIS machine ran bootstrap; uninstall is keyed to the receipt, never the repo manifest. Atomic writes; readManifest never throws (typed states incl. conflict markers).
  • src/core/bootstrap/assets.ts — every template + the question bank embedded via Bun with { type: 'file' } imports (the chunkers/code.ts pattern) so the compiled binary renders with no repo checkout; DERIVED_TOKENS (GITHUB_REPO_URL, CORPUS_RETENTION_DAYS) is the non-bank half of the template token set the CI bijection guard checks.
  • templates/bootstrap/ — the ten {{TOKEN}} identity templates (AGENTS/CLAUDE/SOUL/USER/MEMORY/HEARTBEAT/ACCESS_POLICY/GITHUB/memory-README/gitignore), questions.json (12 asked / 6 required; consent keys; persist:false sink keys), and template-repo/ — the VENDORED deterministic render the release job diffs against before publishing the public template repo. Generic placeholder content only (privacy iron rule; CI-asserted).
  • src/core/bootstrap/interview.ts — interview state at <ws>/state/interview.json (committed; multi-device re-render source). Read-back confirm hash: --confirm must present the hash of the exact answer set shown to the human, and ANY later answer change clears the confirmation — the single-batch self-confirm attack is structurally impossible. Set-time enforcement: length caps, reject-lists, allowed-lists, control-char strip, {{ escaping. Conflict-markered files return agent-readable errors, not stack traces.
  • src/core/bootstrap/render.ts — token substitution with interview values treated as data (line-leading #/<!--/fence escaping), hard-fail on unresolved tokens, never-clobber + timestamped backups on --force, blank-line collapse, byte floors scaled to answered count. --minimal is the deterministic placeholder mode the template-repo generator uses (byte-identical across runs; leaves required tokens as literal fill-me markers; writes initialized:false). --only never writes agent.json.
  • src/core/bootstrap/lock.ts — the bootstrap-run mutex (atomic mkdir + pid liveness + age guard + ownership token; steal requires dead pid AND stale age) and the family's shared typed BootstrapError (GH_MISSING/GH_AUTH carry exit 2 = human action needed).
  • src/core/bootstrap/repo.ts / attach.ts / uninstall.ts — private-repo lifecycle. createPrivateRepo: gh gates, slugified name probe, gh repo create --private --source --push, privacy verified via gh api .private (rate-limit/5xx is VERIFY_UNAVAILABLE, distinct from not-private) before any push, idempotency keyed off the remote URL. A pre-existing origin is adopted (disposition 'adopted') when the authed gh user owns it, there's no recorded repo_url, and it is SAFE — empty or already carrying our history (assertAdoptableOrigin; a foreign-content repo is refused ORIGIN_NOT_EMPTY, never a silent no-op); this is the create-repo-first path. Org-owned origins and anything else are refused and pointed at attach. Repo-local git identity is set in both create and adopt paths before commit; repo_url is recorded only after a successful push. attachWorkspace (machine two): requires an initialized manifest, writes this machine's receipt, returns structured wiring steps. uninstallWorkspace: receipt-keyed, refuses under a live serve (read-only lock probe — never opens the engine), removes exactly receipt-recorded paths + marker-keyed host entries, keeps the brain unless --delete-brain AND bootstrap created it; never wholesale-deletes the gbrain home. All gh/git through an injectable ExecRunner seam.
  • src/core/bootstrap/hooks.ts + host-specs.ts — host wiring. host-specs.ts is the ONE module owning host-format assumptions (dated spec targets with verifiedAt + doc references: claude-code hooks/settings shapes incl. the 10,000-char hook-output cap and the five hook events; codex mcp-add argv, the streamable-HTTP url + http_headers = { Authorization = "Bearer <t>" } config shape (loads on codex-cli 0.147.x and 0.149.x; inline bearer_token is rejected at config load by >=0.149), codexConfigPath() honoring CODEX_HOME, claudeUserSettingsPath() honoring CLAUDE_CONFIG_DIR/HOMEexplicitlyBunshomedir()ignoresaremappedHOME,whichwouldpointsandboxedwritersattheoperatorsrealsettings;thesharedresolutionlivesintheprivateclaudeConfigBase()(CLAUDECONFIGDIRelseHOME explicitly — Bun's homedir() ignores a remapped HOME, which would point sandboxed writers at the operator's real settings; the shared resolution lives in the private `claudeConfigBase()` (CLAUDE_CONFIG_DIR-else-HOME-else-homedir(), the directory playing the role of ~/.claude — NOT for claudeUserMcpConfigPath, whose default lives at the HOME level as ~/.claude.json), which also feeds claudeUserSkillsDir()/claudeProjectSkillsDir() (native SKILL.md discovery, the harness-bridge install targets; user scope attested, project scope provisional-from-docs) — and the opencode shapes: opencodeConfigDir/opencodeGlobalConfigPath (XDG-only resolution; the .jsonc name preferred for parity with opencode mcp add; OPENCODE_CONFIG/_CONFIG_DIR/_CONFIG_CONTENT deliberately NOT honored — observed INERT in opencode 1.18.18, honoring them would write registrations into a file opencode never reads), opencodeProjectConfigPath, opencodeGlobalSiblingPath (the OTHER member of the global filename pair — writers reconcile mcp.<name> across it because opencode merges both), and OPENCODE_HAS_HOOKS=false — 'gbrain does not wire opencode's plugin/event system yet', not 'opencode has no hooks'). The settings writers are path+marker parameterized: writeClaudeHooksAt/removeClaudeHooksAt take an explicit settings file and a marker VALUE (workspace installs stamp bootstrap-v1 in .claude/settings.local.json; harness installs stamp bootstrap-harness-v1 in user-scope settings or a --project dir — the two coexist and each removal strips only its own; refuseOnForeignGbrainMarker blocks same-file double-wiring), fail-closed on broken JSON (a settings file that doesn't parse THROWS with the path — a writer must never relocate or overwrite a config it can't read), mode-preserving realpath-resolved atomic writes (dotfile symlinks survive), and fixed-or-timestamped backups. addPermissionsAllowEntry/removePermissionsAllowEntry manage the harness lane's mcp__<name> headless pre-approval with set semantics and NO marker (ownership rides the harness receipt; foreign entries always survive); the add path fails CLOSED when an existing permissions key or permissions.allow carries a shape it doesn't understand (host security policy is never rewritten on a guess), while removal leaves shapes it can't read untouched and reports nothing-to-remove. registerClaudeMcp/registerCodexMcp build argv only (Claude Code takes --scope, project default; Codex has no scope flag — codex mcp add is always user-global; -e/--env GBRAIN_SOURCE so MCP writes land in the workspace source, and serve --surface full pinned so a pre-existing mcp_surface: verbs config row can't silently narrow the bootstrap op surface). The opencode workspace lane execs NOTHING — registration is the direct opencode-json.ts write with an INVERTED scope default (user-global; opencode spawns project-config servers with no trust prompt, so MCP_SCOPE=project is an explicit opt-in that writes the committed-candidate opencode.json with a PATH-resolved command and prints the sharing warning), verification is config parse-back + a best-effort opencode mcp list --pure probe — the probe spawns via a held Bun.spawn handle from a fresh EMPTY temp-dir cwd (never the invoking cwd: opencode merges a project opencode.json from cwd and spawns its servers with no trust prompt), is SKIPPED entirely for project scope (printed note; parse-back is authoritative) and on plugin-bearing configs (mcp list is a code-execution surface), and on timeout actually kills the child (SIGTERM → SIGKILL, bounded pipe drain, code 124 into the could-not-confirm branch; probeSpawn is the injectable seam). User-scope writes also reconcile the SIBLING global filename first (ours → removed with a note; foreign → refuse naming both files) so a merge-shadow registration can never survive. detectHarness probes OPENCODE/OPENCODE_PID (set in opencode's bash-tool children, observed 1.18.18).
  • src/core/bootstrap/codex-hooks.ts — the codex hooks.json writer (SessionEnd capture lane), built entirely on the dated CODEX_HOOKS_SPEC_TARGET (verified 2026-08-25 against pinned codex-cli 0.147.0, live + tag-source evidence; re-run the observation gate on codex version bumps). The three load-bearing facts: hooks.json is TOP-LEVEL deny-unknown-fields (ownership rides the legal description slot + a command-substring token, never a _gbrain key); user-layer hooks are TRUST-GATED and fail SILENTLY (the writer lands BOTH the hooks.json entry AND its [hooks.state."<path>:session_end:<g>:<h>"].trusted_hash config.toml entry — sha256 of the recursively-key-sorted compact canonical JSON of the normalized handler identity — inside a gbrain-managed marker block, backing config.toml up to .hooks.bak so it never clobbers the MCP writer's .bak rollback anchor); SessionEnd handlers are hard-killed at 3s (the command captures stdin to a mktemp file and detaches a nohup grandchild running the real gbrain hook session-end --harness codex — live-verified end-to-end). Deliberately NO GBRAIN_SOURCE in the command: hooks.json is user-global, so a baked source would stamp every codex session on the machine; session-end resolves everything from the payload. a fresh install APPENDS our group last while a re-run REPLACES it IN PLACE (a single well-formed install never shifts a foreign group's trust index; dropping DUPLICATE gbrain groups can shift foreign groups sitting after them — the writer prints a note naming the re-trust); the whole command is wrapped in a top-level sh -c '…' <bin> so a non-POSIX $SHELL (fish/csh) never parses the POSIX script; the foreign-trust-entry guard is a tolerant [hooks.state…] header match (not byte-exact); a golden known-answer hash vector pins the canonical recipe; removal also deletes the gbrain description when byte-identical to ours and NOTES the inherent index shift for foreign groups that sat after ours; unparseable hooks.json is never touched, on write or removal; the residual stale-trust-index case (user reorders their own groups) degrades to silent non-execution, which doctor's codex_hooks_never_fired rung names. Pinned by test/codex-hooks-writer.test.ts. writeCommittedClaudeHooks is the second, COMMITTED hook carrier: it writes marker-keyed entries into the workspace's checked-in .claude/settings.json using buildPortableClaudeHookCommand (PATH-resolved gbrain, fail-open when the binary is absent) so teammates cloning the repo inherit the hooks; committedHookEvents(ws) feeds the local writer's carriedEvents so the two carriers never double-wire an event, and removeClaudeHooks strips both.
  • src/core/bootstrap/codex-toml.ts — the ONE direct codex-config writer (codex mcp add cannot express an inline credential, and framework-spawned codex inherits no shell profile for the env-var lane). One [mcp_servers.<name>] table between full-line markers carrying url + http_headers = { Authorization = "Bearer <token>" } (NOT inline bearer_token, which codex-cli >=0.149 rejects at config load; parseCodexBlockBearer in harness.ts keeps a legacy bearer_token read fallback so existing tokens still recover); everything outside survives byte-for-byte. Foreign-server detection PARSES the config (Bun.TOML.parse, no dependency) with our block stripped — a header-only regex would false-negative inline-table/dotted/quoted spellings into a codex-bricking duplicate table; rewrites re-anchor at EOF; renders are parse-validated with an ours-keys-exactly assert before rename; damaged markers refuse; 0600 tmp/target/.bak (the .bak carries the previous token on re-runs); CRLF preserved, missing trailing newline repaired. Also exports renderCodexHttpServerBlock({name, url, bearerToken}) — a MARKER-FREE, parse-validated [mcp_servers.<name>] TOML render for paste-into-config surfaces (gbrain agent register's codex block); rendering only, never writes a file. Pinned by test/codex-toml.test.ts.
  • src/core/bootstrap/serve-health.ts — serve /health probe + scopes version-skew floor, re-exported by harness.ts. probeServeHealth(mcpUrl, fetchFn, timeoutMs=3000) GETs <base>/health and returns {ok, version?, engine?, detail?}, never throws; fetchFn is an explicit argument (no ambient fetch, no engine, no config). isServeOlderThanScopes(v) compares against the PINNED SCOPES_MIN_SERVE_VERSION constant — a comparison against the moving CLI VERSION would false-flag every scope-aware serve on the next release. An older serve verifies scoped tokens as FULL ACCESS, so callers print the floor line unconditionally.
  • src/core/bootstrap/opencode-json.ts — the ONE direct opencode-config writer (workspace stdio lane, harness remote lane, connect --install). ALL edits ride jsonc-parser modify/applyEdits (comments/formatting/EOLs survive byte-for-byte outside the edited range — opencode's own mcp add preserves comments, and JSONC is its effective grammar for BOTH .json and .jsonc filenames, which MERGE when both exist). Ownership is a 4-state STRUCTURAL FINGERPRINT (opencodeEntryKind → ours-same-source | ours-other-source | foreign | absent; GBRAIN_SOURCE EQUALITY for local entries, receipt-url match or the {env:GBRAIN_REMOTE_TOKEN} interpolation for remote), never a marker key (a future strict-schema flip must not brick the host). Distinct read-failure classes (ENOENT create / empty-as-{} / unreadable refuse); foreign refusal on write AND remove; post-render validation (our entry round-trips + every other key survives) keeps the original on failure; 0600 for inline-bearer targets; backups are UNIQUE per operation (<config>.bak-<hex>, returned in the result with writtenText, the exact rendered bytes) so overlapping runs can never clobber each other's snapshot, and a backup is chmod'd 0600 whenever the COPIED content carries an inline bearer (write AND remove paths); the ours-other-source refusal is caller-appropriate (url + --force wording on the remote/expect-url path, GBRAIN_SOURCE wording on the local path); removeOpencodeMcpEntry takes a caller expectation + optional skipOtherSource — the uninstall lane passes THIS workspace's source id so an ours-other-source match becomes a calm skip-with-note (another workspace's registration is never deleted; foreign still refuses); reconcileOpencodeSiblingGlobal clears a same-name gbrain entry from the OTHER global filename before a global write (foreign → refuse naming both files) — opencode merges both, so a leftover sibling entry is a shadow registration; the bun-run ownership lane is ANCHORED (an exact gbrain/gbrain-* path segment in some arg — a gbrainy-fork cli path is NOT ours, fail-closed); parseOpencodeEntryBearer recovers the harness --status token url-matched only; opencodeRemoteEntryExists is the stdio-lane ownership arbiter (codexBlockOwnsName analog). Callers hold acquireBootstrapLock (config-dir → opencode-dir ordering). Pinned by test/opencode-json.test.ts (incl. the fingerprint truth-table suite).
  • src/core/bootstrap/atomic-write.ts — the ONE atomic config-file writer for bootstrap host surfaces: symlink-target-resolving, mode-inheriting (freshMode for new files, forceMode for secret-bearing targets), random-suffix tmp + rename, tmp unlinked best-effort when write/chmod/rename throws (ENOSPC/EACCES never strand .tmp- litter), and the existsSync→realpathSync race falls back to fresh-path resolution instead of throwing raw ENOENT. hooks.ts (JSON), codex-toml.ts (TOML+EOL), and opencode-json.ts (JSONC) all swap through it; serialization and EOL policy stay caller-side.
  • src/core/bootstrap/harness.tsgbrain bootstrap harness: machine-level wiring of framework-spawned Claude Code/Codex/opencode sessions to a RUNNING gbrain serve --http, no agent.json. The opencode target mirrors the codex posture: forced-wire on explicit --harness opencode (the JSONC writer needs no CLI), one managed mcp.<name> remote entry with the inline bearer header (0600) via opencode-json.ts, rotation across a url change recognized through the PRIOR receipt's url, failed-smoke rollback restoring the run's UNIQUE backup or removing a fresh entry — guarded by a content compare against the exact text this run wrote, so a NEWER registration that landed after the lock released is never clobbered (rollback skips with a note; the fresh mint is revoked either way), with the backup unlinked once consumed or once the smoke verifies — --remove classifying against the receipt url (not-ours skips with a note), and --status bearer recovery via parseOpencodeEntryBearer (url-matched). Consent block in the honesty register (reach as fact, transcript capture its own numbered item + --no-capture, off-ramps in the same breath; non-TTY requires --yes); /health probe with a loopback guard on --url (remote brains are gbrain connect's charter; --token makes it a pure registrar); mint-first rotation (previous token revoked BY ID only after every target confirms + the bearer smoke passes — clients are never dead mid-swap; revoke-by-name never happens); the smoke is canary-gated: a random same-format bearer must FAIL auth before the real token is sent, so a loopback impostor is caught whichever way it answers, and ANY failed smoke rolls back symmetrically — fresh registrations removed, replaced ones restored (an unrestorable replacement fails the target honestly), the freshly-added pre-approval stripped, and the fresh mint revoked immediately so nothing live stays pointed at an unverified endpoint; the permissions.allow pre-approval only lands after the MCP registration itself confirms; stale prior-target cleanup runs AFTER the smoke passes, so a run that failed to establish its replacement never unwires working prior wiring; write-ahead harness receipt (targets persist as pending at mint time and flip per-target, so a crash leaves consumable state); registration ownership checks (--force to replace a foreign-url server; --remove skips what it no longer owns); user-XOR-project hook scopes; GBRAIN_HOOK_LANE=harness on hook commands so gbrain hook yields to a workspace bootstrap install in the cwd (Claude Code merges settings scopes — same event must not fire twice); the hook GBRAIN_SOURCE, the receipt source_id, and the token grant bind to a validated --source (scalar write floor), else to the source HarnessDeps.resolveHookSource resolves through the SAME chain the serve's resolve-IPC binding runs (resolveSourceWithTier: env → dotfile → local_path → sources.default → sole populated non-default → seed default) with that source's federated read set as the grant, so the hooks' turn_context claim is never source_mismatch; a resolution landing on default/__all__ is the federated floor. Lookup failures are NOT papered over with default: a typo'd --source or stale env/dotfile throws SOURCE_UNRESOLVED before any mint or receipt; an explicit --source with an unopenable engine binds unverified with a warning; a live PGLite serve (engine cannot open) refuses with the LIVE_SERVE escape hatches when no token was supplied and, on the --token lane without --source, wires the hooks UNPINNED with a warning (an unpinned hook resolves through the live serve's own binding; the receipt records the federated floor default plus source_pinned: false); any other lookup failure throws SOURCE_UNRESOLVED naming --source; Postgres degradation + serve-version-skew honesty lines (isServeOlderThanScopes pinned to SCOPES_MIN_SERVE_VERSION, the first scope-aware release, so later CLI bumps never re-trigger the warning); --status probes the live truth with host-config token recovery (redacted; the Claude Code lane recovers a bearer ONLY from a registration whose URL matches the receipt — never another install's credential; the codex fallback reads OUR managed block at the receipt-recorded path, its url key not yet compared — TODOS.md; the opencode fallback IS url-compared inside parseOpencodeEntryBearer) and honest verify: unavailable degrades, with a cron-honest exit contract: 0 only when serve + token + every target verify and the rotation has converged (honest degrades count), 1 on an unreachable serve, a failed token verify, failed/pending targets, unconverged rotations, or a half-removed receipt (zero targets, minted token awaiting deferred revoke — also a doctor FAIL); no receipt prints honestly and exits 0 in plain mode, 2 under --json; --json on apply emits ONLY the final JSON document on stdout (prose → stderr); --remove is engine-free-first and defers the token revoke with exact instructions under a live PGLite serve. Locks on the gbrain HOME, plus the host config dir around codex/opencode writes, removals, rollbacks, and stale cleanup (stale prior-target cleanup nests the codex/opencode config-dir lock inside the held claude config-dir lock — same claude-first ordering as apply/remove, same-dir skip — so its read-modify-write can't interleave with a concurrent config-dir-locked writer); bootstrap uninstall holds the HOME lock across the whole teardown and runs harness removal FIRST (revoke needs the DB alive; --delete-brain would destroy harness.json) and treats NO_RECEIPT/HOME_GUARD/RECEIPT_MISMATCH as "no workspace install" once harness wiring is cleared. Pinned by test/bootstrap-harness.serial.test.ts + test/e2e/bootstrap-harness-lifecycle.serial.test.ts.
  • src/core/token-mint.ts — programmatic legacy-token mint/revoke for the harness lane: mintLegacyToken (scopes → the TEXT[] column; required takesHolders per-token allow-list, harness default ['world']; optional permissions.source_id federation array mirroring the stdio lane's localFederatedSourceIds grant, element 0 = write floor; RETURNING id) and revokeLegacyTokenById (never touches same-name siblings). Exports TOKEN_ID_RE, the canonical token-id shape shared with the auth revoke --id CLI gate. Canonical hashToken/generateToken from src/core/utils.ts. Pinned by test/token-mint.test.ts.
  • src/commands/hook.ts — engine-free gbrain hook {session-start,user-prompt,stop,session-end,compact} (zero engine modules in the import graph; a hook must NEVER contend for the PGLite writer lock). When GBRAIN_HOOK_LANE=harness (set on harness-mode hook commands), each event PARSES the cwd's workspace settings carriers — .claude/settings.local.json AND the committed .claude/settings.json — and yields silently only when a live bootstrap-v1 hook entry wires THAT event: the workspace install wins over user-scope harness wiring so the merged settings scopes can't fire the same event twice, unwired events still run, and a repo committing marker-lookalike strings in unrelated fields can't disable the machine-wide capture lane (fail-open — a read/parse hiccup means the event runs normally). user-prompt: stdin hook JSON → transcript-path confinement → last-4-turns window + cross-turn dedupe (the transcript's hook_additional_context attachments — the blocks WE previously injected — ride priorContextText, deduplicated and capped at PRIOR_CONTEXT_MAX_BYTES (32KB, so the advisory payload can never blow the IPC message cap; one oversized block is skipped without evicting smaller ones), so a page is volunteered once per session, not once per mention; structured extraction only, never raw-turn substring matching) → IPC turn_context (with a feedback-loop channel, --harness <claude-code|codex>, default claude-code) → hookSpecificOutput.additionalContext under an 800ms self-deadline; every path fails open (exit 0, empty stdout) with a typed reason in the heartbeat. Listed in cli.ts's STARTUP_HOOK_SKIP_COMMANDS (per-prompt invocations must never spawn a detached check-update child; membership is pinned by a source grep — the runtime path no-ops under NODE_ENV=test). session-start: file-plane digest (allowlisted MEMORY.md sections, push staleness, prior failures) + crashed-session recovery push gated on an initialized manifest. session-end: confined full-transcript parse → redacted corpus write (session-id filename dedup, retention prune; the corpus scan runs highEntropy whenever the relay is on — the relay child derives its egress task line from the corpus text — and stays vendor-prefix-only otherwise) → parser-drift detection (bytes>0 && turns==0 is loud) → best-effort workspace push. session-start recovery + session-end pushes run in a DETACHED child so the hook returns immediately (a synchronous inline push would block harness startup on a dirty tree); the corpus write is atomic and clears the stale ingested/in-progress sidecars so a resumed session re-ingests its appended transcript. Heartbeat JSONL is counters/reasons only by construction; readHeartbeatTail feeds doctor. GBRAIN_HOOKS=0 kills all events. Session-end capture dispatches per harness through captureSpecFor(io.harness) (src/core/transcripts/capture-spec.ts): claude-code and codex each pin their OWN confinement root + parser; unknown/undefined/opencode resolve to the claude spec (golden-pinned). The codex lane confines to codexSessionsDir() + codexArchivedSessionsDir() (codex moves rollouts into the flat archived store), parses rollouts via parseCodexHookTranscript, and falls back to bounded id-matched discovery across both stores when the SessionEnd payload carries transcript_path: null or a path that no longer exists (missing_path; unreadable is reserved for EACCES/IO faults) — the fallback and prior-run relay failures are DEFERRED heartbeat reasons, applied after current-session reasons so they never mask one (first-degrade-wins). ONE gate covers the whole optional Memorable seam — memorableGateAllowed(cfg) in hook-heartbeat.ts: the config flag (integrations.memorable.enabled === true, file plane), the GBRAIN_MEMORABLE kill switch (any common negative spelling, trimmed; env can only ever disable), AND the gbrain-authored consent stamp (memorable-consent.json — see hook-heartbeat.ts). Off is the default: no tool-call collection, no receipt, no spawn. On, session-end runs the shared redactedToolCallsJson (span-filtered to the corpus window, highEntropy ALWAYS on — those args are the one artifact that leaves the machine) and recordAndRelayReceipt (receipt dedup by post-redaction content hash → CLI-side consent evidence → resolveMemorableBin → detached fire-and-forget memorable record spawn; both capture lanes trim memorable-relay.jsonl). Fail-closed egress guards: a scanner-import failure skips the receipt+relay entirely (memorable_relay_skipped_unscanned — parity with the openclaw lane), and a newest-mtime discovery GUESS (no session-id match) writes the local corpus but never relays (memorable_relay_skipped_newest_guess — the newest rollout can be a different, still-running session). A payload without session_id adopts the transcript's own id post-parse (never a shared unknown.txt corpus), and the gate answer is hoisted BEFORE the parse and collection is opt-in, so the default gate-off population never collects tool calls (collectToolCalls: memorableAllowed). Operator-facing doc: docs/memorable-agents.md (install is npm i -g memorable-cli; closed source, no source install, no embedding model to configure).
  • src/core/transcripts/claude-code-jsonl.ts — the Claude Code transcript parser as a dated spec-target (tool_use/tool_result/thinking/image/sidechain/summary/compact-boundary shapes; placeholders for non-text content); also extracts injectedContextBlocks — the hook_additional_context attachment lines a gbrain hook previously injected (verified live against claude CLI 2.1.224; marker-filtered, so a foreign hook's blocks are excluded and another tool's output can't suppress volunteering — a same-user mislabeling guard, not an authenticity check), the user-prompt hook's cross-turn dedupe input; confineTranscriptPath (contained under ~/.claude/projects, .jsonl, lstat-rejects symlinks, byte cap; under WSL, a Windows drive-literal transcript_path — Claude Code on the Windows host invoking hooks via wsl.exe — is translated through src/core/wsl-paths.ts and then held to the SAME confinement, rooted Windows-side, so translation never widens what a hook may read). Fixtures: test/fixtures/conversation-formats/claude-code.jsonl (synthetic, privacy-guarded) + test/fixtures/hook-transcript.jsonl (real captured hook round-trip). parseTranscript additionally returns toolCalls, but collection is OPT-IN (collectToolCalls: true; the bare parse — every per-prompt lane — collects nothing, so tool INPUTS are never retained for users who never opted in): the tool name + input args for every tool_use block, oldest → newest, each joined to its tool_result outcome by tool_use_id and then stripped of that id so no transcript-internal identifier reaches a consumer; every STRING in a collected input is bounded to TOOL_CALL_VALUE_MAX_CHARS (32k) with an explicit …[N chars omitted] marker (capToolCallInput, shared with the codex lane — one receipt-size ceiling for every harness). It is a parallel extraction, never a change to entryToTurn — that function's placeholder-only [tool: name] rendering is load-bearing for the token-budget-constrained ambient-recall path; existing callers that ignore the field see no behavior change.
  • src/core/transcripts/grok.ts — the Grok Build (grok CLI) transcript adapter: one session directory (~/.grok/sessions/<url-encoded-cwd>/<uuid>/chat_history.jsonl) = one session; GROK_HOME relocates the root. Turn selection is STRUCTURAL (mapGrokLine, exported so the dated SPEC_TARGET mapping is pinned): user turns are type:'user' text blocks WITHOUT synthetic_reason (injected system_reminder/task_completed rows are typed, i.e. intentionally text-free, as are tool_result-only user arrays); assistant turns need non-empty string content (tool-only rows — content ''/null WITH tool_calls — are typed); system/reasoning/tool_result/backend_tool_call rows are typed; unknown row types skip. A recognised HUMAN-turn row whose content this parser cannot decode (user content missing / non-string / non-array / an array holding non-block entries; assistant non-string content with no tool_calls) is malformed and counted as SKIPPED, never typed, so a file made only of such rows is never expectedEmpty — the ingest drift signal fires (bytesRead > 0, sessions 0, cleanScan=false, watermark frozen) and zeroSessionsReason names the malformed row count, instead of an upstream schema change silently advancing the watermark past whole conversations. chat_history.jsonl carries NO per-message timestamps: session times come from the sibling summary.json (created_at/last_active_at; a malformed summary is ignored, an unparseable date is never fabricated) and, because grok writes summary.json at session END, an in-progress session or partial rsync falls back to the log file's own mtime — a real filesystem time — with raw.timestamp_source stamped 'summary.json' vs 'file_mtime' so consumers can tell them apart (without the fallback the render refusal would freeze the --since-last watermark for the whole grok root). Every message carries the session bounds (last message = last_active_at, the rest = start). Session id: summary → UUID dir name → stableIdFromPath (a sha256 prefix of the log path, never the bare chat_history basename, which would collapse every such session onto one page). Sidecars (updates.jsonl, events.jsonl, rewind_points.jsonl, summary.json, prompt_history.jsonl) are not sessions — discover.ts's grok-sidecar filter is scoped to PROVEN grok trees and the head sniff rejects claude-family keys (grok detection runs after claude-code; an oversized system head still detects through the truncated-line sniff); a percent-encoded cwd segment that fails to decode yields cwd undefined with the session id intact.
  • src/core/transcripts/capture-spec.ts + codex-hook-lane.ts — the per-harness session-end capture seam. capture-spec.ts: the satisfies-anchored CAPTURE_SPECS record ({confine, parse, discover?}) for the STDIN-DRIVEN lanes (claude-code, codex); captureSpecFor() resolves unknown/undefined/opencode to the claude spec (golden-pinned by test/codex-hook-lane.test.ts); openclaw is deliberately NOT a member (its lane is in-process, trusted-plane — see context-engine.ts). codex-hook-lane.ts: confineCodexTranscriptPath (the same S3#8 ladder as the claude root, pinned at BOTH codexSessionsDir() AND codexArchivedSessionsDir() — codex MOVES a rollout into the flat archived_sessions store on archive, so fencing only the live store refused a still-valid path; both CODEX_HOME-resolved since the spawner IS codex; an explicitly pinned test-seam root stays single-root; ENOENT/ENOTDIR on the path is missing_path — the rung hook.ts gates its discovery fallback on — while unreadable is reserved for EACCES/IO faults, so the codex session-end heartbeat reason for a nonexistent transcript_path after a failed discovery is transcript_missing_path, not transcript_unreadable; no WSL branch v1), parseCodexHookTranscript (ParsedTranscript shape over the adapter's exported mapCodexLine; same OPT-IN collectToolCalls contract as parseTranscript, and the same capToolCallInput bound on observed args; head+tail over budget so session_meta identity survives; tool calls from the OBSERVED args keys — custom_tool_call.input fixture-verified, function_call.arguments source-verified at rust-v0.147.0; tolerant JSON-string parse; NO result join — 0.147.0 persists no success flag on *_output rows; compacted rows become boundary positions), and discoverNewestCodexRollout (bounded newest-first walk over the dated live store, then a flat pass over the archived store under the SAME 4096 dirent cap — skipped once the cap trips; id-in-filename match, and an id-matched archived hit keeps transcript_discovered (never relabelled _newest, which hook.ts reads as a guess and bars from the relay); symlink-reject — the fallback for transcript_path: null or a moved rollout, and the seam for SIGKILL'd sessions, which never fire SessionEnd). Engine-free by construction; never imports discover.ts. Pinned by test/codex-hook-lane.test.ts + test/codex-hook-lane-archived.test.ts + the runHook-driven matrix in test/memorable-relay.serial.test.ts.
  • src/core/wsl-paths.ts — WSL Windows-drive path translation shared by the doctor asset checks (src/commands/doctor-asset-paths.ts) and hook-transcript confinement (confineTranscriptPath above). Exports WINDOWS_DRIVE_PATH_RE, translateWindowsPath(p, mountRoot) (C:\Users\x\file<mountRoot>/c/Users/x/file; null for non-drive input so callers keep the original untouched), parseWslAutomountRoot(conf) (the /etc/wsl.conf [automount] root value, default /mnt), and detectWslMountRoot() (null off-WSL). Mechanical detection + translation only — POLICY stays with callers (skip vs stat vs confine): the hook lane confines the translated path to the known Claude config tree, doctor decides skip-vs-stat. Pinned by the WSL cases in test/claude-code-jsonl.test.ts.
  • src/core/context/turn-context.ts — server-side per-turn assembly: reflex pointers + volunteered pages (≤3) + hot facts (always visibility=['world'] — the IPC path never widens what MCP would return) under a "data, not instructions" envelope, trimmed to ≤8KB (the harness caps hook output at 10,000 chars). The ambient context_pack and delta modes are world-only across every arm by default; only an explicit trusted-local include_private widens entity cards, changed pages, threads, and facts together. The result exposes pointers AND post-trim volunteered — exactly what the rendered text carries — so the IPC delivery point can log the feedback loop without ever counting a trimmed-out page. Reuses the hot-memory cache keyed by typed sessionId. Engine-agnostic.
  • src/core/context/resolve-ipc.ts (IPC v2) — discriminated-union requests (absent kind = legacy resolve; turn_context carries protocol: 2 + a shared secret from a 0600 file in the data dir, plus an additive channel for feedback-loop attribution — wire channel claims are validated to the harness channels at the logging site, anything else logs as the default hook channel), handler map, named response types, per-kind timeouts/size caps, socket + parent dir permissions set before exposure, server-side source binding (cross-source requests rejected), protocol echo (a response without it = stale serve → loud degradation). Bind policy: startResolveIpcServer connect-probes the path first — a live owner makes it return null (that serve stays the IPC provider; a transient serve never unlinks a live socket), only a dead/absent owner's entry is unlinked (unconditionally, since Bun on win32 cannot stat the AF_UNIX socket file it leaves behind) before listen. The connection handler processes exactly ONE request per connection (trailing bytes mid-await never double-process a line or double-log a delivery); the client clamps a too-big request below the message cap by dropping the advisory priorContextText BEFORE any conversation turn. Delivery seams: onDelivered (resolve kind) and onTurnContextDelivered (turn_context kind) both fire ONLY after the response write succeeds — a block abandoned before the serve responded is never counted (serve's callback logs the delivered block's volunteered pages + pointers to context_volunteer_events under the request channel); write-accept still isn't proof of injection (the client can trim/drop after receipt), which is why the volunteer_channels doctor check reconciles counts against the hook heartbeat. v1 clients and servers interoperate untouched. Serve-delegated sync rides the same socket as three additive secret-gated kinds (sync_start/sync_status/sync_abort, wire shapes + fail-closed option validation in src/core/context/sync-ipc.ts): start+poll, one line per connection, O(1) handlers with no server budget race; a pre-delegation serve answers unknown_kind:* without the protocol echo and the client degrades to the typed stale-serve refusal.
  • src/core/context/turn-context.ts — server-side per-turn assembly: reflex pointers + volunteered pages (≤3) + hot facts (always visibility=['world'] — the IPC path never widens what MCP would return) under a "data, not instructions" envelope, trimmed to ≤8KB (the harness caps hook output at 10,000 chars). delta pages carry updated_at from Page.updated_at_iso (the column's microseconds, projected by listPages), so next_cursor.since and the session last_wake_at (read back via to_char in session-state.ts) resume from the exact row — a millisecond-rounded cursor re-delivers same-millisecond pages on the next wake; the delta op keeps an already-canonical 6-digit ISO since verbatim instead of re-rounding it through a JS Date. The ambient context_pack and delta modes are world-only across every arm by default; only an explicit trusted-local include_private widens entity cards, changed pages, threads, and facts together. The result exposes pointers AND post-trim volunteered — exactly what the rendered text carries — so the IPC delivery point can log the feedback loop without ever counting a trimmed-out page. Reuses the hot-memory cache keyed by typed sessionId. Engine-agnostic.
  • src/core/context/resolve-ipc.ts (IPC v2) — discriminated-union requests (absent kind = legacy resolve; turn_context carries protocol: 2 + a shared secret from a 0600 file in the data dir, plus an additive channel for feedback-loop attribution — wire channel claims are validated to the harness channels at the logging site, anything else logs as the default hook channel), handler map, named response types, per-kind timeouts/size caps, socket + parent dir permissions set before exposure, server-side source binding (cross-source requests rejected), protocol echo (a response without it = stale serve → loud degradation). The connection handler processes exactly ONE request per connection (trailing bytes mid-await never double-process a line or double-log a delivery); the client clamps a too-big request below the message cap by dropping the advisory priorContextText BEFORE any conversation turn. Delivery seams: onDelivered (resolve kind) and onTurnContextDelivered (turn_context kind) both fire ONLY after the response write succeeds — a block abandoned before the serve responded is never counted (serve's callback logs the delivered block's volunteered pages + pointers to context_volunteer_events under the request channel); write-accept still isn't proof of injection (the client can trim/drop after receipt), which is why the volunteer_channels doctor check reconciles counts against the hook heartbeat. v1 clients and servers interoperate untouched. Serve-delegated sync rides the same socket as three additive secret-gated kinds (sync_start/sync_status/sync_abort, wire shapes + fail-closed option validation in src/core/context/sync-ipc.ts): start+poll, one line per connection, O(1) handlers with no server budget race; a pre-delegation serve answers unknown_kind:* without the protocol echo and the client degrades to the typed stale-serve refusal.
  • src/core/facts/visibility.tsresolveDefaultVisibility(engine) / resolveVisibilityParam: the ONE resolver behind all four facts-visibility default sites (facts.default_visibility config key; explicit caller value always wins; invalid values fail closed to private). Bootstrap sets the workspace brain's default to world so the principal's own sessions can recall their facts — a documented, security-relevant knob. NOTE the deliberate asymmetry with the ambient-writeback TEMPLATE resolver (writeback-config.ts below): this READ/extract default fail-closes UNSET to private; write GUIDANCE treats unset as world (the remember verb's own default) because telling agents to write private-by-default on a default install makes their facts invisible to their own later remote sessions.
  • Ambient memory writeback cluster (opt-in, memory.auto_writeback = off|salient|all, default off; guide docs/guides/ambient-writeback.md):
    • src/core/facts/ttl-parse.ts — dependency-free TTL grammar leaf (parseTtlShorthand non-throwing typed results; validateTtlConfig — duration-shorthand-only, positive, ≤365d for the transient-TTL config). ops/facts.ts's parseTtlParam wraps it with the byte-identical frozen verbError copy; the engine-free hook child shares the grammar without touching the gateway-reaching ops graph.
    • src/core/facts/writeback-config.ts — config resolution, fail-closed OFF. gbrain config set memory.* DUAL-WRITES (file mirror first for the engine-free readers, then the AUTHORITATIVE DB plane; src/commands/config.ts MEMORY_DUAL_PLANE_KEYS branch, which also stamps the resolved memory.visibility_posture file mirror for the engine-free harness renderer — and a facts.default_visibility set/unset RE-STAMPS that mirror so installed blocks can always converge; a DB-write/delete failure on these keys exits NON-ZERO naming the unchanged runtime value, and config get reports them DB-first with a drift warn). MOUNT RULE: when the selected brain is not the host (--brain/GBRAIN_BRAIN_ID/.gbrain-mount), every dual-plane lane (set, unset, unset --pattern, the posture re-stamp) writes the DB row ONLY and says so — the machine-local mirror gates the HOST's Stop hook, and enabling a team mount must never opt the host's conversations into banking; an unresolvable brain selection also skips the mirror (fail toward not mutating host state). The engine path resolves mode/ttl from the DB ONLY — the machine-global file mirror never enables a per-brain gap (a mounted/selected brain can't inherit another brain's opt-in); a provided fileCfg serves drift detection (plane_drift: DB row ABSENT + file enabled; explicit DB 'off' is intent, never drift) and the LKG override (a file-mirror explicit 'off' beats a cached ENABLED bundle on read failure). resolveWritebackConfig caches a per-engine LAST-KNOWN-GOOD bundle ATOMICALLY (mode+ttl+visibility together; no LKG + read failure = OFF+read_error). ambientOptsFrom(wb, {remember, extractFacts}) gates the section on the caller's ACTUAL callable set — no remember ⇒ no section (bound-client fences, clamped surfaces). visibilityPostureFromRaw owns the rule: unset → world, only explicit non-world → private.
    • src/core/facts/writeback-instructions.ts — the ONE ~15-line instruction-section builder feeding BOTH the MCP instructions composition and the bootstrap-managed harness blocks (surfaces structurally cannot drift). Mode-conditional candidate policy, literal resolved TTL, surface-honest extract_facts mention, visibility posture with the "world = agents on this brain, not the public internet / never widen" rule.
    • src/core/facts/writeback-gate.ts — the engine-free zero-LLM Stop-hook salience gate: frozen skip-reason vocabulary (empty/too_short [CJK-aware 20→10 floor]/ack_or_greeting/slash_command/question_only/quoted_or_tool_output/bulk_paste >8KB), first hit wins; ok returns the NFC/collapsed normalized text + sha256-hash24 — the turn's idempotency key (same turn ⇒ same .wb- filename ⇒ embedding-free dedup that works keyless).
    • src/core/facts/writeback-audience.ts — personal-vs-shared classifier for the consent nudge. Declaration (brain.audience, stamped by company-brainify's Phase-5 handoff, the bootstrap interview's SURFACE_MULTIUSER answer — applied the moment interview --set records it, BEFORE the agent runs init — or the operator) BEATS the conservative heuristic (≥3 distinct non-automation MCP clients active 30d via mcp-usage.ts, likely_automation excluded); reasons[] always names the evidence. The machine-global file-mirror declaration speaks for the HOST brain only — a mounted/selected brain falls through to its own DB row or heuristic. Fail directions: config read failure → unknown (nudge stays silent), usage-read failure with a reachable DB → personal (missing multi-client infra is personal-shaped). Never enables anything.
    • src/core/onboard/writeback-nudge.ts — the fire-once consent ASK (init epilogue + runPostUpgrade): double gate (sentinel memory.auto_writeback_notice_shown + setting unset), personal-audience only, suppressed on mounts/thin-clients/env bypass, [AGENT]-relayed disclosure + ask (non-TTY IS the relay path — deliberately unlike runInitNudge), sentinel stamped AFTER printing (a decline is permanent; a shared classification does NOT burn the sentinel), whole body try/caught (init/upgrade must survive a crashing nudge). The recurring reminder is src/core/advisor/collect-writeback-consent.ts (info, ask_user, NO dispatch_id — consent is never --apply-able; local-only; fires only AFTER the sentinel so the advisor is never the first ask).
    • src/core/bootstrap/instructions-block.ts — the managed harness block (<!-- gbrain:ambient-writeback:begin/end -->): spliceCompiledBlock discipline verbatim (idempotent replace-interior, THROW on damaged markers, marker-equal body lines neutralized), header names mode + serve endpoint (multi-brain last-wins stays visible), remove/strip helpers never delete a user file. Harness lane (src/core/bootstrap/harness.ts): kind:'instructions' targets (additive under receipt version 1 — old binaries won't unwire them, documented) for user-scope CLAUDE.md + $CODEX_HOME/AGENTS.md (paths + the AGENTS.override.md exclusivity probe are dated PROVISIONAL spec entries in host-specs.ts); enabled ⇒ install, disabled ⇒ CONVERGE (strip blocks + receipt targets + advisory line; a strip FAILURE is recorded as a failed instructions receipt target so --status/--remove/the next converge keep tracking the orphan); registrar mode (non-loopback --url) NEVER installs blocks — the local mirror speaks for the local brain, and the remote brain's own MCP instructions carry the contract when its operator opts in; a block installs only when the SAME host's MCP target confirmed (both lanes — a foreign-registration refusal or failed write must not leave a block directing saves at a server this run never registered), and a failed final smoke STRIPS the blocks this run installed (targets marked failed) so no block outlives its rolled-back registration; --remove/--status wired.
    • Stop-hook backstop lane: hookStop in src/commands/hook.ts (own 2s deadline inside Stop's 10s cap, fail-open exit 0, file-plane gate, 128KB-tail transcript parse with ONE 2MB-cap retry when no user turn is found — turn OFFSET from EOF ≠ turn size; a big tool_result would otherwise silently lose the turn — then gate → secret-scanned bankWritebackTurn [corpus-segments.ts: <session>.wb-<hash24>[.src-<sourceId>].txt — the source segment lets the SWEEP fallback file the turn into the session's own source; scanner-unavailable = fail-closed skip] → requestContextPack(bankOnly, flushCorpusFile, trigger:'writeback-bank', sourceId from GBRAIN_SOURCE); every outcome a typed writeback-bank heartbeat reason — by-design skips [gate reasons, no_user_turn, flush_skip_*] ride outcome:'ok', 'degraded' is INFRA faults only) → src/mcp/context-pack-handler.ts tags .wb- basenames lane:'writeback'checkpoint-harvest.ts's writeback lane (AUTHORITATIVE serve-side memory.auto_writeback re-check with {gate:true} + loadConfig drift input — off ⇒ terminal .ingested {skipped:'writeback_off'}; plane_drift/invalid mode/read_error/non-transport extraction skip [extract_skipped_*: refusal/content_filter/malformed_output class] ⇒ NO sidecar, file survives for the sweep; salient ⇒ notabilityFilter:'medium-and-up'; source:'hook:writeback'; NO manifest publish; per-session prompt-harvest cap WRITEBACK_SESSION_CAP=30 in a true-LRU counter map [delete-before-set on increment], budget burned only on real enqueues, overflow = ACK-only skip the sweep batch-extracts; heartbeats under event:'writeback' with PERSISTED inserted/duplicate/superseded). The sweep's corpus pass recognizes .wb- files, applies the same gate semantics, routes to the FILENAME-banked source (isValidSourceId-checked, pass source as fallback), and records skipped_reason in its terminal sidecar (src/core/sweep.ts) — leftover banked turns never bypass the OFF gate and never silently zero out. wb-file state machine: ack-only skips leave the file for the sweep; writeback_off is the one terminal skip sidecar.
    • Read-time TTL validity (there is no sweeper): ACTIVE fact reads in BOTH engines (listFactsByEntity/Since/BySession, both findCandidateDuplicates branches, countUnconsolidatedFacts, getFactsHealth active buckets) carry AND (valid_until IS NULL OR valid_until > now())valid_until is temporal validity, not retention; history paths (listSupersessions, findTrajectory, --asof) deliberately still see lapsed rows; a re-stated expired fact re-inserts fresh. Pinned by test/facts/ttl-validity.test.ts + engine-parity additions.
    • Diagnostics: src/commands/doctor/checks/memory-writeback.ts (memory_writeback, BRAIN category) — quiet-ok when off, but the OFF branch still probes receipt + canonical instruction paths and WARNS on a lingering block (off-but-still-instructing) with the converge command; plane-compare warns when the file mirror disagrees with the DB row (the Stop hook acting on the wrong truth) naming the re-sync; a read_error reports the state as UNKNOWN, never "off (default)"; resolved bundle + BOTH visibility postures + audience/reasons; instruction blocks receipt-vs-live-probe-vs-drift compare (drift fix names the config set + bootstrap harness combo — the config set re-stamps the posture mirror so the re-render converges even after an out-of-band visibility flip) + AGENTS.override.md detection — and a receipt target in state failed is a STANDING warn regardless of the live probe (the one physical-survival path — a smoke-rollback strip that itself threw — leaves a current-looking block directing sessions at a rolled-back endpoint, and only the receipt state knows) naming the bootstrap harness --yes / --remove converge; validity-lapsed count; 7d counters (verbs usage sidecar's additive remember_status [stamped in src/mcp/dispatch.ts, ALL-MCP-callers semantics labeled honestly] + writeback/writeback-bank heartbeat events; turns_banked counts flush_skip_* — banked, enqueue declined) with the local/lossy disclosure. Null-engine runs report the file mirror honestly.
    • Tests: test/writeback-config.test.ts, test/mcp-instructions-writeback.test.ts, test/facts/writeback-gate.test.ts, test/hook-writeback-stop.serial.test.ts, writeback-lane cases in test/checkpoint-harvest.serial.test.ts, test/bootstrap-instructions-block.test.ts + harness serial additions, test/writeback-nudge.serial.test.ts, test/doctor-memory-writeback.serial.test.ts, the hermetic 5-step test/ambient-writeback-lifecycle.serial.test.ts, and the NON-GATING real-codex door (test/e2e/bootstrap-real-codex.serial.test.ts, ambient-writeback describe).
  • src/core/sweep.ts + src/commands/sweep.ts — the serve-resident maintenance sweep (the lock owner closes the persistence loop): facts-fence reconciliation (zero-LLM, reuses the cycle extractor with a slug subset), deterministic link/timeline extraction over recent workspace pages (the same cores as gbrain extract — remote put_page deliberately skips these, the sweep is where the graph compounds), and spend-gated corpus ingest (skipped keyless; sidecar-marked exactly-once). Bounded, fail-soft, never throws; armed at serve startup (3s, best-effort) and on 10-min idle ticks through the injectable timer seam, everything unref'd; GBRAIN_SWEEP=0 kills it. gbrain sweep --once is the trusted CLI seam bootstrap verify uses (CLI-only, never over MCP). The link pass threads the same link_resolution.cross_source opt-in and configured sources.default the CLI extract lanes pass into resolveCandidateSources, so an edge into another source is created (flag on) or counted as cross_source_link in skipped (flag off) — never silently dropped, and the reconcile never deletes a cross-source edge extract links --source db created.
  • src/core/context/sync-ipc.ts + src/core/serve-sync-runner.ts + src/commands/sync-delegate.ts — serve-delegated sync. sync-ipc.ts is the LEAF wire module: the DELEGATED_SYNC_OPTION_FIELDS table is the single source of truth for the validator, the CLI wire-builder, and the serve-side SyncOpts builder; validateDelegatedSyncOptions rejects unknown keys fail-closed (repoPath/skipLock/lockId/concurrency are unreachable from the socket), requires timeoutSeconds (0 = the explicit --no-hard-deadline unbounded encoding; everything else clamps to 24h), and toWireSyncResult truncates pagesAffected to 50 + a true total under the 256KB message cap. serve-sync-runner.ts is the serve-side module singleton: one job at a time (correctness still rests on the gbrain-sync:<source> row lock inside performSync), clientToken attach (a lost-ack retry finds its own running OR retained-terminal job instead of duplicate-running), per-job deadline timer, [serve-sync] lifecycle stderr lines, an idempotent shared shutdownDelegatedSync() both serve shutdown paths await BEFORE engine.disconnect() (the disconnect-mode drain is allowAbort:false after _db is nulled, so settle writes need the live engine; a registered drainer remains as backstop), and maybeDrainDeferredEmbeds (delegated jobs always run noEmbed — the cost gate lives in runSync — and the serve drains stale embeds afterwards via runEmbedCore, keyless-safe, cleared only when a drain finds nothing left; wire noEmbed records that the USER declined embeds and suppresses the drain). sync-delegate.ts is the CLI half: read-only holder probe (probeLivePgliteHolder), DEFAULT-DENY argv gate (any unclassified token refuses by name — a silently dropped --exclude would perform the wrong sync), engine-free source tiers (resolveSourceIdEngineFree), 1s poll loop tolerant of event-loop-blocked serves with a PID-probe backstop, unknown_job → serve-restarted resume hint, Ctrl-C → sync_abort (second Ctrl-C exits 130), and runSync's pull_failed verdict mirroring. Mounts and non-PGLite configs never delegate; opt-outs --no-delegate / GBRAIN_SYNC_NO_DELEGATE=1 (client) and GBRAIN_SERVE_SYNC_IPC=0 (serve). Pinned by test/context/sync-ipc-validation.test.ts, test/sync-delegate-ladder.test.ts, test/serve-sync-runner.serial.test.ts, test/context/resolve-ipc-sync-kinds.test.ts, and test/e2e/sync-delegation-under-serve.serial.test.ts (gating tier1 step).
  • src/core/capability.ts — config-plane keyless/keyed detection + the honest capability report (per-provider keyless banner: OpenAI = semantic search + auto-extraction, Voyage = semantic search, Anthropic = auto-extraction) rendered by verify and the runbook. The extraction probe resolves through the SAME shared resolveEffectiveChatModel the gateway's reconfigure fallback uses (GBRAIN_MODEL > servable file pin > key-aware tier default), so a stale unservable pin degrades identically in the report and at runtime. Key/env fold comes from mergedProviderEnv (src/core/ai/provider-env.ts). Accepted limitation (documented in the module): DB-plane overrides (models.default, facts.extraction_model) are invisible to this engine-less probe; the runtime gate in facts/extract.ts and the engine-aware pre-enqueue gate in facts/backstop.ts are the backstops.
  • src/core/secret-scan.ts — pattern scanner for USER workspaces (own minimal allowlist + <ws>/.gbrain-scan-allow per-finding overrides — deliberately NOT the repo's .gitleaks.toml, which is a public-repo CI fixture policy); redacted previews only; redactFindings is the corpus-write mode.
  • src/core/backup/status-file.ts — ENGINE-FREE core of the monthly backup-coverage check: the cached verdict (~/.gbrain/backup-status.json, schema gbrain-backup-status-v1; fail-open load, atomic tmp+rename write, invalidateBackupStatus() fired by the fix paths — bootstrap repo receipt write, sources harden, workspacePush finish on ok when the cache carried a warn or failing/unpushed totals (a routine healthy push leaves an ok cache alone, preserving the monthly throttle)), the notice renderer (backupNoticeText(s, 'human'|'aggregate') — human ≤300 chars names assets, aggregate is counts-only for remote surfaces; the remote-privacy pin), and the bounded nag gate shared by every render channel (backupNagGate(channel, s) on backup-nag-state.json, OWN schema gbrain-backup-nag-v1 — skillpack loadNagState would drop the extra fields on round-trip, so only the pure policy fns decideNagAction/recordNagDisplay are reused). Budget = AND of: 24h cross-channel dampener (last_shown_at), per-channel ceiling 3 per pseudo-version (YYYY-MM(checked_at) + no-remote fingerprint — month rollover or a changed verdict re-surfaces; a same-month recompute with an unchanged verdict stays quiet), and a global cap of 3 recorded impressions per month (global_shown_count/global_month). backupNagReadOnlyConsult is the OpenClaw context-engine's never-writes variant; backupSpawnDue/recordBackupSpawn debounce the session-end detached spawn (no sidecar file); maybeEmitBackupNag is the cli.ts startup rail body (skip set incl. serve/call/jobs; BACKUP_LOCAL_ONLY <n> machine marker + human line). Off switches: GBRAIN_BACKUP_CHECK=0, config backup.check_enabled; interval env GBRAIN_BACKUP_CHECK_DAYS > config backup.check_interval_days > 30; values <1 or non-numeric fall back to the 30-day default (DAYS=0 never means "always stale"; config set rejects <1 outright). Tests: test/backup-status-file.serial.test.ts.
  • src/core/backup/coverage.ts — engine-side compute for the backup check. computeBackupCoverage(engine, {localGitProbes}) assesses: source repos (deduped by discoverGitRoot, capped 500 roots with a logged skip count; originRemoteState — a positive tri-state, deliberately NOT sync-git's hasOriginRemote, which collapses probe failures into a false "no remote" — plus hasRemoteTrackingRef (origin configured but never pushed is also no_remote), aheadCount, isWorkingTreeDirty — local read-only git subcommands only, no network), the bootstrap workspace (receipt repo_url + push statuses, file plane only), db_only tiering (info row with the gbrain export --dir fix), harness skill dirs (info — installed copies), and the DB-only-brain worst case (pages>0, nothing git-backed → warn on PGLite / info with different copy on postgres — the engine.kind branch). no_remote alone flips overall: 'warn'. getBackupStatus is the single choke point (fresh cache → file read; probed results persist, probe-less results NEVER persist; a failed compute returns the prior cache — never clobbers). maybeRefreshBackupStatusInProcess(engine) is the serve-side single-flight refresher — called from mcp/dispatch.ts gated on opts.transport === 'stdio' (the locality axis; 'http'/UNSET ⇒ fail-closed, pinned by test), recomputes on stale-or-absent OR warn+>24h, 1h attempt floor. Trust boundary: same as extraction-sync.ts — git probes only in trusted-local contexts. Consumers: commands/backup.ts (gbrain backup status|check, PGLite-lock fallback to cache), advisor/collect-backup-coverage.ts, doctor/checks/backup-coverage.ts (backup_coverage, localOnly probes / remote cache-only), commands/sync.ts post-sync stale-only refresh, hook session-end detached spawn. Docs: docs/operations/backup-check.md. Tests: test/backup-coverage.serial.test.ts, test/advisor-backup-coverage.serial.test.ts, test/doctor-backup-coverage.serial.test.ts, test/hook-backup-notice.serial.test.ts, test/mcp-backup-nag.serial.test.ts.
  • src/core/workspace-push.tsgbrain sources push: deny-glob backstop (tracked *.pglite/.env* refused regardless of .gitignore state) → stage FIRST → secret-scan the STAGED index blobs via git cat-file (closes the scan-then-stage TOCTOU — scanned bytes == committed bytes) → commit FIRST → divergence-safe pull → push, under one cross-platform lock (mkdir-atomic; flock is not a dependency — macOS). The pre-push secret gate FAILS CLOSED: an unreadable, oversized (> PUSH_MAX_SCAN_BYTES), or otherwise unscannable staged blob returns blocked_unscannable (nothing committed) instead of sailing through — only a confirmed staged deletion is skipped, and binary/NUL-sniffed blobs are scanned anyway. Statuses map to exit codes in src/commands/sources.ts: pushed/skipped_in_flight → 0; blocked_secrets/blocked_tracked_deny/blocked_unscannable/refused_visibility → 5; pull_conflict/push_failed/other → 1. Refuses public AND unverifiable remotes (never fail-open); pushes even on clean trees; writes <home>/bootstrap/push-status.json. Parent-repo-aware (a source may be a subdirectory of the workspace repo).
  • src/core/gbrain-home.ts — the single GBRAIN_HOME resolution choke point (delegates to config's parent-dir semantics; 0700 on create) — durability, push, hooks, and bootstrap all route through it so home semantics cannot drift.
  • src/core/bootstrap/verify.ts + status.ts — verify is the definition of done: fail-soft check suite over the REAL write path (put_page op → write-through file under brain/ → in-process sweep → graph floor via link tables → recall), the keyless magic-moment check (## Facts fence → zero-LLM reconciliation → world-visibility read-back), source_id collision resolution (as the one bootstrap subcommand holding an engine: a manifest source_id already registered to a DIFFERENT checkout → derives a stable workspace-<8char-path-hash>, persists it to agent.json, names the re-register steps — every consumer reads manifest.source_id), the embedding_plane check (keyed installs live-embed ONE probe string and compare the RETURNED width to the actual content_chunks.embedding width — a plane split is a named FAIL with the recovery command, keyless passes, a dead probe warns; runs BEFORE the roundtrip so a keyless-passing roundtrip can never certify a brain whose keyed writes all fail), token sweep, byte floors, secret scan, deny globs, repo privacy, hooks smoke (in-process IPC), capability report, first-run tour; snapshots kept last-5 under <home>/bootstrap/. status owns the ordered PHASES list (the runbook defers to it), artifact-first detection, install.jsonl, the runbook version-stamp skew check, and the support blob doctor/agents relay verbatim.
  • src/core/bootstrap/template-repo.ts + scripts/generate-template-repo.ts — deterministic public-template generation (render --minimal + placeholder manifest + stamped README); published only by the release workflow after diffing against the vendored tree.
  • scripts/check-bootstrap-tag.sh / scripts/check-bootstrap-templates.sh — CI guards: sanctioned distribution ref only (latest-stable; the release job advances it after assets publish) + runbook stamp == VERSION; template↔question-bank token bijection + placeholder-only assertion + offline generator↔vendored byte-diff + runbook-phase↔status.ts consistency. Both skip gracefully when their subjects are absent.
  • src/core/cycle/synthesize-concepts.ts concept-quality addendum — eligible groups are processed deterministically by tier, descending atom count, then concept slug so the fixed LLM budget reaches the strongest evidence first regardless of database row order. Every page and phase receipt distinguishes llm, intended deterministic_tier, budget_fallback, and error_fallback synthesis modes. The narrative call's output cap is sized from the resolved models.dream.synthesize model via resolveSynthMaxOutputTokens: 500 for non-thinking models, the gateway's THINKING_MODEL_MAX_OUTPUT_TOKENS (no phase-private number — DeepSeek v4 truncates at 8192-class caps) when the gateway's shared isThinkingModel matches (name-matched Claude 5 or recipe-declared thinking_by_default; unknown providers count as non-thinking), so a tier-reasoning thinking model no longer spends the whole budget on reasoning and persists a template stub or truncated chain-of-thought as the concept narrative. Pinned by test/cycle/synthesize-concepts-token-cap.test.ts. Right after each concept page write the phase banks concept<->member-atom provenance edges through engine.addLinksBatch (link_source: 'concept-provenance'; synthesized_from concept->atom, synthesizes atom->concept; audit site cycle.synthesize_concepts.provenance), both endpoints scoped to the cycle's source, so the concept pages are graph-reachable (backlinks, relational recall, doctor graph_signals_coverage/orphans) even though the prompt forbids enumerating atoms in the body. The dedicated link_source keeps reconcile passes from pruning them; ON CONFLICT DO NOTHING makes re-runs the backfill; a failed or zero-row edge write lands in link_warnings[] (phase warn) — NOT in failures[], which means "LLM-failed → template fallback" downstream and drives the rollup's halt_delta/round_completed_delta — and never aborts the page write. Atom discovery (the type = 'atom' scan) is scoped to the cycle source (opts.sourceId ?? 'default') so same-slug atoms from other sources are never grouped into this source's concepts. Pinned by the #4589 describe in test/cycle/extract-atoms-synthesize-concepts.test.ts and test/cycle/synthesize-concepts-source-scope.test.ts.

Google connector + open-loop engine (key files cluster)

User-facing contracts: docs/guides/google-connect.md (setup + the typed error catalog) and docs/guides/open-loops.md (detection + close semantics). Hosted-relay server design: docs/designs/HOSTED_OAUTH_RELAY.md (the CLI-side seams are frozen in this repo; the server is gbrain.io's build). Agent-facing operation: skills/google-loops/SKILL.md.

  • src/core/creds/vault.ts — the generic credential vault: one home for every outbound credential gbrain holds (Google OAuth today; provider-agnostic by design, providers register under src/core/creds/providers/). Two backends behind one frozen interface: FileVaultBackend (~/.gbrain/credentials.json, 0600, atomic writes — the CLI/self-host default) and the DB-backed EngineVaultBackend shape hosted gbrain.io implements. Custody rules: secrets live ONLY in the vault (never the config plane; sources.config stores a credential-id pointer, mirroring how github sources store an env NAME); list() returns redacted metadata only. No CLI imports — prompts live in src/commands/.

  • src/core/creds/errors.ts — the typed credential error catalog (CredentialError): every connect/refresh failure a user can hit maps to one code with four user-facing fields ({code, problem, cause, fix, doc_url}), rendered two ways — a conversational fix-first one-liner (stderr / [SHOW USER] blocks) and structured JSON (--json envelopes). Single source of truth for the troubleshooting table in docs/guides/google-connect.md — update both together. No CLI imports; hosted reuses it.

  • src/core/creds/providers/google.ts — Google OAuth2 client, BYO + relay-minted; hand-rolled fetch (no googleapis dependency, fetchImpl-injectable). PKCE S256 with access_type=offline&prompt=consent so a refresh token is always minted; client_ref on the vault entry routes refresh (byo → direct against the Google token endpoint with the user's own client; hosted-relay → through the relay). invalid_grant is sub-classified into the catalog: clock skew (local clock vs Google's Date response header), the 7-day Testing-mode expiry (age heuristic on last_refresh_ok_at/connected_at), and plain revocation. GOOGLE_SERVICE_SCOPES is read-only by construction — the connector never writes to Google.

  • src/core/creds/redirect.ts — how the authorization code gets back to us: RedirectStrategy = 'loopback' | 'paste' | 'hosted-callback'. Loopback is an ephemeral 127.0.0.1 listener; paste mode needs NO listener (the fixed PASTE_REDIRECT_URI fails to load, the user pastes the full address-bar URL back) and is auto-selected by sniffHeadless (SSH/WSL/container/no-display). Google's device-code flow is NOT an option — Gmail/Calendar/Contacts scopes are excluded from it; don't re-litigate.

  • src/core/creds/relay-client.ts — the typed CLIENT half of the gbrain.io zero-retention consent relay (createSession → user clicks the consent URL → poll a one-time claim; the relay deletes tokens on first successful claim or at session TTL). Inert unless GBRAIN_OAUTH_RELAY_URL is set (unset = the BYO flow, which always works). test/creds-relay-client.test.ts is the server conformance spec — a server that round-trips those fixtures is compatible.

  • src/core/creds/export.ts — versioned passphrase-encrypted credential bundles (scrypt N=2152^{15}, r=8, p=1 → AES-256-GCM) for machine moves and hosted-upgrade transfer. A bundle carries selected vault entries PLUS the provider client records they depend on (Google refresh tokens are bound to the client that minted them — moving one without the other produces dead tokens). Format frozen here; hosted's import endpoint conforms to it.

  • src/core/google/types.ts — pure data shapes for the google source kind: normalized Gmail/Calendar/People payloads, GoogleSourceConfig (account pointer, services, historyDays, managed dir), and the GoogleSourceState cursor file persisted at <managed dir>/.google-source.json (gmail history id, downward-moving backfill floor, per-service syncTokens). No I/O. Exports DEFAULT_CALENDAR_ID = 'primary' (the Calendar API's own alias; every 'primary' literal in google-clients/google-source/sources/sources-ops resolves through it) and GoogleSourceConfig.calendarId (one calendar per source). GoogleSourceState.calendar_id records which calendar calendar_sync_token was minted for — a token is only valid against its own calendar; absent on legacy state, which is therefore primary's.

  • src/core/google/access.ts — the pluggable Google-access seam: GoogleAccessProvider (getAccessToken/forceRefresh) with CommandAccessProvider (--access command: any CLI that prints a token — gog/gcloud/a gateway's mint command; parsed as a bare token line or JSON {token|access_token, expiry|expires_in}, cached until expiry with a 60s margin, 30s exec timeout, failures = access_command_failed carrying the stderr tail) and EnvAccessProvider (--access env: a live token read from a NAMED env var each call, refreshed outside gbrain; missing = access_env_missing). The vault flow's GoogleTokenProvider satisfies the same interface. The token command executes only in the locally-running sync (google config keys are unreachable over MCP; same trust class as recipe health-check argv).

  • src/core/google/google-clients.ts — hand-rolled Gmail/Calendar/People REST clients (house style, no googleapis dependency): auth via any GoogleAccessProvider (vault-backed GoogleTokenProvider by default), 401 → forceRefresh + single retry, Retry-After honored (delta-seconds AND http-date), 403 accessNotConfigured → api_not_enabled carrying the exact enable deep link (project number extracted from the client id), uniform pageToken pagination with a safety cap, fetchImpl injectable for tests. extractCalendarMethod(part) walks the MIME tree for a text/calendar/application/ics part and reads the iCalendar method= parameter from the PART's own Content-Type header (headers[] on nested parts with format=full — Gmail's MessagePart.mimeType is the BARE media type, so a mimeType-only parse would read '' for every real invite and silently downgrade the structural signal to the subject fallback); a mimeType that still carries params is the fallback parse; a bare .ics filename with a non-calendar MIME type claims nothing. CalendarClient.listEvents({calendarId}) sweeps one calendar (default DEFAULT_CALENDAR_ID).

  • src/core/google/google-render.ts — pure render functions ({relPath, markdown} out; no I/O, no engine): thread pages under emails/YYYY/MM/ (type email), events under calendar/YYYY/MM/ (type meeting), contacts under people/ (type person). Gmail deep links are code-generated via the typed emailCitation scaffold, never LLM-composed. The noise/signature rules recipes/email-to-brain.md specifies as prose for agent-authored collectors are implemented here — keep the recipe and this module in sync. isCalendarSystemMail({calendarMethod, subject}) gates BOTH loop detectors: the PRIMARY signal is a calendarMethod in the CALENDAR_METHODS allowlist (RFC 5546 — REQUEST, REPLY, CANCEL, PUBLISH, COUNTER, DECLINECOUNTER, REFRESH, ADD; case-insensitive); an empty or unrecognised method is NOT a stamp and falls to the subject fallback, which is anchored to the start of the subject, refuses any Re:/Fwd: prefix, and matches only Calendar's OWN localised headers (Invitation:, Updated invitation:, Accepted:, Declined:, Tentative:, Canceled event: + localised equivalents) — a generic word like Notification: is a human/vendor subject and never matches. renderThreadPage stamps participants: (every address on the thread) AND senders: (sorted unique message AUTHORS only — the list loops mute sender gates on, so muting one person cannot silence a whole group thread). Pinned by test/google-render.test.ts.

  • src/core/google/google-source.ts — the google source kind sweep (mirrors github-source.ts: API-backed, materializes markdown under the source's managed dir, flows through the standard import pipeline — chunks, embeds, aliases, links). Sweep order contacts → calendar → gmail: alias rows must exist before the loop detector resolves counterparties. Per-service independent cursors: contacts/calendar syncToken commits only after that service's fully-successful sweep (410 GONE drops the token and re-runs windowed); gmail delta rides history.list with an expired-history windowed fallback; the INITIAL backfill drains newest→oldest with a batch-committed floor cursor so a killed 50k-message backfill resumes at the floor instead of restarting, and the historyId anchor is captured BEFORE the backfill so the delta lane takes over with zero gap (overlap re-renders are idempotent). Emits the sync.google_materialize progress phase (docs/progress-events.md). Access resolves per source config: the vault (default), a token-printing command (g_access: command — gog/gcloud/gateway; identity entry synthesized, scope preflight trusts the configured services, send-as aliases fetched live best-effort), or a named env var (g_access: env). Secrets never land in sources.config (the command/env NAME is config; tokens are not). Honesty invariants: last_sync_at is gated on the GMAIL sweep's success (it feeds gbrain waiting's trust-critical staleness gate, which protects loop freshness — a gmail sweep with real thread failures must not advance it); contacts/calendar failures mark the run partial but do not block the stamp, and a source without gmail in its services stamps unconditionally. A thread whose fetch fails on consecutive sweeps lands in the poison ledger and is skipped (steady-state runs honor the ledger; --full retries with a fresh one), so one bad thread can't wedge the sync while a poison-skip alone doesn't count as failure. Contact reconcile resolves by contact resourceName (DB lookup by contact id FIRST): deletion tombstones carry only resourceName + deleted (no names — slug derivation yields null) and a renamed contact derives a DIFFERENT slug, so both must land on the existing page by id, never by slug. The calendar sweep binds its sync token to its calendar: sweepCalendar compares state.calendar_id (legacy state without it is treated as primary's) with the configured calendarId; on a mismatch it logs [google] calendar changed (<old> → <new>), discards the token and takes the windowed first-sync path (no syncToken on the wire), and a freshly banked token stores the calendar id beside it — pairing a re-pointed source with the previous calendar's cursor would import a foreign delta. Pages from the previous calendar are not reconciled on the switch (logged; a filed follow-up). enqueueLoopsExtraction enqueues EVERY eligible thread (newest first) under a waiting-depth budget LOOPS_EXTRACT_ENQUEUE_CEILING − waiting that counts only THIS source's waiting loops_extract jobs (data->>'sourceId'), so one account's stalled backlog cannot pin another source's budget at 0; overflow is a logged deferral (a deferred thread re-candidates when it changes) and a failed probe fails open. The budget probe counts PENDING rows — status IN ('waiting','delayed','active') — so a retry backlog parked in delayed during a flapping provider cannot stack past the ceiling. With no chat provider available (isAvailable('chat') false) the enqueue is skipped entirely and one stderr line per sweep names the reason and the count; pages still import and the threads re-candidate on their next touch or a --full sweep.

  • src/core/google/loop-detect.ts — the zero-LLM thread-state machine: last substantive message inbound + user in To: + unanswered ≥24h → unanswered_inbound; last outbound + contains a question + unanswered ≥72h → unanswered_outbound; a reply closes the loop (closed_by: reply_detected). Precision IS the product: noise senders, list mail (List-Unsubscribe), CC-only delivery, FYI/forwards without a question, self-threads, and muted senders/threads never open loops — pinned by the labeled fixture corpus in test/google-loop-detect.test.ts (every false-positive class gets a fixture before its fix). Pure verdict function + a thin apply step called per touched thread from the sync; the apply step must never fail the sync.

  • src/core/google/loops-extract.ts — the LLM half of the open-loop engine: ONE extractor per recent thread page projects each commitment into THREE substrates in the same pass — the open_loops row (dedup commit:<sha8>), a facts row via writeSingleFact (kind=commitment, fence-first, deduped; its id lands on open_loops.fact_id so entity cards / recall / context_pack see the commitment through existing read paths with zero new read code), and a typed edge thread-page → person-page (owes_to / awaiting_reply_from) for relational search. extract_facts never runs separately on google-source email pages. Guardrails: injection-hardened input, ALL-or-nothing parse barrier (a malformed model response writes NOTHING), a structural eligibility gate (loopExtractionEligibility — Gmail labels, List-Unsubscribe, calendar part, who wrote the message; no vendor lists) plus the per-source waiting-depth enqueue budget (LOOPS_EXTRACT_ENQUEUE_CEILING, a spend backstop — see google-source.ts), only the last LOOPS_EXTRACT_WINDOW_DAYS (30) of mail (the deep backfill is never extracted), kill switch loops.extraction_enabled (default ON for google sources). Suppressions gate the lane before any model/facts/edge write: a muted thread id, or a muted SENDER — fm.fromfm.senders, every address that AUTHORED a message, so a muted counterparty who wrote earlier in the thread still suppresses — returns reason: 'suppressed'; recipients/CC never count (muting one person must not hide everyone else's commitments in a group thread, and an outside sender must not be able to dodge extraction by CC'ing a muted address). Pages without a senders: field fall back to fm.from alone until their next re-render. Eligibility computes the substantive (non-noise, non-calendar) message set FIRST and owner_participated requires the owner to have written a SUBSTANTIVE message — a pure-calendar thread the owner RSVP'd to stays no_substantive_messages. When the chat provider is unavailable, runLoopsExtract throws LoopsExtractRetryableError (reason llm_unavailable) rather than returning a skipped result: a completed no-work row would hold the revision-keyed idempotency slot forever, while a retried-then-dead row frees it so the same revision re-enqueues on the next sweep.

  • src/core/loops/loops-store.ts — SQL accessors for the open_loops + loop_suppressions tables over engine.executeRaw with IDENTICAL SQL text on both engines (parity by construction, the sources-ops.ts pattern — no per-engine method twins). JSONB discipline: evidence binds through $N::text::jsonb, never a bare ::jsonb cast over JSON.stringify. Loops close by state transition, never delete: reply-driven auto-close flips status to done and stamps closed_by, keeping the audit trail. The upsert's DO UPDATE carries a manual-close guard in its WHERE: a closed row (done/dropped/stale) only reopens on GENUINELY newer activity, so a routine sweep re-seeing the same thread never resurrects a hand-closed loop, and the staleness auto-close (closed_by: 'staleness') is likewise guarded against the upsert's reopen. removeSuppression(engine, sourceId, kind, value) is the exact inverse of the mute write — deletes the one (source_id, kind, value) row with the same lower-casing, returns {removed: boolean} (false = was not muted, callers treat it as a no-op success), and never reopens loops: suppressions gate NEW detection only.

  • src/core/ops/loops.ts — the op surface: open_loops (read), loops_close (write), loops_mute (write). open_loops is deliberately NOT localOnly (hosted serves it over HTTP to the authenticated owner); instead it applies fail-closed evidence redaction for ctx.remote !== false — counts, counterparty, summary, due date only; verbatim quotes, Gmail deep links, and the injectable text digest are trusted-local only. The result carries the google sources' last-successful-sync ages + a stale flag (>24h) so callers can refuse stale-but-confident output on a trust-critical surface. Per-call scope params source_id/all_sources resolve through resolveRequestedScope (an MCP client bound to another source can reach the google source's loops; remote callers stay in-grant, out-of-grant source_id is denied); a scope with NO google source returns no_google_sources: true and the digest says the engine has nothing to read instead of a false "You are clean". Scope is fail-closed too: an UNSCOPED remote open_loops read is refused outright (the op must not rely on transports to enforce it — an unscoped remote read would span every source), and the write ops (loops_close, loops_mute, loops_unmute) require a single-source remote scope that matches the caller's grants — trusting a caller-supplied source_id for a remote write would let any remote client plant (or lift) suppression rows cross-source; loops_unmute reverses a suppression via removeSuppression and reports {removed} honestly.

  • src/commands/google.tsgbrain google connect|status|calendars|disconnect (+ setup dispatch). Agent-first contract: every subcommand supports --json emitting { ok, status, next_action: { command?, user_message? }, error? } (calendars adds account + calendars[] — id/summary/primary/accessRole — and carries the sources add --calendar-id template in next_action.command; it resolves a lone connected account, exits 2 when there are none or several without --account, and throws not_connected for an unknown --account); human copy the harness must relay verbatim is fenced in [SHOW USER] ... [/SHOW USER] blocks; secret intake is --client-json <path|-> (preferred) / env GOOGLE_CLIENT_ID+GOOGLE_CLIENT_SECRET / TTY prompt, with raw argv flags accepted but documented last-resort; connect is an idempotent state machine (detects what exists, performs only the missing step — re-running is the documented fix for most errors). Engine-free except status's best-effort linked-sources listing (degrades without an engine). Funnel events append to ~/.gbrain/integrations/google/heartbeat.jsonl; account addresses are hashed to a short non-reversible tag, never raw. The vault entry's meta.scopes records what Google ACTUALLY granted (the token response's scope — the consent screen lets users uncheck scopes; the requested set is only the fallback), which is what lets downstream preflights report scope_missing instead of opaque per-sweep 403s.

  • src/commands/creds.tsgbrain creds list|remove|export|import: the provider-agnostic vault surface; never prints a secret (list is always redacted). Provider-SPECIFIC connect flows live in their own commands. Export custody: a loud per-credential warning when a byo Google entry's consent screen is not known published-to-Production (its 7-day Testing expiry travels with the tokens).

  • src/commands/loops.tsgbrain waiting [--top N] [--json] [--stale-ok] + gbrain loops list|show|done|drop|mute: all paths dispatch through the trusted-local op layer (handleToolCall, remote:false) so CLI and MCP share one behavior. waiting REFUSES when EVERY google source has gone >24h without a successful sync and prints the exact fix (--stale-ok bypasses; per-source sync ages are always reported) — stale-but-confident output is worse than none. Output per counterparty: what's owed, evidence quotes, Gmail deep links, entity-card context, a paste-ready digest. Reads default to the __all__ brain span (loops live in google sources, not default — a default-scoped read would say "all clean" while people wait); --source <id> narrows explicitly. An unqualified mute/unmute resolves the brain's google source (never default) through the SAME resolution — an unmute can never aim at a different source than the mute it reverses — and refuses with the exact fix when none or multiple exist. unmute sender <email> | thread <id> is exact and forward-only; a repeated unmute is a no-op success (removed: false, exit 0) so scripts can call it unconditionally.

  • src/commands/google-setup.ts + google-setup-tail.ts — the one-command orchestrator behind gbrain google setup: connect (skipped when tokens exist) → source registration (skipped when registered) → first bounded sync under a wall-clock budget (the newest-first backfill floor means whatever lands is the NEWEST mail — exactly what waiting needs; the remainder resumes on every later sync, and setup says so honestly) → the first gbrain waiting digest in the same session as consent. Split in two so the connect half stays engine-free. Every step idempotent; re-running resumes wherever the last run stopped.

  • src/commands/doctor/checks/google-oauth.ts — the google_oauth doctor check: zero-network vault health (live refresh probes belong to gbrain google status; doctor stays fast and offline-safe). fail: a connected account with an expired access token AND no successful refresh in >2 days (refresh is broken — revoked, rotated client, or the Testing-mode expiry already hit); warn: a consent screen not known Production whose last proof of life is ≥5 days old (the proactive day-6 re-auth demand, cheaper than a dead pipeline on day 8); ok: accounts healthy or nothing connected (the connector is optional, not an error).

  • src/core/agent-install/ + scripts/setup-in-agent.sh — Package-versioned non-root installation into a verified persistent root. Absolute launchers isolate ambient routing and credentials, receipts track ownership, pending enablement, non-secret failure codes, and an exact repair command, and repair retains database contents. Upgrades journal pending schema migrations before switching runtime metadata; retries use schema-only initialization without invoking host orchestration. Busy results remain structured and retryable without deleting live locks. Native instructions/routines remain pending until observed in the actual harness.

  • src/core/backup/archive.ts + src/core/backup/snapshot.ts — Private bounded full-PGLite snapshots with file inventories, checksums, credential-reference exclusions and fresh-root restoration. Hold the database lock, detect changing inventories, validate private staging before publication, rebase managed paths, detach external roots, and quarantine nonterminal restored jobs without deleting spending evidence. Never replace the source installation.

  • src/core/harness/registry.ts — Canonical adapter identities, supported transports, credential-delivery mechanisms, guide links and dated evidence; legacy connection/register vocabularies derive their facts here. scripts/build-harness-docs.ts emits the user-facing adapter reference.

  • src/core/harness/credentials.ts + delivery.ts + install.ts — Private credential handoff and host delivery journal, ownership-preserving native configuration and isolated thin launchers. A committed active client and matching current secret are prerequisites for recovery. Configuration removal and server revocation are separate actions.

  • src/core/harness/capabilities.ts + src/mcp/capabilities.ts — Shared effective-grant orientation for OAuth whoami and authenticated gbrain://capabilities. The resource preserves the exact seven-tool surface; source, operation, scope, fence and server-gate restrictions remain enforced at dispatch. No worker probe or per-tool database query during discovery.

  • src/core/harness/verify.ts — Randomized server-side memory verification with separate transport, identity, authority, read, write/readback, delegation and cleanup verdicts. Lost mutation responses are reconciled before retry; server success never certifies native-harness activation or cross-conversation behavior.

  • src/commands/mcp.ts + mcp-provision.ts + harness-connect.ts — Host-side grant/preview/CAS and recoverable private delivery, followed by installation inside the intended harness. Running PGLite hosts are administered through their authenticated API, never by opening a second database process. CLI status uses the delivery-aware exit verdict.