Key files
September 11, 2026 · View on GitHub
On-demand reference. CLAUDE.md (the always-loaded orientation file) routes here via its Reference map. Read a file's entry before editing that file.
Entries describe CURRENT behavior + load-bearing invariants only. Release history
lives in CHANGELOG.md + git log / git blame, NOT here. Do not append
per-release **vX.Y.Z:** narration — CI enforces this
(scripts/check-key-files-current-state.sh).
-
src/core/runtime-version.ts— Bun 1.3.11 minimum enforced by the CLI entrypoint and guarded HTTP transport, with upgrade/restart guidance. Package engines and runtime checks share the same supported floor. -
src/core/data-frontmatter.ts— shared data-only YAML/JSON parsing and serialization. Rejects unsupported language selectors before parsing, uses js-yaml safe schema with existing scalar types, sanitizes parser errors, and keeps body text opaque. Used by markdown, capture, recipes, plugins, and greenfield ingestion. Independent fixtures:test/data-frontmatter.test.ts,test/frontmatter-security.test.ts. -
src/core/oauth-grants.ts— client-row-locked grant transactions shared by OAuth issuance, code/refresh consumption, and revocation. Preserves scope/resource ceilings and rolls back consumption with replacement writes. Bounded pending consent requests snapshot current operation and delegation policies.src/commands/serve-http-oauth.tsauthenticates request details and enforces session-bound CSRF decisions;admin/src/pages/OAuthConsent.tsxrenders the existing admin login/approval flow without exposing protected details in the page shell. -
src/core/minions/submission-authority.ts— versioned authority kept outside caller job data. Submission normalization and worker revalidation bind remote jobs to immutable principal/grant/source/payload ceilings; application authority preserves trusted internal work.src/core/minions/authorize-legacy.tsprovides explicit local preview and transactional snapshot comparison for selected historical rows.src/core/minions/source-filesystem.tsprovides reentrant canonical root/worktree locking and descriptor-based confined reads/writes. Both worker modes enforce the same authority gate. Upgrade protocol:docs/guides/authorization-upgrade.md. -
src/core/guarded-http.ts+src/core/ssrf-validate.ts— DNS answer classification, validated-address connections with original TLS/HTTP hostname and port, shared deadline/redirect policy, header-only probes, and bounded gzip/deflate/Brotli decoding. Ambient proxies fail closed with restart guidance. Consumers: URL reachability, remote images, and HTTP integration checks. Real TLS fixtures run on Bun 1.3.11 and 1.3.13. -
docs/operations/conversation-parser-llm-fallback.md— operator and maintainer contract for the default-off LLM parse fallback: exact config key, deterministic-first dispatch boundary, sampled data surface, untrusted-content prompt handling, page-date/cache-key coupling, timestamp validation, cache/checkpoint behavior, observability, limitations, and focused test commands. -
src/core/operations.ts— Contract-first operation contract, served through a ~300-line assembly façade: the op definitions live in the domain modules undersrc/core/ops/(next entry) and are spread into the single exportedoperationsarray here; the shared contract types (src/core/ops/contract.ts) and the security/scope layer (src/core/ops/context.ts) are re-exported through this file, so every import path below resolves through the façade unchanged. The rest of this entry describes the surface as consumers see it. Same-source graph mutations (add_link,add_timeline_entry) preflight each endpoint against the scalar write source viarequireWritablePage(ops/context.ts): a page readable only from another granted source returns endpoint-specificpermission_deniednaming the boundary, pages outside the caller'sget_pagevisibility scope staypage_not_found(a soft-deleted foreign page is indistinguishable from absence), and a mutation-time engine miss — the typedPageMissingErrorfromsrc/core/engine-errors.ts, thrown by both engines' single-statement endpoint resolution — is reclassified into the same envelope byinstanceof, never message matching. Exports upload validatorsvalidateUploadPath,validatePageSlug,validateFilename, plusmatchesSlugAllowList(slug, prefixes)(glob matcher:<prefix>/*matches recursive children; bare<prefix>matches exact only).OperationContext.remoteis a REQUIRED field flagging untrusted callers;OperationContext.allowedSlugPrefixesis the trusted-workspace allow-list set by the dream cycle;OperationContext.auth?: AuthInfois threaded through HTTP dispatch for scope enforcement inserve-http.tsbefore the op runs. OAuthwhoamiexposes the authenticatedAuthInfo.sourceIdandAuthInfo.allowedSourcesgrants assource_idandfederated_read; absent grants serialize fail-closed asnulland[], while local, legacy, and stdio response shapes stay unchanged.enforceSubagentSlugFence(ctx, slug, opName)is the shared fail-closed subagent write fence: whenviaSubagentandallowedSlugPrefixesis set, the slug must match the allow-list; else the legacywiki/agents/<id>/...namespace check applies. Bothput_pageandadd_timeline_entry(subagent-allowlisted) route through it. Auto-link/timeline hooks are skipped for remote-owned subagents even with explicit prefixes; only local subagents qualify as trusted workspaces.enforceClientSlugFence(ctx, slug, opName)is the OAuth-client write fence: whenctx.auth.boundSlugPrefixesis present (threaded fromoauth_clients.bound_slug_prefixesat token-verification time), every direct slug-mutating write op —put_page,delete_page,restore_page,add_tag,remove_tag,add_link/remove_link(fromendpoint only; linking TO a readable page is a reference),add_timeline_entry,revert_version,put_raw_data— rejects out-of-prefix slugs withpermission_denied, BEFORE each op's dry-run short-circuit. Direct writes use the shared boundary-aware prefix rules; empty-array binding is deny-all (fail-closed); no auth / no binding = no fence. The match rule itself lives in the exportedslugUnderBoundPrefixes(prefixes, slug)so non-op write surfaces reuse it verbatim. It is BOUNDARY-AWARE (a prefix matches whole segments, soemp-alicedoes not admitemp-alice-2/…), lowercases both sides (stored slugs are lowercased byvalidateSlug, so comparing the caller's raw string would let a mixed-case slug commit and only then trip the resolved-slug re-check), accepts BOTH the trailing-slash and the<prefix>/*glob spelling vianormalizeSlugPrefix(direct and delegated fences share normalization while storing independently reviewed spans), and ignores empty-string prefixes.assertValidSlugPrefixes(oauth-provider.ts) rejects empty, whitespace-bearing, non-lowercase, and boundary-less entries at registration and rescope.submit_agentvalidates tools and namespaces against the independently storedbound_tools,delegated_slug_prefixes, anddelegated_namespacegrants. It normalizes accepted prefixes formatchesSlugAllowListand rejects explicit empty remote tool or prefix overrides. Workers interpret explicit empty tool lists as no tools, with defaults reserved for absent trusted-local bindings.put_pageadditionally fences the RESOLVED slug when importFromContent's dedup pre-check redirects the write to a different page (same content_hash /frontmatter.id), since the disk write-through runs against that slug. That re-check applies whichever confinement the CALLER is under — OAuth binding and/or subagent allow-list/legacy namespace — viaslugOutsideCallerFence(ctx, slug), which composesslugUnderBoundPrefixeswith the subagent fence's own match rule: the delegatedsubmit_agent→ subagent context carriesviaSubagent+allowedSlugPrefixeswhile itsauthcarries current read scope without the parent's direct-write fence, so an auth-only check would let a slug-bound client holdingagentscope reach an out-of-fence page simply by delegating the write. Denials never name the resolved slug (it would be a slug-enumeration oracle). Pinned bytest/put-page-dedup-fence.test.ts.CLIENT_FENCED_WRITE_OPS+enforceBoundClientOpAllowList(auth, op)are the fail-closed companion, applied once insrc/mcp/dispatch.ts(the choke point both MCP transports share): a slug-bound client calling ANYwrite/adminop not on the allow-list getspermission_denied. This covers the ops that write by a key other than a slug and therefore cannot be fenced —extract_entities/extract_facts(mutatepeople/*,companies/*),forget_fact(numeric fact id, crosses sources),ontology_propose— and makes a write op added later denied-by-default instead of silently unfenced.thinkis on the allow-list because remote callers cannot persist from it. Pinned bytest/client-slug-fence.test.tsand over-the-wire bytest/e2e/qm-provisioning.test.ts. EveryOperationcarriesscope?: 'read' | 'write' | 'admin'+localOnly?: boolean;thinkis read-scoped for OAuth/MCP because remote callers havesave/takeforced off before persistence, while local CLI can still persist viaremote:false;sync_brain,file_upload,file_list,file_urlareadmin + localOnly(rejected over HTTP). Four trust-boundary call sites (put_pageallowlist, file_upload trust-narrowing, submit_job protected-name guard, auto-link skip) use FAIL-CLOSED semantics:ctx.remote === falsefor trusted-only sites,ctx.remote !== falsefor "untrust unless explicit-false" — anything not strictlyfalseis treated as remote (so a read+write OAuth token over HTTP MCP cannot submitshelljobs).sourceScopeOpts(ctx)encodes the source-scoped read precedence ladder — federated array (ctx.auth.allowedSources) wins over scalar (ctx.sourceId/ctx.auth.sourceId) over nothing; every read-side op handler routes through it so a source-bound OAuth client can't see neighboring sources viasearch/query/list_pages/get_page/find_experts/query's image path, plus the by-slug readsget_tags/get_links/get_backlinks/get_timeline/get_chunks(chunks follow the same ladder asget_page, so a federated grant that can open a page can read its chunks — and the chunk payload never carries embedding vectors) (andget_page's tag fetch, which resolves against the concrete page's ownsource_id).assertExplicitSourceLive(ctx, sourceIdParam)is the async companion for the ops that accept a per-callsource_id(get_page,list_pages,search,query): called right afterfederatedSearchScope(so the grant check has already run and it can only name a granted source), an explicit id with no live unarchivedsourcesrow throwsunknown_sourceinstead of silently scoping the read to an empty source — the CLI's--sourceexistence rule applied to the op path;__all__and an omitted param skip it.linkReadScopeOpts(ctx)is the link-read sibling forget_links/get_backlinks: a link row references three pages (from/to/origin), and the engine's federated (sourceIds[]) branch scopes ALL THREE while its scalar (sourceId) branch scopes only the near endpoint (by design — trusted internal callers likereconcileLinksand back-link validators call the engine with a scalar scope and need the cross-source view). For an UNTRUSTED remote caller (ctx.remote !== false) carrying only a scalar scope it promotes that scope to a single-elementsourceIds:[id], routing them through the all-endpoint branch so a far/origin slug from outside the grant isn't disclosed; a federated array passes through unchanged and trusted local CLI (ctx.remote === false) keeps the scalar cross-source view.thinkSourceScopeOpts(ctx)maps the same precedence ladder ontorunThink's public options (allowedSources/sourceId) so thethinkop's gather and trajectory stages inherit the caller's source grant.put_page's inline disk write-through is the sharedwritePageThroughhelper (src/core/write-through.ts), ATOMIC via temp-sibling + rename so a crash or concurrentgbrain synccan't read a half-written.md; same helper backsgbrain brainstorm/lsd --save. Link provenance surface:add_link(gbrain link/link-add) +remove_link(gbrain unlink/link-rm) exposelink_source/link_type;add_linkrejects the reconciliation-managed built-ins viaMANAGED_LINK_SOURCES(markdown/frontmatter/mentions/wikilink-resolved) and defaults omitted provenance to'manual'(the engine's own default stays'markdown'for internal callers);list_link_sources(gbrain link-sources, read) lists provenances viasourceScopeOpts. CLI aliases register throughcliHints.aliases(collision-guarded insrc/cli.ts). -
src/core/ops/— the operations contract's module directory (the meat behind theoperations.tsfaçade).contract.tsis the foundation contract: the error envelope (ErrorCode/OperationError/verbError), the shared param/logger/auth/context types, and theOperationinterface — re-exported wholesale by the façade.context.tscarries the context validators + scope resolvers: the upload/slug/filename validators, the subagent and bound-client slug fences, and the source-scope resolution ladder (some internal helpers are exported only here and deliberately NOT re-exported from the façade — import those fromcontext.tsdirectly). The remaining modules are one-per-op-domain (pages.ts,search.ts,takes.ts,tags.ts,links.ts,timeline.ts,admin.ts,skills-catalog.ts,sync-status.ts,raw-data.ts,chunks.ts,ingest-log.ts,files.ts,jobs.ts,orphans.ts,calibration.ts,salience.ts,facts.ts,sources.ts,transcripts.ts,insights.ts,image.ts,extraction.ts,code-intel.ts,chronicle.ts,embedding-migration.ts,request-tools.ts,schema-packs.ts,skillopt.ts), each exporting a<domain>Operationsmap that the façade spreads into the singleoperationsexport. Add a new op in the matching domain module — an existing domain needs no façade change; a brand-new domain module gets one spread line inoperations.ts.facadeExpansioninscripts/generate-flag-registry.tsmaps the façade to this whole directory so every module's--flagtext stays on the command flag-scan surface. -
src/core/ops/links.ts— the link-domain op module (add_link,remove_link,get_links,get_backlinks,list_link_sources,traverse_graph). Two load-bearing behaviors. (1) Pack vocabulary at the write surface: an EXPLICITlink_typeonadd_linkmust be declared by the active schema pack when one resolves (schema-pack/write-vocabulary.ts; no resolvable pack = no enforcement; omitted/empty link_type is the untyped-edge default and stays unchecked; the check runs before the dry-run return so dry runs preview the rejection). (2)traverse_graphoutput shape: trusted local no-filter callers (ctx.remote === false, nolink_type/direction) keep the legacyGraphNode[]shape thatgbrain graphrenders; remote callers default todirection: 'both'and always receive explicitGraphPath[]edges, so a page whose typed edges are all inbound does not read as edge absence. An explicitdirectionparam wins for every caller. Depth: default 5 (DEFAULT_TRAVERSE_DEPTH), except a remote call that let direction default tobothALSO defaults depth toREMOTE_BIDIRECTIONAL_DEFAULT_DEPTH(2) — bidirectional path enumeration is combinatorial on entity hubs and this is the per-agent-turn path; alink_type-only remote call still takes both/2; an explicit depth is honored up toTRAVERSE_DEPTH_CAP(10, clamped with a warn). The edge walk runs throughengine.traversePathsDetailed, and a hit onTRAVERSE_PATH_ROW_CAP(engine-constants.ts) surfaces as a stderr warn naming the cap (shallowest edges kept) — theGraphPath[]wire shape stays unchanged. Pinned bytest/traverse-graph-op-default.test.ts+test/traverse-paths-row-cap.test.ts. -
src/core/ops/pages.ts— the page-domain op module (get_page,put_page,delete_page/restore_page,capture, …; the source-scoping contract of the delete/restore trio is described undersrc/core/destructive-guard.ts).get_pageresolution ladder: exact read in the caller's scope → alias hop → fuzzy. The alias hop scopes its lookup to the federated grant (sourceIds[]) > the scalarsourceId> (trusted unscoped only) every LIVE source — archived sources' alias rows count only wheninclude_deletedasks for archived material — throughengine.resolveSlugWithAliasDetailed, then reads the canonical page IN THE SOURCE THAT OWNS THE ALIAS ROW (getPage(canonical, { sourceId: hit.source_id })): a federatedgetPageprefers the anchor source, so an unrelated live page at the canonical slug in another granted source would otherwise shadow the alias owner's page. The redirect target composes with the same private-page gate as the exact read and reportsresolved_slug.capture: an EXPLICIT type (thetypeparam, else a frontmattertype:in the content —explicitCaptureTypeinsrc/core/capture-content.ts, BOM/CRLF tolerant, malformed YAML → none) is validated against the active pack when one resolves (schema-pack/write-vocabulary.ts) and IS the effective type for both the default slug and the merged frontmatter — approved as X, stored as X; no explicit type stampsnote(never checked). Sits at its module-size ratchet ceiling (scripts/module-size-limits.tsv); the next growth peels a submodule. Pinned bytest/get-page-federated-scope.test.ts,test/capture-explicit-type.test.ts, and the engine-parity suite (both engines pick the same owning row). The alias hop has no catch of its own: both engines returnnullfor the pre-v104 missing-table case, so any other rejection (connection reset, timeout) propagates instead of degrading topage_not_found. -
src/core/ops/admin.ts— the admin/diagnostics domain module (get_stats,get_health,get_brain_identity, …). All three aggregate ops confine remote callers via the same ladder reads use:ctx.remote === falsekeeps the trusted brain-wide view; every other caller getssourceScopeOpts(ctx)(federated array > scalar > the unmatchable__all__sentinel, which fail-closes to zeros) threaded intoengine.getStats(scope)/getHealth(scope)— aggregates leak by subtraction, so they scope exactly like reads, and read-scopeget_brain_identityis confined the same way.get_health'smigrations {pending, partial, wedged, skipped_future}block stays GLOBAL for scoped callers by decision: it is a host filesystem ledger with no per-source semantics, a wedged host migration is exactly what a remote agent needs to see to explain degraded behavior, and it is composed at the op layer (not inBrainEngine.getHealth— growing the engine interface would force both engines to duplicate a file read). -
src/core/ops/search.ts— the search-domain op module (search,query,search_stats,search_modes,search_tune,cache_stats).queryis the hybrid entry (hybridSearchCached) plus two non-hybrid legs that share ONE effective-row-contract helper,resolveEffectiveLimit(ctx, p): an explicitlimitwins, otherwise the mode-derivedsearchLimitresolved through the same trust-gated chainhybridSearchuses (resolvePerCallModeignores a remote caller'smode, so a remote client cannot select the tokenmax row count); the config reads run lazily on those paths only. The legs: the image-similarity branch (embedding_imagevector search,test/query-image-mode-limit.serial.test.ts) and the CRAG escalation slice — whensearch.crag_escalationis on andshouldEscalateRetrieval(src/core/search/crag.ts) says the first pass gradedweak, was not already escalated, AND did not already run with the caller's expansion on (callerExpanded), the op re-runs once atlimit: max(effectiveLimit, 50)with expansion + relational on and autocut off, keeping the better-graded run. -
src/core/engine.ts— Pluggable engine interface (BrainEngine).clampSearchLimit(limit, default, cap)takes an explicit cap so per-operation caps can be tighter thanMAX_SEARCH_LIMIT. ExportsLinkBatchInput/TimelineBatchInputfor the bulk-insert API (addLinksBatch/addTimelineEntriesBatch).readonly kind: 'postgres' | 'pglite'discriminator letssrc/core/migrate.tsand others branch withoutinstanceof+ dynamic imports. Methods:batchLoadEmotionalInputs(slugs?)(CTE-shaped read with per-table aggregates so page × N tags × M takes never produces N×M rows),setEmotionalWeightBatch(rows)(UPDATE FROM unnest(\$1::text[],\$2::text[],\$3::real[])composite-keyed on(slug, source_id)),getRecentSalience(opts),findAnomalies(opts).PageFiltershassort?: 'updated_desc'|'updated_asc'|'created_desc'|'slug'+PAGE_SORT_SQLwhitelist consumed by both engines.listAllPageRefs(): Promise<Array<{slug, source_id}>>ordered by(source_id, slug)— cheap cross-source enumeration instead of agetAllSlugs()→getPage(slug)N+1 (which would silently default tosource_id='default'); parity across postgres-engine.ts + pglite-engine.ts; Pinned bytest/e2e/multi-source-bug-class.test.ts.SearchOpts+PageFiltersaddsourceIds?: string[](federated read axis; both engines applyWHERE source_id = ANY($N::text[])when set, preserve scalarsourceIdfast path when unset);traverseGraph(slug, depth, opts?)andtraversePaths(slug, opts?)acceptopts.sourceId/opts.sourceIds.traversePathsDetailed(slug, opts?)returns{ paths: GraphPath[], truncated }— the final SELECT of the path-enumerating recursive CTE is bounded atTRAVERSE_PATH_ROW_CAP + 1rows on both engines (the probe row signals overflow;ORDER BY depthmeans the deepest edges are dropped, so a truncated result is still the complete shallow neighbourhood; the cap counts raw rows before the in-memory edge dedup, sopaths.lengthalone cannot reveal truncation) andtraversePathsis its.pathsprojection.resolveSlugWithAliasDetailed(slug, sourceOrSources)returns{ canonical_slug, source_id } | null— the winning alias row AND its owning source under the same scope/precedence rules (ORDER BY array_position(scope, source_id), id; null when nothing matches or theslug_aliasestable does not exist yet);resolveSlugWithAliasis its canonical-slug projection (falls back to the input slug). A consumer that goes on to read the canonical page must scope that read tosource_id. The by-slug read methods carry the same federated axis:getTags/getLinks/getBacklinks/getChunksopts andTimelineOpts(consumed bygetTimeline) acceptsourceIds?: string[]taking precedence over the scalarsourceId(source_id = ANY($::text[])scoping the slug→page-id lookup);getChunksfalls back to the'default'source when neither is set (importCodeFile's incremental-embedding reuse relies on it) and SELECTs an explicit non-vector column list — embedding vectors never ride the payload since rowToChunk discards them (getChunksWithEmbeddingsstays scalar-only by design: engine-internal, zero remote-reachable callers); the link reads (getLinks/getBacklinks) scope ALL THREE endpoints (from/to/origin) on the federated branch while the scalar branch scopes only the near endpoint for trusted internal cross-source callers.traverseGraphopts hasfrontierCap?: number(per-iteration recursive-CTE cap, approx per-BFS-layer); return typePromise<GraphNode[]>for MCP wire stability; exportTraverseGraphOpts; Postgres uses parenthesizedLIMIT N ORDER BY (slug, id)inside the recursive term, PGLite mirrors with positional params; Pinned bytest/regressions/v0_36_frontier_cap.test.ts. Phantom-redirect methods:refreshPageBody(slug, sourceId, compiled_truth, timeline, content_hash)narrow-UPDATEs three columns + updated_at, skipping soft-deleted rows (content_hash refresh required sogbrain syncsees the canonical as unchanged after fence merge);migrateFactsToCanonical(phantomSlug, canonicalSlug, sourceId)UPDATEsentity_slug+source_markdown_slugon every active fact row keyed on the phantom, preserving embedding/validUntil/kind/status/source_session/confidence; parity attest/phantom-redirect-engine-parity.test.ts.getAdjacencyBoosts(pageIds): Promise<Map<number, AdjacencyRow>>powers the per-query graph-signals stage — one SQL query returning inbound-link counts among top-K plus a cross-source count (links from differingsource_id);COALESCE(p.source_id,'default')null safety,HAVING >= 1, cross-source CASE-WHEN excludes the target's own source; parity SQL across both engines;SearchResultgains optionalbase_score,backlink_boost,salience_boost,recency_boost,exact_match_boost,graph_adjacency_boost,graph_cross_source_boost,session_demote_factor,reranker_delta+ internal staging fields; Pinned bytest/e2e/graph-signals-engine.test.ts. Two REQUIRED methods:deletePages(slugs, {sourceId}): Promise<string[]>(single-batch primitive returning slugs actually deleted) andresolveSlugsByPaths(paths, {sourceId}): Promise<Map<path,slug>>(batch path→slug lookup);sourceIdREQUIRED on both at the type level (asymmetric with single-rowdeletePagewhich keeps optional/'default'); both short-circuit on empty input and throw when> DELETE_BATCH_SIZE.softDeletePages(slugs, {sourceId}): Promise<string[]>is the batch soft-delete twin, mirroringdeletePages' contract exactly (requiredsourceId,> DELETE_BATCH_SIZEthrows, empty input short-circuits, caller owns chunking + decompose-to-one-element-batches on failure): oneUPDATE … SET deleted_at = now() … AND deleted_at IS NULL … RETURNING sluground-trip returning only the slugs that actually flipped active→soft-deleted — thedeleted_at IS NULLpredicate is load-bearing so a re-run never refreshes an already-soft-deleted row's 72h purge clock — and NOTHING cascades (chunks/links/timeline stay behind the read-sidedeleted_atfilters; the autopilot purge phase hard-deletes after 72h; a re-import within the window revives viaputPage's upsert;deletePage/deletePagesremain the purge/teardown primitives).getStats(opts?)/getHealth(opts?)take an optional{sourceId?, sourceIds?}scope (same shape assourceScopeOptsoutput): omitted = brain-wide; scoped = EVERY counter/coverage/degree confines to the grant, including derived-table counts via their page joins, and link-derived numbers count only edges with BOTH endpoints in scope — so an excluded source's numbers can't be recovered by subtraction. Consumer is the remote-caller ladder insrc/core/ops/admin.ts. Embedding-signature stale-detection quartet:countStaleChunks(opts?)gains optionalsignature?: stringwidening the stale predicate fromembedding IS NULLto ALSO include chunks whose JOINed pageembedding_signature IS NOT NULL AND <> $signature(NULL signature is GRANDFATHERED, never counted; omitsignaturefor the legacy NULL-only count);sumStaleChunkChars(opts?: {sourceId?, signature?}): Promise<number>=SUM(LENGTH(chunk_text))over stale chunks (same predicate + embed_skip filter + optional sourceId scope), used bygbrain sync --allcost preview viaestimateCostFromChars;setPageEmbeddingSignature(slug, {sourceId?, signature})stampspages.embedding_signatureafter a page's chunks (re)embed, idempotent no-op when page absent;invalidateStaleSignatureEmbeddings({signature, sourceId?}): Promise<number>NULLsembedding+embedded_aton every chunk whose page signature is set AND differs, returning the count, called BEFORElistStaleChunksso signature-drift pages flow through the NULL-embedding keyset cursor unchanged (NULL never invalidated). WidensfindOrphanPages(opts?: {sourceId?, sourceIds?})(candidate-side scoping only; inbound links counted from any source). Pinned bytest/sum-stale-chunk-chars.test.ts,test/embedding-signature-stale.test.ts,test/e2e/engine-parity.test.ts. Free-text alias layer:resolveAliases(aliasNorms, opts?): Promise<Map<string, Array<{slug, source_id}>>>(READ; maps each normalized alias to declaring(slug, source_id)pairs, source-scoped) andsetPageAliases(slug, sourceId, aliasNorms)(WRITE; replaces the full alias set, delete-then-insert, empty clears, idempotent on the unique triple), called by theimportFromContentingest projection and thereindex --aliasesbackfill; parity across both engines, Pinned bytest/search/page-aliases-engine.test.ts.searchVectorin both engines injects the sharedbuildBestPerPagePoolCteper-page max-pool so a page surfaces on its strongest chunk.executeRawDirect(sql, params?, opts?)is the lock-hot-path sibling ofexecuteRaw: same single-statement contract, but routes to the direct session-mode pool when dual-pool is active (Postgres/Supabase port 5432) so a long-held lock heartbeat survives the transaction pooler's per-transaction connection recycling; PGLite delegates straight toexecuteRaw(no pooler). Both engines implement it; the Minion lock path (claim/renewLock) is the consumer.reconnect(ctx?: {error?})is a REQUIRED lifecycle method on BOTH engines: it recovers a dropped connection using the config captured at the lastconnect(), so callers (autopilot health probe,batchRetry) neverdisconnect()+ bareconnect()(which loses the config and throwsdatabase_url undefinedforever, and opens a null-connection window). PostgresEngine rebuilds its pool with a_reconnectingreentrancy guard and emits pool-recovery audit; PGLite is single-writer in-process so it just restores the saved data dir for parity. Plus two interface members: (1) optionalfindDuplicatePage?(sourceId, {hash, frontmatterId?}): Promise<{slug, id} | null>(identity precedence is content_hash OR frontmatter->>'id', both withdeleted_at IS NULL); (2)resolveSlugs(partial, opts?)extended with{sourceId?, sourceIds?}so the MCP fuzzyget_pagepath scopes by source (field names matchsourceScopeOpts(ctx)output so handlers spread directly; no opts gives the unscoped behavior). Plus a stable tiebreakerORDER BY score DESC, page_id ASC, chunk_id ASCinsearchVectorin both engines: on a score tie (basis-vector eval fixtures) olderpage_idwins, so a new index onpagescannot flip ranking on tied scores. -
src/core/engine-constants.ts— single source of truth for engine batch-sizing constants. ExportsDELETE_BATCH_SIZE = 500consumed by both engines'deletePages+resolveSlugsByPathsand by the sync delete + rename loops. Lives outsideengine.ts(the interface module) to avoid circular-import worry — bounded per-statement work for predictable lock hold time + write amplification. Also exportsTRAVERSE_PATH_ROW_CAP = 5000, the raw-row bound ontraversePaths/traversePathsDetailed(both enginesLIMIT CAP + 1; the extra row is the truncation probe; counted before edge dedup). -
src/core/background-work.ts— process background-work registry: the single owner of "drain every fire-and-forget DB-write sink before the engine goes away," with TWO exit points.registerBackgroundWorkDrainer({name, order, drain(timeoutMs, mode), abort?})over aMap<name, BackgroundWorkDrainer>(idempotent registration by name;__registerDrainerForTestreturns an unregister handle);modeis'exit' | 'disconnect'('exit'= CLI teardown, engine still live, residual buffers may flush;'disconnect'= an engine is mid-disconnect, sinks await only IN-FLIGHT work and never start new writes).drainAllBackgroundWorkForCliExit({timeoutMs})runs mode'exit'with abort allowed;drainBackgroundWorkBeforeDisconnect({timeoutMs})is called by BOTH engines'disconnect()so an in-flight statement settles before the underlying handle closes — PGLite'sclose()deadlocks PERMANENTLY with a statement in flight — and it NEVER callsabort()(permanent process state, wrong for a long-livedgbrain servedisconnecting one engine); a partial disconnect-drain warns once per sink to stderr. The module stays a zero-import leaf on purpose (both engines import it statically). Drains in explicit(order, name)order — facts FIRST (order 0) so its abort-path DBlogIngestruns against the freshest live engine — and AWAITSabort()only whendrain()reportsunfinished>0in'exit'mode. Best-effort per drainer: one sink's failure never blocks the others or the disconnect. SIX sinks register at module import:facts/queue.ts(order 0;abort=shutdown()cancels a hung facts:absorb Haiku via internalAbort),last-retrieved.ts(order 1),search/hybrid.ts(order 2;awaitPendingSearchCacheWritesbounded viaPromise.race),eval-capture.ts(order 3;captureEvalCandidateself-tracks its promise viaawaitPendingEvalCaptures),context/volunteer-events.ts(order 4 — batched volunteer-event INSERTs, drained like the rest),search/telemetry.ts(order 5 —awaitPendingTelemetryFlush: awaits the in-flight flush in both modes, flushes residual buckets only in'exit'mode so short-lived CLI calls land insearch_telemetryon clean exit). Every cli.ts teardown site reaches it throughfinishCliTeardown(src/core/cli-force-exit.ts), which drains the registry beforeengine.disconnect(), sodb.close()cannot race an in-flight job and pin the PGLite single-writer lock. ExportsbackgroundWorkSinkCount()so the teardown helper computes its backstop deadline from the registered sink count, plus the shared teardown budgets:MAX_TIMER_DELAY_MS(the −1 setTimeout ceiling;process-watchdog.tsaliases it asMAX_WATCHDOG_TIMER_MS),SINK_DRAIN_TIMEOUT_MS(the per-sink drain bound, used as therunDrainersdefault), andpgliteCloseTimeoutMs()(the env-tunable in-loop close bound, defined here socli-force-exit's computed deadline budgets the SAME bound the engine honors). CLI-EXIT-ONLY: the factsshutdown()abort is permanent process state, never call in a long-livedgbrain serve. Companion changes:src/core/ai/gateway.tswithDefaultTimeout(caller, ms)bounds every outbound AI call (chat 300s, embed+multimodal 60s; envGBRAIN_AI_{CHAT,EMBED,MULTIMODAL}_TIMEOUT_MS; composed with caller signals viaAbortSignal.any) and the teardown backstop honors an errored op's exit code so a hung disconnect can't mask failure as success (seecli-force-exit.ts);src/core/postgres-engine.tsreconnect()module-mode branch re-establishes via idempotentdb.connect()+connectionManager.setReadPoolrefresh instead ofdb.disconnect()(no null window for concurrent ops; fail-loud on real connect failure);src/core/search/hybrid.tsembedQueryBounded+ a sharedQueryEmbedDeadline(6s, floored 2s per embed viaMIN_QUERY_EMBED_BUDGET_MS; envGBRAIN_QUERY_EMBED_TIMEOUT_MS) bounds the cache-lookup AND inner query embeds so a stalled provider falls back to keyword instead of stalling the whole op. Pinned bytest/core/background-work.test.ts,test/search/query-embed-deadline.test.ts,test/eval-capture-drain.test.ts,test/e2e/postgres-reconnect-singleton.test.ts,test/e2e/pglite-cli-exit.serial.test.ts,test/fix-wave-structural.test.ts. -
src/core/search/read-policy-sql.ts— shared internal concrete-page SQL policy (PageReadScopeintypes.ts): nonempty federated grants precede scalar source;excludePrivateis resolved byops/context.ts:readPolicyOpts, never an MCP override. Optional live-page filtering excludes deleted/quarantined pages and archived sources. Both engines apply policy inside content, version, timeline, graph and identity reads before limits. While page privacy is enforced, history requires both current-page and snapshot visibility. Links/timeline projections independently authorize originating pages. -
src/core/search/read-enrichment.ts— narrow query helpers shared by PGLite and Postgres. Recursive relational reads authorize seeds, hops and link origins before limits; seed identity constraints remain separate from caller grants. Batched IDs or(source_id, slug)pairs authorize backlinks, adjacency, dates, flags, extraction state and take-derived salience. Restricted holders use permitted active takes and zero stored emotional-weight contribution; required expert admission runs after optional enrichment, uses an authorized effective-date map and propagates query failures. Callers omit failed optional signals or apply their existing neutral defaults. -
src/core/search/safe-chunks.ts— internal remote chunk-read policy and current safe-index version. Remote reads require a completed index built from sanitized inputs; existing chunks remain available to trusted local callers while awaiting rebuild. Both engines apply the predicate before chunk ranking and limits, independently of page-visibility opt-outs. Import seals the index only after replacing chunks in its transaction; body/chunk mutations invalidate it. Uses the existingpages.chunker_versioncolumn, with no schema migration. -
src/core/remote-body.ts— linear protected-body sanitizer over the existing Facts parser/renderer. Processes every protected fence, retains only world Facts, removes all Takes and drops malformed protected sections. Chunk creation sanitizes full bodies before splitting. Page/history responses apply it whenever trust is not explicitly local, independently of page-visibility opt-outs and holder grants. -
src/core/search/query-cache.ts— stored semantic-cache data and maintenance remain, butsemanticResultCacheAvailable()is false.hybridSearchCachedbypasses both result lookup and writes regardless of configuration oruseCache; request-local query embeddings still work. Search metadata, cache_stats and the mode dashboard report effective caching as disabled. Historical hit counts and search_telemetry remain intact. Restoration requires complete response-dependency provenance. -
src/core/ops/insights.ts— stored contradiction reports load only for explicitly trusted, unscoped local callers. Other callers receive the fixed availability note and an empty contradictions list before any report storage access. Expert search threads page and holder policy and performs final authorized-page admission after enrichment. -
src/core/search/graph-signals.ts— per-query graph-signals helper.applyGraphSignals(results, engine, opts)runs as the 4th post-fusion stage (after backlink/salience/recency). Three boosts:ADJACENCY_BOOST=1.05(page linked from 2+ OTHER top-K results — local hub for THIS query),CROSS_SOURCE_BOOST=1.10(page linked from 2+ DIFFERENT sources — corroborated across team brains, dormant in single-source brains),SESSION_DEMOTE=0.95(3+ results from same chat session — keep the highest-scoring at full score, demote the rest). All three inherit the floor-ratio gate preventing weak pages from being boosted past strong ones via popularity.computeScoreDistribution(results)emits min/p25/p50/p75/p95/max +reorder_band_width.sessionPrefix(slug)extracts the chat-session anchor (chat/2026-05-15-...). PurepairedBootstrapPValue(deltas, resamples, rng)exported for eval gates. Test seam viaadjacencyFnDI. Fail-open: any error logs vialogGraphSignalsFailure(JSONL audit viaaudit-writer) and returns the input array unchanged. Pinned bytest/search/graph-signals.test.ts(incl. the IRON-RULE floor-gate guard). -
src/core/search/explain-formatter.ts— rendersSearchResult[]as a multi-line per-result breakdown forgbrain search --explain. Reads every boost-stamping field; also prints the raw query↔chunk cosine (SearchResult.cosine, the calibrated signal evidence keys off) next to the blended score when present — absent on keyword-only / no-embedding paths. Handles the "no boosts applied" empty path. 4-decimal precision with trailing-zero strip.formatResultsExplain(results, meta?)prepends header lines from the captured retrieval meta when present —formatAutocutSummary(the autocut decision) andformatDegradedSummary(meta.degraded), which renders the closeddegraded[]vocabulary asdegraded: reranker_skipped (no_key)(null when the run was clean);cli.ts:formatResultthreadslastRetrievalMetainto it, so a silently skipped reranker is visible from the CLI. Pinned bytest/search/explain-formatter.test.ts+test/cli-explain-degraded-render.test.ts. -
src/core/search/mode.ts— Named search-mode bundles and retained semantic-cache key machinery; persisted result reuse remains disabled.MODE_BUNDLES(conservative/balanced/tokenmax) and the resolution chain (per-callSearchOpts→ per-keysearch.*config → bundle → balanced fallback) resolve every search knob;knobsHashfolds registered search knobs into thequery_cachekey, andKNOBS_HASH_VERSION(exported from this file — the single source of truth for the current cache-key version) is bumped whenever a new knob shapes results so stale cache rows become unreachable.graph_signals: booleanknob inModeBundle(defaults:conservative=false,balanced=true,tokenmax=true).KNOBS_HASH_VERSIONappends ags=parts entry per the cache-key contamination convention so a graph-on cache write can't be served to a graph-off lookup.SearchKeyOverrides+SearchPerCallOpts+loadOverridesFromConfig+SEARCH_MODE_CONFIG_KEYS+resolveSearchMode+attributeKnoball carry the field. Opt-out:gbrain config set search.graph_signals false.query_cacherows written under an older hash version hash differently — natural row segregation, cleared withincache.ttl_seconds(3600s default).title_boost: number | undefinedknob inModeBundle(default1.25for all three modes; multiplier for the post-fusion title-phrase boost). Override chain: per-callSearchOpts→search.title_boostconfig (clamped[1.0, 5.0]) → bundle.KNOBS_HASH_VERSIONappends atib=parts entry so a title-boost-on cache write can't be served to a title-boost-off lookup.SEARCH_MODE_CONFIG_KEYSgainssearch.title_boost. Cross-modal knobs inModeBundle:cross_modal_both_text_weight/cross_modal_both_image_weight(weighted RRF for 'both' modality, defaults 0.6/0.4),image_query_text_refinement_weight/image_query_image_refinement_weight(hybrid intersect forsearchByImagequery refinement, defaults 0.4/0.6),unified_multimodal+unified_multimodal_only(unified-column routing flags),cross_modal_llm_intent(opt-in LLM escalation).SEARCH_MODE_CONFIG_KEYScarries the corresponding config keys, and the modality knobs participate inknobsHashso a cached text-mode result can't be served to an image-mode caller. Retrieval-quality knobsautocut_min_top(default 0.35 in all three bundles; configsearch.autocut_min_top; folds intoknobsHashas anacm=part) andevidence_cosine_floor(default 0.8 in all three bundles; configsearch.evidence_cosine_floor; labels evidence — result-set-shape-neutral, so not hashed) ride the same bundle → config → per-call chain.keywordOrFallback: booleanknob inModeBundle(default true in all three bundles; configsearch.keywordOrFallback): the keyword arm's AND→OR zero-recall retry — corpora the FTS config can't stem (CJK/agglutinative text under'english') can turn it off because the OR retry has no IDF demotion and its common-token hits read as noise in the RRF blend; folds intoknobsHashas akof=part.KnobsHashContextadditionally carries the EFFECTIVE per-call salience/recency boost modes (sal=/rec=— explicitSearchOpts?? the classifier's auto-suggestion, resolved by the same chain barehybridSearchuses) and the per-enginesearch.intent_patternsconfig fingerprint (ipat=, fromquery-intent.ts'sintentPatternFingerprint; threaded through ctx, never read process-globally, so a multi-engine process can't key one brain's rows under another brain's patterns) — a reordered or reclassified write can never serve a differently-configured lookup. Cache HITS slice the stored page to the same resolved result count as misses (per-calllimit→ the mode'ssearchLimit) inhybridSearchCached, so a mode's result-count knob holds on both paths; nonzero offsets (positive OR negative) skip the cache entirely.expansion_variant_budget: number | nullknob inModeBundle(nullin all three bundles; configsearch.expansion_variant_budgetacceptslegacy/nullor a number in (0, 4] through the ONE range contractnormalizeExpansionVariantBudgetinfusion-lists.ts— out-of-range falls through to the bundle; per-callHybridSearchOpts.expansionVariantBudget, normalized by the same function in both the inner search and the cache resolver): the total RRF weight the NON-EMPTY expansion variant/clause lists share at fusion (weight_i = b / n_voting_arms; the original list always keeps weight 1),null= legacy weight 1 on every list (byte-identical fusion), a no-op whenexpansionis off. It folds intoknobsHashas the last, append-onlyevb=part (legacyor the budget to 3 decimals) —KNOBS_HASH_VERSIONis 29 with that part in — so a budget-weighted write can never serve a legacy lookup.KNOB_NULL_LABELS+formatKnobValueinmodes-report.tsrender its legitimate null aslegacy (null)ingbrain search modes(plain(undefined)stays reserved for genuinely unset knobs).relational_rerank_pin: numberknob inModeBundle(3 in all three bundles — a no-op underconservative, which has no reranker; configsearch.relational_rerank_pinacceptsoff/0or an integer in [0, 10] through the ONE range contractnormalizeRelationalRerankPininrelational-rerank-pin.ts, out-of-range falls through to the bundle; per-callSearchOpts.relationalRerankPin, normalized by the same function in both the inner search and the cache resolver): how many relational-arm rowspinRelationalRowsre-pins above the reranked text rows afterapplyReranker. It folds intoknobsHashas the append-onlyrrp=part, ridingKNOBS_HASH_VERSION29 together withevb=(one bump per wave — both parts landed before v29 shipped; a partial-knobs literal without the field hashes as the bundle default), so a pin-3 write can never serve a pin-0 lookup. Pinned bytest/search-mode.test.ts(bundle snapshots + hash pins),test/search/knobs-hash-reranker.test.ts(the version comment chain),test/config-adaptive-return-keys.test.ts,test/search/modes-report-coverage.test.ts. -
src/core/search/modes-report.ts—buildModesReport(engine)→SearchModesReport(schema_version: 2), the read-only dashboard behindgbrain search modesand thesearch_modesMCP op:active_mode+ validity, the per-knobresolvedattribution (attributeKnoboverKNOB_DESCRIPTIONS, including the fivereranker_*knobs), the three frozenbundles,config_keys, andreranker_readiness—{model, enabled, ready, required_key, key_present, sunset_passed, self_hosted, fix}fromrerankerReadinessForEngine+describeRerankerFix(a thrown readiness check still yields aready: falseverdict whosefixsays to rungbrain doctor; the block never vanishes silently).redactReadinessForRemote(report)is what thesearch_modesop returns whenctx.remote !== false:required_key,key_presentandfixare dropped andself_hostedis forced false — which env vars exist on the host is fingerprinting data and the fix names them;readystays because it is observable anyway (reranked results carryrerank_score).src/commands/search.ts:formatModesTextrenders the runtimeReranker:line (off (resolved) — …/<model> (enabled) — <KEY> present/<model> (enabled but NOT running) — <fix>) and a per-bundlereranker=… topNIn=… autocut=…line. Pinned bytest/modes-report-reranker.test.ts. -
src/core/context-engine.ts+src/openclaw-context-engine.ts— the deterministic context engine OpenClaw loads on every turn (assemble()injects the Live Context block, zero-LLM).createGBrainContextEngine({workspaceDir, resolveEntities?})accepts an OPTIONAL host-injected resolver (ENGINE_API_VERSION0.3.0, additive — older hosts work unchanged; the plugin entry mapsctx.resolveEntities/ctx.brainQueryonto it). Checkpoint compaction:compact()runs a time-bounded (8s), fail-open, lazily-imported checkpoint step BEFORE delegating — spools the since-last-boundary window (openclaw tail reader over the exported adapter mapper, 40-turn cap = the no-prior-boundary fallback) as a content-addressed corpus segment + ledger entry, then rung 2 (PGLite: onebankOnly+flushCorpusFileIPC round trip to serve) or rung 3 (Postgres: inline harvest over the reflex ladder's exportedgetDirectPostgresEnginesingleton, under sweep claim fencing + capability/kill-switch gates, abort post-check) — and rides an additiveresult.gbrain_checkpointbag on the delegate's return (ownsCompactionstays false). After the segment is spooled, the Memorable receipt lane runs (gate + stamp viamemorableGateAllowed, span-filtered redacted tool calls,recordAndRelayReceiptwithharness: 'openclaw'— per-compaction capture, content-hash dedup on retries, fail-open for the checkpoint; the whole block is skipped once the host's compact() deadline has fired; the rendered segment text is re-scanned with highEntropy at receipt time and the relay REFUSED fail-closed on any hit — the segment rendering feeds the ledger hash and cannot change, so the next compaction window re-evaluates; this lane NEVER compacts the receipts file — the hook lane is the ONE compactor, single-rewriter rule — but DOES trim the relay file, since an openclaw-only host has no hook lane and converging newest-keeping trims make concurrent trims safe).assemble()consumessessionId ?? sessionKeyand splices a deterministic envelope-bearing Compaction-checkpoint block at parts[1] (after Live Context) from the banked manifest via an in-process memo + hash-keyed polls (≤5; stale manifests can't satisfy a poll for the new segment; no manifest ⇒ byte-identical output).assemble()runs the Retrieval Reflex after the Live Context block: extracts the current turn's user text, builds prior-context text (every message EXCEPT the current turn — suppression must not see the triggering mention), passes the rolling window (getWindowTurns, last 12 user/assistant turns; the reflex slices to its configuredretrieval_reflex_window_turns), and appends the pointer block.warmReflex()fires at construction. -
src/core/context/— Retrieval Reflex (Layer 1).entity-salience.ts: pure, zero-LLM, precision-biasedextractCandidates(text)(capitalized runs +@handles, STOPWORDS + soft COMMON_WORDS + sentence-start guard, deterministic, capped; a lowercase weak pass additionally emitsweak: truecandidates — lowercase words ≥3 chars on a separateMAX_WEAK_CANDIDATES=32budget that never evicts strong candidates — which downstream may resolve through the alias arm ONLY) +extractCandidatesFromWindow(turns)(merges per-turn extraction across the last N turns by normalizeAlias form with occurrence/newest-turn/user-mention metadata; salience-ordered — recency > frequency > user-role — so the cap drops stale assistant chatter first).retrieval-reflex.ts:resolveEntitiesToPointers(engine, sourceId, candidates, opts)— alias arm (resolveAliases, caught per-arm for pre-v110 brains) + exact title/slug-suffix arm (real slugs are namespacedpeople/xbutslugifydrops the prefix) + two lexical identity arms behindopts.lexicalArms(kill switch: configretrieval_reflex_lexical_arms/ envGBRAIN_RETRIEVAL_REFLEX_LEXICAL_ARMS, default on): weak lowercase candidates probe the ALIAS arm ONLY (never title/slug-suffix, where ordinary lowercase words would fabricate pointers) and require GLOBAL uniqueness across all sources in play; the surname arm resolves a strong single capitalized token ≥3 chars via alower(title) LIKE '% <token>'predicate on the same query, with exact-arm precedence and ambiguity counted over ALL person rows carrying the surname — ambiguity in either arm injects nothing; pointers carrysource_id/arm/confidence/matchedNorm(ARM_CONFIDENCEalias 0.9 / title 0.8 / title-surname 0.72 (deliberately above the volunteer layer's 0.70 gate, below title) / slug-suffix 0.6 lives next to the arm definitions; arm-2 provenance classified in JS since the combined OR can't report which predicate matched); opts:sourceIds?federated scope (alias arm loops per source, arm 2 usessource_id = ANY),suppression?('slug-and-title' default; 'slug-only' REQUIRED under windowing — the title rule would suppress every entity merely mentioned in a prior window turn), ambient-channel event logging is DELIVERY-side, not in-resolver —logDeliveredReflexPointers(engine, pointers)fires only once a block is actually handed to the consumer (serve's resolve-IPConDeliveredhook post-write;buildReflexAdditionpost-timeout on the direct rung), so abandoned/timed-out blocks never pollute the volunteered-vs-used stats; its event write is registered synchronously before return so the CLI background-work drain cannot miss it; synopsis runs throughstripTakesFence/stripFactsFence(the same privacy boundaryget_pageapplies) so private facts never reach the prompt; capped atMAX_POINTERS.reflex.ts: the orchestrator + engine-aware resolver ladder (hostresolveEntities→ PGLite serve IPC → Postgres cached process-singleton → disabled), zero-candidate fast path, fail-open + timeout, heartbeat write for the doctor check,reflexEnabled(cfg)(file/env gate, default ON; DB-plane does NOT gate —assemble()is sync); windowed extraction whenwindowTurnspresent andretrieval_reflex_window_turns(default 4; 1 = single-turn behavior) > 1 — switches suppression to slug-only; accept-side reflex-channel logging fires after the per-turn timeout admits the block (direct-Postgres rung only — IPC logs server-side at delivery; host-injected resolvers are a documented gap).resolve-ipc.ts: local unix-socket resolve protocol (client + server) so PGLite resolves through the single connectiongbrain serveholds (a second opener would hit the exclusive lock; a subprocess would force-steal it past the 5-min staleness window and crash). Wired into both serve transports viasrc/mcp/resolve-ipc-binding.ts(engine-uniform; socket + secret keyed offhash12(database_url)under~/.gbrain/run, cleaned up on shutdown). Doctor surface:retrieval_reflex_healthinsrc/commands/doctor.ts(reads the heartbeat for truthful runtime status; categorized indoctor-categories.ts) +volunteer_channels(engine-aware sibling: groupscontext_volunteer_eventsby channel over 7 days so operators see which push channels — reflex/op/watch/claude-code/codex — actually fire; info-only; the LOCAL doctor runs it brain-wide while the remote report path threads the caller's source scope, so a source-bound token never sees other sources' activity counts/timestamps; counts are reconciled against the hook heartbeat over the same 7-day window — a mostly-degraded week gets a CAUTION note, since a server-side delivery count isn't proof of injection; quiet-channel guidance is engine-aware — Postgres brains are told the hook lane is quiet by design rather than to chase registration — and walks both quiet classes (installed-but-unregistered vs registered-but-quiet; the check can't inspect registration itself); pre-v117 tolerant, and transient DB errors are reported as such, never as an old schema; pinned bytest/doctor-volunteer-channels.test.ts). Config:retrieval_reflex+retrieval_reflex_max_pointers+retrieval_reflex_window_turns+retrieval_reflex_lexical_armsinsrc/core/config.ts(envGBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS,GBRAIN_RETRIEVAL_REFLEX_LEXICAL_ARMS).volunteer.ts:parseWindow(lenientuser:/assistant:prefixes, unprefixed → one user turn),volunteerContext(extract → resolve → +0.05 multi-turn/newest-turn boost →min_confidence0.7 gate → cap 3/5; deterministic rationale strings, never raw conversation text; slug-only suppression),volunteerUsageStats(per-arm/channel precision from thepages.last_retrieved_at > volunteered_atjoin — APPROXIMATE: the 5-min last-retrieved throttle causes false negatives, unrelated reads false positives).volunteer-events.ts:insertVolunteerEvents(ONE multi-row parameterized INSERT),logVolunteerEventsFireAndForget+ bounded drain registered as thevolunteer-eventsbackground-work sink (order 4),purgeStaleVolunteerEvents(90-day GC, called from the dream cycle's purge phase). Policy layer ships as theretrieval-reflexrecipe (recipes/retrieval-reflex/). Pinned bytest/context/entity-salience.test.ts,test/retrieval-reflex.test.ts,test/retrieval-reflex-pre-v110.test.ts,test/context/resolve-ipc.test.ts,test/doctor-retrieval-reflex.test.ts,test/volunteer-context.test.ts,test/e2e/volunteer-context-postgres.test.ts. Checkpoint-compaction modules in this dir:corpus-segments.ts— engine-free content-addressed compaction segments (<session>.seg-<hash24>.txt, idempotent by name; parsers accept 12–64 hex), the hash-keyed per-session ledger (atomic tmp+rename, fail-open, entry order = the only ordinal),sliceBoundaryWindow/splitByBoundaries, the redactedrenderSegmentText(a segment is NEVER written unscanned — unlike the session-end full write, which degrades-and-writes), exact-setcoverageComplete/decideCorpusMode(count equality can be fooled by a duplicated boundary), the compact-timebankCompactSegmentstep (per-step deadline degrades, segment-then-ledger crash order), the openclaw tail boundary reader (readOpenclawBoundaryTail, delegates mapping to the adapter's exportedmapOpenclawLine; also returns turn-stampedtoolCalls/toolCallTurnIndexesfor the Memorable receipt — name-only v1,input: nullby design until an observation run characterizes OpenClaw's args field), and orphan-sidecar/aged-ledger GC — pinned bytest/corpus-segments.test.ts.checkpoint-harvest.ts— the serve-side prompt harvest of a segment: bounded FIFO (cap 8, 60s abort), sweep-shared claim fencing, capability-then-kill-switch gates,signal.abortedPOST-check (the pipeline returns partials on abort; an aborted run writes nothing and stays retryable), receipt sidecar before idempotent manifest publish (source-scopedgetPageverification — a link that resolves to nothing is never banked),.ingestedlast, explicitshutdownCheckpointHarvest()called by serve BEFOREengine.disconnect()(the background-work drain is CLI-exit-only by contract) — pinned bytest/checkpoint-harvest.serial.test.ts.hook-heartbeat.ts— the hooks telemetry JSONL extracted from hook.ts (hook.ts re-exports; serve-side writers never import the command module); allowlist carries the checkpointsegment/inserted/duplicate/linkscount-only fields.session-state.tsalso carries the v132checkpoint_manifesthelpers (getCheckpointManifest/appendCheckpointManifest: newest-first, dedup-by-slug, cap 20, seg-hash completion key, fail-open on pre-v132 schema) — pinned bytest/checkpoint-manifest.test.ts.sensitivity-scan.ts+compile-view.ts— the compile-context stack: composed detector (secret-scan + orderedPII_PATTERNS+ path/blocklist families + operator pattern file; uniform.gbrain-scan-allowfingerprint escape hatch; CONTENT hits drop an entry, CONFIG/loader failures THROW so the caller aborts without writing) and the deterministic compiled-view builder (recency decay anchored to the newest candidateupdated_at, never wall-clock; total-order (score desc, slug asc); source-scoped listPages/getPage reads that skip op-layer write-backs; whole-filepackToBudgetmath that never passes a <=0 budget) — pinned bytest/sensitivity-scan.test.ts+test/compile-view.test.ts+test/e2e/compile-context-pglite.test.ts; the CLI shell issrc/commands/compile-context.ts(targets claude-code|codex|openclaw, AGENTS.md managed-marker splice that throws on damaged markers, atomic writes,--checkrecompile-and-compare exit codes; the guide isdocs/guides/checkpoint-compaction.md). It also owns the whole Memorable seam's engine-free plumbing: the session-receipt JSONL (session-receipts.jsonl,appendSessionReceipt— dedup by post-redactioncontent_hashso a resumed session neither duplicates its receipt nor re-fires the relay; tail-rewrite compaction triggered by the 32 MB byte ceiling alone — the read fires exactly when the pass will act, with the 2000-line budget enforced inside the pass — and askipCompactionseam the openclaw compact() lane sets unconditionally: the hook lane is the ONE receipts compactor, so two processes can never race the read-filter-rename rewrite); ONE bounded adaptivereadJsonlTailreader shared bylastReceiptMatches,lastRelayResultandreadSessionReceiptsTail(a receipt line can exceed the 1 MB window — the window doubles rather than reading "empty");maybeTrimRelayResults(BOTH capture lanes trim the child-appendedmemorable-relay.jsonl— an openclaw-only host has no hook lane, and converging newest-keeping trims make concurrent trims safe: mtime-stale preferred, hard 8 MB force backstop, line-boundary cut keeping the newest lines — the child appends open-by-path, so the residual loss window is one in-flight write); the CONSENT STAMP (memorable-consent.json, 0600, written ONLY by config-set's disclosure flow — deliberately outside config.json, which the external CLI full-file-rewrites; scope-bound to the disclosure sha256 +MEMORABLE_CAPTURE_HARNESSES, so widening the capture surface invalidates old stamps);memorableGateAllowed(cfg)({allowed, reason: kill_switch|disabled|disclosure_missing} — one reason vocabulary for hook, context-engine and doctor);memorableConsentEvidence()(the CLI-side opt-in read fail-closed from ~/.memorable/config.json, dated provisional spec note, GBRAIN_MEMORABLE_CONFIG test seam);redactedToolCallsJson(the security-load-bearing span-filter + highEntropy redaction, ONE implementation for both capture lanes; string LEAVES are redacted raw before serialization — scanning the serialized JSON would let quoted secrets through via " escaping — with a post-serialization re-scan as belt-and-braces);recordAndRelayReceipt(per-receipt harness scope-binding first — a harness outsideMEMORABLE_CAPTURE_HARNESSESis refused withmemorable_harness_undisclosed→ receipt → prior-run outcome surfacing with the child's reason clamped via the sharedclampRelayCause(32 chars, so the 16-char prefix keeps composites inside the 48-char reason bound) and appended LAST → consent evidence →resolveMemorableBin→ detached spawn; never throws;spawnFntest seam); andpriorRelayFailure, which surfaces the PREVIOUS relay run's self-reported exit status so a relay that records nothing is never reported healthy.resolveMemorableBindoes the pre-spawn PATH/MEMORABLE_BINresolution so a missing CLI is a named heartbeat reason rather than a silent async ENOENT. -
src/commands/watch.ts—gbrain watch: the push transport. Reads turns from stdin as they arrive (user:/assistant:prefixes; unprefixed = user turn), keeps a rolling in-process window (--window-turns, default 4), callsvolunteerContextper turn, streams pointers to stdout (--jsonfor JSONL with turn attribution), logschannel: 'watch'events with a per-session id. Session dedupe feeds already-pushed slugs back as priorContext so the core's slug-only suppression dedupes. Blocks in the stdin iteration (interactive alive until Ctrl-C/Ctrl-D; piped ends at EOF) — deliberately NOT in DAEMON_COMMANDS; SIGINT closes the stream so teardown flows through finishCliTeardown. Per-turn resolution failures are fail-open. Registered in CLI_ONLY + CLI_ONLY_SELF_HELP + THIN_CLIENT_REFUSED_COMMANDS (thin clients use thevolunteer_contextMCP op). Pinned bytest/watch-command.test.ts. -
src/commands/integrations.ts— recipe install. The resolver-row install fence is keyed bymanifest.recipe(gbrain:<recipe>:resolver-rows), so a secondcopy-into-host-reporecipe never writes a block mislabeled with the first recipe's name. Pinned bytest/integrations-install.test.ts. Health-check DSL includes the staleness-awareheartbeat_max_agetype: declares the sense's expected cadence (max_age: 48h), andintegrations doctorFAILS when the newest heartbeat event is older — the only check type that catches a green-but-dead sense (all others are point-in-time). Not embedded-gated (reads only the local heartbeat file). Recipe frontmatter carriesoutput_paths(repo-relative dirs the collector writes, e.g. calendar-to-brain →daily/calendar/);getConfiguredCollectorOutputs()surfaces them for the db_only-collision check/warning. Pinned bytest/integrations-heartbeat-max-age.test.ts. Standalone integration recipe management (no DB needed). ExportsgetRecipeDirs()(trust-tagged recipe sources), SSRF helpers (isInternalUrl,parseOctet,hostnameToOctets,isPrivateIpv4). Only package-bundled recipes areembedded=true;$GBRAIN_RECIPES_DIRand cwd./recipes/are untrusted and cannot runcommand/http/string health checks. -
src/core/audit/audit-writer.ts— shared JSONL audit primitive behind the audit modules. ExportscreateAuditWriter({kind, recordSchema})returning{log, readRecent}plus shared helperscomputeIsoWeekFilename(kind, now?)andresolveAuditDir()(honorsGBRAIN_AUDIT_DIR). ISO-week file rotation; best-effort writes (stderr warn on failure, never throws); read-path scans current-week + previous-week files for boundary spans. Built on it:src/core/rerank-audit.ts,src/core/audit-slug-fallback.ts,src/core/minions/handlers/shell-audit.ts,src/core/minions/handlers/supervisor-audit.ts,src/core/facts/phantom-audit.ts. Thegraph-signals-failuresaudit (logGraphSignalsFailure) uses the same primitive.src/core/skillpack/audit.tsis the one audit that does not use it. Pinned bytest/audit/audit-writer.test.ts. -
src/core/cli-force-exit.ts— single owner of one-shot CLI exit + teardown, designed as a PAIR with theimport.meta.mainseam at the bottom ofsrc/cli.ts.finishCliTeardown({engine, drainTimeoutMs?})is teardown-ONLY (never exits on the clean path): arms a REF'D backstop (unref'd would let a hung teardown exit naturally, skipping the flush and exiting with whatever PGLite scribbled intoprocess.exitCode) whose deadline is COMPUTED from the bounds it guards (computeTeardownDeadlineMs= sinks × drainTimeoutMs + sinks ×SINK_DRAIN_TIMEOUT_MSdisconnect-drain bound + the RESOLVED PGLite close bound (pgliteCloseTimeoutMs()from background-work.ts — an operator-raisedGBRAIN_PGLITE_CLOSE_TIMEOUT_MSwidens this backstop too, never a hardcoded copy of the default) + facts-abort grace + 2 × pool-end bound + slack, floor 10s — the disconnect-drain and close terms budgetengine.disconnect()'s own drain pass and bounded close so the backstop can't fire while every component honored its own bound;GBRAIN_TEARDOWN_DEADLINE_MSenv override is the incident escape hatch), drains every background-work sink, disconnects the engine (a throw is warned + swallowed — the exit code reports the OPERATION, not the cleanup), then returns. The exit VERDICT lives in a gbrain-owned channel (setCliExitVerdict/currentExitCode; mirror-writesprocess.exitCodebut NEVER reads it back) because PGLite's Emscripten runtime scribbles its own status intoprocess.exitCodeat arbitrary points mid-run — every writer that means to set the CLI exit code (op-dispatch catch, reindex, frontmatter, transcripts, brainstorm, autopilot, doctor's FAIL verdict, extract, and cli.ts's swept inner exits — friction, claw-test, smoke-test, the no-DB eval runners, status/status-thin, whoknows-thin) callssetCliExitVerdict;test/cli-exit-verdict-pin.test.tsgreps src/ so the next rawprocess.exitCode =write fails CI instead of silently reporting success on failure. The deadline arms at TEARDOWN start, never before the op handler (arming before the handler would measure handler + teardown combined, taxing PgBouncer deployments with a flat force-exit deadline on every query and killing any long op mid-run with exit 0). All nine cli.ts disconnect sites route through it; the ONE process exit happens in cli.ts'smain().then/catchviaflushThenExit(currentExitCode()), gated byshouldForceExitAfterMain()(daemon list:serve) — the CLI never waits for Bun's event loop to drain, becauseendPoolBoundeddeliberately races past stuck PgBouncer sockets that would keep it alive.flushThenExit(code)fences stdout+stderr (write('', cb)raced with an unref'd guard, EPIPE-safe both sync and async) then holds a REF'D aliveness grace for non-TTY stdio beforeprocess.exit— Bun delivers queued pipe writes only while the process is alive (no flush API reachesprocess.stdout's native queue; write callbacks fire on accept, not delivery), so the grace IS the flush. Scope claim is deliberately cli.ts-only: command modules' mid-run engine lifecycles stay local (process-exit semantics inside them would be wrong) and are absorbed by the final explicit exit. Pinned bytest/cli-finish-teardown.test.ts,test/flush-then-exit-harness.test.ts(real spawned-Bun pipe semantics),test/cli-should-force-exit.test.ts,test/cli-pipe-truncation.test.ts(real-CLI piped --tools-json byte-stable),test/cli-exit-verdict-pin.test.ts, the teardown describes intest/fix-wave-structural.test.ts+test/e2e/pglite-cli-exit.serial.test.ts, andtest/e2e/pgbouncer-teardown.test.ts(CI transaction-mode pooler). -
src/commands/search.ts:gbrain search stats—graph_signalssection (enabled/source/failures_count/failures_by_reason). JSON envelope adds agraph_signalssibling property;_meta.metric_glossaryaddsgraph_signals.enabled+graph_signals.failures_by_reason. Human output prints the section after the existing block. Readssearch.graph_signalsconfig first, falls back to the mode default. Pinned bytest/search/search-stats-graph-signals.test.ts. Bothgbrain search statsandgbrain search tunealso surface acoveragedisclosure (JSON:{cli_invocations: 'recorded_on_clean_exit', reason}; human: a one-line caveat) sourced fromtelemetryCoverage()/TELEMETRY_COVERAGE_CAVEATinsrc/core/search/telemetry.ts: a short-lived CLI call's buffer flushes during the bounded CLI teardown drain, so a CLEAN exit is recorded; hard kills, drains that exceed their bound, and anything buffered when an engine disconnects outside the CLI teardown path still drop — the disclosure is display-only and reads its truth from the telemetry module's single-source constants. Pinned by the coverage-disclosure tests intest/commands-search.test.ts. -
src/core/engine-factory.ts— Engine factory with dynamic imports ('pglite'|'postgres'). -
src/core/pglite-engine.ts— PGLite (embedded Postgres 17.5 via WASM) implementation, all BrainEngine methods. The facts/takes/code-edges/salience method clusters are implemented in narrow-deps modules undersrc/core/pglite-engine/(see the engine-module-dirs entry below); the class methods delegate to them and the façade keeps its full public surface.listLinkSources({sourceId?, sourceIds?})returns distinctlink_sourceprovenances + counts (ORDER BY count DESC, link_source ASC NULLS LAST; scalar + federated scoped; parity with postgres-engine.ts) poweringgbrain link-sources.addLinksBatch/addTimelineEntriesBatch/addTakesBatchpass the whole batch as one JSONB document viajsonb_to_recordset((\$1::jsonb)->'rows')(bound throughexecuteRawJsonbwith a{ rows }wrapper; rows built by the sharedsrc/core/batch-rows.tshelpers, NUL-stripped), and arebatchRetry-wrapped.connect()wrapsPGlite.create()in a try/catch that classifies the failure and, for thewasm-abortverdict on a persistent data dir (torn WAL/checkpoint state after an unclean shutdown; it presents like a macOS WASM bug but is not one), runs in-place auto-repair viaattemptWalRepairAndRetry(static import frompglite-repair.ts, per the engine-live rule; the retry create ispreservingProcessExitCode-wrapped; success sets the publicwalRepairReceiptfield + printsbuildWalRepairNoticeto stderr and returns with the lock held). The seam never throws, so every non-repaired path funnels through the single lock-release-then-throw site; repair refuses when the lock was acquired by reaping (LockHandle.reaped→'possibly-live-writer'), when disabled (GBRAIN_PGLITE_WAL_REPAIR=off), on layout-validation failure, or inside the post-failure cooldown.disconnect()early-nulls the handle (so no NEW statement can reach it), then runsdrainBackgroundWorkBeforeDisconnect()so statements ALREADY in flight settle against the still-open handle — PGLite'sclose()deadlocks permanently (close's promise AND the in-flight query's promise never settle) with a statement in flight; the ordering is load-bearing (drain above the null would let a new statement race the close). Teardown defense is layered and honestly scoped: the drain PREVENTS the known wedge; the in-loop close bound (GBRAIN_PGLITE_CLOSE_TIMEOUT_MS, default 5000ms, floor 1s, ceiling −1, read per call) covers ONLY a close that still yields to the event loop — armed BEFOREclose()is called and deliberately ref'd, a timed-out close degrades to a once-per-process stderr warning naming both env knobs and teardown proceeds (an abandoned close's later rejection is swallowed; the WAL-repair path covers a zombie instance on next open); a close that WEDGES the loop (blocked or microtask-starved) can never be caught by any same-loop timer and is observable/killable only by the opt-in out-of-band watchdog — a diagnostic/incident instrument, not ambient production protection:GBRAIN_PGLITE_CLOSE_WATCHDOG_MS(unset/0 = off; a positive value clamps UP to with a warn so a units typo never SIGKILLs a healthy slow teardown) +GBRAIN_PGLITE_CLOSE_WATCHDOG_GRACE_MS(default 30000) arm the sharedprocess-watchdog.tsworker around a PGLite disconnect with a live handle (armed after the early-return, before the drain; disposed in a nested finally so areleaseLockthrow can't leak it; SIGTERM at deadline, SIGKILL at deadline+grace; lock-only teardown and postgres pool teardown are out of scope). Pinned bytest/pglite-engine-disconnect.serial.test.ts+test/search-telemetry-disconnect-hang.serial.test.ts+test/pglite-disconnect-watchdog.serial.test.ts(spawned fixture that genuinely starves the loop) + the structural pins intest/fix-wave-structural.test.ts.searchKeyword/searchKeywordChunksmultiplyts_rankby the source-factor CASE at chunk grain;searchVectoris a two-stage CTE — inner CTE keepsORDER BY cc.embedding <=> vecso HNSW stays usable, outer SELECT re-ranks byraw_score * source_factor, inner LIMIT scales with offset to preserve pagination.searchTakes/searchTakesVectortake fullSearchOptsand apply the standard source-scope predicates (federatedsourceIds[]wins over scalarsourceId, via the joined page'ssource_id) alongside the holder allow-list — parity SQL in postgres-engine.ts; pinned bytest/e2e/think-source-isolation-pglite.test.ts.initSchema()callsapplyForwardReferenceBootstrap()BEFORE replaying SCHEMA_SQL — probes for forward-referenced state the embedded blob needs (pages.source_id,links.link_source,links.origin_page_id,content_chunks.symbol_name,content_chunks.language,sourcesFK target, plusfiles.source_id,files.page_id,oauth_clients.source_id,oauth_clients.federated_read,sources.archived,sources.archived_at,sources.archive_expires_at,timeline_entries.event_page_id— column-only, migration v121 stays the source of truth for its FK + indexes) and adds only what's missing; threads the DDL connection frominitSchemaso probes run inside the advisory-lock scope; no-op on fresh installs and current brains (this is what keeps an upgrade from wedging on a forward-referenced column).getBrainScorereturns 100/100 with full breakdown (35/25/15/15/10) whenpageCount === 0(vacuous truth — empty brain has no coverage problem); Pinned bytest/brain-score-breakdown.test.tsempty-brain assertion +test/doctor-report-remote.serial.test.ts.disconnect()uses snapshot+early-null (snapshot_db/_lock, null instance fields BEFORE anyawaitso a concurrentconnect()can't see a partial mid-close state) wrapped in try/finally guaranteeing lock-release even ifdb.close()throws; KEEPS close-then-release order (release-then-close was rejected: it would widen the window where a sibling process connects to a still-closing brain); Pinned bytest/pglite-engine-disconnect.serial.test.ts.PGlite.create()runs insidepreservingProcessExitCode: PGLite's Emscripten runtime writes its own status intoprocess.exitCode(99 at create; in-memory brains run initdb whose status lands on a later tick; the exit status at close — and assigningundefinedcannot clear a prior value), which would otherwise silently clobber an errored op's exit 1 back to 0. The wrapper keeps the global tidy for external readers;db.close()stays unwrapped (its 0-write is baseline behavior test runners depend on). The CLI's exit verdict is immune either way — it lives in the gbrain-owned channel incli-force-exit.tsand never readsprocess.exitCodeback. ExportsclassifyPgliteInitError(message): 'bunfs' | 'wasm-abort' | 'corrupt' | 'unknown'+buildPgliteInitErrorMessage(verdict, original, platform?, ctx?)+stringifyPgliteInitError(err)+buildWalRepairNotice(receipt)+ thePgliteInitRepairContexttype, routing the catch-block hint by failure shape (bunfsmatches literal$$bunfsORENOENT[\s\S]*pglite\.dataco-occurrence, surfaces a paste-readybun upgrade+ Node fallback;corrupt— 58P01/internal_load_library/missing vector type, catalog corruption WAL repair can't fix — stays matched BEFORE the wasm arm and routes toreinit-pglite;wasm-abortmatches the real production shapesAborted()/RuntimeError/unreachableplus legacy signatures, names the corrupt-WAL root cause + the recovery ladder (pglite-repair→ rebuild → engine switch) + what auto-repair did perctxincl. the honesty-criticalfailed-not-restoredarm, and links the upstream tracking issue;unknownis platform-gated).stringifyPgliteInitErroralso surfaces message-less Emscripten objects (ErrnoError (errno N)) instead of[object Object]. Pinned bytest/pglite-init-classifier.test.ts+test/pglite-wal-repair.serial.test.ts+test/fix-wave-structural.test.ts. ImplementsdeletePages(slugs, {sourceId})+resolveSlugsByPaths(paths, {sourceId})viaslug = ANY(\$1::text[])array-param binding, caller-chunking primitive throwing when input exceedsDELETE_BATCH_SIZE,deletePagesreturnsRETURNING slugrows so callers filterpagesAffectedto confirmed deletes. Implements the embedding-signature stale-detection quartet —sumStaleChunkChars({sourceId?, signature?}),setPageEmbeddingSignature(slug, {sourceId?, signature}),invalidateStaleSignatureEmbeddings({signature, sourceId?}), widenedcountStaleChunks({sourceId?, signature?})(thesignatureopt widens viaJOIN pages p ... WHERE cc.embedding IS NULL OR (p.embedding_signature IS NOT NULL AND p.embedding_signature <> $signature), NULL grandfathered); parity SQL with postgres-engine.ts. PGLite-specific DDL (pgvector, pg_trgm, triggers). Engine-path helper dependencies (retry, ontology, recency decay) avoid dynamicimport(); the only lazy dynamic imports areai/gateway.tsininitSchemaand_upsertChunksOnce, line-marked because their local catches preserve compiled-default and stored-config fallbacks that eager module evaluation would bypass. -
src/core/pglite-embedded-assets.ts+src/core/pglite-embedded-asset-paths.ts— PGLite runtime-asset supply for every run mode (compiled binary, source checkout, bun-global install).pglite-embedded-asset-paths.tsis the bundler ANCHOR: five literalwith { type: 'file' }imports of the wasm/data/extension-tarball assets via repo-relative node_modules paths (the package'sexportsmap hides./dist/*) — keep the specifiers literal and byte-stable, an expression or indirection silently stops the bytes embedding and the compiled binary falls into the Bun-vfs ENOENT failure.resolvePgliteAssetPaths()inpglite-embedded-assets.tsis the tiered resolver: tier 1 dynamicallyimport()s the anchor (the rejection under a hoisted install — bun-global upgrades dedupe@electric-sql/pgliteto the global root, making the anchor's specifiers unresolvable — is EXPECTED and routes to tier 2, which is why the anchor is not a static import); tier 2 derives the dist dir via module resolution of@electric-sql/pglite(import.meta.resolvethencreateRequire— the SAME specifier the engine imports, so assets can never come from a different pglite copy than the loaded JS); tier 3 throws one actionable error naming everything tried, with the canonical GitHub reinstall command (never the npm-registry name, which is squatted — see the README install warning).getEmbeddedPgliteOptions()memoizes one build per process and hands PGLite compiledWebAssembly.Modules + fs-bundle Blob + materialized extension tarballs viaPGliteOptions. Pinned bytest/pglite-embedded-assets.test.ts+test/pglite-hoisted-install.serial.test.ts(realbun add -g-shaped hoisted layout). -
src/core/pglite-lock.ts— advisory data-dir lock so two processes never open the same embedded PGLite (single-connection WASM Postgres) at once. Atomicmkdirof.gbrain-lock/+ alockfile carrying{pid, acquired_at, refreshed_at, command, subcommand}. A held lock HEARTBEATS itsrefreshed_atevery 30s (.unref()ed timer; informational). A waiting acquirer reaps a holder ONLY on affirmative proof of death — ESRCH from kill-0, or aps-/proc-read command line proving the PID was recycled by a non-gbrain program under AFFIRMATIVE same-namespace proof (on Linux the lock's recordedpid_nsmust be readable and equal ours,boot_idtoo when recorded; legacy locks without markers are never cmdline-reaped; unreadable cmdline and same-process PID also read as alive) — a LIVE holder is NEVER stolen, regardless of how stale its heartbeat is. Every reap serializes on an atomic claim dir (tryReapLockDir) and re-validates the victim's ownership token while the lock is still in place, so concurrent reapers can't delete each other's freshly installed locks; a crashed claimant's dir is broken after a 30s TTL / dead PID. A livegbrain serveholder is identified from the parsedsubcommandand reported immediately with separate CLI-retry and MCP-tool choices; other live holders keep the bounded wait.gbrain syncnormally never reaches this error under a live serve — the cli.ts preflight (src/commands/sync-delegate.ts) probes the lock file read-only and delegates the sync through the serve's IPC socket instead. The heartbeat runs on the JS event loop, which is blocked during long synchronous WASM imports/CHECKPOINTs, so a workingdream/embedholder can look stale while alive; a steal-on-stale-heartbeat grace would let a second OS process open the same data dir and corrupt the catalog + pgvector extension (58P01 /internal_load_library/type "vector" does not exist), recoverable only by wipe+restore. A wedged-but-alive holder is never stolen: serve-tagged holders report immediately, while other holders time out with a message naming the PID. Each holder carries an ownership token (<pid>:<acquired_at>); the heartbeat andreleaseLockverify the on-disk lock is STILL theirs before touching it. In-memory engines take no lock. There is deliberately NO same-process reentrancy or same-PID special case: a secondacquireLockfrom the process that already holds the lock waits out the timeout like any other live holder (the classic cause is a command double-connecting a second engine on the same data dir; the fix is to reuse the connected engine at the dispatch layer, never to soften the lock).LockHandle.reapedmarks an acquisition that reaped a prior holder's lock (dead-PID reap, PID-recycling reap, or corrupt-lock-file removal — the only reaps that exist); the WAL auto-repair gate refuses to run surgery on a reaped acquisition since a corrupt lock file cannot prove its holder is dead. A corrupt-lock reap ALSO writes a persisted marker (<dataDir>.lock-reap.json, read via exportedmsSinceLastReap) so the NEXT process's clean acquisition is still repair-quarantined for 10 minutes — the in-process flag alone would let the reaper's successor run surgery under a possibly-live writer; dead-PID reaps (affirmative ESRCH verdict; EPERM reads as ALIVE) deliberately skip the marker so dead-holder recovery stays one-failed-command-plus-one-re-run. Heartbeat refreshes write via tmp+rename (a torn in-place write could be read mid-flight by a polling acquirer and misclassify a HEALTHY live holder as a corrupt lock). Pinned bytest/pglite-lock.test.ts. A corrupted store surfaces areinit-pgliterecovery hint viaclassifyPgliteInitError'scorruptverdict inpglite-engine.ts. ExportedinspectLockHolder(dataDir): LockHolderInfois the READ-ONLY inspection seam for status surfaces (gbrain engine status, smoke-test): never reaps, never mkdirs, never blocks; a dead-PID holder reads as not-held, a live serve holder reads asserve: trueso a probe can reportlocked_by_serveinstead of hanging on the single-writer lock.peekLock(dataDir)is the pure read of the same lock — nomkdir, no acquisition side effect, never throwsLiveServeLockError— so a third-party caller can tell "held by a live serve" apart from "available" without taking the lock itself; it reports not-held for a missing lock dir, a dead-PID holder, or an unreadable lock file, and is deliberately outside gbrain's own acquisition path, which is unchanged. Exported asgbrain/pglite-lock. -
src/core/pglite-resetwal.ts— pg_resetwal for PGLite NodeFS data dirs, in TypeScript (a port of the electric-sql/pglite pg_resetwal contribution, PR #994, Apache-2.0). Validates the PG17 pg_control layout fail-closed (WalResetUnsupportedErroron any unsupported shape — PG_VERSION ≠ 17, control ≠ 8192 bytes, control version ≠ 1700, bad seg/block size), removes stale postmaster.pid + old WAL segments + archive_status/summaries entries, writes a replacement shutdown-checkpoint WAL segment + CRC32C'd pg_control. Both file writes are atomic + durable (tmp cleared then opened'wx'so a pre-planted symlink at the predictable tmp name can never redirect the write, + fsync(tmp) + rename + fsync(parent dir)); write order is segment-first/control-last so a mid-write kill leaves a state that still fails startup and the next attempt re-runs (idempotent — a torn pair can never claim success). WAL segment size is capped at 64MB (pglite ships 16MB; the Postgres-general 1GB bound would let a corrupt-but-plausible control field drive a 1GB allocation on the repair path). Exports the shared PG17 layout literals (PG_CONTROL_FILE_SIZE,isWalSegmentName) consumed by pglite-repair.ts. LAYOUT COUPLING: any pglite bump past PG17 must revisit this file together with the./vectorexport blocker (TODOS.md "pglite upgrade blocker" entry). Pinned bytest/pglite-resetwal.test.ts. -
src/core/pglite-repair.ts— WAL-repair orchestrator wrapping the resetWal port with the safety layers that make it runnable automatically fromconnect():validateWalRepairTarget(read-only, fail-closed; refuses symlinked dataDir/pg_wal/global/pg_control — lstat follows INTERMEDIATE symlinks, soglobal/itself must be checked or surgery would write pg_control through it into a foreign dir; tolerates the in-dir.gbrain-lock), rename-based backup (the ENTIREpg_wal/dir + postmaster.pid renamed into a sibling<dataDir>.wal-repair-backup-<ts>/, only the 8KB pg_control copied — zero transient disk cost),restoreWalBackup(overwrite order: control first via atomic tmp+rename, then a pg_wal dir swap with the reset dir set ASIDE inside the backup — nothing is ever deleted during restore; mtime guard refuses when a foreign segment is newer than the backup; a missing/empty backup NEVER reportsrestored:true),WalRepairError(thrown when resetWal fails AFTER the backup — carries the receipt + the best-effort restore's REAL result so the seam'srestoredflag and thefailed-restored/failed-not-restoredmessage arms never lie), a cooldown sidecar<dataDir>.wal-repair-attempt.json(skip'recently-failed'insideGBRAIN_PGLITE_WAL_REPAIR_COOLDOWN_SECONDS, default 3600 — bounds the autopilot/supervisor reconnect loops) with episode-scoped backups (attempts within one corruption episode REUSE the episode's first backup — the pre-damage forensic state, honored only when the sidecar's path is a real non-symlink<dataDir>.wal-repair-backup-*sibling since the sidecar is user-writable JSON; retention keeps the newest 3 episodes, never pruning the open episode's), andattemptWalRepairAndRetry— the engine seam that NEVER throws (gates: kill-switch → reaped-lock'possibly-live-writer'→ 10-minute reap-marker quarantine (msSinceLastReap, cross-process) → validation → cooldown; then repair → retry create once → restore-and-record on failure; prints a repair-start stderr line so a timeout-killed attempt is self-explaining).inspectPgliteDataDiris the read-only diagnosis forgbrain doctor+pglite-repair --dry-run. Imports runtime values only frompglite-lock.ts/pglite-resetwal.ts/node:fs — never frompglite-engine.ts(no cycle; the engine statically imports THIS file per the engine-live rule). Pinned bytest/pglite-repair.test.ts+test/pglite-wal-repair.serial.test.ts. -
src/commands/pglite-repair.ts—gbrain pglite-repair: the manual surface for WAL repair (--dry-run | --yes | --json | --path <dir>; CLI_ONLY + SELF_HELP; returns an exit code viasetCliExitVerdict, neverprocess.exit). Never connects an engine — works when the DB won't open and when auto-repair is disabled.--dry-runis strictly read-only. Its confirmation prompt (andsrc/commands/reinit-pglite.ts's) writes to stderr so--jsonstdout stays clean, refuses non-TTY stdin in-prompt (defense-in-depth behind the caller-side "Non-TTY environment requires --yes" guard), resolvesfalseon EOF/close instead of parking forever on a closed or piped stdin, and cleans up its listeners;--yes/-ystays the non-interactive path. The real run validates BEFORE locking (acquireLockmkdirs the data dir — a typo'd--pathmust not create directories), refuses a live lock holder (pre-lock diagnosis names the PID; a livegbrain serveis called out), refuses a reaped acquisition (refused_reaped_lock— no--forceby design: force-removing.gbrain-lockwould reopen the concurrent-writer hole), re-validates under the lock, repairs with episode-backup reuse, and records the attempt in the sidecar. Pinned bytest/pglite-repair-command.serial.test.ts. -
src/core/pg-access-classify.ts— Postgres ACCESS-error classifier, the reason/remediation layer of the db-availability loop (detect →GBRAIN_DB_ACCESSmarker → skills/db-repair →gbrain db-repair). Two entry points, one diagnosis type:classifyPgAccessError(err, ctx?)for thrown errors (data-driven orderedREASON_ROWStable, first match wins — specific rows before general ones) anddiagnoseDbConfig(ctx)for errorless config-plane reasons (no_url,env_shadowed— there is no error object at connectEngine's no-config exit or under an env shadow; never fake one). ThePgAccessReasonunion (16 reasons) is APPEND-ONLY — a compatibility surface once the bundled skills are in the wild, like progress phase names; consumers switch exhaustively with aneversentinel. Invariants:message/remediationare ALWAYS pre-redacted (safe for transcripts/receipts/issues); everyfixdescriptor is hardcoded or derived from the CURRENT config URL (deriveSessionPoolerUrl/deriveDirectUrl), never from anything parsed out of error text, andrun_commandargv[0] is alwaysgbrain; remediation copy has exactly ONE home: this module — db-repair, doctor, and MCP dispatch all renderdiagnosis.remediation, skills reference the command and never duplicate recipe text. TWO-AXIS design note (mirrored inretry-matcher.ts's header — do not "fix" one side to match the other): retry-matcher answers "should I retry?" and deliberately treatspassword authentication failedas retryable (auth race during DNS failover); this module answers "what went wrong?" and reports the same errorauth_failed+transient: false— both correct on their own axis; unknown reasons defertransienttoisRetryableConnError. Also exportsDB_ACCESS_MARKER_PREFIX/formatDbAccessMarker(the single source of the marker literal that skills/db-repair pins in bothdescription:andtriggers:; the action a reader takes is ALWAYS the hardcodedgbrain db-repair, never a command parsed from the marker),isDbAccessFailure(the ACCESS-class subset fail-open recall arms use to tell "one arm degraded" from "the DB is down"), and Supabase enrichment{pooler, projectRef, pausedSuspect}(paused projects aren't a distinct wire error — modeled as a suspicion flag on dns/timeout/tenant reasons). Pinned bytest/pg-access-classify.test.ts. -
src/commands/engine-status.ts—gbrain engine status [--json] [--probe] [--brain <id>]: the ENGINE-FREE detection primitive (CLI_ONLY, dispatched in handleCliOnly BEFORE connectEngine like pglite-repair; deliberately NOT an operations.ts op — an MCP server can't serve a status call when its DB is down). Zero round-trips without--probe: reports (JSONschema_version: 1)effective_enginevsconfig_file_engine(can differ under a transient env URL),db_url_source, env-shadow + both-env-URLs precedence note, redacted URLs only, brain id from the engine-free resolver (NOTget_brain_identity, which needs an engine), thin-client flag, and a Postgres pooler block (supabase-pooler detection,resolvePrepare, direct/session-pooler derivability,GBRAIN_DISABLE_DIRECT_POOL, pool sizes).--brain <id>/mount resolution reports the MOUNT's engine/URL, never the host's.--probeis a SINGLE bounded connect +SELECT 1(the driver's built-inconnect_timeout— never a custom race;noRetry, never the 3-attempt ladder); Postgres success also reportsConnectionManager.describeMode(); failure returns a classifiedPgAccessDiagnosis. PGLite probe is LOCK-AWARE: a live serve holding the single-writer data-dir lock reportslocked_by_serve(healthy-with-note) instead of hanging ~30s and misreporting a healthy brain. Exit 0 healthy, 1 on config diagnosis or failed probe, 2 on flag errors. Pinned bytest/engine-status.test.ts. -
src/commands/db-repair.ts—gbrain db-repair [--yes] [--apply-rewrites] [--json] [--force] [--undo-last-rewrite] [--dry-run]: ENGINE-FREE Postgres-access repair, sibling of pglite-repair (PGLite brains are redirected there). Never calls connectEngine; default invocation is DIAGNOSE-ONLY (--dry-runis an explicit alias). Consent is tiered and flag-gated, never TTY-dependent: auto tier under--yes(bounded reconnects, pending migrations in-process,CREATE EXTENSION vector,docker startof gbrain's own container viadocker-postgres.ts); rewrite tier under--yes --apply-rewritesonly (config-filedatabase_urlrewrites — pooler form, session pooler,?sslmode=require— printed first, candidate connect-probed BEFORE persisting, receipted, undo-able); manual tier never applied (credentials/paused-project/env recipes printed — relaying the recipe IS the repair).--apply-rewriteswithout--yesis an error. Invariants: fix targets derive only from the CURRENT config URL, never error text; the prober uses exactly ONE connection (poolSize: 1— diagnosing pool exhaustion with a 10-connection pool would worsen the outage); rewrite cooldown is ≤24h per (reason, action) with--forcebypass while auto-tier fixes are NEVER cooldown-blocked (availability first); every rewrite stores the prior URL in the 0600~/.gbrain/db-repair-undo.json(a secret — never the redacted receipts) and--undo-last-rewrite(--yes-gated) restores it — undo runs INSIDE the advisory lock, emits the same--jsonenvelope, validates the record's postgres:// scheme before restoring, and is itself reversible (the outgoing URL becomes the new undo record); anO_EXCL-created advisory lockfile blocks concurrent double-rewrites (check-then-write race closed; one stale-holder retry; other write failures stay fail-open); a healthy probe exits 0 "nothing to fix" (the forged-marker defense) and additionally REPORTS (never rewrites) stale DB-planeengine/database_urlrows; the schema probe treats a ZERO-ROWpg_extensionresult aspgvector_missing(the query never errors on an absent extension); the docker arm matches the container's REAL inspected host port (deps.docker.hostPort) as well as the default 5434; a successful rewrite prints the "restart long-lived gbrain processes" note (config rewrites reach NEW processes only). Refusals: thin-client (no local DB), non-host brain resolution (a mount outage must never rewrite host config — mount-targeted repair is a filed TODO), PGLite (→ pglite-repair). InjectableDbRepairDepsprober/executor seam for tests;defaultDeps.probeAccessis also reused by the init ladder (one prober, one contract). Pinned bytest/db-repair.serial.test.ts. -
src/core/db-repair-receipts.ts— the shared seam betweengbrain db-repair(writer) and doctor'sdb_repair_recurrencecheck (reader — it must run in doctor's dead-DB filesystem lane, exactly when repairs are repeating). JSONL at~/.gbrain/db-repair-receipts.jsonl; row shape{ts, brain_id, reason, action, outcome: 'applied'|'refused'|'diagnose'}— onlyappliedrows count toward the recurrence threshold (3+ same-reason per brain in 7 days). Redacted before write (the classifier redacts its own copy; actions are fixed strings), fail-open (a receipts problem never blocks a repair), capped at 200 rows ON EVERY WRITE (the recurrence check reads the file on every doctor run — it must not grow unbounded on exactly the machines having chronic problems) — EXCEPTappliedrows inside an 8-day retention window, which are exempt from the flat cap (separately bounded at 200): a diagnose flood from an agent loop must never evict the cooldown/recurrence memory. The reader is per-line tolerant (one torn row never discards every other receipt). Also ownsrewriteCooldownBlocked(24h per (reason, action), rewrite tier only). -
src/core/degraded-engine.ts+src/core/degraded-marker.ts— degraded-mode serve, STARTUP-SCOPED by design: when Postgres is unreachable atgbrain servestartup, serve boots oncreateDegradedEngine(a concrete object whose method set is enumerated from PostgresEngine's prototype at construction — no dynamicgettrap;kindis an explicit getter, read synchronously at ~29 branch sites;disconnect()is a no-op while dead so shutdown never triggers a reconnect) instead of dying. Every tool call attempts ONE reconnect (single-flight across concurrent callers, min-interval gated ~5s — no connect storms against a dying pooler); calls inside the window throw the STORED original error (stale-but-honest, refreshed on every real attempt) so MCP dispatch classifies the REAL error shape into thedatabase_errorenvelope + marker. Thereconnectcallback is the FULL deferred connectEngine (config merge, gateway, migrations — guarded in cli.ts: a vanished config.json throws instead of connectEngine's process.exit killing the live server); first success swaps in the live engine permanently and fires recovery callbacks (serve re-runs deferred boot + sendstools/list_changed— clients that handshook degraded hold the reduced gate-hidden catalog and must refresh; the callback is a no-op once shutdown has started) — but the TRIGGERING call getsDegradedRecoveredRetryError, never a result: its source scope was resolved under the degraded fallback, so one client retry buys scope correctness (no call ever executes under the fallback scope against a live engine). Callers waiting on an in-flight attempt are latency-capped (callerWaitMs, default 2s — they get the stored error while the attempt finishes in the background, so a timeout-class failure can't stall the MCP handshake). Prototype GETTERS delegate too (throw the stored error while dead — never a silentundefined);DEGRADED_LAST_ERRORis a read-only accessor for the stored diagnosis (drives the HTTP/healthdegraded reason without consuming a reconnect). Serve-side degraded posture: source scope honors a validatedGBRAIN_SOURCE(tierenv) before seed_default, andstdioVisibleToolsfail-closes every gated op WITHOUT touching the engine. The degraded gate keys on the RESOLVED brain's engine (a PGLite mount on a postgres host keeps die-on-startup). Mid-session outages ridePostgresEngine.reconnect()+ per-call classified envelopes — degraded mode adds no mid-session state; PGLite startup failures keep die-on-startup (single-writer lock makes a lazy proxy wrong there; that lane's repair is pglite-repair). Kill switchGBRAIN_SERVE_DEGRADED=0(orfalse); structured[gbrain-serve] DEGRADED/RECOVEREDstderr lines.degraded-marker.tsis the dependency-free seam (Symbol.for-registered markers +isEngineDegraded/onEngineRecovered/degradedLastError) so server.ts/serve.ts/http-transport.ts can ask about degraded state without importing the heavy Postgres engine module. -
src/core/docker-postgres.ts— gbrain's OWN docker Postgres container, the shared seam between the init ladder's docker rung and db-repair'sconn_refusedauto arm.GBRAIN_PG_CONTAINER = 'gbrain-postgres'/GBRAIN_PG_IMAGE = 'pgvector/pgvector:pg16'/GBRAIN_PG_HOST_PORT = 5434live HERE and only here (DRY — the init rung and the repair arm must never carry two string literals that can drift). Ownership contract: gbrain starts and reuses this container, NEVER stops or removes it, and never touches a container whose credentials it cannot recover viainspectCredentials(docker inspect →POSTGRES_PASSWORDenv + the container's ORIGINAL port mapping — a freshly generated password can never match a surviving container; unrecoverable → refuse with a recreate-with-consent recipe). Hardening: the port binds loopback-only (127.0.0.1:5434— a single-machine brain store, not a network service); data lives on the namedgbrain-pgdatavolume so a container recreate never loses the brain; the password rides the child ENVIRONMENT (bare-e POSTGRES_PASSWORD— argv is world-readable inps);docker rungets a 10-minute timeout (the first run synchronously pulls the image; 30s stays for cheap ps/inspect/start calls).isGbrainDockerUrlis the one predicate for "does this URL point at our container" (loopback + dedicated port — never error text). -
src/commands/init-prefer-postgres.ts—runPreferPostgresLadder: the Postgres-first install ladder behindgbrain init --prefer-postgres [--allow-docker] [--allow-create-db] [--local-postgres] [--json](the zero-configgbrain initdefault stays PGLite; this flag is the harness lane). Five rungs, first usable wins, every reachable-but-unusable rung prints a one-line note and falls through (only the PGLite floor is terminal): (1) env URL; (2) Supabase Management-API DISCOVERY viasupabase-admin.ts—SUPABASE_ACCESS_TOKEN(+SUPABASE_PROJECT_REFon multi-project accounts, auto-selected on single-project ones) +SUPABASE_DB_PASSWORD(the API can't return it), ~10s timeouts, the discovered URL is a CANDIDATE connect-probed before anything persists, creation stays dashboard guidance, the token is never persisted/logged/echoed; (3) local Postgres, opportunistic (only when PG* env vars are set or--local-postgres— a blind localhost probe on peer-auth dev installs just burns a timeout) and detection-only (CREATE DATABASE gbrainneeds explicit--allow-create-db); (4) docker viadocker-postgres.ts(explicit--allow-docker; idempotent reuse recovers real credentials via docker inspect; readiness-polled); (5)initPGLite+ explicit upgrade-later note. CallsinitPostgresCore(typedInitPostgresFailure, noprocess.exit— the returnable core that makes rung fall-through possible) and reuses db-repair'sdefaultDeps.probeAccessprober. Prints the rung-entry criterion line (Postgres wins on concurrency/multi-machine/1000+ pages; PGLite keeps the per-turn hook lane) so the preference is informed, not silent. Guards: an ALREADY-CONFIGURED brain refuses the whole ladder (rung choice is environment-dependent, so a re-run during an outage would let the PGLite floor overwrite a healthy Postgres config — outages are db-repair's lane); a bareDATABASE_URL(vs the stated-intentGBRAIN_DATABASE_URL) is adopted only when the target is already a gbrain brain or holds no tables at all (classifyDbContent, fail-closed on foreign/unknown — deploy platforms export DATABASE_URL pointing at the APP's database); the docker rung content-guards BOTH reuse and fresh-create (a fresh container can attach a survivinggbrain-pgdatavolume) and detects the surviving-volume auth mismatch (fresh password rejected → adoption/clean-start recipe instead of "never became reachable"); rung-note error text is redacted.--jsonemits ONE final{status, engine, ladder_rung, url_source}envelope and stdout is EXACTLY that document —withStdoutToStderrreroutes bothconsole.log(Bun writes it to fd 1 directly) and bareprocess.stdout.writearound the inner init cores. -
src/commands/doctor.ts—gbrain doctor [--json] [--fast] [--fix] [--dry-run] [--index-audit] [--no-migrate]: health checks.--no-migratekeeps doctor observational: the CLI connects withprobeOnlyso a clean-or-behind schema is reported as-is instead of being auto-migrated before the checks run. The file is a façade carryingbuildChecks/runDoctorand output rendering; the check-function library lives in bundles undersrc/commands/doctor/checks/plus four tail-cluster modules undersrc/commands/doctor/(see that entry), all re-exported here so the full surface is unchanged — structural guards pin its source text viatest/helpers/doctor-source.ts, never by reading this file alone. Checks includejsonb_integrity+markdown_body_completeness(reliability),schema_version(fails loudly whenversion=0, routes togbrain apply-migrations --yes; a version AHEAD of this client'sLATEST_VERSIONwarns "upgrade this client"; and when the ledger reads current, a read-onlydetectMissingColumnsdiff fromsrc/core/schema-verify.tsruns INSIDE the ledger-current branch — a PgBouncer-swallowed ALTER TABLE can advanceconfig.versionover a physically narrower table, so missing live columns downgrade the ok to a warn naming them with thegbrain init --migrate-onlyhint; diff failure is best-effort and the ledger ok stands; positional wiring pinned bytest/doctor-schema-column-diff.test.ts),upgrade_errors(asynccheckUpgradeErrors(engine)over~/.gbrain/upgrade-errors.jsonl: the warn downgrades to an EXPLICIT status-ok line — never silence — only when the past failure is provably superseded, i.e. the running binary is at/past the failed target version AND the schema verifies current viaengine.getConfig('version'); a missing engine or unverifiable schema keeps the warn, fail-closed, becauseself-upgradeswaps the binary beforepost-upgraderuns migrations and the binary alone can lie; pinned bytest/doctor-upgrade-errors.test.ts+test/doctor-upgrade-errors-stale.test.ts),queue_health(Postgres-only: stalled-forever active jobs started_at > 1h, waiting-depth-per-name > threshold default 10 viaGBRAIN_QUEUE_WAITING_THRESHOLD, and dead-lettered subagent jobs withlast_errormatching theprompt_too_longclassifier in last 24h),sync_failures([CODE=N, ...]breakdown for unacked-warn + acked-ok; severity comes from the shareddecideSyncFailureSeverityinsrc/core/sync-failure-ledger.tsso the LOCAL and REMOTE/thin-client doctor surfaces can never drift — a stuck bookmark escalates to FAIL once an OPEN failure has blocked past the staleness window or ≥10 files block, while alreadyauto_skippedrows stay a visible WARN),rls_event_trigger(healthyevtenabledset is('O','A')only; fix hintgbrain apply-migrations --force-retry 35),graph_coverage(short-circuits to ok whenSELECT COUNT(*) FROM pages WHERE type IN ('entity','person','company','organization')returns 0; WARN hint isgbrain extract all),embedding_column_registry(probes each declared column via Postgresformat_type(atttypid, atttypmod)to catch dim mismatch with a paste-readygbrain config set embedding_columns '{...}'hint, probes HNSW index presence viapg_indexes, computes default-column population viaCOUNT(*) FILTER (WHERE <col> IS NOT NULL) / COUNT(*)warning below 90% except empty brains where chunk_count=0 short-circuits to ok; PGLite parity viaexecuteRaw), andskill_brain_first(walks SKILL.md viaautoDetectSkillsDirReadOnly, callsanalyzeSkillBrainFirst()fromsrc/core/skill-brain-first.tsper file with structuredCheck.issues[]; warn statesmissing_brain_first/brain_first_typo, ok statescompliant_callout/compliant_phase/compliant_position/exempt_frontmatter/no_external; snapshot+diff audit at~/.gbrain/audit/skill-brain-first-YYYY-Www.jsonl).--fixdelegates inlined cross-cutting rules to> **Convention:** see [path](path).callouts viasrc/core/dry-fix.ts(and MISSING_RULE_PATTERNS for the brain-first callout);--fix --dry-runpreviews.--index-audit(Postgres-only, informational, no auto-drop) reports zero-scan indexes frompg_stat_user_indexes. Every DB check runs under a progress phase;markdown_body_completenessruns under a 1s heartbeat.runDoctorusesautoDetectSkillsDirReadOnly(fromsrc/core/repo-root.ts; install-path fallback socd ~ && gbrain doctorfinds bundled skills);--fixcarries a D6 install-path safety gate that refuses auto-repair whendetected.source === 'install_path'(would rewrite the bundled tree). The Lane D supervisor check atdoctor.ts:1011-1043consumessummarizeCrashes(events)fromsrc/core/minions/handlers/supervisor-audit.ts(warn at>=1real crash; ok message hasclean_exits_24h=N; warn message hasruntime=A oom=B unknown=C legacy=Dper-cause breakdown) so OOM/runtime/unknown crashes are distinguishable from clean code=0 worker drains; cross-surface parity withgbrain jobs supervisor statusis pinned by source-grep wiring assertions requiring the breakdown substrings in BOTHdoctor.tsandjobs.ts.checkSyncFreshness(exported, inrunDoctorlocal +doctorReportRemotethin-client) is a staleness probe: warns at 24h, fails at 72h or never-synced; future-last_sync_atwarns ("clock skew") instead of falling through ok; env overridesGBRAIN_SYNC_FRESHNESS_WARN_HOURS/GBRAIN_SYNC_FRESHNESS_FAIL_HOURS(invalid fall back with once-per-process stderr warn via_resolveSyncFreshnessHours); failure messages embedsource.idso the printedgbrain sync --source <id>matches. A source holding a LIVE, non-expired per-source sync lock (inspectLock(engine, syncLockId(source.id))fromsrc/core/db-lock.ts) is reported as actively syncing (the message names the holder pid + host) and counted insynced_recently_count, NOT flagged stale — the live lock is the only honest in-progress signal (checkpoint banking can't distinguish in-progress from wedged: a blocked sync banks its files but writes no anchor). A blocked/failed sync's process has exited (no lock row) and a wedged holder stops refreshing (TTL lapses), so either falls through to the stale path and is never masked; the dynamicdb-lockimport is swallowed to a no-op on a stub engine or pre-lock-table brain, so this can only ADD an in-progress verdict, never suppress a real stale one. The in-progress note is appended to whatever verdict the buckets produce and is empty when nothing is syncing, so steady-state messages stay byte-for-byte unchanged. It has alocalOnly-gated git short-circuit (runDoctorpasseslocalOnly: true;doctorReportRemoteruns in the HTTP MCP serversrc/commands/serve-http.tsand keeps defaultfalseso that path never walks DB-suppliedlocal_pathvia subprocess — trust boundary). The local predicate mirrors sync's "do work?" gate (HEAD ==last_commitAND working tree clean viarequireCleanWorkingTree: 'ignore-untracked'so a quiet repo with only untracked dirs isunchangednot SEVERE, ANDchunker_version === CURRENT); the inline SELECT carrieslast_commit + chunker_version + newest_content_at. The REMOTE path computes lag vialagFromContentMs(newest_content_at, lastSync, now)from the stored column, NO git subprocess; LOCAL fall-through and the< 0clock-skew check stay on raw wall-clock. Three-bucket count math populatesCheck.details = {unchanged_count, synced_recently_count, stale_count}with the invariantsum === sources.length.checkCycleFreshnessis DELIBERATELY NOT git-short-circuited or content-relativized (last_commit == HEADcan't answer "did the full cycle complete?"; a sync can succeed while later cycle phases fail; different axislast_full_cycle_at). Pinned bytest/doctor.test.ts(incl. the IRON-RULE guard banning stale verb names, the sync_freshness boundary matrix, the D4 guard verifying git probes are NEVER called whenlocalOnlyis unset/false, the three-bucket invariant, and the untracked-folders / remote-never-shells-out trust-boundary cases).pglite_data_dircheck: fs-only check that runs when a PGLite brain FAILS to connect (!fastMode && !engine && config.engine === 'pglite', placed afterorphan_clones, before the DB-checks gate):computePgliteDataDirCheck(dataDir, diagnosis)(exported pure fn,computeWorkerOomLoopCheckconvention) maps theinspectPgliteDataDirverdict to a Check — corruption-likely/looks-healthy-but-unopenable/unsupported-layout →failnaminggbrain pglite-repair --dry-run/--yesor the rebuild path, live-lock/missing-dir →warn; allremediation_status: 'human_only'(Minion remediation needs the DB that is down). Escalates when ≥2 repair attempts failed inside 7 days (unclean-shutdown genesis still active → engine-switch pointer) and reports retained backup-dir inventory (orphan_clones disk-visibility class). Registered indoctor-categories.tsOPS_CHECK_NAMES. Pinned bytest/doctor-pglite-datadir.test.ts. silent-failure checks:content_hash_duplicates(single GROUP BY over(source_id, content_hash)with FILTER aggregates — never N² — flagging hash groups that hold BOTH a bare and a path-prefixed slug, the wrong-import-root pattern; warn carries sample pairs + thepages delete→purge-deleted --older-than 0remediation);undeclared_db_only_pages(per source with a local repo: markdown pages with no backing file outside every declared + derive-phase-default db_only prefix — the one check deliberately allowed to stat the repo);db_only_collector_collision(configured recipeoutput_pathsinside a declared db_only dir — auto-gitignore means sync AND import silently skip the collector's files; same warning fires in sync'smanageGitignoreat config-write time). All warn-level, engine-parity pinned bytest/e2e/doctor-silent-death-parity.test.ts; units intest/doctor-silent-death-checks.test.ts.graph_signals_coveragecheck wired into bothrunDoctor(local) anddoctorReportRemote(HTTP/JSON thin-client path). Readssearch.graph_signalsconfig first, falls back to mode default; silentokwhen disabled. Computes inbound link coverage on the page set; warns at <10% withgbrain extract allfix hint;okat ≥30% ("fire on most queries") and 10-29% ("fire occasionally"), each with the percentage embedded. Pinned by cases intest/doctor.test.ts.subagent_providercheck (layer 3 of 3). Resolves subagent model config in runtime order (models.subagent>models.default>models.tier.subagent> built-in default) and warns when the selected model lacks native tool-loop capability (message names the bad value + paste-ready fixgbrain config set models.tier.subagent anthropic:claude-sonnet-4-6); also warns whenmodels.defaultwould sneaksubagentinto a non-Anthropic provider via tier inheritance. OK when subagent tier resolves to Anthropic. Tests intest/doctor.test.ts.computeWorkerOomLoopCheck(engine)is the single authoritative OOM-loop signal, unioning supervisedsummarizeCrashes(readRecentSupervisorEvents(24)).by_cause.rss_watchdog(cross-week read viareadRecentSupervisorEventsso a Monday window can't lose Sunday) + bare-workerminion_jobs error_text='aborted: watchdog'count (Postgres-only; the same sourcequeue_healthsubcheck 3 reads). Cap comes from the latestrss_watchdog_loopbreaker alert'smax_rss_mb, elseresolveDefaultMaxRssMb()fallback. fail at breaker-tripped or oomKills≥5, warn at ≥1, null otherwise.computePoolReapHealthCheck(engine)is the Postgres-onlypool_reap_healthcheck readingreadRecentPoolRecoveries(1)— fail when reconnect failures>0 (reconnect throwing is the actionable signal), warn at ≥10 reaps/hr (pooler thrash), null otherwise. Both registered inbuildChecksafter thesupervisorblock. ThesupervisorcauseStr carriesrss=N (see worker_oom_loop)andqueue_health's watchdog message cross-referencesworker_oom_loop.DoctorReport.top_issues+ the cause-ranked render header.worker_oom_loop+pool_reap_healthregistered under ops indoctor-categories.ts. Pinned bytest/doctor-worker-oom-loop.test.ts,test/doctor-pool-reap-health.test.ts.supervisor_singletoncheck, a SEPARATE check fromsupervisor(same split precedent as the niceness check) so a singleton-divergence warn can't clobber the crash/liveness precedence. Runs only when astartedsupervisor event was seen in the last 24h and a live engine is available. Reads the queue-scoped DB lock row (gbrain_cycle_locks WHERE id = supervisorLockId(queue)) and compares the lock holder (holder_host:holder_pid) against the local pidfile holder via the pureclassifySupervisorSingleton.mismatch→ warn (a second supervisor may be running with a different--max-rss; message names both holders, the effective cap from thestartedevent'smax_rss_mb, and the fixgbrain jobs supervisor stop);single→ ok (names holder + cap);no_lock→ no check emitted. Best-effort try/catch (silent skip on brains without the lock table). Registered under ops indoctor-categories.tsassupervisor_singleton. Pinned bytest/supervisor-db-lock.test.ts+test/doctor.test.ts.checkBatchRetryHealth:batch_retry_healthcheck surfacing Supavisor circuit-breaker incidents. Wired into bothrunDoctor(local) anddoctorReportRemote(thin-client). Reads last 24h. States:ok(zero exhausted in 24h OR <3 from a single site),warn(>=3 same-site OR >=5 cross-site),fail(>=20 sustained breaker). Surfaces badGBRAIN_BULK_*env at doctor startup. Corrupt-JSONL tolerant. Paste-ready fix hints in every warn/fail message. Also readsreadRecentDbDisconnects(24)and appendsDisconnect-call audit: N call(s) in 24h (most recent caller: <frame>).to ALL three message paths so connection-incident signal is greppable from onegbrain doctor --jsoncall (module-import wrapped in try/catch so older brains without the audit file degrade silently). Pinned bytest/doctor-batch-retry.test.ts(10 cases). three checks wired intorunDoctor()and the JSON envelope, all warn-only with paste-ready fix hints. (1)checkSourceRoutingHealth(engine)scans up to 200 pages on federated brains and flags pages whosesource_iddoesn't match whatresolveSourceWithTier()would have picked for theirsource_path; single-source brains short-circuit took; the 200-page cap is total across the brain so doctor stays under 5s. (2)checkOauthConfidentialHealth(engine)probes registered confidential clients for/tokenreachability. (3)checkAutopilotLockScope()(pure, no engine) compares the resolved lock path to$GBRAIN_HOME; warns when set but the lock lives elsewhere, with a PID-safe inspection hint (kill -0 <pid>before deletion). Pinned bytest/doctor-v0_37_7_checks.test.ts.buildChecks(engine, args, dbSource, connectError?): Promise<Check[]>exported as a test seam; the optionalconnectErroris the connect failure captured by the CLI's dead-DB fallback, which the null-engine path turns into a SYNTHESIZED classifiedconnectionfail entry (sochecks[name=="connection"]exists in every failure shape — smoke-test branches on it). Theconnectioncheck's failure path classifies viasrc/core/pg-access-classify.tsinto a redacted message +details: {reason, transient, fix_hint}naminggbrain db-repair(deliberately NOTremediation[]/makeRemediationStep— that lane feeds--remediate, whose Minion jobs need the very DB that's down; applicator boundary:--remediate= brain-DATA quality with a live engine,db-repair= DB ACCESS engine-free). The URL-onlypgbouncerPrepareCheckhelper and the engine-freedb_repair_recurrencecheck run BOTH before the connection check and in the dead-DB filesystem lane;pglite_scale(engine-fit, warn at ≥1000 pages on pglite) runs in the DB phase.computeDoctorReport(checks, extras?)accepts optional{engine, db_url_source}which land as additiveDoctorReport.engine/DoctorReport.db_url_sourceJSON fields (schema_version stays 2).runDoctoris a thin wrapper:buildChecks → computeDoctorReport → render + process.exit. All 10process.exitsites stay in the wrapper; the two early-return paths (no engine, connection failure) return partial check lists instead of inline exits (observable output identical). Pinned bytest/doctor-behavioral.test.ts(13 cases: pure aggregation math overcomputeDoctorReport, orchestrator cases for--fastskip set +--jsonflag + no-engine partial path + snapshot of load-bearing check names) andtest/doctor-cli-smoke.serial.test.ts(1 subprocess case spawningbun run src/cli.ts doctor --jsonagainst a fresh PGLite tempdir, asserting schema_version=2 envelope, status enum, non-empty checks array — the render-path coverage buildChecks-only tests miss; quarantined.serialbecause PGLite write-locks don't play with parallel runners). three checks wired intorunDoctor()and the JSON envelope:oversized_pages(warns on pages exceedingcontent_sanity.bytes_warn),scraper_junk_pages(warns on live DB pages matching any junk pattern that escaped ingest), andcontent_sanity_audit_recent(reads the last 7 days of audit events, aggregates by pattern+source). Default scans the 1000 most-recent pages;--content-auditopts into a full scan. All three warn-only with paste-ready fix hints (junk →gbrain sources audit <id>+git rmsource-of-truth, oversize → split or accept). two checks wired intorunDoctor()+ the JSON envelope:quarantined_pages(counts pages carrying thequarantinemarker viaengine.executeRawJSONB?existence, works on PGLite + Postgres; warn-only with agbrain quarantine listhint) andflagged_pages(countscontent_flagpages — searchable but odd; warn-only). Both skip gracefully (status ok, "Skipped") on engines/brains where the probe errors. Pinned bytest/doctor.test.ts.home_dir_in_worktree: filesystem check walking up fromgbrainPath()toward$HOMElooking for a.gitdirectory (main repo) or.gitfile (linked worktree pointer; Conductor + git-worktrees topology). Walk terminates at$HOMEso a.gitabove the user's home doesn't false-positive. HonorsGBRAIN_HOME(appends.gbrainto the override). Warn (not fail) with worktree-root path + paste-ready fix pointing atGBRAIN_HOMEoverride or moving the brain.--remediation-plan [--json] [--target-score N]prints what would run (stableid,idempotency_key,severity,est_seconds,est_usd_cost,depends_on);--remediate [--yes] [--target-score N] [--max-usd N]submits each plan step as a Minion job in dependency order, re-checking score between steps.--target-score Ndefaults to 90; refuses to start when target exceedsmaxReachableScore()and lists what's missing.--max-usd Nis the cron-safety guard — submission refuses when the plan'sest_total_usd_costexceeds the cap. JSON envelope adds aCheck.remediationfield (additive, schema_version unchanged). Pinned by tests intest/doctor.test.ts. 4 checks:abandoned_threads,calibration_freshness,grade_confidence_drift(mitigation surface; math ships later),voice_gate_health. Schema health is a two-check pair:schema_versionresolves throughschemaVersionHealth(src/core/schema-version-health.ts, shared with the remote doctor report; a DB AHEAD of the client warns with an upgrade-this-client hint rather than suggesting migrations), and on ledger-current brains a read-onlyschema_columnscheck diffs live columns against the expected schema (detectMissingColumns, dynamically imported inside the ledger-current branch — positional guard intest/doctor-schema-column-diff.test.ts) and warns naming the missing columns with thegbrain init --migrate-onlyhint. -
src/commands/doctor/— the doctor module directory (the meat behind thedoctor.tsfaçade).checks/holds the check-function library in bundles grouped by concern (core-health.ts,queue-jobs.ts,extraction-sync.ts(itsatom_provenance_driftcheck scopes drift to page-bound atoms and splits it intosource_changed/source_gone; atoms with nosource_slug— transcript-originsource_path-only rows, whose hash is over a file and whose page liveness cannot be resolved by slug — are reported as their ownslug_unboundcount and never enter the drift total or the WARN ratio),graph-embedding.ts,search-eval.ts,calibration.ts,consolidation-cycle.ts,pglite-worker.ts,routing-federation.ts,verbs-reflex.ts,stale-mentions.ts,engine-fit.ts— the last carryingpglite_scale+ the engine-freedb_repair_recurrenceover the db-repair receipts);schema-pack-checks.ts,report-remote.ts,bootstrap-checks.ts, andskill-checks.tsare the four tail-cluster modules (schema-pack checks, the remote/thin-client doctor report, bootstrap checks, skill checks). All are re-exported by the façade. Structural guards that pin doctor source text load it throughtest/helpers/doctor-source.ts:doctorSource()concatenates the façade plus everysrc/commands/doctor/**/*.ts(façade first, then sorted) so a module move can never take a pinned string out of a guard's sight;doctorFileSource(rel)reads one named file for positional/ordering assertions; the helper also feeds thedoctor-source-helperdetector inscripts/classify-tests.ts.facadeExpansioninscripts/generate-flag-registry.tskeeps this directory on the doctor command's flag-scan surface. -
src/commands/doctor/checks/search-eval.ts— search, model, and AI-config health checks.checkChatFallbackChainInertreturns one warning when either the effective file/environment config or the DB plane has a non-emptychat_fallback_chain; it returnsnullwhen both are empty, so clean doctor reports add no line. The local and remote doctor registries both consume the check.checkSearchMode(search_mode) reports the resolved mode and itssearch.*overrides; when the only override is an explicitsearch.reranker.modelrow equal to the mode bundle's own default (a row an init may have written on Voyage installs) it is called redundant with the precisegbrain config unset search.reranker.model— never a--reset, which would also wipe every other tuned search knob — and overrides that keep a brain OFF a sunsetting bundle default are never reset-nagged (a reset would re-arm a dying provider). -
src/core/default-source-path-check.ts— pureassessDefaultSourcePath(input)verdict for thedefault_source_local_pathdoctor check (no DB/FS access — caller supplies gathered inputs, the npm-squat-check.ts shape). INVARIANT:default.local_path: nullis the DESIGNED fallback topology (write-through nests undersync.repo_path), NOT an error — the check warns ONLY when the null pointer provably breaks something: the fallback repo is another source's own working tree (the leak-guard collision: every unscoped default write-through silently skipped), or default has file-backed pages and no resolvable root. Everything else (no pages, resolvable fallback, deliberate DB-only brain) is ok. The repair it names is the non-destructivegbrain sources set-path default <path>. -
src/commands/doctor/checks/default-source-path.ts— the gathering wrapper on the doctor surface: reads thedefaultsources row, page counts (live + file-backed),sync.repo_pathresolution, and the leak-guard collision, then delegates the verdict toassessDefaultSourcePath. Only gathers the fallback-topology inputs when the pointer is actually null; returns null for the skip verdict. -
src/commands/doctor/checks/home-worktree.ts— thehome_dir_in_worktreecheck: walks up from the gbrain home looking for an enclosing.git(dir or linked-worktree file), warning that agit addfrom the worktree root could stage the brain; stops at HOME) arepath.resolve()d before the containment test, so a trailing-slashHOME=/home/user/(a common shell/launchd spelling) cannot make a brain inside a worktree grade ok.isValidGitMarkerstructurally validates a candidate before it counts (a.git/dir must contain HEAD; a.gitfile must start withgitdir:— git itself rejects anything else), and invalid candidates CONTINUE the walk so a valid repo higher up still warns. Cheap (stat + read, no subprocess) — runs on every doctor invocation including--fast`. -
src/core/postgres-engine.ts— Postgres + pgvector implementation (Supabase / self-hosted). The facts/takes/code-edges/salience method clusters are implemented in narrow-deps modules undersrc/core/postgres-engine/(see the engine-module-dirs entry below); the class methods delegate to them and the façade keeps its full public surface.addLinksBatch/addTimelineEntriesBatch/addTakesBatchpass the batch as one JSONB document —INSERT ... SELECT FROM jsonb_to_recordset((\$1::jsonb)->'rows') AS v(...) JOIN pages ...bound throughexecuteRawJsonb({ rows })— which encodes arbitrary free text safely (anunnest(${arr}::text[])array-literal path would crash Postgres with "malformed array literal" on free text such as calendar/Zoom context) and sidesteps the 65535-parameter cap; takes declares native recordset column types (page_id int, weight real, active boolean, …) so no per-element casts; all three arebatchRetry-wrapped.disconnect()runsdrainBackgroundWorkBeforeDisconnect()before pool teardown (engine parity with PGLiteEngine — mode'disconnect', so residual telemetry buffers drop symmetrically on both engines; guarded so a never-connected/already-torn-down engine skips the drain).searchKeyword/searchVectorscopestatement_timeoutviasql.begin+SET LOCALso the GUC dies with the transaction instead of leaking across the pooled postgres.js connection.getEmbeddingsByChunkIdsusestryParseEmbeddingso one corrupt row skips+warns instead of killing the query.searchKeyword/searchKeywordChunks/searchVectorapply source-aware ranking by inlining the source-factor CASE andNOT (col LIKE …)hard-exclude fromsrc/core/search/sql-ranking.ts;searchVectoris a two-stage CTE (HNSW-safe inner ORDER BY, source-boost re-rank in outer SELECT) carryingp.source_idinner→outer._savedConfigretains the connect config;reconnect()tears down + recreates the pool from saved config (called by supervisor watchdog after 3 consecutive health-check failures, and bybatchRetryon a retryable connection error). Concurrent callers share one in-flight_reconnectPromise(they await the single reconnect rather than racing a half-rebuilt pool); ownership re-samples through the atomicdb.connect()token on the connect leg.reconnect(ctx?)accepts the triggering error and records a pool-recovery audit event (reap_detected/reconnect_other/reconnect_succeeded/reconnect_failed) for thepool_reap_healthdoctor check.executeRawis a single-statement passthrough — no per-call retry (unsound for non-idempotent statements; recovery is supervisor-driven).connect()appliesresolveSessionTimeouts()fromdb.tsas connection-time startup parameters (statement_timeout,idle_in_transaction_session_timeout) so orphan pgbouncer backends can't hold locks for hours.countStaleChunks()+listStaleChunks()server-side-filter onembedding IS NULLforembed --stale(eliminates ~76 MB/call client-side pull);upsertChunks()routes text-embedding writes through the column registry (both engines in parity): a caller-resolvedopts.embeddingColumndescriptor wins; otherwise the DB-planesearch_embedding_column+embedding_columnsconfig rows resolve viaresolveWriteColumnFromConfigRowsto the SAME active column + cast the read side searches (legacyembedding::vector on pre-registry brains; so a registry-routed brain never writes with a dimension mismatch against the legacy column). The INSERT column list, the$N::vector|::halfvec(N)cast, and everyembeddingreference in the ON CONFLICT CASE branches use the resolved column; it resets both the active column ANDembedded_atto NULL when chunk_text changes without a new embedding. Pinned bytest/e2e/upsert-chunks-registry-column.test.ts(PGLite always; Postgres DATABASE_URL-gated).initSchema()callsapplyForwardReferenceBootstrap()BEFORE replaying SCHEMA_SQL — a superset of PGLite's probe set: the shared column-only forward-reference cases (files.source_id,files.page_id,oauth_clients.source_id,oauth_clients.federated_read,sources.archived/archived_at/archive_expires_at,timeline_entries.event_page_id) plus Postgres-blob-only probes for state the PGLite blob doesn't carry (dream_verdicts.expires_at— see the forward-reference-bootstrap.ts entry); the entire probe path runs on the DDL connection threaded frominitSchema(so concurrent bootstraps cannot race on a Supabase pooler).disconnect()is idempotent —_connectionStyletracks whether the engine owns its pool (worker engines) or shares the module-level singleton; second call on an instance-pool engine is a no-op rather than clobbering the singleton; and a module-style engine only callsdb.disconnect()when it owns the singleton (_ownsModuleSingleton, set from thedb.connect()creation token), so a borrower probe engine's teardown leaves the cycle owner's connection intact. Pinned bytest/e2e/postgres-engine-disconnect-idempotency.test.ts+test/postgres-engine-singleton-ownership.test.ts.getBrainScoreempty-brain parity with PGLite — 100/100 with breakdown 35/25/15/15/10 whenpageCount === 0(both engines must agree to keepdoctor-report-remote.serial.test.tsdeterministic). ImplementsdeletePages(slugs, {sourceId}): Promise<string[]>viaDELETE FROM pages WHERE slug = ANY(\$1::text[]) AND source_id = \$2 RETURNING slug(single round-trip; caller chunks);resolveSlugsByPathsdoesSELECT slug, source_path FROM pages WHERE source_path = ANY(\$1::text[]) AND source_id = \$2; FK cascades throughcontent_chunks/links/tags/raw_data/timeline_entries/page_versions,files.page_id+links.origin_page_idgo SET NULL; throws when input exceedsDELETE_BATCH_SIZE(fromsrc/core/engine-constants.ts); both short-circuit on empty input. Implements the embedding-signature stale-detection quartet (sumStaleChunkChars,setPageEmbeddingSignature,invalidateStaleSignatureEmbeddings, widenedcountStaleChunks, all accept optionalsignatureextending "stale" to model/dims-swap drift via thepages.embedding_signatureJOIN, NULL grandfathered; theembedding IS NULLserver-side filter is preserved as the no-signature fast path); Pinned bytest/e2e/engine-parity.test.ts. Retry classifiers, ontology/recency helpers, and disconnect/pool-recovery audit writers bind statically; only the twoai/gateway.tsfallback lookups stay lazy and line-marked, in parity with PGLite.insertFact+insertFactsdo not hardcodetx.unsafe(\'${embedLit}'::vector`)for the embedding column.resolveFactsEmbeddingCast()(private) probespg_attributeonce per engine instance (cached in_factsEmbeddingCastSuffix) and returns'::halfvec'when migration v40 created the column as halfvec, else'::vector'; both insert paths use the cached suffix so the cast matches the actual column type (works on older pgvector that lacks implicit auto-cast). Test seam__resetFactsEmbeddingCastCacheForTest()clears the per-engine cache.withReservedConnectionroutes long-hold reserved work (CREATE INDEX CONCURRENTLY,transaction: falsemigration DDL, backfill write batches) to the DIRECT session lane when dual-pool is active, so multi-minute holds do not pin the worker's shared read pool — never rerouted inside an open transaction (same guard shape asexecuteRawDirect), semaphore-capped belowdirect_pool_sizewith deliberately NO minimum-1 floor (atdirect_pool_size=1a floor would let one long reserve consume the only direct session and starve the claim/renewLock heartbeats — the same starvation on the direct lane), overflowing to the shared read pool when the direct lane has no spare capacity or is unavailable; the permit is released on both fn throw and reserve failure.getPoolDiagnostics()(duck-typed, no BrainEngine change) surfaces theCheckoutGaugein-flight counters fromsrc/core/pool-gauge.tsat the raw/direct/reserved/tx seams — a tracked SUBSET (tagged-template traffic is untracked) thatdb-probe.tslabels honestly in health-probe failure lines. Pinned bytest/postgres-engine-reserved-routing.test.ts`. -
src/commands/doctor/checks/integrations-memorable.ts—memorable_relay_health(OPS category), the doctor surface for the optional third-party relay. Engine-free (file-plane reads only), pushed unconditionally so it survives--fast/--scope. Rung ladder: gate off → quiet ok; enabled WITHOUT the gbrain disclosure stamp → FAIL naming the exact fix (thememorable enableout-of-band state); CLI-side consent absent → warn; no runnable binary → FAIL; last relay run failed → warn with the CLAMPED cause (the sharedclampRelayCause; child text never lands in a doctor message verbatim) — EXCEPT the documented openclawno_decisive_stepsrejection, which becomes an ok-with-note (expected_openclaw_rejection, name-only capture refused until argument capture lands) held as a DEFERRED note that surfaces only after the codex rung below passes: on a mixed openclaw+codex host the rejection is the persistent last-relay state, and an early return would paint the check green while a trust-stale codex hook sits silently dead; receipts written but the child never reported → warn; codex hooks wired but no codex-harness receipt in the recent receipt window (tail-200 read) → warn (codex_hooks_never_fired— codex hooks fail silently on a stale trust entry); healthy → ok with structured details (out_of_band_settable: truealways rides — the enable flag can be flipped by the external CLI, which is exactly why the gate also demands the stamp). Pinned bytest/doctor-memorable.test.ts. -
src/core/postgres-engine/+src/core/pglite-engine/— narrow-deps engine module directories, mirrored in lockstep (the engine-parity discipline applies to these dirs exactly as to the façades). Each holdsfacts.ts,takes.ts,code-edges.ts, andsalience.ts: the corresponding BrainEngine method clusters implemented as free functions over a typed deps interface (e.g.PgliteFactsDeps) that the engine class satisfies, so each module depends only on the narrow slice it uses. The façade methods delegate to them via static top-level imports (the engine-live no-runtime-dynamic-import rule applies) and keep the full public surface. -
src/core/postgres-engine/forward-reference-bootstrap.ts—applyPostgresForwardReferenceBootstrap(conn): the forward-reference bootstrap for the RAWSCHEMA_SQLreplay path (probe + patch every column/table the embedded schema blob forward-references but older brains don't have yet — single probe round-trip, idempotent, fast no-op when nothing is missing). Peeled out ofPostgresEngineso BOTH replay entrypoints run it before the blob:PostgresEngine.initSchema()AND the standalone module-singletondb.ts:initSchema()(replaying with no bootstrap would wedge an upgrade-boundary brain onCREATE INDEXover a column the replayed blob forward-references). Callers MUST hold the initSchema advisory lock (key 42) onconnso concurrent bootstraps can't race on a transaction pooler. Mirror ofPGLiteEngine#applyForwardReferenceBootstrapin shape — keep in sync; covered bytest/schema-bootstrap-coverage.test.ts(PGLite A2 static check + the Postgres-blob CREATE-INDEX class-closure gate, which parsesSCHEMA_SQLand requires every migration-added blob-indexed column to have a probe here or an explicit exemption) +test/e2e/postgres-bootstrap.test.ts(Postgres side, live convergence cases). Probes are column-only (nullableADD COLUMN, plusSET DEFAULTwhere hardening the upgrade window needs it —dream_verdicts.expires_at); the owning migration stays the source of truth for backfill/constraints/indexes: on the engine path it runs right after the bootstrap under the same advisory lock, while the standalonedb.ts:initSchemapath replays the blob WITHOUT running migrations — safe because the NULL-tolerant read predicates treat pre-backfill rows as valid until the next full engine init. -
src/core/cjk.ts— Single source of truth for CJK detection. ExportsCJK_RANGES_REGEX,CJK_SLUG_CHARS(character-class fragment for embedding inside other regexes),CJK_SENTENCE_DELIMITERS(。!?),CJK_CLAUSE_DELIMITERS(;:,、),CJK_DENSITY_THRESHOLD = 0.30,hasCJK(s),isCJKDominant(s)(the ONE density test: CJK chars ≥ 30% of non-whitespace chars),countCJKAwareWords(s)(routes throughisCJKDominant— English docs with one Japanese term stay whitespace-tokenized; Chinese-dominant docs get char-counted), andescapeLikePattern(s)(escapes%,_,\\forILIKE ... ESCAPE '\\'). BMP-only ranges (Han / Hiragana / Katakana / Hangul Syllables). Consumers:expansion.ts,sync.ts:slugifySegment,operations.ts:validatePageSlug + validateFilename,chunkers/recursive.ts:countWords + DELIMITERS + extractTrailingContext(the overlap extractor branches onisCJKDominantso it counts the same unitcountWordsdoes),pglite-engine.ts:searchKeyword + searchKeywordChunks. -
src/core/latin-fold.ts— leaf text primitive (no engine/graph imports):NON_DECOMPOSING_LATIN, the lowercase-keyed table of the eleven Latin stroke/bar/ligature letters Unicode gives no decomposition (đ ð ø ł ħ ŧ ı ß æ œ þ), +foldNonDecomposingLatin(s). Call it AFTER lowercasing and AFTER the combining-mark strip so composed forms reduce in one pass (ǿ → ø → o). Consumers:entities/resolve.ts:slugify(entity fallback slugs — pre-foldĐăng Examplecollided ontoang-exampleand an all-stroke name slugged to empty) andlink-extraction.ts:normalizeBasename(basename-index keys + dir-hint candidates). Deliberately NOT applied tosync.ts:slugifySegment— the page-slug grammar keeps these letters (#3417) — so the dir-hint candidate step inmakeResolver/ the FS resolver tries both forms. -
src/core/chunkers/recursive.ts— base chunker: 300-word chunks, 50-word sentence-aware overlap, 5-level delimiter hierarchy. AppliessanitizeRemoteBodybefore splitting so only world Facts and ordinary prose enter retrieval text; all Takes and malformed protected sections are omitted. Non-overlapping portions reassemble to that sanitized input. The overlap extractor counts chars (not whitespace tokens) for CJK-dominant chunks — same unit ascountWords— and aligns to。!?or a whitespace-followed ASCII.!?; the L4 char-slice fallback advances throughsafeSplitIndexso astral pairs (emoji, non-BMP CJK) are never halved. Both are gated onisCJKDominant, so English output is byte-identical (pinned bytest/chunkers/recursive-cjk-overlap.test.ts). The markdown chunker version records the current index format. -
src/core/chunkers/semantic.ts— embedding-based topic-boundary detection: embeds sentences, computes cosine-similarity valleys, smooths with a Savitzky-Golay filter (5-window, 3rd-order polynomial) to find chunk boundaries. Sanitizes the full body before sentence embedding and boundary detection. -
src/core/chunkers/llm.ts— LLM-guided chunking: pre-splits into 128-word candidates via the recursive chunker, then asks a Haiku-class model "where does the FIRST topic shift occur?" per window. -
src/core/search/dedup.ts— 4-layer result dedup + compiled-truth guarantee: (1) top 3 chunks per page by score, (2) drop chunks >0.85 Jaccard-similar to already-kept chunks of the SAME page (cross-page near-dups survive — two legitimately similar pages both return), (3) no page type exceeds 60% of results — applied only when the candidate set actually contains multiple page types; a homogeneous set passes through unchanged because dropping (for example) one of three distinct LongMemEvalnotesessions cannot introduce diversity and destroys multi-session recall (pinned by the homogeneous-set case intest/dedup.test.ts), (4) max 2 chunks per page (default; the two-pass structural expansion inhybrid.tswidens it), (5) ensure at least 1 compiled_truth chunk per page. Page identity is the compositepageKey()(source_id, slug) — the one canonical key helper every layer uses, so slug collisions across sources can't collapse recall. -
src/core/audit-slug-fallback.ts— Weekly ISO-week-rotated audit JSONL at~/.gbrain/audit/slug-fallback-YYYY-Www.jsonl.logSlugFallback(slug, sourcePath)fires whenimportFromFilefalls back to a frontmatter slug becauseslugifyPathreturned empty (emoji / Thai / Arabic / non-CJK exotic-script filenames).readRecentSlugFallbacks(days)reads the last N days forgbrain doctor'sslug_fallback_auditcheck. HonorsGBRAIN_AUDIT_DIRvia the sharedresolveAuditDir(). Separate surface fromsync-failures.jsonl— that file carries bookmark-gating semantics that info events shouldn't trigger. -
src/core/embedding-pricing.ts—EMBEDDING_PRICINGmap keyedprovider:modelfor the post-upgrade reindex cost estimate. Sibling toanthropic-pricing.ts; EMBEDDINGS only — chat/completion pricing lives inmodel-pricing.ts(different unit) and is never mixed in. Every entry carries its official source URL + the date it was last read. Entries: OpenAI text-embedding-3-large ($0.13/1M), 3-small ($0.02/1M), ada-002 ($0.10/1M); Voyage 4-large ($0.12/1M), 4 ($0.06/1M), 4-lite ($0.02/1M), legacy 3-large ($0.18/1M), 3 ($0.06/1M); ZeroEntropy zembed-1 ($0.05/1M), zerank-2 ($0.025/1M); Mistral mistral-embed ($0.10/1M); Perplexity pplx-embed-v1-4b ($0.03/1M), 0.6b ($0.004/1M).voyage-4-nanois deliberately unpriced (open-weight variant, no published hosted rate) so it degrades to "estimate unavailable" rather than a fabricated 0.lookupEmbeddingPrice(modelString)returns a tagged union (knownwith price +unknownwith provider name);estimateCostFromChars(charCount, pricePerMTok)uses 3.5 chars/token. Unknown providers degrade to "estimate unavailable" instead of fabricating numbers. -
src/core/post-upgrade-reembed.ts— Pure functions backing thegbrain upgradechunker-bump cost prompt.computeReembedEstimate(engine, model)queries real SQL (COUNT(*)+COALESCE(SUM(LENGTH(compiled_truth)) + SUM(LENGTH(timeline)), 0)) onpages WHERE chunker_version < MARKDOWN_CHUNKER_VERSION.formatReembedPrompt(est, graceSeconds)is the stderr-line formatter.runPostUpgradeReembedPrompt(engine, model, opts)orchestrates the 10-second Ctrl-C window; TTY-only wait (non-TTY auto-proceeds for CI / cron);GBRAIN_NO_REEMBED=1bails with a doctor-warning marker;GBRAIN_REEMBED_GRACE_SECONDS=0skips the wait. -
src/commands/reindex.ts—gbrain reindex --markdown [--type PAGE_TYPE] [--limit N] [--dry-run] [--json] [--no-embed] [--repo PATH]. Walks markdown pages with stalechunker_version(plus unstamped contextual-retrieval mode when embedding is enabled) in 100-row id-keyset batches; a failed row cannot starve later batches in the same invocation.--typeadds a bound-parameterpages.type = $Nscope for focused backfills such as atom pages and is rejected by reindex modes that do not consume it. Current-version chunkless healing deliberately remains owned by the native, boundedembed --stalepath. Rows with non-nullsource_pathre-import viaimportFromFile; rows without fall back toimportFromContent. Both paths passforceRechunk: trueto bypassimportFromContent'scontent_hashshort-circuit — without it the chunker version bump never reaches pages whose source content hasn't changed, AND the stripFactsFence privacy strip never applies to chunks stored without it. Wired intosrc/commands/upgrade.ts:runPostUpgradeafterapply-migrations. The DB-only fallback (no source file on disk) does NOT pass body-onlycompiled_truthtoimportFromContent(that path re-parses with EMPTY frontmatter and OVERWRITES the page's real frontmatter/title/timeline); itgetPage+getTags, reconstructs FULL markdown viaserializeMarkdown(frontmatter, compiled_truth, timeline, {type, title, tags}), and re-imports THAT so re-chunking a DB-only page preserves everything while bumpingchunker_version. Pinned bytest/reindex-preserve-tags.test.tsandtest/reindex.test.ts. -
src/commands/reindex-code.ts—gbrain reindex --code [--source ID] [--dry-run] [--yes] [--json] [--force] [--no-embed]. Walkspages WHERE type = 'code'in 100-row batches, replays throughimportCodeFilefor chunk + embed + content_hash folding. Idempotent unless--forcebypasses the content_hash early-return. Cost-preview model field readsgetEmbeddingModelName()from the gateway so preview reflects what the gateway will actually embed with. An informational stderr nudge insiderunReindexCode(so dry-run + execute both surface it): when the configured embedding model isn't code-tuned (allowlist{'voyage-code-3'}, case-insensitive bare match), prints a recommendation to switch tovoyage:voyage-code-3; suppress withGBRAIN_NO_CODE_MODEL_NUDGE=1,--no-embed, or--json. PureshouldNudgeCodeModel(bareName)returns a taggedNudgeDecisionunion (takes the bare model name, emits qualifiedvoyage:voyage-code-3for the paste-readygbrain config setline). When--yesis absent and the caller is non-TTY or passed--json, the cost gate refuses (exit 2, no spend) via the pure exportedbuildCostRefusal({json, previewMsg, preview, costUsd, model}): {stdout?, stderr?}— JSON envelope only when--jsonis explicit, otherwise a human refusal on stderr (the spend guardrail is independent of the output format).spend.posture=tokenmaxOR an explicit--max-cost off/unlimitedmakes the gate informational and proceeds;--max-cost offalso disables the runtime BudgetTracker cap. Pinned bytest/ai/voyage-code-3-recipe.test.ts,test/reindex-code-nudge.serial.test.ts,test/reindex-code-model-source.serial.test.ts(IRON-RULE guard for the cost-preview model source),test/reindex-cost-refusal.test.ts. -
src/core/fts-language.ts— Single source for the Postgres text-search configuration name used by FTS.getFtsLanguage()resolvesGBRAIN_FTS_LANGUAGE(defaultenglish), validates against/^[a-z][a-z0-9_]*$/(tsvector config names can't be bound as parameters, so the value is interpolated into raw SQL — the allowlist regex is the injection guard; invalid values warn once and fall back toenglish), and caches on first read (resetFtsLanguageCache()is test-only). Consumed by both engines'searchKeyword/searchKeywordChunks(websearch_to_tsqueryquery side), theconfigurable_fts_languagemigration, andreindex-search-vector(write-side trigger functions). Pinned bytest/fts-language.serial.test.ts+test/fts-language-migration.serial.test.ts(includes the'; DROP TABLE pages; --injection cases). -
src/commands/reindex-search-vector.ts—gbrain reindex-search-vector [--dry-run] [--yes] [--json]. Escape hatch for changingGBRAIN_FTS_LANGUAGEafter theconfigurable_fts_languagemigration has run (the migration shows applied and is skipped): recreatesupdate_page_search_vector+update_chunk_search_vectorwith the configured language — bodies mirror the migration's and KEEP theSET search_path = pg_catalog, publichardening (CREATE OR REPLACE resets proconfig) — then backfillspages(UPDATE-to-self re-fires the trigger) andcontent_chunks(direct vector recompute) in id-keyset batches ofBACKFILL_BATCH_SIZE(5000) viaUPDATE … WHERE id IN (SELECT … LIMIT n) RETURNING id, streaming phasesreindex_search_vector.pages/.chunksthrough the shared progress reporter (stderr). Confirmation gate:--yes, or an interactive TTY [y/N];--jsondoes NOT bypass the gate (non-TTY without--yesrefuses with a ConfirmationRequired envelope, exit 2). Idempotent. Pinned bytest/reindex-search-vector.serial.test.ts. -
src/commands/sync.ts—gbrain syncCLI + theperformSync/performFullSynclibrary entrypoints (consumed by the autopilot cycle and the Minion sync handler).performSyncInnerresolves the source's persistedconfig.strategy(markdown/code/auto) when the caller passes nostrategy— the autopilot lane, the dream cycle, the MCPsyncop and the single-source CLI path all omit it, and without this the walk fell back tomarkdown, importing nothing from a code source while still advancing the anchor and soft-deleting its modified code pages; an explicit--strategystill wins, so the--allfan-out is unchanged. Pinned bytest/sync-index-matches-tree.serial.test.ts(index == working tree across first/incremental/full sync). Six pure-function clusters live insrc/core/sync-{cost-gate,git,anchor,lock,reconcile,status-report}.ts(see the grouped entry below) and are re-exported through this façade, so the full import surface is unchanged.performSyncruns under a writer lock: per-sourcegbrain-sync:<sourceId>wheneveropts.sourceIdis set, wrapped inwithRefreshingLockfromsrc/core/db-lock.tsso long-running sources (250K+ chunks) don't lose the lock at the 30-min TTL mid-run; the bare no-source path uses the SAME refreshing lock;SyncOpts.lockId?: stringis the explicit override. The lock refresh AND its health probe route through the DIRECT session pool so Supavisor transaction-pool exhaustion (EMAXCONNSESSION) can't kill renewal; takeover is heartbeat-aware (it will NOT steal a holder whoselast_refreshed_atis withinGBRAIN_LOCK_STEAL_GRACE_SECONDS, defending an alive-but-starved holder); the import loop yields the event loop everyGBRAIN_SYNC_YIELD_EVERYfiles (setTimeout(0), notsetImmediate— Bun starves the timers phase) so the refreshsetIntervalheartbeat fires mid-import. This lock-identity invariant prevents async --allper-source worker racingsync --source fooon the global lock from corrupting the same source.performSyncthrows a typedSyncLockBusyErrorwhen the writer lock is held; the Minionsynchandler (src/commands/jobs.ts) catches it and marks the job SKIPPED (not failed) so a cron/autopilot tick defers to the holder without polluting crash metrics.performSyncInneris RESUMABLE (incremental path): it drains a PINNED target commit (lastCommit..pin), banking drained file paths viaappendCompleted(append-only delta into theop_checkpoint_pathschild table, migration v115 — one row per path, O(delta) rather than an O(N²) full-array rewrite), keyed bysyncFingerprint({sourceId, lastCommit})fromsrc/core/op-checkpoint.ts(paths underop:'sync'; the pinned target underop:'sync-target'), and advanceslast_commit/last_sync_atONLY at full import completion. Checkpoint writes route through the DIRECT session pool + bounded retry so they surviveEMAXCONNSESSION; the flush cadence is first-file then everyGBRAIN_SYNC_CHECKPOINT_EVERY(default 1000) files ORGBRAIN_SYNC_CHECKPOINT_SECONDS(default 10s), with a race-safependingCheckpointPathsdelta (single-flight swap, re-merge on failure) under parallel workers; a SIGTERM banks the in-flight delta via a no-retry one-shot (appendCompletedOnce, ordered before lock release throughregisterCleanup); and sustained flush failure aborts the run withreason:'checkpoint_unavailable'afterGBRAIN_SYNC_MAX_CHECKPOINT_FAILURESconsecutive fails rather than importing work it can never bank (every partial/blocked exit logs the banked-file count). A sync killed mid-import banks its progress, leaves the anchor unmoved (the source stays correctly stale to the autopilot scheduler —last_sync_atis never bumped on a partial), and the next runresumeFilters the same fixed diff to skip done files. The pin is the checkpoint's stored target when still reachable from HEAD, else live HEAD (a history rewrite / reset re-pins); completion advances to the pin, NOT live HEAD, so commits landing past the pin are a clean next-sync diff and there is no cross-run staleness window. After import a pin-reachability gate (git merge-base --is-ancestor pin HEAD) decides: forward commits on top of the pin (e.g. a background enrich process committing to the same repo every ~2 min) are SAFE and do not block the run; only a real rewrite (pin not an ancestor of HEAD) blocks and discards the checkpoint. A file added inlastCommit..pinbut gone from disk (deleted by a commit after the pin) is SKIPPED and marked completed, not recorded as a failure. Downstream extract/facts/embed are size-gated: inline only fortotalChanges <= 100; large syncs defer to the resumableextract --stalewatermark +embed --stale/backfill + the facts cycle phases, so a 44K-page facts/embed pass never re-blocks import convergence (sync convergence == import convergence). Worker engines wrap in try/finally so disconnect always fires; both PGLite-detection sites useengine.kind === 'pglite'. CLI accepts--workers N(alias--concurrency N) validated viaparseWorkers(explicit bypasses the file-count floor; auto path defers toautoConcurrency()). The newest-first descending-lex order usessortNewestFirst(addsAndMods)fromsrc/core/sort-newest-first.ts(shared withgbrain import).gbrain sync --allruns a continuous worker pool:parseWorkers-validated--parallel N(defaultmin(sourceCount, --workers, DEFAULT_PARALLEL_SOURCES=4)), long-lived async workers pulling from a shared FIFO queue (no head-of-line blocking), per-sourcewithSourcePrefix(src.id, ...)so everyslog/serrline carries[<source-id>];--skip-failed/--retry-failedare scoped per source (acknowledgeFailures(sourceId);--allacks every source, single-source acks only its own) and run UNDER parallel — the failure ledger is per-(source_id, path)and serialized throughwithLedgerLock, so recovery syncs never need--serial; a connection-budget stderr warning fires when (the factor: each per-file worker opens its ownPostgresEnginewithpoolSize=2). ExportsresolveParallelism,syncOneSource,buildSyncStatusReport,printSyncStatusReport,SyncStatusReportback thegbrain sources statusdashboard.--jsonenvelope{schema_version: 1, sources, parallel, ok_count, error_count, skipped_count}on stdout; human banners route to stderr viahumanSinksojqparses cleanly. Exit matrix: 0 all ok (sources skipped by--missing-path skipcount as ok), 1 any error.--missing-path <fail|skip>(default fail) handles sources whoselocal_pathdoes not exist on this machine — machine-specific state in a brain-wide table, so a brain registered from several machines fails every foreign source on every run;skipclassifies themskipped_missing_path(⊘ line, envelope entry withlocal_path, excluded fromerror_countand the rc gate) via the exported pure helpersparseMissingPathMode+partitionMissingPathSources, pinned bytest/sync-all-missing-path.test.ts; defaultfailstays loud because on a single-machine brain a missing path usually means an unmounted volume. (The non-TTY cost gate does not exit 2: a worker-backed engine can auto-defer, while a runtime without a worker surface reports a manual drain instead of claiming a queued job.) The dashboard SQL iscontent_chunks ch JOIN pages pg ON pg.id = ch.page_id WHERE pg.deleted_at IS NULLwitharchived = falseat the caller; embedding column resolved viaresolveEmbeddingColumn(undefined, cfg)fromsrc/core/search/embedding-column.tsso Voyage/multimodal/non-default-column brains count against the column they use; errors propagate (no swallow-catch). Every sync delete lane (removed-file drain, full-sync reconcile, un-syncable-page cleanup, rename-stale reconcile) SOFT-deletes viaengine.softDeletePages:deleted_atis set for a 72h recovery window, the autopilot purge phase owns the eventual hard delete, and a re-import within the window revives the page throughputPage's upsert (lanes without a threadedsourceIdfall back toDEFAULT_SOURCE_ID, matchingdeletePage's'default'scope). The removed-file drain is interleaved per-batch resolve+soft-delete usingengine.resolveSlugsByPaths+engine.softDeletePagesfromsrc/core/engine.ts(a 73K-delete commit takes ~292 SQL round-trips instead of ~146K, so one big-delete commit cannot jam every other source's sync); per-batch try-catch decomposes batch failures to one-elementsoftDeletePagesbatches per slug (same primitive, per-slug isolation), unrecoverable per-slug failures land infailedFilesand the run continues;pagesAffectedfilters to slugs that actually transitioned (phantom slugs and already-soft-deleted rows are excluded by the primitive'sdeleted_at IS NULLpredicate). The rename loop gets the same batched slug-resolve; a rename whose destination is un-syncable folds the source path into the delete set so the old page can't orphan. An entry-time bookmark-reachability guard distinguishes a gc'd anchor (cat-filefails →performFullSync) from a history-rewrite anchor that is merely no-longer-an-ancestor but still on disk: the latter is diffed tree-to-tree directly (git diff lastCommit..pinis an endpoint-tree compare, ancestry not required) so a force-push /master→mainconsolidation imports only the real delta instead of re-walking the whole tree forever; an oversized or failed diff degrades toperformFullSync.performFullSyncis itself authoritative for deletes — after an advancing full import it soft-deletes file-backed pages (source_path != nullAND strategy-awareisSyncable) whose source file no longer exists (samesoftDeletePages+ 72h-window semantics as the incremental lanes; already-soft-deleted rows don't inflate the reconcile count), sparingput_page/manual pages (nullsource_path) and metafiles. The stale-file decision routes through the pure, exportedplanReconcileDeletes(rows, currentFiles, isSyncablePath): it normalizes path separators on both sides of the membership test (a Windowspath.relativebackslash path vs a git-derived forward-slashsource_pathwould otherwise mark every page stale and wipe the source) and computes a mass-delete signal — when the reconcile would delete more thanMASS_RECONCILE_RATIO(50%) of the file-backed pages the strategy manages, on a source holding more thanMASS_RECONCILE_MIN_PAGES(20) of them, the delete is REFUSED with a loud stderr warning (that shape is almost always a path-comparison bug or the wrong repo path, not a genuine bulk deletion);GBRAIN_ALLOW_MASS_RECONCILE=1restores the unguarded delete for genuinely intended bulk removals. Pinned bytest/sync-reconcile-mass-delete.test.ts. Below the valve, stale pages are partitioned by git history via exportedlistEverCommittedPaths(repoPath)(onegit log --all --no-renames --diff-filter=A --name-onlypass; null on non-git dirs → unchanged behavior): a stale path that EVER existed in history was genuinely deleted → reconciled; a path with NO history is DB-only write-through (never committed/pushed, e.g. lost to a fresh clone) → the page is KEPT and its markdown re-exported to the working tree viawritePageThrough, with a stderr hint to commit it ("absent from git" is the symptom of the missing write-through commit, not evidence the content is disposable). Pinned bytest/sync-reconcile-db-only.serial.test.ts.resolveSlugByPathOrSourcePath(insrc/core/sync-git.ts— see its dedicated entry below) delegates toengine.resolveSlugsByPathswhensourceIdis set, keeping the legacyexecuteRawfallback for the no-sourceId path.failedFilesis hoisted to the top ofperformSyncInnerso both delete-decompose and import loops feed the same bookmark gate. The cost gate is the sharedrunInlineCostGate(one implementation on BOTH the--alland single-source paths; runs at the command layer, never insideperformSync), mode-aware viaresolveWorkerBackedSyncEmbedMode+ posture-awareshouldBlockSyncfromsrc/core/embedding.ts. The DEFERRED path (v2 on, explicitly Postgres/worker-backed, and not--serial; source count controls fan-out only) is INFORMATIONAL (embedding goes to per-sourceembed-backfilljobs with their own$X/source/24hcap, default $25 viaSPEND_CAP_CONFIG_KEYfromembed-backfill-submit.ts; prints the cap + backlog + queued-job count, NEVER exits 2). The INLINE path (v2 off,--serial, or PGLite/unknown engine) gates on the DELTA estimate vssync.cost_gate_min_usd(default $0.50): below floor proceeds; above floor in a TTY prompts[y/N]; above floor in a non-TTY/--jsonsession auto-defers only when a worker-backed queue is actually available. Without that worker surface it exits 0 without wedging import convergence, writes no queue row, reportsmanual_drain_requiredwith reasonno_worker_surface, and emits one exactgbrain embed --stale --source <id>command per source; a worker-backed runtime with--no-auto-embedreports the distinctauto_submit_disabledpolicy reason without denying queue capability; intrinsic >100-file incremental deferral reaches the same final manual outcome on no-worker runtimes;spend.posture=tokenmaxmakes it informational and proceeds inline. The estimate MIRRORS EXECUTION instead of pricing the whole tree:estimateInlineNewTokensroutes through the sharedcomputeSyncDelta(src/core/sync-delta.ts) — fetch-first againstorigin/<branch>, prices only the committed delta (a dirty-but-caught-up tree → $0), with a full-tree CEILING only on the fail-open rungs (chunker drift, first sync, git-unavailable) honestly labeled;--fulladds the stale backlog (full sync sweeps it inline). Return shape carriesestimateKind: 'delta'|'ceiling'|'mixed'|'unchanged'+ceilingReasons. HelpersresolveCostGateFloorUsd(engine)+resolveBackfillCapUsd(engine)resolve viaparseUsdLimit(off/unlimited→Infinity; floor accepts0= block-on-any-spend). JSON envelopes carrymode+gatediscriminators (dry_run | deferred_notice | below_floor | auto_deferred_embeds | manual_drain_required | posture_tokenmax) + a paste-readyhint; terminal single-source and--allenvelopes carry per-sourceembed_backfilloutcomes so machine readers see queued/manual/skipped state;Infinityfloors/caps render as the string'unlimited'(never raw, which JSON-serializes tonull);SyncStatusReportSourcegainsbackfill_queued/backfill_active/backfill_last_completed_at; cost previews readgetEmbeddingModelName()(no hardcoded OpenAI). Format splits on the explicit--jsonflag only (human text otherwise).SyncOpts.noSchemaPack(CLI--no-schema-pack, threaded throughperformSyncANDsyncOneSource) skipsloadActivePackso pages fall back to legacy prefix typing — an escape hatch when a suspect pack regex wedges a sync. A per-file BEGIN heartbeatif (process.env.GBRAIN_SYNC_TRACE) serr('[sync] begin import: <path>')fires BEFOREimportFile(theprogress.tickfires only AFTER) so a stuck file is a begin-line with no matching completion. Triage doc:docs/architecture/serve-sync-concurrency.md(serve-delegated sync + theGBRAIN_SYNC_TRACE+--no-schema-packrecipes). On a PGLite host brain with a livegbrain serve, the cli.ts pre-connect hook routesgbrain syncthroughsrc/commands/sync-delegate.ts(delegation over the resolve-IPC socket; default-deny flag gate; 1s status polls; Ctrl-C →sync_abort; embeds always deferred to the serve's drain) —SyncOpts.onProgressis the seam performSync fires at phase boundaries and checkpoint flushes so the serve-side job record carries live progress;printSyncResultis exported for the delegated result path. Pinned bytest/e2e/sync-status-pglite.test.ts(IRON-RULE: PGLite seeds 2 sources × pages × chunks, soft-deletes 1 page, archives 1 source, validates the SQL excludes both AND uses the active embedding column),test/sync-cost-gate.serial.test.ts,test/sync-cost-preview.test.ts. Runaway-sync protection:resolveSyncHardDeadline(args, {isTty, env, defaultNonTtySec?})resolves a wall-clock hard deadline (precedence--no-hard-deadline>--hard-deadline <s>>--timeout <s>(non---all, which auto-arms the backstop) >GBRAIN_SYNC_MAX_RUNTIME_SECONDSenv > non-TTY default 3600s > none;HARD_DEADLINE_GRACE_SEC=30).src/cli.tsinstalls the out-of-band watchdog (seesrc/core/process-watchdog.ts) for the sync command BEFOREconnectEngineand disposes it in the dispatchfinally, so even an event-loop-starved sync — or a connect-phase hang — is SIGTERM-then-SIGKILLed by the deadline instead of orphaning under cron.runSyncregisters a SIGINT handler that aborts an interruptAbortControllercomposed viacomposeAbortSignals(...)(anAbortSignal.anywrapper over the defined signals) with the per-source--timeoutsignal, so Ctrl-C returns a cleanpartialand releases the lock through the normalfinally(process-cleanup.ts owns SIGTERM lock-release; the watchdog owns the hard kill).withRefreshingLockunref()s its refreshsetInterval. The spin's own root cause is not yet pinned (leading lead is catastrophic-backtracking in a pack link-inference regex, bounded by the redos-guard); the watchdog heartbeat plus the existing[gbrain phase]breadcrumbs are the diagnosis surface. Pinned bytest/sync-hard-deadline.test.ts(resolution precedence +composeAbortSignals). Monorepo subdir sources:--src-subpath <dir>(or a repo path that IS a subdir — auto-discovery viadiscoverGitRoot, i.e.git rev-parse --show-toplevel) splits the repo path intogitContextRoot(all git ops: pull/diff/rev-parse/cat-file) andsyncScopeRoot(walk/import/delete/rename scope); scoped syncs use git-root-relative slugs +source_path(full sync threadsslugRootintorunImport) so full and incremental agree; NAV-1/NAV-2 realpath containment rejects../-traversal and symlinked scopes resolving outside the repo BEFORE any git op, and a per-file realpath guard (isPathSafe) refuses symlink-escape files in the incremental drain and rename reimport (fail-closed intofailedFiles, so the bookmark can't advance past an escape); the full-sync reconcile is scope-restricted so a scoped sync never sweeps out-of-scope pages.--exclude <glob>(repeatable) filters scope-relative paths in both full and incremental paths; exclusion never deletes previously-imported pages (conservative, matching the metafile posture); an all-excluded run warns loudly (NAV-4). The persistedsync.excludeconfig key (comma- or newline-separated patterns; a trailing/normalizes to a<dir>/**subtree glob) is UNIONED with per-call--excludeon EVERY sync path — union, not override, so an ad-hoc flag narrows further but never silently re-opens a scope the operator persisted, and internal callers with no flag surface (autopilot, minion sync jobs, dream cycle) inherit the same indexing scope. CAVEAT: the key is BRAIN-GLOBAL — one exclude list scopes EVERY source, includingsync --all(where the per-call flag is forbidden), so a pattern meant for repo A silently narrows repo B; per-source scoping is a filed follow-up. The union resolves at the top ofperformSyncInner, ABOVE theperformFullSyncearly returns — position is load-bearing: the first sync's full walk is exactly where ignoring the persisted scope would permanently import excluded files that no later incremental revisits. Best-effort config read (an unreadable config never breaks a sync). A warn-and-continue internalgit pullfailure (non-timeout class — e.g. a local-path origin rejected byprotocol.file.allow=never) still falls through to sync the local working tree, but a ZERO-import run after a failed pull returnspartialwithreason: 'pull_failed'instead ofup_to_date:last_commitAND thelast_sync_atheartbeat stay frozen (so doctorsync_freshness/sources statusstaleness fires), the single-source CLI exits non-zero,sync --allexits non-zero if any source hit it (JSON envelope carries the per-sourcereason), and the autopilot cycle's sync phase maps it towarn. Timeout-class partials keep their pre-existing exit-0 / phase-oksemantics (they converge on retry; a failing pull does not). Pinned bytest/sync-pull-failed-anchor.serial.test.ts.resolveSlugByPathOrSourcePathis threaded into all 4 delete/rename call sites — see its dedicatedsrc/core/sync-git.tsentry below for the resolution contract. Above the size gate (totalChanges > 100) the deferred link/timeline extraction is DURABLY QUEUED, not just hinted — the defer branch submits anextractMinion job{stale: true, sourceId?, deferred_commit: pin}keyedextract-stale:<sourceId|default>:<pin>(repeat submissions toward the same drained pin coalesce; deliberately NO maxWaiting — an unscoped payload's coalesce filter matches ANY waiting extract job and would silently drop the sweep),timeout_msderived from extract.ts's exportedSTALE_TIME_BUDGET_MS+ headroom. The returned row is verified to be a live{stale:true}job (waiting/delayed/active) before the log claims "queued"; a finished row occupying the key slot (a prior sweep toward the same pin that completed before this run's pages landed — the checkpoint-resume / blocked-advance re-sync case) triggers a fresh submission under a run-unique key so those pages never strand stale. Submission is best-effort (failure falls back to the hint; pages stay stale + doctor-visible, never mis-stamped). Pinned bytest/sync-deferred-extract-queue.serial.test.ts. Unscoped default-write guard: when the resolver lands on tierseed_defaultfor a single-source sync (not--all) andGBRAIN_ALLOW_DEFAULT_WRITEis unset,assessDefaultWriteGuard(src/core/source-resolver.ts) decides whether the brain's pages overwhelmingly live outsidedefault; if so the run printsformatDefaultWriteRefusal('sync', …)and exits 1 — except under--dry-run, which prints the same text prefixed[dry-run] a real run would be refused:and still previews (a dry run writes nothing; refusing it only hides information). Escapes:--source default(tierflag),--all, orGBRAIN_ALLOW_DEFAULT_WRITE=1; a failed assessment is fail-open. Pinned bytest/sync-default-write-guard.serial.test.ts+test/sync-default-write-guard-dry-run.test.ts. -
src/core/sync-git.ts:resolveSlugByPathOrSourcePath— Resolves a slug bypages.source_pathfirst (returns the stored slug for frontmatter-fallback pages whose path doesn't derive a slug), then falls back toresolveSlugForPath(path). Three call sites: the un-syncable cleanup (which passesopts.sourceId), and the bare no-sourceIddelete and rename-source paths; whensourceIdIS set the delete and rename loops batch throughengine.resolveSlugsByPathsinstead. Thesource_pathlookup is scoped to the DEFAULT source on the bare path, matching every write that path performs (updateSlug,deletePage, the rename bookkeeping repair are all default-scoped) — an unscoped lookup could return a row from another source sharing thesource_path, and the rename loop consumes this value directly as its cheap-move source slug. Without the helper, emoji-only / Thai / Arabic filenames whose slug came from frontmatter would orphan on delete/rename (that path would compute the wrong path-derived slug). Best-effort query — pre-migration brains fall through to the legacy path. -
src/core/sources-ops.ts— Multi-source registration + clone-lifecycle ops (addSource,recloneIfMissing,defaultCloneDir,isOwnedClone,unownedHint). Reclone-ownership invariant (must-never-violate): gbrain may only delete/re-clone a clone it created, NEVER a user working tree.recloneIfMissingdeleteslocal_path, so it gates onisOwnedClone(src)and throws aSourceOpError('unmanaged_path', ...)BEFORE any filesystem op when ownership is unprovable — fail-closed. Ownership is proven byconfig.managed_clone === true(written byaddSource's--urlpath, covering default-location and--clone-dirclones) ORlocal_path === defaultCloneDir(id)(back-compat for pre-marker clones, via exact normalized-path equality, symlink-free). A row withremote_url+ an unownedlocal_path(a user-registered working tree, e.g.sources add --path) is refused untouched; re-add with--urlto regain auto-reclone. The reclone is EXDEV-safe: clone into a SIBLING temp oflocal_path(not the sharedclones/.tmp, which may sit on a different mount than a--clone-dirtarget), then swap (move old aside → move new in → drop old) solocal_pathis never left missing-and-unrecoverable; on swap failure the original is restored, and if restore fails the error names theasidepath so it's never reflexively deleted. A TOCTOU re-check re-confirms ownership immediately before the destructive move and rejects a symlink leaf swapped in after the entry check (symlink_escape).unownedHint(src, state)is the shared recovery message used by both the core error and thegbrain sync --sourceCLI error;gbrain sources restorespecial-casesunmanaged_pathto print "DB row restored; gbrain syncs this path read-only" instead of the misleading "try sync to recover" guidance.SourceOpErrorCodeincludesunmanaged_path. Pinned bytest/sources-ops.test.ts,test/sources-resync-recovery.test.ts.assertNoOverlappingPath(engine, id, path)is the ONE overlapping-path guard shared by every surface that binds alocal_path(addSource,gbrain sources set-path): a path equal to, nested inside, or enclosing another source'slocal_paththrowsSourceOpError('overlapping_path')— overlapping trees make sync/write-through attribute files to the wrong source, so no add or repair path may skip it. The guard compares by spelling AND by realpath (both the candidate and each siblinglocal_patharerealpathSync'd when they exist, fail-open to the spelling otherwise), so a symlink resolving into or over another source's tree cannot slip past it; the stored pointer and the error message keep the operator's spelling. -
src/core/utils.ts— Shared SQL utilities used by both engines. ExportsparseEmbedding(value)(throws on unknown input, used by migration + ingest paths where data integrity matters) andtryParseEmbedding(value)(returnsnull+ warns once per process, used by search/rescore paths where availability matters more than strictness).isUndefinedColumnError(err)predicate — pattern-matches Postgres SQLSTATE 42703 / "column ... does not exist" with engine-driver shape variation tolerated; used byoauth-provider.tsso genuine errors (lock timeout, network blip, permission denied) propagate while column-missing falls through to the legacy fallback.validateSourceId(id)throws on anything outside^[a-z0-9_-]+$, used by the per-source disk-layout fix in patterns.ts/synthesize.ts before anyjoin(brainDir, '.sources', source_id, slug+'.md')so source_id can't traverse out of brainDir.rowToSearchResultprojects emailmessage_id/thread_idmetadata and exposessource_subjectonly when a non-empty Message-ID proves the page is an email, so generated page titles never become authoritative email subjects.rowToPagepopulates the requiredPage.source_idfrom the SELECT projection (scripts/check-source-id-projection.shenforces every projection feedingrowToPageincludes the column).bigintToStringReplacer(key, value)— the JSON replacer turningbigintinto strings (the postgres.js int8 wire shape), shared by cli.ts's local-result normalizer (which re-exports it) and the commands that stringify results themselves (gbrain call,extract --explain); commands import it from here, never from the dispatcher. Its doc comment deliberately carries no double-dash flag literals — the flag-registry generator harvests them from every module a command transitively imports, and utils.ts is imported by nearly all. -
src/core/llm-json.ts— tolerant JSON extraction from LLM output.parseLlmJson<T>(raw, {array?})walks a strategy ladder (strip ```json fences → direct parse → first{...}/[...]substring), then — only after the raw parse has failed — retries the same ladder with reasoning blocks stripped, so a payload that parses raw is untouched and a payload legitimately containing<think>is unaffected; returns null on any failure (adversarial throws swallowed). ExportsstripReasoningBlocks(raw): removes closed<think>…</think>pairs AND a truncated never-closed<think>tail (output-budget exhaustion) — reasoning models draft their JSON inside the think block, which defeats the greedy first-to-last-brace substring scan.src/core/facts/extract.ts's hand-rolledparseExtractorJsonDetailedapplies the same raw-first fallback ladder with the sharedstripReasoningBlocks. -
src/core/db.ts— Connection management, schema initialization.resolveSessionTimeouts()returnsstatement_timeout+idle_in_transaction_session_timeout(defaults 5min each, env-overridable viaGBRAIN_STATEMENT_TIMEOUT/GBRAIN_IDLE_TX_TIMEOUT/GBRAIN_CLIENT_CHECK_INTERVAL). Bothconnect()(module singleton) andPostgresEngine.connect()(worker pool) consume the result via postgres.js'sconnectionoption, sending GUCs as startup parameters that survive PgBouncer transaction mode (setSessionDefaultskept as a back-compat no-op shim).connect()returnsPromise<boolean>—trueiff THIS call created the module singleton,falseif it joined an existing one; the decision is atomic (noawaitbetween theif (sql)null-check and the synchronoussql = postgres(...)assignment), so two concurrent module connects can't both claim creation.PostgresEnginestores the return as its_ownsModuleSingletontoken and only the creating engine maydb.disconnect()the singleton — a borrower probe engine (lint/doctor config-lift) no-ops its disconnect, so its teardown can't null the connection the long-lived cycle owner is still using (the dream-cycle "connect() has not been called" failure). The modulesqlis only ever nulled bydb.disconnect()(postgres.js auto-reconnects its own internal pool and never touches our reference).disconnect()snapshots + nullssqlbefore awaiting the pool end so a concurrent connect can't join a pool that's already closing. The end routes throughendPoolBounded(pool)— a gbrain-ownedPromise.raceofpool.end({ timeout: POOL_END_TIMEOUT_SECONDS })against a hard timer — so a PgBouncer transaction-mode drain that never settles can't hang teardown — the CLI teardown contract (finishCliTeardown's computed-deadline backstop + flushThenExit's fence-and-grace exit in cli-force-exit.ts) bounds it and delivers pending stdout before exit.connection-manager.tsends its direct + read pools concurrently through the same helper so the per-pool bounds don't stack.resolveMaxLifetimeSeconds(env?)— explicit client-poolmax_lifetimefor all four postgres() call sites (matches the postgres.js implicit 30-60min jittered default;GBRAIN_POOL_MAX_LIFETIME_Soverrides, 0 disables; warn-once on invalid). Pinned bytest/db-pool-max-lifetime.test.ts. -
src/core/pool-gauge.ts—CheckoutGauge: approximate in-flight counters at the engine's raw/direct/reserved/tx seams, surfaced via duck-typedPostgresEngine.getPoolDiagnostics()(no BrainEngine change, no PGLite stub). HONESTY CONTRACT in the module doc: tagged-template traffic is untracked; consumers must label counts as a subset and never derive waiter/available figures. Fail-open (clamped release, try/finally around sync-throwing runUnsafe). Consumed by db-probe.ts. Pinned bytest/pool-gauge.test.ts. -
src/commands/migrate-engine.ts— Bidirectional engine migration (gbrain migrate --to supabase/pglite). Copies the complete source catalog FIRST (copyMigrationSources— everysourcesrow incl. archived rows and sync/routing metadata,ON CONFLICT (id) DO UPDATE,defaultordered first) so every page write has a validpages.source_idFK parent and the target preserves per-source behavior; pages copy afterward, tracked in the resume manifest by composite(source_id, slug)key. Link copy preserves each row'sto_source_id(falling back to the origin source only for legacy rows without it), and failed-target filtering uses that same target composite key, so cross-source links migrate without being rebound to the origin source. The resume manifest is target-aware:migrationTargetId(config)hashes(engine, locator)(database_urlfor Postgres, resolveddatabase_pathfor PGLite) andmanifestMatchesTargetrequiresschema_version === 2plus a matchingtarget_id— a legacy engine-only manifest, or one from a DIFFERENT target of the same engine kind, starts fresh instead of skipping "completed" pages the new target never received. After links,copyMigrationFactscarries the facts table verbatim (conversation facts have no markdown fence to re-sync from): source∩target column intersection via information_schema (older source schemas still copy), delete-and-recopy so every re-run converges to source truth, two-passsuperseded_byrestore (the self-FK points old→newer id, so id-ordered single-pass inserts would violate it), embeddings round-tripped as text into whichever physical type (vector/halfvec) the target column has with a NULL-embedding retry on per-row cast failure, and asetvalbump so post-migration inserts don't collide with copied ids; fact-row failures block the config flip and force a non-zero exit, same contract as page failures. Takes are deliberately NOT copied (fence-canonical, keyed on target-reassignedpage_id; the next extract cycle rebuilds them).copyMigrationConfigthen copies EVERY DB-plane config row except the explicit engine-local denylistMIGRATE_CONFIG_ENGINE_LOCAL_KEYS(engine,version= target-owned schema ledger,embedding_columns+search_embedding_column= physical-column registry the copy doesn't materialize); skipped keys print in the end-of-run per-table copied-count summary, so nothing is dropped silently. Pinned bytest/migrate-engine-resume.test.ts(manifest identity) +test/e2e/migrate-engine-sources-postgres.test.ts(source catalog lands before overlapping-slug pages, PGLite → real Postgres) +test/e2e/multi-source-bug-class.test.ts(cross-source links) +test/migrate-engine-completeness.serial.test.ts(facts chains/embeddings/sequence, config copy-all + denylist, summary). -
src/core/import-file.ts— importFromFile + importFromContent (chunk + embed + tags).importFromContentandimportCodeFilestamppages.embedding_signatureviasetPageEmbeddingSignature(slug, {sourceId, signature: currentEmbeddingSignature()})when the import actually embedded (not--no-embed) so a model/dims swap is detectable as stale;importCodeFileonly stamps when every chunk was freshly embedded this call (needsEmbedIndexes.length === chunks.length), mixed reuse-by-hash pages stay unstamped (reindex --code --force/embed --stalehandle those).importFromContent's tag reconciliation is ADD-ONLY: it onlyaddTag(idempotent, ON CONFLICT DO NOTHING). Thetagstable has no provenance column and frontmatter tags are stripped from storedpages.frontmatter(markdown.ts:118), so a frontmatter-origin tag can't be distinguished from a DB-enrichment tag (auto-tag / dream synthesize / signal-detector) at re-import — deletion is unsafe (would wipe enrichment undergbrain reindex --markdown). Accepted trade-off: removing a tag from frontmatter does not remove it from the DB on next sync (needs atag_sourceprovenance column). Pinned bytest/reindex-preserve-tags.test.ts+test/import-file.test.ts. identity-based dedup pre-check at:427-490. Callsengine.findDuplicatePage?.(sourceId, {hash, frontmatterId})(optional?so test doubles compile). Posture: SKIP whenfrontmatter.idmatches (true external duplicate from overlapping ingest roots), WARN-ALWAYS on content_hash collision with different/missingfrontmatter.id(templates and daily logs may legitimately share text), FAIL CLOSED on lookup error, bypass via--force-rechunk. Soft-deleted pages excluded at the engine layer so tombstones don't block legitimate re-imports under new slugs. Pinned bytest/import-dedup-frontmatter-id.test.ts(11 cases).importFromContentis the narrow waist every ingest path passes through (gbrain import,gbrain sync,put_pageMCP,/ingestwebhook). It runs a three-tier content-quality disposition viaassessContentSanityfromsrc/core/content-sanity.tsBEFORE chunking: (1) high-confidence junk (built-in Cloudflare/CAPTCHA interstitial patterns + operator literals) → QUARANTINE (stamps thequarantinefrontmatter marker, writes ZERO chunks, hides the page from search) OR REJECT (throw → sync-failure) whencontent_sanity.junk_dispositionisreject; (2) fuzzy markup-heavy (prose-vs-markup ratio abovecontent_sanity.max_markup_ratio, warn-tier byte window, code pages exempt) →content_flag:markup_heavymarker (page stays fully searchable, marker rides search results + get_page to warn the agent); (3) oversize →embed_skipsoft-block viabuildEmbedSkipMarker()PLUS acontent_flag:oversizedmarker, AND deletes any pre-existing chunks in the same transaction so search can't surface stale chunks. Gate-owned markers (quarantine,content_flag) are STRIPPED from untrusted (remote MCP,ctx.remote !== false) frontmatter so a write-scoped client can't hide pages or forge the warning channel; markers are excluded fromcontent_hashso a flagged page doesn't re-embed every sync.gbrain importhonorserrors > 0for non-zero exit.classifyErrorCodeinsrc/core/sync.tsrecognizes thePAGE_JUNK_PATTERNcode so sync-failures.jsonl grouping bins these.extractEntityRefs(canonical; matches both[Name](people/slug)markdown links and Obsidian[[people/slug|Name]]wikilinks),extractPageLinks,inferLinkTypeheuristics (attended/works_at/invested_in/founded/advises/source/mentions),parseTimelineEntries,isAutoLinkEnabledconfig helper. Link candidates match any dir-shaped path (existence-checked at persist). Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts. Pinned bytest/import-file-content-sanity.test.ts.importCodeFilewritessource_path: relativePathon every code page (the same repo-relative path markdown imports record) so the full-sync reconcile — which only considerssource_path IS NOT NULLrows — can retire a deleted code file's page;putPageCOALESCEs the column, so rows imported before this keep NULL until the file changes,sync --force, orgbrain reindex-code --force. -
src/core/sync.ts— Pure sync functions (manifest parsing, filtering, slug conversion). ExportedpruneDir(name: string): booleanis the single source of truth for descent-time directory exclusion across walkers — blocksnode_modules(no leading dot, so a naive walker would descend into it and inflate MISSING_OPEN counts via vendor packages),vendor/dist/build/venv, dot-prefix dirs, and*.rawsidecars — NOTops/, which is ordinary user content (the bundled daily-task-manager storesops/tasksthere);isSyncableapplies it per path segment, andwalkMarkdownFilesinsrc/commands/extract.ts+listTextFilesinsrc/core/cycle/transcript-discovery.tsconsult it BEFORE recursing to save the IO of walking thousands of vendor files.manageGitignoreworktree discriminator matches the gitdir path segment (/modules/<name>= submodule,/worktrees/<name>= worktree, per Git's documented layout) so Conductor worktrees (first-class repos) get.gitignoremanagement for storage-tiering. The sync-failure ledger (failure store, error classifier, the shared bookmark gate, and the doctor severity rule) lives insrc/core/sync-failure-ledger.ts;sync.tsre-exportsclassifyErrorCode,summarizeFailuresByCode,loadSyncFailures,unacknowledgedSyncFailures,acknowledgeSyncFailures,recordSyncFailures,decideSyncFailureSeverity,applySyncFailureGate, and theSyncFailuretype for backward-compatible imports — see its entry below.isSyncablefactored through privateclassifySync(path, opts): SyncableReason | null; exported companionunsyncableReason(path, opts)returns the same tagged reason or null when syncable.SYNC_SKIP_FILESis a named export (the four canonical metafile basenamesschema.md,index.md,log.md,README.md).SyncableReasonunion:'metafile' | 'strategy' | 'pruned-dir' | 'include-glob-miss' | 'exclude-glob-hit' | 'malformed-path'. Malformed filenames are TWO-TIER:hasMalformedPathSegment(ADMISSION — control chars on any path; square brackets on.md/.mdxpaths only, so code-strategy lanes keep indexingapp/[id]/page.tsxframework layouts) vsisPoisonedPath(DESTRUCTION — only the injection signature](or control chars; sync's row-DELETING lanes gate on this so a bare-bracket markdown row already in the DB survives reconcile while its file exists).sanitizePathForDisplayscrubs control bytes + caps length before echoing such paths. Thecommands/sync.tscleanup loop guards onunsyncableReason(path)being'metafile'OR'pruned-dir'so previously-indexed metafile pages AND deliberately put-created pages under pruned dirs survive every re-sync. Does NOT covermanifest.deleted(the upstream filter already strips metafiles). Pinned bytest/sync-isSyncable-shape.test.ts(15 cases, duality contract) +test/sync-metafile-skip.serial.test.ts(3 PGLite cases incl. the renamed.md → .txtnegative).pruneDir:pruneDir(name, parentDir?)extended with optionalparentDir. When provided, additionally rejects directories containing.gitas a FILE — the git submodule gitfile pattern (regular repos have.gitas a DIRECTORY; submodules as a file pointing into the parent's.git/modules/). Sync + extract walkers threadparentDirso the gitfile-as-FILE check fires per descend step. Best-effort:statSyncfailures fall through and treat as a normal dir. Prevents phantom imports from a worktree-with-submodules sync walking into submodule trees. Pinned bytest/sync-walker-submodule.test.ts. -
src/core/sync-failure-ledger.ts— the bounded auto-skip sync failure ledger. A LEAF module (imports only fs/path/crypto/config) sosync.tscan re-export it without a circular dependency. State lives in~/.gbrain/sync-failures.jsonl, one JSON object per line, keyed by(source_id, path)with a per-keyattemptscount and a 3-state machine:open(fresh/blocking) →auto_skipped(chronic, still doctor-visible) oracknowledged(human resolved viagbrain sync --skip-failedfrom either unresolved state).classifyErrorCode(errorMsg)regex classifier with 23 named codes (RENAME_RECONCILE,SLUG_MISMATCH,DB_DUPLICATE_KEY,STATEMENT_TIMEOUT,YAML_PARSE,YAML_DUPLICATE_KEY,MISSING_OPEN,MISSING_CLOSE,EMPTY_FRONTMATTER,NULL_BYTES,NESTED_QUOTES,INVALID_UTF8,FILE_TOO_LARGE,SYMLINK_NOT_ALLOWED,TAKES_TABLE_MALFORMED,TAKES_HOLDER_INVALID,EMBEDDING_NO_CREDS,EMBEDDING_NO_TOUCHPOINT,EMBEDDING_TIMEOUT,EMBEDDING_RATE_LIMIT,EMBEDDING_QUOTA,EMBEDDING_OVERSIZE,PAGE_JUNK_PATTERN) plusUNKNOWN(the default fallthrough);summarizeFailuresByCode(failures)returns sorted[{code, count}];MISSING_OPEN/EMPTY_FRONTMATTERregexes match themarkdown.tsvalidator strings;MISSING_CLOSE's regex is stale against currentmarkdown.tsoutput — it matches the no-heading-hint case (No closing --- delimiter found) but not the heading-hint case (No closing --- before heading at line N), which falls through toUNKNOWN.FILE_TOO_LARGEcoversimport-file.ts:395, 1236, 1436(markdown/text, plain-file, and code-file import paths),SYMLINK_NOT_ALLOWEDcovers:1231, 2038(file and image import paths).EMBEDDING_INFRA_CODESis the subset (EMBEDDING_TIMEOUT/EMBEDDING_RATE_LIMIT/EMBEDDING_QUOTA) that means the embedding PROVIDER is unhealthy rather than the file being poison —isEmbeddingInfraCode(code)is consulted insidedecideGateAction's chronic-eligibility loop so these three are never counted toward auto-skip (treated as fresh, forcingblockinstead ofadvance_then_autoskip); an explicit--skip-failedstill advances past them like any other failure, since that check runs before the loop. All mutations run underwithLedgerLock(cross-process file lock) with an atomic rename write. The auto-skip threshold resolves viaresolveAutoSkipThreshold()fromGBRAIN_SYNC_AUTOSKIP_AFTER(defaultDEFAULT_AUTOSKIP_AFTER = 3;0disables the valve = pure fail-closed). Two pure decision functions are the unit-test surface:decideGateAction({fileFailures, sentinels, attemptsByPath, threshold, skipFailed})returnshard_block | block | advance | advance_then_autoskip(sentinels like<head>ALWAYS hard-block, even with--skip-failed, so a history rewrite can't auto-skip; any FRESH failure withattempts < thresholdblocks fail-closed; only when ALL failures are chronic does itadvance_then_autoskip), anddecideSyncFailureSeverity({entries, nowMs, failHours})returns thesync_failuresdoctor status (okwhen zero unresolved;failwhen ≥10 OPEN-blocking or the oldest OPEN failure'sts(last attempt, NOTfirst_seen) is older thanfailHours— a failure retried inside that window keeps refreshingtsand never trips this leg even iffirst_seenis much older; otherwisewarn—auto_skipped-only rows stay WARN-visible regardless of count because the bookmark already advanced).applySyncFailureGate(input)is the one orchestrator BOTH sync paths (incremental + full/runImport) call: it records/clears ledger rows, runsdecideGateAction, then executes effects in the crash-safe order (advance the bookmark FIRST via the injectedadvance()callback, THEN auto-skip the chronic set) so a crash can never mark a file skipped while leaving sync wedged.isSkippablePathrejects<…>sentinels. Pinned bytest/sync-failure-ledger.serial.test.ts+test/sync-failures.test.ts. -
src/core/sync-cost-gate.ts+sync-git.ts+sync-anchor.ts+sync-lock.ts+sync-reconcile.ts+sync-status-report.ts— six pure-function clusters behind thesrc/commands/sync.tsfaçade, which re-exports them;facadeExpansioninscripts/generate-flag-registry.tskeeps exactly these six (NOT the othersync-*siblings, which are ordinary deps) on the sync command's flag-scan surface.sync-cost-gate.ts: the inline-embed cost gate + token estimation forgbrain sync.sync-git.ts: git plumbing — invocation building, repo discovery, baseline-commit self-heal, path-containment guards, and the delete/rename slug resolvers (resolveSlugByPathOrSourcePath— see its dedicated entry above — plus the verifiedresolveSlugsForRemovedPaths/resolveRemovedPathSlug).sync-anchor.ts: sync anchor + chunker-version state helpers (source-scoped vs legacy global-config storage).sync-lock.ts: the lock layer — typed lock-busy error, the rich busy message,--break-lockhandling, the partial-result envelope (performSyncitself stays in the façade).sync-reconcile.ts: full-sync reconcile planning (the mass-delete valve and ever-committed gate) + sync deadline/stall resolution.sync-status-report.ts: the per-source sync status report backinggbrain sources statusand theget_status_snapshotop. Its queue context is engine-aware: only an explicitly worker-backed surface may promise job deferral; otherwise non-interactive cost deferral returnsmanual_drain_requiredwith exact per-sourcegbrain embed --stale --source <id>commands and never writes or claims an embed-backfill row. -
src/core/sync-embed-backfill.ts— Sync command’s engine-aware embedding-delivery seam. It derives separate worker-backed deferral and multi-source fan-out eligibility, keeps automatic-submission policy distinct from capability, invokes the shared cost gate for--alland single-source runs, propagates intrinsic large-sync deferral into final delivery outcomes, resolves backfill results through the centralized submitter, formats one truthful human status, and builds the single-source JSON envelope. Keeping these pieces together prevents cost-gate, queue, human, and machine surfaces from drifting while the size-ratchetedsrc/commands/sync.tsfaçade stays at its committed ceiling. -
src/core/storage.ts— Pluggable storage interface (S3, Supabase Storage, local). -
src/core/storage-config.ts— Storage tiering:loadStorageConfigreadsgbrain.yml, normalizes deprecated keys (git_tracked/supabase_only) to canonical (db_tracked/db_only) with once-per-process deprecation warning, and runsnormalizeAndValidateStorageConfig(auto-fixes missing trailing/, throwsStorageConfigErroron tier overlap). Path-segment matcher:media/x/does NOT matchmedia/xerox/foo. Uses a dedicated parser for thegbrain.ymlshape rather than gray-matter (broken on delimiter-less YAML). Also carriesDERIVE_PHASE_DB_ONLY_DEFAULTS(life/events/,atoms/,extracts/,dream-cycle-summaries/) +effectiveDbOnlyDirs— the engine's derive-phase output prefixes treated as implicitly-declared db_only by theundeclared_db_only_pagesdoctor check but deliberately NOT merged intoloadStorageConfig(a global merge would auto-gitignore those dirs and silently kill ingestion for brains that file-back them) — andfindDbOnlyCollisions(pure collector-output vs db_only overlap detector shared by thedb_only_collector_collisiondoctor check and sync'smanageGitignorewarning). Pinned bytest/storage-config.test.ts+test/doctor-silent-death-checks.test.ts. -
src/core/disk-walk.ts—walkBrainRepo(repoPath)returnsMap<slug, {size, mtimeMs}>from one recursivereaddirSync. Skips dot-dirs,node_modules, non-.mdfiles. Used bygbrain storage statusinstead of per-pageexistsSync + statSync(~400K syscalls on 200K-page brains → tens). -
src/core/git-head.ts— local git HEAD freshness probe forgbrain doctor.isSourceUnchangedSinceSync(localPath, lastCommit, opts?)returns true ifflocalPathis a git repo whose current HEAD matcheslastCommit; whenopts.requireCleanWorkingTreeis true also requires a clean working tree (mirrorsgbrain sync's force-walk gate atsync.ts:1075so doctor and sync agree on "is there work to do?").requireCleanWorkingTreeisboolean | 'ignore-untracked'— in'ignore-untracked'mode the clean probe runsgit status --porcelain --untracked-files=noso a quiet repo with stray untracked dirs (?? companies/,?? media/) is still "unchanged" (sync's incremental path keys off the commit diff and does not import untracked files by default — it counts them asuncommitteddrift with a stderr warning;--working-treeopts into importing them);GitCleanProbegains anignoreUntracked?second arg. Two probe seams (_setGitHeadProbeForTests,_setGitCleanProbeForTests) keep unit tests R2-compliant (nomock.module). UsesexecFileSyncwith array args so shell metachars inlocal_pathcannot escape to a shell (the test runs realexecFileSyncagainst'/nonexistent/$(touch <sentinel>)/repo'and asserts the sentinel is never created). Fail-open on every error (missing path, not a git repo, git not installed, timeout, NULL inputs, dirty-probe errored → false) so the caller falls back to its time-based check. The chunker-version-match check lives in the caller (doctor.ts) because it depends on engine state (sources.chunker_versionvsCHUNKER_VERSIONfromsrc/core/chunkers/code.ts). Pinned bytest/core/git-head.test.ts(incl. the shell-injection guard). -
src/core/env-number.ts— numeric env-var resolution with ONE shared warn-once memo.resolveEnvNumber(varName, fallback, {unit?})(positive numbers only; zero/negative/garbage → warn once + fallback),resolveHoursEnv(hours unit pre-applied),warnOnceForEnv(varName, message)for disabled-unless-set vars that need the same memo without the fallback shape (e.g.GBRAIN_EXTRACTION_LAG_FAIL_PCT). Lives in core becausesource-health.tsneeds the hours resolver for the staleness ceiling whiledoctor.tsalready imports FROMsource-health.ts— reaching back would cycle, and duplicating would fork the memo so one bad var warns twice.doctor.tsre-exports it as_resolveEnvNumberforsync.ts's dynamic import._resetEnvNumberWarnedForTests()is a test seam. -
src/core/source-health.ts— per-source health metrics forgbrain sources status+ doctor'sfederation_health. Commit-relative staleness:newestCommitMs(localPath)= HEAD committer time viagit log -1 --format=%ct(fail-open null; NO working-tree mtime parsing — committed content only, robust against the porcelain-mtime bug farm);lagFromContentMs(contentMs|null, lastSyncMs|null, nowMs, ceilingSeconds?)= remote/column comparator (null lastSync → null; negative wall-clock → skew passthrough;contentMs <= lastSync→max(0, wallClock - ceiling); else/null-content → wall-clock). Core logic is pure; only the DEFAULTceilingSecondsreads env, viaresolveStalenessCeilingSeconds()(GBRAIN_STALENESS_CEILING_HOURSoverridingGBRAIN_SYNC_FRESHNESS_FAIL_HOURS, default 72) — pass the 4th arg for determinism. The caught-up branch RAMPS rather than returning a flat 0: a flat 0 meant a source whose clone vanished reported fresh forever (the dead-daemon class), while a hard step to the ceiling would tripfederation_health(24h) andsync_freshness(24h/72h) in the same instant and skip the warn tier — the alert-storm shape the caught-up branch exists to prevent. Ramping keeps the escalation ordered.computeAllSourceMetrics(engine, sources, {probeContent?}): LOCAL (probeContent:true,gbrain sources status) →isSourceUnchangedSinceSync(..., {requireCleanWorkingTree:'ignore-untracked'}) ? 0 : wall-clock(live commit-hash catches HEAD moving to an old-dated commit a timestamp compare would miss); REMOTE (default,federation_healthon the HTTP MCP path) →lagFromContentMs(row.newest_content_at, ...), NO git subprocess (trust boundary).commitTimeMs(localPath, sha)is thenewestCommitMssibling pinned to an arbitrary commit (committer time viagit show -s --format=%ct <sha>, fail-open null, execFileSync array args) — the resumable sync stampsnewest_content_atagainst its pinned target commit, not whatever HEAD raced to. Pinned bytest/source-health.test.ts. -
src/core/npm-squat-check.ts— classifiesgbrainPATH entries as real, foreign npm package, broken, or unknown for doctor'snpm_squatcheck. On Windows it normalizes Git Bash/MSYS drive paths (/c/...→C:/...) and tries the native.exesuffix before reporting a broken entry; non-Windows classification keeps the original single-candidate behavior. Pinned bytest/npm-squat-check.test.ts. -
src/core/git-remote.ts— SSRF-hardened git invocations for remote-sourcecloneRepo,pullRepo, andfetchRemote(repoPath, branch)(the last used by the sync cost-estimator's fetch-first path, so a cost preview / dry-run fetches through the same hardened flags +GIT_TERMINAL_PROMPT=0as real sync rather than a less-protected route). Exports two distinct flag constants becausegit's argv grammar treats them differently:GIT_SSRF_FLAGS(3-cconfig flags —http.followRedirects=false,protocol.file.allow=never,protocol.ext.allow=never) is global config, spread BEFORE the subcommand verb;GIT_SSRF_SUBCOMMAND_FLAGS = ['--no-recurse-submodules']is subcommand-scoped, spread AFTER the verb (a combined array would spread--no-recurse-submodulesbefore the verb where real git rejects it exit 129).cloneRepoargv:git <GIT_SSRF_FLAGS> clone <GIT_SSRF_SUBCOMMAND_FLAGS> --depth=1 [--branch X] <url> <dir>.pullRepoargv:git -C <dir> <durableSsrfFlags()> pull <GIT_SSRF_SUBCOMMAND_FLAGS> --ff-only—durableSsrfFlags(), not the hardcodedGIT_SSRF_FLAGS: identical-cflags exceptprotocol.file.allowhonors theGBRAIN_GIT_ALLOW_FILE_TRANSPORT=1escape hatch.fetchRemoteuses the same helper. Pinned bytest/git-remote.test.tsposition-anchored guard (argv.indexOf('--no-recurse-submodules') > argv.indexOf(verb)). Also exports the durability-side helpers that powergbrain sources harden/pull:GIT_ENV_AUTH(the no-prompt env minus the askpass/bin/falseoverrides, so an auth'd push/fetch can consult the repo's configured credential helper whileGIT_TERMINAL_PROMPT=0still fails fast on a missing credential),divergenceSafePull(repoPath, branch)(fetch +pull --rebase; returnsskipped_dirtyon a dirty tree,conflict_abortedon ANYpull --rebasefailure (the catch doesn't classify the cause — it's treated as a conflict) after up to two best-effortrebase --abortattempts (each swallows its own failure;conflict_abortedis still returned even if rebase state remains), elseup_to_date/advanced),detectDefaultBranch(origin/HEAD → current branch →main),pushProbe(repoPath, branch)(authenticatedpush --dry-runthat proves push access and classifiesauth/protected/unreachable), andisWorkingTreeDirty. These auth'd paths route theirprotocol.file.allowthroughGBRAIN_GIT_ALLOW_FILE_TRANSPORT(defaultnever; set=1for self-hosted filesystem remotes), unlikecloneRepo, which still uses the hardcodedGIT_SSRF_FLAGSand stays strict. -
src/core/brain-repo-durability.ts+src/commands/sources-harden.ts— brain-repo git durability.hardenBrainRepo(opts)makes a brain's working tree durable, idempotently: divergence-safe pull, a LOCAL untracked.git/hooks/post-commitauto-push safety net (never committed — a pulled commit can't rewrite executed code next to the token; installed into the activecore.hooksPathdir and excluded via.git/info/excludewhen that dir is tracked), a committedscripts/brain-commit-push.shthat refuses to exit 0 without a confirmed push and stages+commits BEFORE any pull so a dirty tree of modified pages (the write-through shape) can still be committed — the push-retry's rebase-on-reject handles a remote that advanced (hook + helper render from ONE bash push-retry template — DRY at the TS source, not by the hook sourcing a repo-controlled script), durability rules patched into the active resolver file (findResolverFile→ RESOLVER.md > AGENTS.md; taxonomy rendered from the bundled_brain-filing-rules.json), a minimal DB-free pull cron (launchd/crontab runninggbrain sources pull --path <dir>so it never opens the PGLite single-writer lock), and a push-probe verify (no heartbeat commit). Credential is REPO-scoped (acceptPatfrom--pat-file/GBRAIN_GITHUB_PAT, warns on loose perms; reuses an existing repo-localcredential.helper, else a0600store wired via repo-local config); the token is redacted everywhere viaredactSecretsInTextand never enters the repo, remote URL, logs, orDurabilityReport.unhardenBrainReporemoves the cron/hook/credential wiring (ownership-fingerprinted);sources removeruns it only AFTER the source row's DELETE commits (a refused or raced delete leaves the scaffolding intact; post-commit teardown failure is loud but non-fatal). CLI:gbrain sources harden <id|--all>/pull <id>|--path <dir>/unharden <id>; auto-harden fires onsources add --url ... --pat-filefor managed clones (--no-hardenopts out).sources pull --pathis dispatched insrc/cli.tsBEFOREconnectEngineso the cron stays DB-free. CLI-only (writes executables + an OS cron + a credential helper on the host); never exposed over MCP. Tests:test/brain-repo-durability.serial.test.ts,test/git-remote-durable.serial.test.ts,test/brain-durability-hook.serial.test.ts,test/durability-cron.test.ts.GBRAIN_PUSH_LOCK_WAIT_SECONDS(default 30, env-only) tunes theflock -wwait in the rendered push-retry block that both the committed helper and the local hook share; on lock timeout the synchronous helper returns rc 1 (fail-loud, no push was attempted) while the detached post-commit hook returns rc 0 (another holder is already pushing — the designed coalescing outcome). -
src/commands/storage.ts—gbrain storage status [--repo P] [--json]. Split into pure data (getStorageStatus) + JSON formatter + human formatter (ASCII-only) matching theorphans.tspattern.PageCountsByTierandDiskUsageByTierare distinct nominal types so swaps fail at compile time. -
gbrain.yml(brain repo root) — Optional storage tiering config. Top-levelstorage:section withdb_tracked:anddb_only:array-valued keys.gbrain syncauto-manages.gitignorefordb_onlypaths on successful sync (skips on dry-run, blocked-by-failures, submodule context, orGBRAIN_NO_GITIGNORE=1).gbrain export --restore-only [--repo P] [--type T] [--slug-prefix S]repopulates missingdb_onlyfiles from the database. -
src/core/supabase-admin.ts— Supabase Management-API client (listProjects,discoverPoolerUrl,extractProjectRef— refs are lowercase ALPHANUMERIC,[a-z0-9]+). Consumed by theinit --prefer-postgresladder's rung 2 (DISCOVERY only — project creation stays dashboard guidance; discovered URLs are candidates, connect-probed before persisting) and by the classifier's Supabase hints. The access token is never persisted, logged, or written to receipts/config. -
src/core/file-resolver.ts— File resolution with fallback chain (local -> .redirect.yaml -> .redirect -> .supabase). -
src/core/chunkers/— 3-tier chunking (recursive, semantic, LLM-guided).code.tsis a tree-sitter-based semantic chunker for 30 languages (plus SQL via DerekStride/tree-sitter-sql) with embedded-asset WASMs (src/assets/wasm/),@dqbd/tiktokencl100k_base tokenizer, small-sibling merging.CHUNKER_VERSIONis folded intoimportCodeFile'scontent_hashso chunker shape changes force clean re-chunks across releases.extractSymbolNamehas an inline SQL branch (extractSqlSymbolName) diving through DerekStride'sstatementwrapper into the inner DDL child (create_table/create_function/create_view/create_index/create_procedure/create_type/create_schema/create_database/create_trigger/alter_table/alter_view) and extracting the target identifier via thenamefield with identifier-shaped fallback; DML kinds (select/insert/update/delete/merge/with) deliberately return null so chunks emit unnamed (code-def is a DDL signal).normalizeSymbolTypehas parallel SQL branches mappingcreate_table → 'table',create_view → 'view', etc.DEF_TYPES(owned bysrc/core/chunkers/def-types.ts, re-exported bysrc/commands/code-def.ts— see that entry) carries the SQL kinds ('table' | 'view' | 'index' | 'procedure' | 'schema' | 'database' | 'trigger') so the new chunks surface ingbrain code-def <name>queries. -
src/core/chunkers/def-types.ts—DEF_TYPES: the ONE list of definition-shaped normalized (post-normalizeSymbolType) symbol types, shared bygbrain code-def's lookup allowlist (src/commands/code-def.tsre-exports it) and the code chunker's small-sibling merge guard so the two can't drift — a symbol type code-def can resolve never has itssymbol_nameerased by merging a small definition into a neighbor. Includes the fallthrough formsnormalizeSymbolTypeemits verbatim for tree-sitter node types it doesn't canonicalize ('method declaration','struct specifier','record declaration', …) — most of an OO codebase's symbols arrive in those forms. Pinned bytest/chunkers/code-merge-defs.test.ts. -
src/core/errors.ts—StructuredAgentError+buildError+serializeError. Every agent-facing surface (code-def, code-refs, usage errors) uses this envelope; matches theCycleReport.PhaseResult.errorshape. -
src/assets/wasm/— 37 tree-sitter grammar WASMs + tree-sitter runtime. Committed to the repo sobun --compileembeds them deterministically viaimport path from ... with { type: 'file' }. The CI guardscripts/check-wasm-embedded.shfails the build if the compiled binary ever silently falls through to recursive chunks.tree-sitter-sql.wasm(DerekStride/tree-sitter-sql @ c2e1e08db1ea20dc23bdb8d228a81a8756e9c450, built with tree-sitter-cli@v0.26.3 --abi 14) adds SQL coverage at 11 MB — larger than peers because the grammar covers PostgreSQL + MySQL + SQLite + T-SQL basics (40 MB generated parser.c); the compiled binary grows ~6%. -
src/commands/code-def.ts+src/commands/code-refs.ts— symbol definition + references lookup. Querycontent_chunks.symbol_nameor chunk_text ILIKE withpage_kind='code'filter. Auto-JSON when stdout is not a TTY (gh-CLI convention). Bypass the standardsearchKeywordDISTINCT ON (slug)collapse so multiple call-sites from the same file surface. The JSON envelope (CLI + thecode_def/code_refsMCP ops) carriesstatus+readyfromsrc/core/code-graph-readiness.tsso acount:0result is distinguishable asnot_built(no code indexed) vsready(genuinely no match); human output prints a one-line hint when not ready. Both commands resolve--source <id>(space or inline=spelling) and the ambient source scope through the sharedcode-scope.tsresolver, matchingcode-callers/code-callees;--all-sourcesrestores the brain-wide read. TheAND p.source_id = $Nfragment comes fromcode-scope.ts'spushSourcePredicate(params, opts)(numbered offparams.lengthso it composes with--langand any other optional predicate;''when spanning every source) — code-def's lookup, its filtered-types probe, and code-refs all use it. -
src/commands/code-scope.ts— shared--source/--all-sourcesresolution for the four code-* CLI commands (code-def, code-refs, code-callers, code-callees), extracted so the four can't drift.positionalArgsskips value-taking flags AND their values (--source,--limit,--lang— a naive scan looked up the flag's value as the symbol); inlinename=valuespellings are one token and consume nothing. A bad.gbrain-source/GBRAIN_SOURCEpin is recognized through the sharedisResolverUserErrorpredicate (exported bysrc/core/source-resolver.ts, next to the messages it matches) and exits 2 with a cleaninvalid_source_pinenvelope instead of an uncaught stack. Also exportspushSourcePredicate(params, opts), the source-scope SQL fragment builder code-def/code-refs share. -
src/core/code-graph-readiness.ts— typed readiness signal shared by the code-* surfaces (code-def/code-refs/code-callers/code-callees, plus thecode_blast/code_flowwalk ops).resolveCodeReadiness(engine, {kind:'symbol'|'edge', count, sourceId?, allSources?, remote?})returns{status:'not_built'|'no_symbols'|'indexing'|'ready'|'out_of_scope'|'unknown', ready, has_code, pending_edges, scoped_source_id?}(6-stateCodeGraphStatus;scoped_source_idis set only forout_of_scope, naming the excluded scope).count>0short-circuits toreadywith no query; on empty it runsEXISTSprobes againstcontent_chunksJOINpages(page_kind='code') — nopage_kindindex needed, and the pending probe rides the partialidx_content_chunks_edges_backfill.kind:'symbol'(code-def/refs) is 3-state + brain-wide, because symbol metadata is set at chunk time and edge resolution is irrelevant to it: no code chunks →not_built; code chunks but none carrysymbol_nameyet →no_symbols(chunks indexed before symbol extraction; hints atreindex-code); symbol-bearing chunks →ready(it never reportsindexing).kind:'edge'(code-callers/callees) is 3-state + source-scoped (not_built/indexing/ready), with the pending predicate mirroring the resolver (edges_backfilled_at IS NULL OR < EDGE_EXTRACTOR_VERSION_TSfromsrc/core/chunkers/symbol-resolver.ts) so a resolver-version bump never falsely reportsready. Both grains additionally shareout_of_scope: when the scoped probe finds no code but an unscoped rerun finds code exists brain-wide (a chunk-existence check only — it doesn't verify symbol metadata or edge completeness), it's the caller's resolved scope that excludes it, not an unbuilt graph. That unscoped rerun runs ONLY for trusted local callers (remote === false); a scoped remote caller seesnot_builtinstead, never a brain-wide code-existence disclosure. Probe scope matches each command's result-querydeleted_atposture (def/refs don't filterdeleted_at, so neither do the probes). Any DB error returnsstatus:'unknown'(fail-open; never breaks the command).readinessHint(r)renders the human one-liner. Wired intocode-def.ts/code-refs.ts(brain-wide),code-callers.ts/code-callees.ts(resolvedsourceId/allSources), and all sixcode_*MCP op handlers insrc/core/ops/code-intel.ts(operations.tsimportscodeIntelOperationsand spreads it into the canonicaloperationsarray, not a re-export).code_blast/code_flowstampstatus/ready(+scoped_source_id) onto the walk result through the module-privateattachWalkReadinesshelper: computed AFTER the traversal cache (readiness is never cached),not_foundprobes at symbol grain,ok/ambiguousat edge grain with count = nodes/candidates, andunsupported_languageis passed through untouched. Pinned bytest/code-graph-readiness.test.ts+ readiness-envelope cases intest/e2e/code-intel-mcp-ops-pglite.test.ts. -
src/core/search/— Hybrid search: vector + keyword + RRF + multi-query expansion + dedup.searchKeyword/searchKeywordChunks/searchVectorapply source-aware ranking at the SQL layer (curated content likeoriginals/,concepts/,writing/outranks bulk content like<fork>/chat/,daily/,media/x/).searchVectoruses a two-stage CTE so source-boost re-ranking doesn't kill the HNSW index. Hard-exclude prefixes (test/,archive/,attachments/,.raw/by default) filter at retrieval, not post-rank. Both gates honordetail !== 'high'so temporal queries surface chat pages normally. -
src/core/search/query-intent.ts— Deterministic query classifiers (pure, no LLM):classifyQuery/classifyQueryIntent(entity/temporal/event/concept/general → auto-selects detail level, salience/recency/modality axes;conceptfires on definitional-paraphrase OR landscape/quantifier cues with a proper-noun name-guard — capitalized names, quoted phrases, slugs, and sub-3-word queries never trigger — and ranks vector-lean via the RRF-k tilt inintent-weights.ts),isAmbiguousModalityQuery(LLM-escalation gate), and the concept-shape pair —looksConceptShaped(fuzzy-quantifier/landscape cues minus exact-identifier anti-signals, tuned to favor false-negatives; cues owned by other routers like "who are the"/find_experts and bare "anything"/salience are deliberately excluded) +conceptNudge(full one-line CLI hint string steering a concept-shapedsearchtowardquery; consumed bymaybePrintConceptNudgeinsrc/cli.tson BOTH the local-engine and thin-client result paths, stderr-only,--quiet-gated). Pinned bytest/query-intent-concept.test.ts+test/cli-concept-nudge.test.ts. -
src/core/search/llm-intent.ts— opt-in LLM modality tie-break.classifyModalityWithLLM(query, fallback)routes throughgateway.chat()with a fixed single-word-output system prompt; 1s timeout via AbortController.parseModality(raw, fallback)is the pure parser (tolerates trailing punctuation + casing). Fail-open on every error (gateway unavailable, timeout, parse failure, unrecognized output) — returns the fallback so a misbehaving LLM can never break search. Cost-bounded byisAmbiguousModalityQueryinquery-intent.tsso the LLM call fires on only a small fraction of queries when on. -
src/core/search/image-loader.ts—loadImageInput(input, opts)accepts a local path,data:URI, orhttp(s)://URL. Magic-byte sniff for PNG/JPEG/WebP. Hard size cap (default 10 MB, configurable viasearch.image_query.max_bytes). URLs route throughfetchWithSSRFGuardso DNS rebinding + redirect chains are defeated; shared guarded transport pins validated DNS answers and streams decoded bodies with a 2 MiB remote cap (10 MiB local cap).ImageLoadErrorwith discriminatedcode(INVALID_FORMAT / OVERSIZED / INVALID_URL / FETCH_FAILED / TIMEOUT / SSRF_BLOCKED / NOT_FOUND). -
src/core/search/by-image.ts—searchByImage(engine, input, opts). Always runs the image branch (embedQueryMultimodalImage+searchVector(embedding_image)). Hybrid intersect: when the caller provides an optionalquery, runs a parallel text branch viaembedQueryMultimodal(query)and merges viarrfFusionWeightedwitheffectiveRrfK(baseRrfK, weight)from the resolved mode's refinement weights. Widens to the unified column whensearch.unified_multimodal=true(transparently upgrades retrieval quality post-reindex). -
src/core/ssrf-validate.ts— DNS-rebinding-defended URL validation.validateAndResolveUrl(url)resolves the hostname viadns.lookup({all: true, family: 0}), checks EVERY A and AAAA record against the internal-IP deny list, and returns the resolved IP so callers fetch by IP (validation IP === fetch IP defeats DNS rebinding).fetchWithSSRFGuard(url, opts)does redirect-aware fetching with per-hop re-validation (max 3 hops by default). Reusable across all URL-fetching features. Test seam__setDnsLookupForTestsfor hermetic tests. -
src/core/spend-log.ts— per-OAuth-client paid-API spend tracking against themcp_spend_logtable.checkBudget(engine, clientId, capCents)is the pre-flight gate; throwsBudgetExceededErrorwhen today's spend has hit the cap.recordSpend(engine, entry)is best-effort post-call. UTC day-aligned aggregation so caps roll over deterministically regardless of server timezone. Local CLI callers (no clientId) bypass the gate; brains without the table fail open to spend=0.VOYAGE_MULTIMODAL_3_PER_IMAGE_CENTS= 0.12 cents per image embed. -
src/commands/reindex-multimodal.ts—gbrain reindex --multimodal [--limit N] [--dry-run] [--cost-estimate] [--no-embed] [--yes] [--json]. Walkscontent_chunks WHERE embedding_multimodal IS NULL, batches viaembedMultimodalSafe(partial-failure-aware), persists. Lock viatryAcquireDbLock(360min) so a concurrent autopilot embed phase can't race it. Cost prompt + Ctrl-C grace window in TTY.GBRAIN_NO_REEMBED=1bypass. Checkpoint at~/.gbrain/reindex-multimodal-checkpoint.jsonfor resume. Auto-flip prompt at coverage=100% completion (TTY: interactive; non-TTY: stderr hint with a paste-ready command). -
src/core/backfill-registry.ts— registry of idempotent data backfills. Themodalitybackfill flipsmodalityto'image'on image-asset chunks the ingest path missed; its SQL filter requireschunk_source='image_asset'ANDembedding_image IS NOT NULLAND(modality IS NULL OR modality != 'image')— thechunk_sourceguard ensures a non-image chunk that happens to haveembedding_imagepopulated is never flagged. A second run finds zero rows. -
src/core/search/eval.ts— Retrieval eval harness: P@k, R@k, MRR, nDCG@k metrics + runEval() orchestrator. -
src/core/eval/ranked-docs.ts— dependency-free page-level ranking primitive shared by the retrieval eval harness, NamedThingBench, and the qrels correctness gate.dedupeRankedKeys()preserves the first/best occurrence of each${source_id}::${slug}or slug key before any cutoff, so repeated chunks neither inflate relevance gain nor consume document ranks. Precision retains its fixed-kdenominator; Recall and nDCG are structurally bounded to[0,1]. Pinned acrosstest/eval.test.ts,test/retrieval-quality-harness.test.ts,test/bench/qrels-file.test.ts, andtest/bench/correctness-gate.test.ts. -
src/core/search/source-boost.ts— Source-type boost map keyed by slug prefix.DEFAULT_SOURCE_BOOSTS(originals/ 1.5, concepts/ 1.3, writing/ 1.4, people/companies/deals/ 1.2, daily/ 0.8, media/x/ 0.7,/chat/ 0.5, archive/ 0.5, extracts/ 0.3) and DEFAULT_HARD_EXCLUDES(test/, attachments/, .raw/).archive/is DEMOTED (findable, ranked below curated), not hard-excluded — archive holds high-signal history users expect to retrieve; the demote is a prior at the SQL/fusion layer and the cross-encoder reranker can still promote a strongly-matching archive page.parseSourceBoostEnv/parseHardExcludesEnvparse comma-separatedprefix:factorpairs fromGBRAIN_SOURCE_BOOST/GBRAIN_SEARCH_EXCLUDE.resolveBoostMapandresolveHardExcludesmerge defaults + env + callerSearchOpts.exclude_slug_prefixes/include_slug_prefixes. The surviving exclude policy is auditable via thehidden_by_search_policydoctor check (src/commands/doctor.ts, local + remote paths) which counts chunked pages withheld per active exclude prefix, reusingresolveHardExcludes+buildVisibilityClause+ the exportedescapeLikePattern. -
src/core/search/sql-ranking.ts— Pure SQL string builders.buildSourceFactorCase(slugColumn, boostMap, detail)emits a CASE with longest-prefix-match wins (returns literal'1.0'whendetail === 'high'for temporal-bypass parity with COMPILED_TRUTH_BOOST).buildHardExcludeClause(slugColumn, prefixes)emitsNOT (col LIKE 'p1%' OR col LIKE 'p2%')— OR-chain wrapped in NOT, NOTNOT LIKE ALL/ANY(those quantifiers don't express set-exclusion). LIKE meta-character escape covers all three of%,_, AND\(backslash is Postgres LIKE's default escape char). Single-quote doubling on SQL string literals so injection-style inputs are inert text.buildBestPerPagePoolCte(...)is the shared per-page max-pool CTE both engines'searchVectorinject — instead of returning the single best chunk per page from an innerORDER BY embedding <=> vec LIMIT N(which lets a page lose to a neighbor on ONE weak chunk while its strong chunk sits just below the inner cut), the CTE pools the BEST chunk score per(source_id, slug)composite key so a page surfaces on its strongest evidence; composite key (not bare slug) keeps multi-source brains correct; single source of truth so the two engines can't drift. -
src/core/search/title-match.ts— pure, zero-I/O title-phrase matcher shared by the production title boost AND NamedThingBench (no drift).isTitlePhraseMatch(query, title)returns true when the normalized query is a contiguous token run inside the title with>= MIN_CONTENT_TOKENS=2non-stopword tokens, OR an exact full-title match (covers deliberate 1-word chosen names like "Helios"). Token-boundary matching (never raw substring, so "art" doesn't match "Bartholomew"); small conservative English stopword set excluded from the content-token floor (guards against promoting generic pages on stopword-y queries); NFKC normalize so CJK / width variants converge. ExportstokenizeTitle+__test__internals. -
src/core/search/alias-normalize.ts— ONE normalizer shared by the WRITE path (ingest projects frontmatteraliases:intopage_aliases) and the READ path (search matches query againstpage_aliases), so stored aliases can't silently fail to match queries via divergent normalization (same single-source posture ascjk.ts).normalizeAlias(raw)does NFKC + lowercase + whitespace-collapse + trim + strip one layer of wrapping quotes/brackets; returns''for empty (callers MUST skip empty aliases).normalizeAliasList(value)coerces a frontmatter scalar / array / comma-list / garbage into a deduped list of normalized non-empty aliases — used by both the ingest projection and thereindex --aliasesbackfill. -
src/core/search/evidence.ts— the agent-facing why-it-matched contract (so an agent never reads one blended score, decides "no strong match, safe to create", and writes a duplicate over a fully-developed page).classifyEvidence(r, opts?)names the strongest signal (precedence:alias_hit>exact_title_match>high_vector_match(real query↔chunk cosineSearchResult.cosine≥DEFAULT_HIGH_COSINE_FLOOR=0.8, overridable viaEvidenceOpts.cosineFloor/ configsearch.evidence_cosine_floor— never the blended score, so a keyword+boost pile-up can't earn the label; keyless/hermetic runs have no cosine and degrade to keyword-based labels; legacyHIGH_MATCH_FLOOR=0.85stays exported for back-compat only) >keyword_exact(base ≥SOLID_MATCH_FLOOR=0.6) >weak_semantic).createSafetyFor(evidence)derives the don't-duplicate hint (exists/probable/unknown) the agent keys off INSTEAD of a raw threshold (a blended RRF/cosine score is not a calibrated probability).stampEvidence(results, opts?)stampsevidence+create_safetyin place once at pipeline end (after the alias hop, before slice); idempotent. -
src/core/search/private-visibility.ts— shared page-visibility resolver and SQL predicates. Explicit local trust disables page filtering; operator env/config opt-outs retain their existing semantics and failed config reads enforce privacy.privatePagesFilterFragmentauthorizes concrete page rows; link origins and timeline projections use independent ID-based predicates. Legacy slug-set helpers are for name enumeration, never authorization of data from another row/source.ops/context.ts:readPolicyOptscombines canonical source precedence, resolved privacy and holder permissions. Both engines filter content/history/graph/identity/analytics before limits or aggregation; current and historical visibility are checked independently while page privacy is enforced. Semantic response caching remains disabled, so visibility-key folding alone never authorizes reuse. -
src/commands/search-diagnose.ts—gbrain search diagnose "<query>" --target <slug> [--json] [--source <id>]: Phase-0 retrieval diagnostic. Traces WHERE a target page surfaces (or fails to) across keyword / vector (per-page max-pool) / alias / hybrid layers and names the layer responsible for an incident, so an operator can pin whether the fix is max-pool/innerLimit (vector) vs title/alias. The verdict names the layer that DOES surface the target (or "none"). Pinned bytest/search/search-diagnose.test.ts. -
src/commands/reindex-aliases.ts—gbrain reindex --aliases [--limit N] [--dry-run] [--json] [--source <id>]: backfills the free-text alias layer for EXISTING pages whose frontmatteraliases:predate the alias table (the import-time projection covers new + changed pages). Reads each page's frontmatteraliases:, writes viaengine.setPageAliases. Idempotent + convergent (setPageAliases replaces a page's alias set) so no op-checkpoint needed; walkslistAllPageRefs(cheap cross-source enumeration),--sourcenarrows. Pinned bytest/search/reindex-aliases.test.ts. -
src/eval/retrieval-quality/harness.ts+src/commands/eval-retrieval-quality.ts+test/fixtures/retrieval-quality/namedthing.jsonl+test/fixtures/retrieval-quality/namedthing/corpus.ts— NamedThingBench, the retrieval-quality eval that pins the named-thing-miss failure class. Seven query families, each a distinct failure class:title-substring(the named-thing miss itself),generic-to-named(tourist label → named thing),alias-synonym(declared alias / romanization → canonical),multi-chunk-dilution(one strong chunk among many weak — stresses max-pool),short-vs-rich,graph-relationship(guardrail),hard-negative(precision guard, must NOT return a page).gbrain eval retrieval-quality <fixture.jsonl>runs it with hard gates (e.g. title-substring Hit@1 ≥ 0.95, alias Hit@1 ≥ 0.98, multi-chunk-dilution Hit@3 = 1.0). Pure: caller injects aSearchFn(CLI useshybridSearch, tests stub) so it's engine-agnostic. Metric glossary entries (hit@1/hit@3) added tosrc/core/eval/metric-glossary.ts. Pinned bytest/eval-retrieval-quality.test.ts+test/retrieval-quality-harness.test.ts. The seed corpus (NAMEDTHING_CORPUS,seedNamedThingCorpus(engine, { embed? }),loadNamedThingQuestions()) lives intest/fixtures/retrieval-quality/namedthing/corpus.ts— the ONE brain the fixture is written against (sibling ofrelational/corpus.ts), seeded without vectors by the hermetic gate and with real vectors by the R1 A/B script, so a paid receipt and the CI gate describe the same pages. The committed fixture is 12 queries — 11 non-relational core queries plus onegraph-relationshipquery, which R1 scores together withrelational/corpus.ts's 38RELATIONAL_QUESTIONSas the 39 relational questions.seedNamedThingCorpusseeds exactly what the gate always seeded (putPagewithcompiled_truth= the chunks joined, oneupsertChunksrow per chunk withtoken_count: 10, lower-casedsetPageAliaseswhen declared) and, whenembedis supplied, embeds every chunk text in ONE batch and stores the vectors (a count mismatch throws); the result reportspages,chunks,embedded,embedded_charsfor spend accounting. Change pages, chunk boundaries or aliases only together with the fixture and its gate expectations. -
scripts/r1-namedthing-rerank-ab.ts— Phase C′ rule R1 receipt: balanced reranker ON vs OFF on NamedThingBench, paired per query, in ONE in-memory PGLite brain seeded ONCE with real embeddings (seedNamedThingCorpus;--relationaladdsrelational/corpus.ts+ its 38RELATIONAL_QUESTIONS, which with the fixture's owngraph-relationshipquery are the 39 relational questions the receipt reports beside the 11 non-relational core queries). Arms are applied withengine.setConfig(ARM_PINS: OFF =search.mode balanced,search.reranker.enabled false,search.autocut false; ON addssearch.reranker.enabled true+search.reranker.model voyage:rerank-2.5; autocut pinned off in both by default — a rerank-only comparison — and--autocut on|offis an overlay applied to BOTH arms throughapplyArmPins(engine, arm, overlay)so the ON arm can run in the pre-R2 shipped balanced shape (autocut on) without changing the pinned defaults the tests pin), so barehybridSearchresolves them the way a production balanced brain would; scoring reusesrunRetrievalQuality/evaluateGate(no local hit@k). Per query it records deduped top-3, the rank-1evidence/create_safetytier, finite-rerank_scorerow count and thedegradedstamp. INVARIANTS: readiness (rerankerReadinessForEngine) is checked BEFORE any spend and the ON arm must show noreranker_skipped/rerank_passthroughstage and ≥1 reranked row per non-empty query, else exit 2 (fail-open would silently turn ON into OFF); embedding degradation (embed_unavailable/embed_timeout/vector_arm_failed) in a live arm is exit 2 too;--embed-cache PATH(installEmbedCache) makes both arms see byte-identical query vectors and the receipt says whether that held (identical_query_vectors).r1Verdictis pure: PASS iff hit@1 losses == 0 and hit@3 losses ≤ 1, losses counted per query (OFF hit, ON miss) and never offset by wins; create_safety downgrades are counted and named, not gated. A pass-through rerank transport tees the API-echoed model id + usage into the receipt.--stub-embedis the hermetic dry run (embed transport throws, OFF arm only, ON reported as skipped — needsVOYAGE_API_KEY).--jsoncarries_meta.metric_glossary(hit@1,hit@3,mrr,create_safety); exit 0 PASS/dry-run · 1 FAIL · 2 integrity/usage. Receipt (shippedbalanced,voyage:rerank-2.5): the 11 core queries show 0 hit@1 / 0 hit@3 losses in every cell; the 39 relational queries WITHOUT the relational re-pin fell hit@1 21→3 (19 losses) and hit@3 27→5 (22 losses) — R1 FAIL; WITHsearch.relational_rerank_pinat its default 3 (measured with--autocut on, the shape that shipped before rule R2 turned autocut off) 0 losses, hit@1 21/39 and hit@3 27/39 — R1 PASS, the balanced reranker stays ON.--relational-pin N|offoverlayssearch.relational_rerank_pinon BOTH arms: the no-pin cell is--relational-pin off(or0); the default cell omits the flag and resolves the bundle default 3 exactly as production does. Pinned bytest/r1-namedthing-rerank-ab.test.ts. -
docs/architecture/RETRIEVAL.md+docs/incidents/RETRIEVAL_MAXPOOL_INCIDENT.md— retrieval-pipeline architecture reference + the named-thing-miss incident write-up (root cause, the five-layer fix, the eval that pins it). -
src/eval/brainbench/+src/commands/eval-brainbench.ts— BrainBench, the cross-harness memory conformance suite (gbrain eval brainbench; methodology indocs/eval/BRAINBENCH.md).types.tscarries the PUBLISHED interchange shapes (fixture/gold/result/baseline — mirrored as JSON Schemas inevals/brainbench/schema/; breaking changes bump the schema versions).fixtures.ts: strict loader/validator + corpusfixtures_hash(covers fixture AND gold files); agoldkey inside a fixture turn is a validation error — gold is SEALED in the gold dir and adapters only ever see sanitizedPublicTurns.seed.ts: fail-fast hermetic seeding (importFromContentnoEmbed + NULL-embeddinginsertFact; any non-importedstatus ⇒SeedError⇒ fixtureseed_failed⇒ run exit 2).adapters/shared.ts: ONErunReflexPipelineall three adapters drive with declarative config (pointer budget, suppression mode) — cross-harness comparability is structural;openclaw.ts(seamproduction, the shipped pipeline),claude-code.ts(seamproduction; drives the shippedgbrain hook user-promptpath end-to-end — fixture turns becomeUserPromptSubmitstdin JSON, synthesized Claude Code JSONL transcripts feed the real window parse + cross-turn dedupe, and resolution rides a run-scoped resolve-IPC server with the real shared secret; bench-pinned deviations disclosed indocs/eval/BRAINBENCH.md: generoususerPromptDeadlineMs, push-failure banner suppressed),codex.ts(seamcontract; static entity-index preamble whose slugs deliberately don't count as injections + ≤1 per-turn fragment; fixture conversations round-trip through the real rollout format + the shipped parsersrc/core/transcripts/codex.tsfor turn selection — fragment DELIVERY remains harness-shaped until a shipped codex injection path lands).metrics/: know-to-ask (+false-fire anti-gaming companion), push (micro-averaged P/R), write-back (drives the PRODUCTION conversation→facts pipeline via the injectable-extractor seam; gold extractor in CI, real extractor under--llm), continuity (writer→reader pairs on a shared brain through DIFFERENT adapters; pointer-injection OR stored-fact keyword probe).harness.ts: ONE in-memory PGLite per run +resetTablesbetween fixtures (longmemeval engine-sharing pattern); read-only suites share one seeding across all adapters; emits per-(harness×suite) cells + re-scoreable turn rows;source_isolation_violationscounted per turn and gated at zero.scoreboard.ts: markdown render, canonical diff-stable committed baseline (4-decimal rounding, sorted keys, receipts excluded),compareBaselineswith main-baseline governance — same-hash count-aware gate vs corpus-bless mode (the committed baseline must byte-match the run; regressions vs main require ajustification). The CLI brings its own PGLite (cli.ts routes before connectEngine), writes--outas the canonical CI artifact, and terminates via an explicit grace-tickprocess.exit(verdict)(0 pass / 1 regression / 2 error) because PGLite stompsprocess.exitCodeand Bun discards queued stdout on exit.runBrainBenchCore()is the in-process entryeval run-alluses (one record per sweep,EvalRunRecordschema_version 3,mode: 'n/a'). Pinned bytest/brainbench-*.test.ts+test/eval-brainbench-e2e.slow.test.ts. -
evals/brainbench/— the committed BrainBench corpus: 141 fixtures (135 generated + 6 hand-authored spike) / 241 gold-annotated turns across 7 categories (kta-pos/kta-neg/push/write-back/continuity/multi-source/adversarial), ~15% holdout (excluded from the CI gate, scored in published--include-holdoutruns).generator/gen.tsrebuilds the corpus byte-identically (Mulberry32, seed 42; whole-cloth fictional universe from curated synthetic name pools so scenario privacy is structural; prose is template-synthesized with PRNG-selected variants — deliberately no LLM pass, difficulty stays controlled; several know-to-ask variants intentionally exercise documented v1 reflex limits so the baseline measures the roadmap).gold/is sealed;schema/is the foreign-runner contract (gbrain-evals drives the suite as a subprocess via--fixtures DIR --gold DIR --json --out FILE);baselines/main.jsonis the committed gate baseline;_ledger.jsonrecords counts/seed/rebuild command. CI: the.github/workflows/test.ymlbrainbenchjob +scripts/ci-brainbench-gate.sh(fetches MAIN's baseline viagit show origin/master:…— a PR cannot rewrite what it's compared against; first-landing path runs ungated) +scripts/render-brainbench-delta.ts(compact step-summary/PR-body delta block from the--outartifact). Privacy:scripts/check-synthetic-corpus-privacy.shscansevals/brainbench/{fixtures,gold}inbun run verify. -
src/core/types.tsextension +src/core/operations.ts:search+src/core/import-file.ts+src/cli.ts+src/core/search/telemetry.ts— the wiring layer for the retrieval-quality stack.SearchResultcarriesevidence,create_safety,title_match_boost,alias_hit(all optional; evidence/create_safety reference the union types inevidence.ts). ThesearchMCP op uses a cheap-hybrid path by default and accepts a per-callmode(conservative|balanced|tokenmax) honored ONLY for trusted/local callers (resolvePerCallMode(ctx, ...)— remote callers use the configured mode so a remote provider can't force tokenmax spend); every search path stamps evidence fail-soft.importFromContentprojects frontmatteraliases:intopage_aliasesvianormalizeAliasList+engine.setPageAliasesso new + changed pages register aliases at ingest.src/cli.tscarries thegbrain search diagnosedispatch (lazy import) and reconciles thesearchCLI path with the cheap-hybrid op.src/core/search/telemetry.tscarries in its rollup the rank-1 base_score drift signal (sum/count + 3 coarse buckets, aggregate not per-query), surfaced viagbrain search stats, backed by migration v111'ssearch_telemetrycolumns. Tests:test/cli-search-dispatch.test.ts,test/search/per-call-mode.test.ts,test/search/telemetry-rank1.test.ts,test/search/title-boost-stage.test.ts,test/search/alias-hop.test.ts,test/search/evidence.test.ts,test/search/searchvector-maxpool.test.ts,test/search/pre-migration-failopen.test.ts. -
src/commands/eval.ts—gbrain evalcommand: single-run table + A/B config comparison. Sub-subcommand dispatch onargs[0]routesgbrain eval export+gbrain eval prune+gbrain eval replayinto session-capture handlers; baregbrain eval --qrels …fall-through preserves the legacy IR-metrics flow.gbrain eval cross-modalis in the dispatch (the user-facing path is the cli.ts no-DB branch —src/commands/eval.ts:cross-modalonly fires when callers re-enter with an existing engine). -
src/commands/eval-cross-modal.ts— multi-model quality gate. Three different-provider frontier models score the OUTPUT against the TASK on a 5-dim list. Verdictpass(exit 0) /fail(exit 1) /inconclusive(exit 2; <2/3 model successes). Reusessrc/core/ai/gateway.ts:chat()so config/auth/aliasing comes from the gateway recipe registry — no parallel provider stack. Self-configures the gateway (configureGateway(loadConfig() + process.env)) since the cli.ts dispatch bypassesconnectEngine(). Default cycles 3 in TTY, 1 in non-TTY (partial cost guardrail) via the sharedresolveCycleDefault(explicit, isTty)insrc/core/eval/cycle-default.ts; the cost-estimate banner appendscycleDefaultSuffix(...)(for 1 cycle(s) (non-interactive default; --cycles N for more)) when the value is the silent non-TTY fallback, so the 1-vs-3 difference isn't hidden. Receipts land atgbrainPath('eval-receipts')/<slug>-<sha8-of-output>.json.--batch <jsonl> [--limit N] [--concurrent N] [--max-usd FLOAT] [--yes]fans out cross-modal scoring across a LongMemEval-shape JSONL; mutually exclusive with--task(fail-fast usage error if both set); filterskind: "by_type_summary"rows; pre-flight cost estimate refuses if> --max-usdwithout--yes(default cap 5.00 USD). Semaphore-bounded fan-out via inlinerunWithLimit<T>(items, limit, fn)$ (\text{exported} \text{for} \text{unit} \text{tests}): \text{max} \text{N} \text{questions} \text{in}-\text{flight} \times 3 \text{model} \text{slots} = \text{ceiling} \text{of} 3\text{N} \text{parallel} \text{API} \text{calls} (\text{default} $--concurrent 3→ 9). Per-question receipts land in a per-batch tempdir and are deleted at end of run; the summary receipt inlines per-question verdicts as JSON, not file paths. Exit precedence (batch-level policy, NOT inherited from aggregate.ts): ERROR > FAIL > INCONCLUSIVE > PASS. DI seam:runEvalCrossModal(args, opts?: {runEval?: typeof runEval})mirrorsrunEvalLongMemEval(args, {client?}); tests passopts.runEvalto bypass real LLM calls AND the gateway availability check. Pinned bytest/eval-cross-modal-batch.test.ts. -
src/core/eval/cycle-default.ts— single source of truth for the eval cycle-count default. ExportsDEFAULT_CYCLES_TTY = 3,DEFAULT_CYCLES_NONTTY = 1,resolveCycleDefault(explicit, isTty): {cycles, usedNonTtyDefault}, andcycleDefaultSuffix(r)(returns(non-interactive default; --cycles N for more)only when the non-TTY default was applied, else''). Consumed byeval-cross-modal.ts,eval-takes-quality.ts(run + regress), andtakes-quality-eval/runner.ts(core uses only the constant — library stays TTY-agnostic; the CLI owns the TTY=3 upgrade + banner annotation).eval-suspected-contradictions.tsapplies the same transparency to its$5/$1budget default via abudgetUsdExplicitflag (the budget is overwritten in-place so explicitness can't be inferred post-hoc). Not shared withresolveWorkersWithClamp(different domain, no engine, no dedup). Pinned bytest/eval/cycle-default.test.ts,test/eval-suspected-contradictions-budget-default.test.ts. -
src/core/cross-modal-eval/json-repair.ts—parseModelJSON(raw)named export with a 4-strategy fallback chain (direct parse → fence-strip → trailing-comma + single-quote + embedded-newline repair → regex nuclear option). Adversarial input throws rather than fabricating scores — the aggregator treats a throw as "this model contributed nothing this cycle" so the gate stays correct at >=2/3 successes. -
src/core/cross-modal-eval/aggregate.ts— pure verdict logic. Judge dimension keys are normalized with trim + lowercase before cross-model roll-up, soCORRECTNESSandcorrectnesscannot split into separate one-model dimensions. Pass criterion:(successes >= 2) AND (every dim mean >= 7) AND (every dim min across models >= 5). Inconclusive when <2/3 models returned parseable scores (guards theObject.values({}).every(...) === trueempty-array PASS trap). -
src/core/cross-modal-eval/runner.ts— orchestrator. The judge prompt pins the exactscoresJSON keys per dimension (dimensionScoreKey= the label before the em-dash) so judges cannot invent spellings; aggregate.ts's normalization is the backstop.buildPromptwraps the task and the candidate in<task_to_grade>/<candidate_output>data blocks, andneutralizeClosingTagrewrites every closing form of those tags inside the untrusted text (case-insensitive, whitespace-tolerant) to the visibly-escaped<\/tag, so a candidate carrying its own closing delimiter cannot terminate the block and land text outside the data boundary — the post-candidate grading-only instruction stays the last thing the judge reads.callSlotsendsEVALUATOR_SYSTEM_PROMPTassystemand the bounded prompt as the single user turn. Each cycle runsPromise.allSettled([gwChat(slotA), gwChat(slotB), gwChat(slotC)])(bare allSettled, no rate-leases for the CLI path). Stops early on PASS or INCONCLUSIVE; runs up to 3 cycles. Default slots:openai:gpt-5.2/anthropic:claude-opus-4-7/deepseek:deepseek-v4-pro.estimateCost()prices via the canonical model-pricing table;test/cross-modal-default-slots.test.tspins recipe support, pricing coverage, and three distinct providers. -
src/core/cross-modal-eval/receipt-name.ts— receipt filename binds (slug, SKILL.md sha-8).findReceiptForSkill(skillPath, receiptDir)returns'found' | 'stale' | 'missing'. Skillify-check surfaces the status as informational; the audit does NOT fail on missing/stale receipts. -
src/core/cross-modal-eval/receipt-write.ts— wrapsfs.writeFileSyncwithmkdirSync({recursive:true})ahead of every write (gbrainPath()does NOT auto-mkdir). -
src/commands/eval-export.ts— streamseval_candidatesrows as NDJSON to stdout withschema_version: 1prefix on every line. EPIPE-safe, progress heartbeats on stderr, stable id-desc tiebreaker so--sincewindows never dupe/miss rows. -
src/commands/eval-prune.ts— explicit retention cleanup. Requires--older-than DUR.--dry-runreports would-delete count. -
src/commands/eval-replay.ts— contributor-facing replay tool. Reads NDJSON fromgbrain eval export, re-runs each capturedquery/searchop against the current brain, computes set-Jaccard@k between captured + currentretrieved_slugs, top-1 stability rate, and latency Δ. Stable JSON shape (schema_version: 1) for CI gating; human mode prints a regression table. Pure Bun, zero new deps. The dev-loop half of BrainBench-Real. Seedocs/eval-bench.md.parseNdjsonskips lines where_kind === 'baseline_metadata'sogbrain bench publishbaselines parse cleanly without the metadata header polluting row counts. ExportsreplayCore(engine, opts): Promise<{summary, results}>+ReplaySummarytype sogbrain eval gatecalls replay in-process (NOT subprocess — avoids gbrain-version-drift for source-tree CI). CLIrunEvalReplaywrapsreplayCore. -
src/core/bench/baseline-file.ts+src/core/bench/qrels-file.ts+src/core/bench/correctness-gate.ts+src/commands/bench-publish.ts+src/commands/eval-gate.ts— the eval loop.gbrain bench publish --from <captured.ndjson> --to <X.baseline.ndjson>writes a baseline (stamps stablequery_hashper row; metadata header carries_kind: 'baseline_metadata'+ thresholds +source_hash+baseline_mean_latency_ms; deterministic sort by(tool_name, query_hash); strict: empty=fail, dupes=fail with paste-ready hint,--toexists=refuse without--force).gbrain eval gate [--baseline X] [--qrels Y]is the two-gate dispatcher (regression gate via in-processreplayCore, correctness gate via barehybridSearchfor determinism, both must pass when both flags set, exit 0 PASS / 1 FAIL / 2 USAGE). Source-id-aware:bench publishdedup key is(tool_name, source_ids, query_hash); qrels compare keys are${source_id}::${slug}everywhere (so multi-source brains are keyed correctly at the file-shape layer). Latency math:(baseline + delta) / baseline <= multiplier. Fail-closed: ANY in-process throw flips verdict to fail with named breach inbreaches[]— never silently exit 0..qrels.jsonpreserves the 12-rowtest/fixtures/eval-baselines/qrels-search.jsonfixture (slug-onlyrelevant_slugs+first_relevant_slugauto-promote tosource_id='default') AND supports the federated shape (explicitrelevant: [{source_id, slug}]+expected_top1).correctness-gate.tsruns each qrels query via barehybridSearch; per-query throw recorded aserrored: trueand flagged as gate failure. Audit JSONL at~/.gbrain/audit/bench-publish-YYYY-Www.jsonl. Hermetic mode:--embedder deterministic(correctness gate ONLY — rejected with--baseline, requires--qrels) embeds each query as the qrels fixture's basis vector viasrc/eval/deterministic-embed.ts(basisEmbeddingunit vectors; FNV-1a-derived fallback dim for off-fixture query texts) and threads it into barehybridSearchthrough thequeryEmbedFnseam — no API keys, no network; barehybridSearchneither reads nor writes the semantic query cache (both live inhybridSearchCached), so deterministic runs cannot poison cached production results.scripts/run-eval-canary.ts(check:eval-canarypackage script, on-demand; in CI the same runner executes viatest/eval-canary.test.tsin the unit matrix, not in the verify battery) is the hermetic CLI retrieval canary built on it: boots a throwaway PGLite brain under a temp GBRAIN_HOME, seeds the qrels fixture corpus (the expected-top1 page carries its query text intimelinetoo — page-grain FTS indexes title(A) + timeline(C) only,compiled_truthis deliberately unindexed), spawns the REAL CLI with engine-reroute/provider env stripped, and asserts exit 0 + metric floors;--recordadditionally appends an EvalRunRecord-shaped line to.gbrain-evals/eval-results.jsonl. Honest scope: the canary gates the hybrid ranking pipeline (keyword/title/alias arms + RRF against gold qrels) with synthetic vectors — semantic-embedding regressions remain the keyed eval suites' job. Pinned bytest/bench/baseline-file.test.ts,test/bench/qrels-file.test.ts,test/bench/correctness-gate.test.ts,test/bench-publish.test.ts,test/eval-gate.test.ts,test/eval-canary.test.ts,test/eval-replay-metadata-skip.test.ts,test/cycle/nightly-probe-adapters.test.ts,test/autopilot-nightly-probe-wiring.test.ts,test/e2e/eval-loop.test.ts. -
src/core/cycle/nightly-probe-adapters.ts— bridges the autopilot's object-shapeNightlyProbeDepsto the argv-shaperunEvalLongMemEval+runEvalCrossModalCLI functions. Cross-modal adapter argv MUST include--output summaryPath(without it the summary lands at the default receipt path and the adapter reads nothing fromsummaryPath). In-process invocation (NOT subprocess) — avoids gbrain-version-drift for source-tree CI. Pinned bytest/cycle/nightly-probe-adapters.test.ts(incl. the argv-shape guard for the--outputrequirement). -
test/eval-replay-gate.test.ts+test/fixtures/eval-baselines/qrels-search.json— hermetic retrieval qrels gate running in the standard PR unit-shard CI matrix (.github/workflows/test.yml, NOT the fixed-file E2E workflow). Uses the canonical PGLite block (test-isolation R3+R4) and the basis-vector embedding pattern fromtest/e2e/search-quality.test.ts:23-28for fully hermetic retrieval. The qrels fixture (12 queries) uses PLACEHOLDER names only (alice-example, widget-co-example, etc. — privacy rule) and embeds each query at a deterministic basis dimension so retrieval is reproducible. Each query listsrelevant_slugs[]+first_relevant_slug; the test computestop1_match_rate(top-1 == first_relevant) andrecall@10(fraction of relevant_slugs in top-10), asserting both meet floors (defaults>= 0.80and>= 0.85). Env-overridable floorsGBRAIN_REPLAY_GATE_TOP1_FLOOR/GBRAIN_REPLAY_GATE_RECALL_FLOOR(viawithEnv()per R1). Refresh discipline: when ranking changes intentionally move expected slugs, editqrels-search.jsondirectly with aWhy:line in the commit body or the gate degrades to rubber-stamp. Pinned bytest/eval-replay-gate.test.ts(incl. a privacy-grep guard against real names). -
src/core/cycle/nightly-quality-probe.ts+src/core/audit-quality-probe.ts+test/fixtures/longmemeval-nightly.jsonl+test/nightly-quality-probe.test.ts— opt-in nightly cross-modal quality probe. The phase runsgbrain eval longmemeval --by-typeagainst the committed 10-question placeholder fixture, pipes output throughgbrain eval cross-modal --batch --max-usd 5 --yes, and writes one event per run to~/.gbrain/audit/quality-probe-YYYY-Www.jsonl(ISO-week-rotated, mirrorsaudit-slug-fallback.ts; honorsGBRAIN_AUDIT_DIR). Default DISABLED — opt-in viagbrain config set autopilot.nightly_quality_probe.enabled true(prevents surprise API spend). 24h rate limit (pureshouldRunNightly(now, recentEvents, windowMs?)) skips with audit rowoutcome: rate_limited. Embedding-key short-circuit: longmemeval needsgateway.embedQuery(), so the phase exits early withoutcome: no_embedding_key+ stderr warn when no provider configured. Full DI surface viaNightlyProbeDeps(isEnabled,hasEmbeddingProvider,resolveMaxUsd,resolveRepoRoot,runLongMemEval,runCrossModalBatch,now) so the unit test stubs every external effect. Cost ceiling: $5/run × 30 nights ≈ $150/month worst-case; expected ~$10.50/month. Thenightly_quality_probe_healthdoctor check (src/commands/doctor.ts, right afterslug_fallback_audit) reads last 7 days: SKIPPED when flag off (with enable command); OK when enabled + all PASS; WARN on any FAIL / ERROR / BUDGET_EXCEEDED with per-outcome counts. Pinned bytest/nightly-quality-probe.test.ts. -
src/commands/eval-trajectory.ts+src/commands/founder-scorecard.ts+src/core/trajectory.ts— temporal trajectory + founder scorecard.gbrain eval trajectory <entity>shows the chronological typed-claim history (mrr/arr/team_size/etc) with regressions auto-flagged inline;gbrain founder scorecard <entity>rolls up claim_accuracy / consistency / growth_trajectory / red_flags into one JSON. Pure-function math intrajectory.ts:detectRegressions(points, threshold)walks consecutive metric-value pairs per metric (10% drop default, env overrideGBRAIN_TRAJECTORY_REGRESSION_THRESHOLD);computeDriftScore(points)returns1 - mean(cosine(emb[i], emb[i-1]))over existing embeddings (null when <3 embedded points). Backed byBrainEngine.findTrajectory(opts)— both Postgres and PGLite, single SQL query, deterministicORDER BY valid_from ASC, id ASC. Source-scoped via thesourceIdscalar /sourceIdsarray dual pattern; visibility-filtered for remote callers. MCP opfind_trajectory(read scope, NOT localOnly) registered afterfind_experts. Migration v67 adds optional typed-claim columns (claim_metric,claim_value,claim_unit,claim_period) + a partial index on(entity_slug, claim_metric, valid_from) WHERE claim_metric IS NOT NULL. Fence widens from 10 to 14 cells when any row has typed data; renderer stays at 10 cells when none do (no churn diff on existing fences). Metric labels normalize to lowercase snake_case vianormalizeMetricLabel(15-entry seed map). Theconsolidatecycle phase does semantic upsert keyed on(page_id, claim, since_date)(so re-running the cycle afterextract_factsclearsconsolidated_atcannot append duplicates viaMAX(row_num)+1) and writes chronologicalvalid_untilon each cluster's older facts. Theextract_factscycle phase batch-embeds viagateway.embed()before insert AND threadspages.effective_dateas thepageEffectiveDatefallback forvalid_from(precedence: fence-row > pageEffectiveDate > now()). The contradiction probe MUST NOT writevalid_until— grep guard attest/eval-contradictions/no-valid-until-write.test.ts. Haiku extraction lives insrc/core/facts/extract.ts(not theextract-facts.tscycle phase); its output cap is configfacts.extraction_max_tokens(default 4000), astopReason: 'length'response retries once at 2× the cap, and persistent truncation warns loudly on stderr instead of silently extracting zero facts. Mixed extractor arrays salvage valid candidates and warn with the dropped malformed count; all-malformed output still fails retryably, while an explicitly empty facts array remains a successful empty result.pageEffectiveDateis OPTIONAL becausefence-write.tscallers have no Page object. Migration v89 adds a nullableevent_type TEXTcolumn onfactsso the substrate carries event-shaped rows (event_type='meeting'/'job_change'/'location_change') alongside metric rows.TrajectoryPoint.event_type: string | nullprojected by both engines.TrajectoryOpts.kind?: 'metric' | 'event' | 'all'filter (default'all');founder-scorecard+eval-trajectorypasskind: 'metric'explicitly. Back-compat pinned bytest/regressions/v0_40_2_0-trajectory-backcompat.test.ts(byte-identicalcomputeFounderScorecard+computeTrajectoryStatswith and without event rows); engine parity intest/engine-parity-event-type.test.ts. -
src/core/trajectory-format.ts— sharedformatTrajectoryBlock(points, entitySlug, opts)consumed by bothgbrain think(production) and the LongMemEval harness (benchmark). Groups by(metric ?? event_type), per-metric cap 20, total cap 100, knowledge_update intent annotates value-change rows with(superseded prior). Emits a<trajectory entity="...">XML envelope —INJECTION_PATTERNSinsrc/core/think/sanitize.tsescapes</trajectory>,<trajectory ...>open tags, and attribute injection so adversarial fact text can't break out. Pinned bytest/trajectory-format.test.ts. -
src/core/think/intent.ts+src/core/think/entity-extract.ts— pureclassifyIntent(question)returns'temporal' | 'knowledge_update' | 'other'(regex-first, no LLM,'other'fast path short-circuits with zero SQL).extractCandidateEntities(question, retrievedSlugs)pulls high-precision candidates from retrieved entity-prefix slugs (people/,companies/,organizations/) and medium-precision noun phrases. Stop-word boundaries + leading-verb stripper handle "When did I last meet Marco" →marco. Both consumed byrunThinkand the LongMemEval harness so the two paths cannot drift. Pinned bytest/think-intent.test.tsandtest/think-entity-extract.test.ts. -
src/commands/eval-suspected-contradictions.ts+src/core/eval-contradictions/{judge,runner,types,date-filter,cost-tracker,cache,severity-classify,cross-source,trends,calibration,judge-errors,auto-supersession,fixture-redact}.ts—gbrain eval suspected-contradictions [run|trend|review]. Probe samples top-K retrieval pairs per query (cross-slug + intra-page chunk-vs-take), date pre-filters (3-rule layered — same-paragraph-dual-date overrides separation rule), LLM judge (query-conditioned; UTF-8-safe truncation; confidence-floor double-enforcement; resolution_kind output drives paste-ready commands), persistent cache keyed on(chunk_a_hash, chunk_b_hash, model_id, prompt_version, truncation_policy)(prompt edits cleanly invalidate prior verdicts), Wilson 95% CI calibration on the headline percentage withsmall_sample_notewhen n<30, judge_errors as first-class typed counters (parse_fail/refusal/timeout/http_5xx/unknown — avoids bias from silent skip), trend writes toeval_contradictions_runs, source-tier breakdown reusesDEFAULT_SOURCE_BOOSTSprefix logic, deterministic sampling (combined_score DESC + lex tiebreaker for stable cache hit-rate). Hermetic viajudgeFn+searchFnDI in the runner; never touches the real gateway in tests. Engine surface:BrainEngine.listActiveTakesForPages(batched),writeContradictionsRun+loadContradictionsTrend,getContradictionCacheEntry+putContradictionCacheEntry+sweepContradictionCache. Schema migrations v51 + v52. MCP opfind_contradictions(read scope, NOT localOnly, NOT in subagent allowlist — user-initiated only). Doctor check surfaces high-severity findings with paste-ready resolution commands; synthesize phase pre-fetches the latest probe's top-5-by-severity findings and threads them intobuildSynthesisPromptas an informational block. Architecture doc:docs/contradictions.md. -
src/core/think/index.ts—runThinkbuilds its internalLLMClientvia a small adapter wrappinggateway.chat()fromsrc/core/ai/gateway.ts(notnew Anthropic()directly) so stdio MCP launches (Claude Desktop, Cursor) that don't inherit shell env still find a key set viagbrain config set anthropic_api_key(the gateway reads~/.gbrain/config.jsonAND env). Test seam:opts.client?: ThinkLLMClientinjection works (test/think-pipeline.serial.test.ts,test/think-gateway-adapter.test.ts);opts.stubResponseshort-circuits before any LLM call. When neither key nor client is available, the "no LLM available" stub fires withNO_ANTHROPIC_API_KEY. Trajectory injection (default ON):runThinkorchestratesclassifyIntent(question)→extractCandidateEntities(question, retrievedSlugs)→findTrajectory(5sPromise.racetimeout per candidate, concurrency cap 3) →formatTrajectoryBlock.buildThinkUserMessage(insrc/core/think/prompt.ts) has atrajectory?: ThinkTrajectoryBlockOptsslot honoring BOTH prompt orderings (calibration mode: retrieval → calibration → trajectory → question; default mode: question → retrieval → trajectory → instruction). The MCPthinkop handler mapssourceScopeOpts(ctx)ontoRunThinkOptsviathinkSourceScopeOpts(ctx)(operations.ts), andrunThinkthreads the scope intorunGather(src/core/think/gather.ts) — so every gather stream (hybrid retrieval, takes keyword + vector via the engines' scopedsearchTakes/searchTakesVector, graph walk viatraversePaths) AND trajectory resolution stay within the caller's source grant (federatedsourceIds[]wins over scalarsourceId); pinned bytest/e2e/think-source-isolation-pglite.test.ts. Config keythink.trajectory_enabled(defaulttrue). Any error in the trajectory path degrades to "no block injected" +TRAJECTORY_INJECTION_FAILEDwarning — the think call never crashes from trajectory. Production path skipsfallback_slugifyresolutions (avoid querying invented slugs); the LongMemEval harness accepts them. Pinned bytest/think-trajectory-injection.test.ts. Debug:GBRAIN_THINK_DEBUG=1 gbrain think "..."prints the spliced prompt to stderr. -
src/commands/eval-longmemeval.ts+src/eval/longmemeval/{harness,adapter,sanitize,extract,intent,reader,emit,capture,gateway-client,trajectory-route}.ts—gbrain eval longmemeval <dataset.jsonl>runs the public LongMemEval benchmark against gbrain's hybrid retrieval and is the repo's receipt producer for the strict retrieval metric and, with--judge, for judged answer accuracy (both reproduction paths are indocs/eval-bench.md). One in-memory PGLite per run viacreateBenchmarkBrain+withBenchmarkBrain; between questions,TRUNCATEover runtime-enumeratedpg_tableswith the infrastructure tables (sources,config,gbrain_cycle_locks,subagent_rate_leases) preserved — which is why the run's pins, written once viaengine.setConfig, hold for every question.cli.tspre-dispatch bypass skipsconnectEngine(), so~/.gbrainis never opened. Metric (metrics.ts): recall joins on RAW dataset session ids through a per-question slug→raw map;recall_all@k(every gold session among the distinct sessions of the top-k CHUNK rows returned atlimit: k) is the headline,recall_any@kthe diagnostic, and the per-rowrecall_hitis a deprecated alias ofrecall_any_hit;_absabstention questions are emitted withabstention: truebut stay out of the recall denominators unless--include-abstention; a slug collision touching a gold id aborts that question with an error row. Pins:--mode,--reranker on|off,--autocut on|off,--expansion(OFF for every mode — the per-call setting wins over the bundle, so--mode tokenmaxalone never expands),--expansion-variant-budget legacy|(0,4],--search-pin KEY=VALUE(repeatable; anysearch.*key, written verbatim viaengine.setConfig); precedence is explicit flag >--search-pin> injectedRunOpts.searchConfigSnapshot> bundle, so a flag wins over a pin on the same key. The raw--search-pinmap folds intoretrieval_config_hash(only when non-empty, so pin-free runs keep their hash identity) while the knobs hash covers only the resolved mode knobs. The reranker gate keys on the RESOLVED pin (flag,--search-pin, snapshot or bundle), not only on an explicit--reranker on: a run that resolves to reranker on preflightsrerankerReadinessForEngineand exits 2 with the fix when the reranker cannot run (abalancedrun with noVOYAGE_API_KEYexits with the fix text —--reranker offor set the key — instead of quietly scoring un-reranked rows). Gates (exit 1): a run in which every question errored, a row that fell through un-reranked under a resolved reranker-on pin (reranker_skipped_rows), a row whose vector arm silently degraded to keyword-only on a non---keyword-onlyrun (vector_degraded_rows:vector_enabled:false/embed_unavailable/embed_timeout), an--expansionrow that did not expand as configured (expansion_failed_rows), an--expansion-replaymiss, and--by-type-floor Fbreaches — which gate onrecall_allby default (--by-type-floor-metric recall_anyselects the lenient rate). The run-end gates andGBRAIN_LME_DEBUG=1prints per-question wall time to stderr;--recordrun through ONEfinishRunfrom both the main path and the no-op resume path. Rows:retrieved[](every returned chunk row:slug,chunk_id, RAWsession_id,rank,score,rerank_score?,alias_hit?),retrieved_session_ids,distinct_sessions_in_top_k,gold_total/gold_found,search_meta(vector_enabled,expansion_applied,degraded, derivedreranked= some finitererank_scoreand no skip stage,autocut),retrieval_config_hash, the reader pins on every answered row (reader_model= the requested id,reader_model_snapshot= the provider-reported id when it differs — a dated API snapshot — else null,reader_prompt_sha,reader_max_tokens;--retrieval-onlyrows carryretrieval_only: trueinstead so a judge backfill can refuse them),expansion_variantswhen expansion ran (--expansion-replay FILEserves them back and stampsexpansion_replayed, so every cell differs only in its knobs), and with--capture-poolarerank_pool— the exact pre-autocutreturnPoolhybridSearchhands toapplyAutocut(viaHybridSearchOpts.onRerankPool; unscored alias/exact-lookup injections included;rrf_rank,pool_rank,est_tokensper row;autocut_kept_keyswhen the returned rows are the kept set) for the offline autocut-floor replay. Summary:--by-typeemits aschema_version: 2by_type_summary(per-type + aggregate{total, all_hit, all_rate, any_hit, any_rate},excluded_abstention,mean_distinct_sessions,legacy_rows,gold_missing_from_haystack,slug_collisions, andrun_config: every pin, embeddermodel@dims, dataset sha256 + question count,knobs_hash+KNOBS_HASH_VERSION,retrieval_config_hash, the embed-cache receipt, the degradation counters); resume-replace keeps exactly ONE summary at the tail (emit.ts:emitByTypeSummary: the summary is always the FINAL line — any priorby_type_summaryis removed before the new one is appended, and the file rewrite is atomic,<path>.summary.tmp+ rename, so a kill mid-write never truncates the paid rows — its_meta.metric_glossaryis the ONE glossary block per response and names exactly the metrics carried,recall_all@k,recall_any@k, andqa_accuracywhen the judge lane ran; a CR inside any emitted line throws rather than splitting a JSONL record). Reader (reader.ts):READER_SYSTEM_TEXTis a module constant — every per-question input (question,Current Date: {question_date}when the dataset row carries one, the trajectory block, the sanitized<chat_session>blocks) lives in the USER message — soREADER_PROMPT_SHA(sha256 of the system text) is a run-level pin and two rows with equal shas saw the identical instruction;READER_MAX_TOKENSis 512 (official 500). Disclosed deviations from the officialrun_generation.pyprompt: an abstention instruction (say the information is not available / "I don't know" when the retrieved sessions lack it — without it the 30_absquestions are answered and judged wrong by construction), the #4338 data-boundary framing + pattern stripping, and the 512 cap.generateAnswerreturns{text, response_model},response_modelbeing the provider-reported snapshot when it differs from the requested id (the harness's gateway client mapsChatResult.responseModel ?? modelinto the Anthropic-shapedmessage.model). Judge lane (--judge): implies--by-type;--judge --retrieval-onlyis a usage error. Preflight (judge-lane.ts:judgePreflight): no usable chat provider for the judge model → exit 1;--max-usdagainst an unpriced model → exit 2 (pass--max-usd off); estimate over the cap without--yes→ exit 2. The estimate (judge.ts:estimateJudgeRunUsd) assumesREADER_MAX_TOKENSper live hypothesis and the stored hypothesis for backfill rows; theBudgetLedgersoft-stops at the cap and the remaining rows are stampedjudge_skipped: 'budget'. Live rows are judged inline after each reader call.--judge --resume-from FILEis the judge-only backfill:selectBackfillRowspicks every prior row with a hypothesis and no settled verdict (judge_errorrows are re-judged,--retrieval-onlyrows are refused, rows absent from the dataset are left unjudged with a WARN); a same-file resume ALWAYS appends (makeEmitter(outputPath, append)has no atomic-rewrite mode — appending is the only write path) — judged backfill rows and retries land as newer duplicates the moment they finish, so a timeout or kill loses at most the in-flight question — andcompactJsonlByQuestionId(emit.ts) rewrites the file atomically (<path>.compact.tmp+ rename) to one row per question_id (last wins, first-seen order) before the summary is emitted;readJsonlRows/loadResumeSetread appended files last-wins and--judge-concurrency Nparallelizes the backfill. Every judged row carriesjudge_config_hash— the judge pins (model,JUDGE_PROMPT_VERSION, max_tokens, temperature) plus the reader pins the row was produced under; a prior row hashes from its OWN recordedreader_model/reader_prompt_sha/reader_max_tokens, so a file answered by another reader is never relabelled as this run's, and rows already judged under a different hash are refused unless--allow-mixed-run-config. Run end:qa_accuracy(qa-accuracy.ts) is rebuilt from ALL rows (prior + new, last row perquestion_idwins) whenever the lane ran or any row carries a verdict; a run withjudge_errors > 0,skipped_budget > 0orunjudged > 0(theqa_accuracy.completepredicate) prints aFAIL … NOT publishableline and exits 1 unless--allow-incomplete-judgments(WARN, exit 0) — the fix is--judge --resume-from FILEuntil all three are 0; a row whose judge call threw is stampedjudge_error: 'provider_error'by the backfill, never silently left unjudged. Judge row fields: exactly one ofjudge_correct/judge_error(+ secret-redactedjudge_error_detail) /judge_skipped, plusjudge_model,judge_model_snapshot,judge_raw(first 200 chars),judge_cost_usd,judge_attempts,judge_prompt_kind,judge_prompt_version,judge_config_hash.RunOpts.judgeClient/judgeBackoffMsare the test seams. Embed cache: on by default at~/.cache/gbrain-eval/longmemeval-embed.sqlite(--embed-cache FILE/--no-embed-cache; never installed on--keyword-only), installed through the gateway embed-transport seam so every arm sees byte-identical vectors; the cache transaction wraps only the embed-producing section of a question (import + search), never a reader call. Resume:--resume-from FILEre-scores prior rows fromretrieved[]+ the dataset's gold (stored booleans are never trusted) and refuses a file whose rows carry a differentretrieval_config_hashunless--allow-mixed-run-config; error rows without a hypothesis are retried.--question-ids FILErestricts the run to a listed slice (unknown ids or an empty file exit 1);--recordappends a secret-redactedEvalRunRecord(schema 3, suitelongmemeval, params =run_config) viapersistRunRecord. Flags live in ONE table,LME_FLAGS, that drives bothparseArgsandprintHelp, so the flag-registry scan sees every literal and help cannot drift from the parser. Sanitization parity:harness.tsreusesINJECTION_PATTERNSfromsrc/core/think/sanitize.ts; retrieved chat content is wrapped in<chat_session id="..." date="...">and the answer-gen system prompt declares it UNTRUSTED.RunOptsseams —client,extractorClient/extractorModel,engine,searchConfigSnapshot,expandFn,embedTransport,rerankerReadiness,recordDir— let the full pipeline run hermetically without an API key. Trajectory routing (default on;--no-trajectorybypasses BOTH the extractor and the intent routing, the like-for-like retrieval setting):extract.tsruns the Haiku claim extractor over each haystack session into the benchmark brain'sfactstable (content-hash cache, per-question alias map, fail-open on every error path),intent.tsprefers the dataset'squestion_typebefore the SHARED regex set fromsrc/core/think/intent.ts, temporal/knowledge_update questions splice afindTrajectoryblock into the reader prompt, and rows carryintent,trajectory_points,entity_resolved,resolution_source,methodology_note(extractor=haiku-preprocess-full-haystack-v1— that number is "gbrain + Haiku-preprocess", not "gbrain alone"). Pinned bytest/longmemeval-metrics.test.ts,test/eval-longmemeval-mixedcase.slow.test.ts(raw-id join, strict/any split, abstention, pins, gates over the placeholder fixturetest/fixtures/longmemeval-mixedcase.jsonl),test/eval-longmemeval-cli-smoke.test.ts,test/longmemeval-embed-cache.test.ts,test/longmemeval-judge.test.ts+test/eval-longmemeval-judge.slow.test.ts(the judge lane, reader pins andqa_accuracydenominators),test/eval-longmemeval.slow.test.ts,test/eval-longmemeval-e2e.slow.test.ts,test/eval-longmemeval-search-config.test.ts,test/longmemeval-extract.test.ts,test/longmemeval-intent.test.ts,test/longmemeval-trajectory-routing.test.ts. -
src/eval/longmemeval/metrics.ts— LongMemEval strict-recall metrics: the raw-id join,recall_all@k/recall_any@k, per-type buckets, the schema-v2by_type_summary, and the per-question row assembler. PURE: no engine, no I/O, no LLM, so the harness and the tests score the same bytes. The join is on RAW session ids:haystackToPageslowercases and hyphenates ids to build slugs, so a slug tail compared againstanswer_session_idsnever matches on the public_ssplit;buildSlugToRawMapinverts the slug construction per question,distinctRetrievedSessionsjoins through it,detectSlugCollisions/collisionsTouchingGoldname the ambiguous slugs (a gold-touching collision makes the harness emit an error row),goldMissingFromHaystackcounts dataset defects, andnormalizeSessionIdexists only for slug construction — never for the gold compare. k semantics:recall_*@kis scored over the DISTINCT sessions among the top-k CHUNK rows returned atlimit: k(the caller slices first;scoreRecalltreats k as a guard); empty gold scores both hits false, not vacuously true, and such rows stay out of the denominator.addRowToBucketfolds a v2 row intototal/all_hit/any_hit; a row carrying only the deprecated any-onlyrecall_hitcounts towardtotal+any_hitand bumpslegacy_rows(itsall_rateis therefore a lower bound);buildByTypeSummaryV2emits sorted type keys,nullrates on empty buckets,mean_distinct_sessions, and the caller'srun_config.buildRowassembles the JSONL row — scored fields overresults.slice(0, k),retrieved[]+retrieved_session_idsover EVERY returned row so replay can re-score at any smaller k — and harness passthroughextrakeys never override the scored fields. Pinned bytest/longmemeval-metrics.test.ts. -
src/eval/longmemeval/run-config.ts— the resolved retrieval pins,retrieval_config_hash, therun_configreceipt block, secret redaction, and the--question-idsloader. Pure given its inputs (the harness resolves the mode bundle and the embedder string and passes them in).RetrievalPins={mode, keyword_only, reranker: {enabled, model}, autocut, expansion, expansion_variant_budget, embedder (model@dims or 'unconfigured'), top_k, trajectory};retrievalConfigHash(pins, {knobs_hash, knobs_hash_version})is sha256 overstableStringify(sorted keys at every level) of the pins PLUS the resolvedknobsHash, so two runs hash equal regardless of flag order, dataset, output path or clock, and a resume file written under a config snapshot that differs in ANY result-shaping knob — not just a pin — is a different run.redactSecrets(URL userinfo, bearer tokens, secret-lookingkey=valuepairs, provider key prefixes,?password=) is applied to every error string BEFORE it lands in a receipt row or the eval ledger.loadQuestionIdsreads one id per line with#comments, dedupes, and throws on a missing or empty file.buildRunConfigstamps the summary'srun_config(pins,expansion_replay, dataset sha256 + count,question_ids_file, both hashes, theCacheReceipt{path, hits, misses, bypassed, infra_faults, canonical_sha256, sha256}orcache: null+cache_skippedreason, and the countersreranker_skipped_rows,vector_degraded_rows,expansion_failed_rows,expansion_replay_miss,gold_missing_from_haystack,slug_collisions,excluded_abstention,errors). -
src/eval/longmemeval/resume.ts— reading a prior run's JSONL back: row scanning, the mixed-run check, expansion-variant replay maps, and per-type bucket re-seeding. INVARIANT: recall is RECOMPUTED, never trusted —seedBucketsFromRowsre-derives both metrics for every prior row fromretrievedIdsAtK(prefersretrieved[]sliced to k; falls back to the olderretrieved_session_idsshape) joined against the dataset's gold, so a file written before the raw-id join simply scores false instead of poisoning the summary, and there is no legacy any-only counter.readJsonlRowsskips corrupt lines (a SIGKILL tail);isScoredQuestionRowexcludes summaries and hypothesis-less error rows (those are retried);checkResumeConfigHashreports rows stamped with a foreignretrieval_config_hash(refused by the harness unless--allow-mixed-run-config) and tolerates unstamped rows;loadExpansionReplaymapsquestion_id→ recordedexpansion_variants. Pure given its inputs (file reads are the only I/O). -
src/eval/longmemeval/judge.ts+judge-lane.ts+qa-accuracy.ts— the LongMemEval LLM-as-judge lane behindgbrain eval longmemeval --judge. judge.ts is a faithful port of the officialevaluate_qa.py::get_anscheck_prompt: the per-type instruction (standard forsingle-session-user/single-session-assistant/multi-session; temporal-reasoning adds the off-by-one-days clause; knowledge-update; the single-session-preference rubric) and the abstention instruction for_absids (the id suffix beats the type; an unknown type falls back to standard), ONE user message per question,DEFAULT_JUDGE_MODELopenai:gpt-4o,JUDGE_TEMPERATURE0,JUDGE_MAX_TOKENS16 (the OpenAI API's minimum; the official 10 is rejected, and a one-token yes/no verdict is unaffected), and the official verdict rule (parseJudgeVerdict:'yes'substring of the lowercased completion). Disclosed deviations, carried verbatim inJUDGE_METHODOLOGY_NOTEon every summary: (1) the question / reference / response sit inside<judge_input>data-boundary framing underJUDGE_DATA_BOUNDARY_INSTRUCTION(#4338) —escapeJudgeDataneutralises tag closures inside the data and the response text is otherwise unaltered, so the judge grades what the reader actually said; (2) thejudge_errorclass —classifyJudgeResponsereturns null for a completion that is neither a yes nor a standalone no (malformed, re-judged) instead of a silentno; (3) abstention detected by the_abssuffix; (4) the unknown-type fallback.JUDGE_PROMPT_VERSIONbumps whenever any instruction text, framing or field label changes.judgeConfigHash= sha256 over the stable JSON of{judge_model, prompt_version, max_tokens, temperature, reader_model, reader_prompt_sha, context: {k, max_tokens}}.judgeRowjudges one row from its stored hypothesis: budget-skips on the PROJECTED cost (estimateJudgeCallUsd) BEFORE spending, records actual cost after, never throws on a transport failure, andstripJudgeFieldsremoves every priorjudge_*key first so a re-judge leaves no stale field. judge-lane.ts is the harness-side orchestration, pure exceptrunJudgeBackfill(whose only effects are the injected client and in-place row updates; no engine, no file I/O):parseMaxUsd(N, oroff/none/unlimited→ no cap),makeJudgeConfigHasher(per-row hasher — a prior row's recorded reader pins win over this run's; k is the run's--top-k, already gated byretrieval_config_hash),hasJudgeAttempt/hasSettledVerdict,selectBackfillRows(candidates = rows with a hypothesis and no settled verdict; countssettled,mismatched+ the foreign hashes,retrievalOnly,missingFromDataset),judgePreflight(availability → pricing → estimate → cap: exit 1 no provider, exit 2 unpriced-with-budget or over-cap without--yes) andrunJudgeBackfill(runWithLimitat--judge-concurrency; the dataset's answer is the reference, the row's ownanswerthe fallback). qa-accuracy.ts builds theqa_accuracyblock from ALL rows (pure; last row perquestion_idwins):accuracy_headline(aliasaccuracy) = correct /total_questionsover EVERY question including_abs— ajudge_error, a budget-skipped row, a reader-error row and a never-judged row all score INCORRECT, so it is never more lenient than the official scorer;accuracy_excluding_errors= correct / judged (secondary);accuracy_470= the headline rule over the non-_absquestions (the retrieval-metric denominator);by_typeand anabstentionsub-block;judge_error_classes;ci95_bootstrap(percentile bootstrap over the headline 0/1 vector,label: 'question-sampling only');complete(no error, no skip, no unjudged row — the publishability bit the run-end gate reads);mixed_judge_config;est_cost_usd,actual_cost_usd(summed over the rows' ownjudge_cost_usdacross resumes),run_cost_usd(this run's ledger);methodology_note. Pinned bytest/longmemeval-judge.test.tsandtest/eval-longmemeval-judge.slow.test.ts. -
src/eval/shared/judge-runner.ts+src/eval/shared/bootstrap.ts— dataset-agnostic judge mechanics shared by the LongMemEval lane and any later judged eval. judge-runner.ts owns the chat-client seam (JudgeChatFn— the gatewaychatin production, a canned fn in tests), bounded retries with exponential backoff (default 2 retries, base 500 ms,sleepseam), the closedjudge_errorvocabulary (timeout | rate_limit | empty | refusal | malformed | provider_error, exported asJUDGE_ERROR_CLASSES), usage summed over every attempt, per-call cost through the ONE canonical pricing table (canonicalLookup; an unpriced model yieldscost_usd: null, never 0, andisJudgeModelPricedlets a budget refuse it), and theBudgetLedger(maxUsd === null= no cap;canAfford(next)is PROJECTED so a run never overshoots by more than the calls already in flight; unpriced calls are counted, not summed;snapshot()for receipts). INVARIANT: a judge malfunction is an error outcome, never averdict: 'incorrect'—runJudgenever throws on a transport failure (only an already-abortedAbortSignalor a bug in the caller'sparsecan), classifies 429 / rate-limit prose →rate_limit, abort / timeout →timeout, everything else →provider_error, and maps arefusal/content_filterstop, an empty completion and a null fromparsetorefusal/empty/malformed; the caller decides how the headline scores them. It passestemperatureandmaxTokensthroughChatOptsand recordsChatResult.responseModelasresponse_model. bootstrap.ts:bootstrapMeanCi(xs, {resamples = 10000, seed = 42, alpha = 0.05})is a seeded (mulberry32) percentile bootstrap of the mean, byte-reproducible; n = 0 → nulls, n = 1 → a degenerate[x, x]. INVARIANT: every block carrieslabel: 'question-sampling only'— the interval quantifies how much the number would move under a different draw of questions and says nothing about reader / judge nondeterminism, dataset revision or prompt drift. Pinned bytest/longmemeval-judge.test.ts. -
src/eval/longmemeval/diagnostics.ts+scripts/lme-miss-diagnostics.ts— LongMemEval miss diagnostics: locate WHERE a strict miss is lost before any fix is chosen. Input is a harness receipt (ndjson) plus the dataset; for every question withrecall_all_hit=false(every scored question under--all) the question's brain is re-created EXACTLY as the harness built it (one in-memory PGLite per run,resetTablesper question,haystackToPages→importFromContent, the embed cache installed through the same seam so every page / question vector is a cache hit) and each gold session missing from the receipt's top-k is located per arm:vector_rank(first chunk row of the session inengine.searchVector, paged by offset todepth, default 200, because one call caps atMAX_SEARCH_LIMIT),keyword_rank(engine.searchKeyword, OR-fallback as hybrid),title_rank(engine.searchTitles),fused_rank_*(the pre-rerank RRF order of ONEhybridSearchcall atfusedLimit, default 50, under the same pins, captured viaonRerankPool(pool, preRerank)),post_rerank_*(that call's post-rerank pool) andfinal_rank_rows(what the call returned after autocut / limit). Frozen classes (classifyMiss): (i) absent from every arm's top-depth, (ii) in an arm pool but outside the fused top-k pre-rerank, (iii) in the fused top-k pre-rerank but reranked out, (iv) ceiling (more gold sessions than k); two observational classes keep the receipt honest —rerun_hit(the re-created run places the gold inside the top-k: the receipt's miss did not reproduce under these pins) andautocut_dropped/post_fusion_dropped(the gold survived fusion + rerank and a later trim removed it). Frozen hypothesis probes:h1Signature(≥ 2 gold, one at fused session rank 1–3, the missing one at 6–15), the counterfactual clause sub-queries (splitClauses/splitClausesDetailedover the frozen pattern listhow_many_between | between | first_or | before_after, with the content-token and quoted-span guardrails; H1 is supported when a clause's own vector top-5 contains the missing gold) and theh3Candidatessplit — H3a candidate generation (gold in the vector top-depth but outside the pre-fusion pool,computeInnerLimit) vs H3b reranker depth (in the fused pool but beyondreranker_top_n_in).splitMembershiptags each question with the committedevals/longmemeval/splits-seed42.jsonsplits (dev40 | decision430 | halfA430 | halfB430 | halfA470 | halfB470). INVARIANT: the clause sub-query embeds BYPASS the embed cache, so a diagnostics run never adds rows to the shared like-for-like cache (its canonical hash is a receipt field). INVARIANT: every helper exceptrunDiagnostics(the one engine-touching entry point;applyPinswrites the pins) is pure over plain data —classifyMiss,splitClauses,h1Signature,h3Candidates, the rank builders,parseReceipt/pinsFromReceipt/receiptTopKSessions,summarizeDiagnostics,renderDiagnosticsMarkdown— so the tests pin them without PGLite, and every printed metric routes through the shared glossary (glossFor, withLOCAL_GLOSSARYfor the diagnostics-only names). The script owns argv, gateway bootstrap, file I/O and printing (<receipt.ndjson> --dataset FILE [--splits FILE] [--mode M] [--reranker on|off] [--autocut on|off] [--expansion-variant-budget legacy|B] [--embed-cache PATH | --no-embed-cache] [--k N] [--depth 200] [--fused-limit 50] [--all] [--question-ids FILE] [--limit N] [--out-ndjson FILE] [--out-md FILE] [--json]); pins default to the receipt's flatrun_configon theby_type_summaryline (mode,reranker.{enabled,model},autocut,expansion,expansion_variant_budget,topK,embedder; a legacy nestedpinsblock is still accepted) and explicit flags win; it is NOT agbrainsubcommand, so its flags sit outside the CLI flag registry by design. Spend: with the receipt's cache every page / question embed is a hit; the ≤ 2 clause sub-query embeds per miss and, under--reranker on, one rerank call per miss are the only paid calls. Exit 0 ok · 1 bad input / run error · 2 gateway or reranker not ready. Pinned bytest/longmemeval-diagnostics.test.ts. -
src/eval/shared/embed-cache.ts— content-addressed embedding cache for fixed-corpus evals (dataset-agnostic; the LongMemEval harness is its first consumer). INVARIANT: a cached vector is served ONLY for the exact(model@dims, text, side)it was computed for, and every row is integrity-checked on read (declared dims == stored bytes / 4 == vector length); a mismatch is a HARDEmbedCacheIntegrityErrornaming the file and the key — never a silent re-embed, because a silently corrupted cache makes every arm's vectors non-comparable. Key =${model}@${dims}(+#queryfor query-side asymmetric embeds, so zembed/Voyage query vectors can never be served a document-side row) +:+ sha256(text). Storage:bun:sqlite, WAL journal +synchronous=NORMAL, busy retry, one transaction per question's embeds; local filesystem only.installEmbedCacheinstalls it through the gateway's__setEmbedTransportForTestsseam (gateway.ts sits at its module-size ceiling; a namedsetEmbedTransport()hook is a filed follow-up) and restores the caller-suppliedrealTransport ?? nullon uninstall. Stats:hits,misses(genuine misses PLUS every value of a batch that fell open on an infrastructure fault),bypassed(SDK model id ≠ the installed model — never cached),infra_faults(file deleted mid-run / table dropped / disk error → uncached re-embed or lost write-back, so a run that lost its cache can never show a cleanmisses: 0). The canonical hash (PRAGMA wal_checkpoint(TRUNCATE)then sha256 over the sorted(key, dims, sha256(vector))rows) goes intorun_config.cache.canonical_sha256so two runs can prove they saw the same vectors. Pinned bytest/longmemeval-embed-cache.test.ts. -
src/eval/shared/autocut-replay.ts+scripts/replay-autocut-floor.ts— pure replay of the autocut weak-top-floor sweep over captured rerank pools, and its CLI. INVARIANT: the replay uses the SAMEapplyAutocutthe live path uses, over the SAME pre-autocut reranked pool (gbrain eval longmemeval --capture-pool→rerank_pool, unscored alias/exact-lookup rows included — dropping them would changedecision.total), with the SAME preserve predicate (alias_hit === true || exact_lookup === true), and slices to k AFTER autocut exactly like hybrid.ts, sovalidateLive(rows, floor)can demand byte-for-byte agreement with the recorded live decision (search_meta.autocut, plusautocut_kept_keyswhen present) before any other floor cell is trusted. Every cell — including flooroff— comes from ONE capture; no second reranker call. Metric semantics mirror the harness (distinctsessions among the first k kept chunk rows;recall_all= gold ⊆ distinct;recall_any= non-empty intersection), with the benefit metricsmean_returned_results/mean_returned_est_tokens(why autocut exists) and recall as the guardrail;pairedDeltagives wins/losses/net per type vs the first floor,splitHalf(rows, seed)gives a seeded half-A/half-B selection/confirmation split,topScoreHistogrampublishes the reranker's top-score distribution.normalizePoolRowthrows on a row withoutslug/session_id(a silently dropped row shifts every cliff). The script owns argv (<capture.ndjson> --floors off,0.10,… [--k 5] [--validate-live F] [--split-half SEED] [--jump 0.2] [--min-keep 1] [--json];--validate-livetakes exactly one floor), file I/O and printing; every metric it prints routes throughsrc/core/eval/metric-glossary.ts(REPLAY_GLOSSARY_KEYS); exit 0 ok · 1 validate-live mismatch or bad input · 2 usage. Pinned bytest/replay-autocut-floor.test.ts. -
scripts/eval-spend-guard.sh— hard spend cap for paid eval runs:scripts/eval-spend-guard.sh <cap_usd> <estimate_usd> -- <command...>. INVARIANT: a paid command never launches when ledger total + estimate would exceed the cap, and every launch is appended to the ledger whether or not it succeeded, so the running total can only under-state spend if the command itself lies about its cost. FAILS CLOSED on anything it cannot account for: a missing ledger (a missing file is NOT a $0 ledger — the first run setsGBRAIN_EVAL_SPEND_LEDGER_INIT=1, which prints a loud NEW LEDGER line), an unparseable ledger line (named by line number), a signed or malformed amount (a negative estimate would drive the ledger backwards). Ledger$GBRAIN_EVAL_SPEND_LEDGER(default~/gbrain-lme-receipts/spend.jsonl): every launch writes TWO rows sharing arun_id— astatus: runningreservation appended BEFORE the command starts (cost = the estimate, so a concurrent guard already counts it) and astatus: donereconciliation after it exits (actual cost, exit code); the sum is reconciled rows + unreconciled reservations, so a guard killed by INT/TERM/HUP (trap-reconciled at the estimate) or SIGKILL (reservation left standing) never under-counts, and an unwritable ledger refuses the launch — the child gets TERM, then KILL afterGBRAIN_EVAL_SPEND_GUARD_KILL_GRACE_SECONDS(default 10). Legacy single-row lines{ts, estimate_usd, cost_usd, exit_code, command}still count. The recorded cost comes from$GBRAIN_EVAL_ACTUAL_COST_FILE(exported to the child as a scratch path when unset; bare unsigned number or JSON withcost_usd) only when positive, else the estimate (over-stating is the safe direction). Amounts normalize to%.6f; awk/grep/sed only — no jq or bun at runtime. Exit: the wrapped command's code · 2 usage · 3 refused. Pinned bytest/eval-spend-guard.test.ts. -
evals/longmemeval/splits-seed42.json+evals/longmemeval/dev-slice-seed42.txt— the committed, seeded question-id splits that carry the LongMemEval decision discipline (ids only, never question text or session content).schema_version: 1;dataset_sha256+scored: 470/abstention_excluded: 30pin the exact cleaned_scorpus the splits were drawn from;prng= mulberry32 over question_ids sorted ascending with Fisher-Yates; seedsdev: 42,halves470: 4242,halves430: 4243. Lists:dev40(the 40-question dev slice, per-type counts indev40_type_counts),decision430(the held-out remainder every pre-registered success rule is decided on),halfA470/halfB470(235 each) andhalfA430/halfB430(215 each) for select-on-A / confirm-on-B decisions;type_countsrecords the per-type totals of the 470.dev-slice-seed42.txtisdev40one id per line forgbrain eval longmemeval --question-ids. Invariant: mechanisms are chosen on the dev slice or half A and decided ondecision430or half B; a full-470 row published alongside is labelled as including the dev slice. -
docs/eval-bench.md— contributor guide for using captured data to benchmark retrieval changes before merging. Linked from CONTRIBUTING.md under "Running real-world eval benchmarks (touching retrieval code)". -
src/core/eval-capture.ts— op-layer capture wrapper called fromsrc/core/operations.tsquery+searchhandlers (catches MCP + CLI + subagent tool-bridge from one site). Fire-and-forget; failures route toengine.logEvalCaptureFailuresogbrain doctorsees drops cross-process. Capture is off by default —isEvalCaptureEnabledresolution: explicitconfig.eval.capture(true/false) wins, elseprocess.env.GBRAIN_CONTRIBUTOR_MODE === '1', else off. Contributors setexport GBRAIN_CONTRIBUTOR_MODE=1. PII scrubber gate is independent and defaults to true regardless of CONTRIBUTOR_MODE. -
src/core/eval-capture-scrub.ts— zero-deps PII scrubber: emails, phones, SSN, Luhn-verified credit cards, JWT-shaped tokens, bearer tokens. -
src/core/search/hybrid.ts—Promise<SearchResult[]>return shape.onMeta?: (m: HybridSearchMeta) => voidcallback so op-layer capture records what hybridSearch actually did; callers that don't need it leave it undefined.HybridSearchOpts.queryEmbedFn?: (text) => Float32Array | Promise<Float32Array>is the hermetic eval seam: when set, the TEXT vector arm's query embedding comes from this function instead of the gateway's query-embed path AND the no-embedding-provider keyword-only short-circuit is bypassed, so deterministic eval canaries (gbrain eval gate --embedder deterministic,scripts/run-eval-canary.ts) run the vector arm with no provider key; never set on production paths — absent, the production path is untouched, and barehybridSearchnever touches the semantic query cache so the seam can't poisonquery_cache.HybridSearchOpts.types?: PageType[](onSearchOpts) threads a multi-type filter into per-enginesearchKeyword+searchVector+searchKeywordChunksasAND p.type = ANY($N::text[])(primary consumergbrain whoknows, filters to['person','company']); AND-applies alongside the single-valuetypefilter.hybridSearchresolves the embedding column at the boundary viaresolveColumn(loadRegistry(cfg), opts.embedding_column, cfg)fromsrc/core/search/embedding-column.ts, threads theResolvedColumndescriptor (not a raw string) into per-enginesearchVector, and usesisCacheSafe(resolved, cfg)for the cache-skip decision so a repointedembeddingbuiltin doesn't leak across vector spaces.cosineReScorecallsengine.getEmbeddingsByChunkIds(ids, resolved.name)so rerank uses vectors from the active column, not the hardcoded OpenAIembedding, and hydrates each result's raw query↔chunkcosineontoSearchResult(the calibrated signal evidence and--explainconsume).SearchOpts.onVectorPoolMetais the engines' out-channel for searchVector's bounded pagination escalation (one dense page filling the inner candidate pool escalates the pool ×4 up to 3 times; HNSW-backed columns additionally cap at theef_searchceiling, while exact-scan columns above the index dim ceiling are bounded by the escalation count alone); hybrid passes the collector and owns the emit, fired when the loop ends with the pre-DISTINCT pool still full — at the substrate cap or after the escalation budget. ThequeryMCP op acceptsembedding_columnfor per-call A/B;search(keyword-only) rejects it. Two post-fusion stages + evidence stamp:applyTitleBoost(results, query, titleBoost, floorThreshold)multiplies a result's score by the resolvedtitle_boostwhenisTitlePhraseMatchfires, stampstitle_match_boost, inherits the floor-ratio gate so a title match can't shove a much-stronger page below it;applyAliasHop(engine, results, query, opts)normalizes the query, callsengine.resolveAliases, and on exact normalized-alias match surfaces that page at top-of-organic + epsilon withalias_hit=true;stampEvidence(...)runs LAST (after the alias hop, before slice) on every path — keyword-only, no-embed, and full hybrid — so MCP callers and--explainread the sameevidence+create_safetycontract.title_boostresolved from the mode bundle and threaded in.runPostFusionStageshas a 4th stage (graphSignalsEnabled,onGraphMeta,onScoreDistribution).base_scorestamped at function entry idempotently (captured ONCE before any boost stage mutatesscore). Each post-fusion stage stamps its multiplier:applyBacklinkBoost→backlink_boost,applySalienceBoost→salience_boost,applyRecencyBoost→recency_boost.applyReranker(earlier in the pipeline) stampsreranker_deltaas a rank delta (positive = improved).applyExactMatchBoostinsrc/core/search/intent-weights.tsstampsexact_match_boostwhen fired. Per-stage attribution powersgbrain search --explain— every boost surface carries its own field soformatResultsExplainreads them all without coupling to internal stage ordering. withsrc/core/search/sql-ranking.ts+src/core/operations.ts+src/core/types.ts: agent-warning channel.SearchResult.content_flag?: {reason, detail}(optional field intypes.ts) is stamped post-fusion bystampContentFlags(thestampEvidenceprecedent) inhybridSearchAND in the keyword-onlysearchMCP op so both retrieval paths surface the marker.get_pagereturns a top-levelcontent_flagparallel field viagetContentFlag(page.frontmatter).buildVisibilityClause(sql-ranking.ts) ANDs inQUARANTINE_FILTER_FRAGMENTso quarantined pages are excluded from all six search call sites (alongside soft-delete + archived-source filters). Pinned bytest/sql-ranking.test.ts+test/e2e/quarantine-search-exclusion.test.ts. Cross-modal routing at the embed step:effectiveModalityresolves per-callopts.crossModal(literal'auto'→ undefined) →suggestions.suggestedModality→'text'. Image route:embedQueryMultimodal+searchVector({embeddingColumn: 'embedding_image'}), skipping expansion + keyword.'both'route: parallel text + image vector searches merged viarrfFusionWeightedwitheffectiveRrfK(baseRrfK, weight)from the configured cross-modal weights. Unified routing fires whensearch.unified_multimodalis true — bypasses dual-column branching, runsembedQueryMultimodal+searchVector({embeddingColumn: 'embedding_multimodal'}), fail-open on zero rows (non-strict falls through to dual-column). LLM modality escalation fires only when no explicit per-call opt is set AND the regex returned'text'ANDsearch.cross_modal.llm_intentis on ANDisAmbiguousModalityQueryfires; fail-open on every error.compiledTruthBoost(result, applyBoost)is exported for direct predicate tests: a synthetic chunkless title row (chunk_id === 0AND blankchunk_text) never receives the 2x compiled-truth boost (test/search/compiled-truth-boost.test.ts). The reranker pass-through callback is typed withRerankPassThroughReasonimported fromrerank.ts— one union, not a re-declaration. RRF inputs are assembled by ONEcomposeFusionListscall (next entry) over role-taggedvectorArmsbuilt withpushVectorListat every assembly site (unified, image-only, text/both, and theallSettledsalvage path — where theoriginalrole additionally requires the original'ssearchVectorto have succeeded); afterexpandFn,queries[0]is re-enforced to be the caller's query and repeats are deduped, so a variant can never take the anchor role.HybridSearchOpts.onRerankPool(eval-only, best-effort) fires immediately beforeapplyAutocutwith the exact pre-autocutreturnPool— post alias-hop / exact-lookup / adaptive-return, unscored injected rows included, even when autocut is off — plus the deduped pre-rerank order, so an offline autocut replay is byte-faithful. Immediately afterapplyReranker— only when it actually reordered (reranked !== deduped) and never for image modality —pinRelationalRows(relational-rerank-pin.ts) re-pins the relational arm's rows above the reranked text rows bounded byresolvedMode.relational_rerank_pin, stampsrelational_pinned, and emitsHybridSearchMeta.relational_rerank_pin;ensureRelationalEvidenceSlotstill runs after autocut (usually a no-op on relational queries now — it remains the page-1 guarantee for pin 0, reranker fail-open, and arm rows dropped before the pin). -
src/core/search/fusion-lists.ts— the ONE composition point forhybridSearch's RRF inputs. Every vector recall list is a ROLE-tagged arm (VectorArm {list, role: 'original'|'variant'|'clause'|'image'}, appended only viapushVectorList) — never a parallel index-aligned array: thePromise.allSettledsalvage path drops failed arms and the both-mode image branch fails open, so any positional rule ("last list is the image", "index 0 is the original") mis-tags lists under partial failure.composeFusionLists({arms, keywordFusionList, titleFusionList, relationalList, includeRelational, ks, knobs})returns the complete weighted list set (FusionListEntry {list, k, weight?}) in the fixed order vector arms → keyword (keywordK) → title (keywordK, only if non-empty) → relational (baseRrfK, only if non-empty and not image modality): text arms fuse attextRrfKand the image arm atimageRrfKonly when BOTH kinds are present, otherwise every arm atvectorK(text, image-only, unified, and a both-mode whose image branch fell open).variant/clausearms shareexpansionVariantBudgetas total RRF weight —weight_i = b / n_voting_armsover the NON-EMPTY expansion arms (an empty list casts no vote), theoriginalarm always weight 1; anullbudget emits noweightkey, byte-identical to unweighted fusion; when the original is missing (its embed orsearchVectorfailed) every surviving text arm is avariantsharing the budget.rrfFusionWeighted(hybrid.ts) scoresweight / (k + rank)— the literature weighted-RRF form, a list-level multiplier that holds at every rank (a k-penalty would fade at deep ranks).textArmsNonEmptyis the role-based text-arm health gate behind the OR-relaxed keyword demotion (image arms never count, so a fell-open image branch can't veto the lexical rescue).normalizeExpansionVariantBudgetis the single range contract for the knob: finite(0, 4](or a string parsing to one) → number,null/legacy/null-literal → null, anything else →undefined(fall through to config → bundle); shared bymode.ts:loadOverridesFromConfigand both per-call seams inhybrid.ts, so an invalid per-call value never reaches fusion or the cache key. Pure, no engine/IO. Pinned bytest/search/fusion-lists.test.ts(knife-edge arithmetic at budget 1.0, legacy deep-equals the pre-role text mapping, role alignment under rejected arms),test/search/expansion-variant-budget.test.ts(hermetic directional PGLite test), and the both-mode demotion-gate regression intest/keyword-relaxed-fusion.serial.test.ts. -
src/core/search/relational-rerank-pin.ts— relational-arm rows bypass reranker DEMOTION (ranker wave, receipt R1 inscripts/r1-namedthing-rerank-ab.ts: on NamedThingBench's 39 graph-relationship questions the shippedbalanceddefault lost hit@1 21/39 → 3/39 and hit@3 27/39 → 5/39 with the reranker on — 19 and 22 paired losses — because the cross-encoder scores chunk TEXT and an edge-derived answer's text need not mention the query's entity; with the pin at its default 3, measured with--autocut on, the shape that shipped before rule R2 turned autocut off, the same paired run shows 0 hit@1 / 0 hit@3 losses, 21/39 and 27/39, and the 11 non-relational core queries show 0 losses — one hard-negative gain, hit@1 10/11 → 11/11).pinRelationalRows(reranked, relationalList, {max, fusedOrder?, onPin?})is a pure PERMUTATION of the post-rerank pool: relational rows (page key(source_id, slug)in the arm's list; ONE row per page — its first, highest-ranked occurrence) claimmin(fused_rank, reranked_rank)(fused rank = position among relational rows infusedOrder, the pre-rerankdedupedpool; arm order for rows absent from it), claims sort ascending with ties resolved to the FUSED order then the reranked position, the firstmaxclaimants form the top block in that order, every other row follows in reranked order (an unpinned row lands at its reranked position or up topinned.lengthlower, never higher). Pinned rows are shallow copies stampedrelational_pinned: true; no row is added or removed (re-injecting rows dropped upstream staysensureRelationalEvidenceSlot's job). Every no-op path (max <= 0, empty pool, empty arm, no relational page in the pool) returns the input array itself.normalizeRelationalRerankPinis the ONE range contract for the knob (non-negative integer<= 10or a string parsing to one → that integer; the literalsoff/false(any case) or booleanfalse→ 0; anything else →undefined= fall through), shared bymode.ts:loadOverridesFromConfigand both per-call seams inhybrid.ts;DEFAULT_RELATIONAL_RERANK_PIN(3) andRELATIONAL_RERANK_PIN_MAX(10) are exported.RelationalRerankPinDecision({max, relational_in_pool, pinned:[{slug, source_id, from_rank, to_rank, fused_rank}], moved}) is surfaced asHybridSearchMeta.relational_rerank_pinfor--explain. Wired inhybridSearchimmediately afterapplyRerankerand before the alias hop, gated on the reranker having actually reordered (reranked !== deduped—applyRerankerreturns its input on every fail-open / skip / pass-through path, and the fused order already carries the arm) and on non-image modality; autocut'sscoreOfignores pinned rows and its preserve predicate keeps them, so text-row autocut is byte-identical to the pre-pin behavior. Known cost: a false-positive arm (relational shape parsed AND a real seed resolved, but wrong/stale edges) now puts up tomaxedge pages at ranks 1..max instead of one atlimit; mitigations are the arm'sfallback_slugifyconfidence gate, the tier-2 resolution-margin TODO, configsearch.relational_rerank_pin off, and per-callSearchOpts.relationalRerankPin. Pinned bytest/search/relational-rerank-pin.test.ts(pure contract + tie policy) andtest/search/relational-rerank-pin-hybrid.serial.test.ts(hermetic PGLite on the relational corpus through the REAL gateway rerank path behind__setRerankTransportForTestswith an inverted-relevance stub: pin 0 reproduces the regression, pin 3 restores hit@1/hit@3 on all 8 who-invested questions, non-relational queries and reranker-off runs are byte-identical). -
src/core/search/metadata-boost-gate.ts— the ONE parse + decision contract forsearch.metadata_boost_gate(always|lexical; every bundle islexical,DEFAULT_METADATA_BOOST_GATEstaysalwaysso a knobs literal without the field keeps its pre-wave hash identity).lexicalArmsVotedanswers "did a strict keyword, title-phrase or relational row fuse?" from the composed fusion lists (relaxed OR-fallback rows do not count — they are already demoted);decideMetadataBoosts({gate, lexicalVoted})returns{gate, lexical_voted, boosts_applied, reason: 'gate_always'|'lexical_voted'|'vector_only_voter'|'image_modality'}. Image-modality queries are exempt from the gate:hybridSearchnever runs the keyword / title arms for an image query and excludes the relational arm, so their lexical arms never vote by construction andlexicalwould silently disable the boosts for the whole modality — the caller passesmodality(fromeffectiveModality) and an image decision applies the boosts as before (reasonimage_modality). Underlexical,hybridSearchpassesskipMetadataBoostsintorunPostFusionStagesso the backlink, salience, recency (+chronicle), graph-signal and alias-resolved boosts are skipped when the vector arm was the only voter; supersede downrank, exact-match boost, title-phrase boost, compiled-truth boost, cosine re-score, dedup, reranker and autocut are untouched either way. Receipt (Cat 13 conceptual recall, sibling repo): the E1 localization showed hub pages carrying 1.03–1.12x backlink / graph-adjacency / recency boosts over gold concept pages that carried none whenever both lexical arms were empty; the pre-registered E3 held-out arm moved gbrain from 53.0 to 57.8 nDCG@5 (rule ≥ 57.0) with NamedThingBench, BrainBench, the retrieval canary and the LongMemEval dev slice byte-identical, so the gate flipped tolexicalin all three bundles. The decision is surfaced asHybridSearchMeta.metadata_boost_gate(--explain); knobs-hash partmbg=; override chain per-callmetadataBoostGate→ config → bundle (normalizeMetadataBoostGateis the shared parser, garbage falls through). Pinned bytest/search/metadata-boost-gate.test.ts(pure contract, hash participation, resolution chain) andtest/search/metadata-boost-gate-hybrid.test.ts(hermetic PGLite hub-vs-gold corpus:alwaysreproduces the hub-first regression, the default yields gold-first, config and per-call planes, unparseable falls through). -
src/core/search/arm-confidence.ts— arm-confidence-weighted fusion for the keyword arm (search.keyword_arm_confidence_floor;null= off in every bundle — the pre-registered Cat 13 E2 mechanism FAILED its held-out rule at the calibrated floor 0.6121 (53.0 → 53.0), so the knob ships default off for operators and the receipt is published).keywordArmConfidence(rows)reads the strict keyword arm's rawscorecolumn into a scale-free statistic (top,second,margin_ratio; TypeScript-side — engines still return rawts_rank × sourceFactor, no SQL change);decideKeywordArmWeight({keywordList, floor, vectorArmVoted, relationalQuery})down-weights the keyword AND title fusion entries toKEYWORD_ARM_WEAK_WEIGHT(0.5) only when the floor is set, the keyword list is non-empty, a text vector arm voted, the query is not relational, andmargin_ratio < floor; otherwise it emits no weight (byte-identical fusion). It is called fromcomposeFusionLists(fusion-lists.ts) so the weight lands through the same per-listweightprimitive as the expansion budget, never a k change.normalizeKeywordArmConfidenceFlooris the ONE range contract (number in(0, 1]or a string parsing to one;null/false/off→null; anything else →undefined= fall through), shared bymode.ts:loadOverridesFromConfigand both per-call seams inhybrid.ts. The decision ({top, second, margin_ratio, downweighted}) is surfaced asHybridSearchMeta.keyword_arm_confidenceeven with the floor off, so a calibration run can pick a floor from per-probe receipts; knobs-hash partkacf=. Pinned bytest/search/arm-confidence.test.ts(pure contract) andtest/search/arm-confidence-hybrid.test.ts(hermetic PGLite: a weak keyword arm is down-weighted only with the floor set; off is byte-identical). -
docs/eval-capture.md— stable NDJSON schema reference for gbrain-evals consumers. -
test/public-exports.test.ts— runtime contract test (R2). Imports each of the 17 public subpaths via package name and pins a canary symbol per module. Paired withscripts/check-exports-count.sh. -
src/core/embedding.ts— OpenAI text-embedding-3-large, batch, retry, backoff.BATCH_SIZE=100(per-recipe pre-split + recursive halving + adaptive shrink-on-miss live in the gateway; the outer paginator is for progress-callback granularity, not batch protection).estimateEmbeddingCostUsd(tokens)prices against the currently-configured model's rate viacurrentEmbeddingPricePerMTok()(resolves the per-1M-token rate vialookupEmbeddingPrice(gatewayGetModel())fromembedding-pricing.ts, falling back to the OpenAI 3-large rate 0.13 only when the gateway is unconfigured or the model is unknown to the pricing table).EMBEDDING_COST_PER_1K_TOKENSretained for back-compat with direct importers/tests.currentEmbeddingSignature(): stringreturns the embedding-provenance signature<provider:model>:<dims>(e.g.openai:text-embedding-3-large:1536) stamped ontopages.embedding_signatureat every embed-write site; DELIBERATELY excludes the chunker version (tracked separately viapages.chunker_version) — this signature is strictly the EMBEDDING space, so a model OR dimension swap makes the stored signature differ from current and a page becomes stale. Same unconfigured-gateway fallback as the cost helpers. (Seesrc/core/sync-delta.ts+src/core/spend-posture.tsfor the cost-gate supporting modules.)willEmbedSynchronously({v2Enabled, serialFlag, noEmbed}): SyncEmbedModeretains the package-exported compatibility contract for downstream TypeScript and JavaScript consumers. The command-internalresolveWorkerBackedSyncEmbedMode({deferEligible, noEmbed})uses the engine-aware admission result, so PGLite/unknown runtimes stay inline while worker-backed Postgres preserves v2 deferral even with one runnable source; source count separately controls fan-out.shouldBlockSync(costUsd, floorUsd, mode, posture='gated'): booleanis the pure cost-gate decision: blocks ONLY whenmode === 'inline' && costUsd > floorUsd— deferred mode never blocks (the backfill's $X/source/24h cap is the real money gate), andposture === 'tokenmax'never blocks (the operator declared cost isn't the constraint; anoff/unlimitedfloor isInfinityand so is never exceeded). Pinned bytest/sync-cost-preview.test.ts+test/embedding-signature-stale.test.ts. -
src/core/embed-retry.ts— embed retry/backoff primitives + the title-tier restamp helper, kept in core becausesrc/core/import-file.tsandsrc/core/embed-stale.tsconsume these (a core→commands import would make every import-file consumer eagerly load the embed COMMAND module — one value import away from a real ESM cycle).embedBatchWithBackoffwrapsembedBatchwith rate-limit-aware retry: detects 429s via the wrapped error'scause.status(message-match fallback), also retries transient gateway 502/503/504, parses provider retry-delay hints, jitters ±30% so concurrent workers don't resynchronize, passesmaxRetries: 0through so the AI SDK's own retry stack doesn't multiply attempts, and threads an external AbortSignal into both the sleep and the in-flight HTTP call.restampIfDemotedToTitleTierrestamps aper_chunk_synopsispage's CR state to'title'after a plain re-embed socontextual_retrieval_modekeeps describing the vectors actually in the column (the reindex sweep restores the synopsis tier later).src/commands/embed.tsre-exports everything here (façade rule) so import sites and tests never chase the peel. -
src/core/embedding-input-limit.ts— per-model embedding INPUT token caps. Some hosted encoders reject any single input over a hard per-input limit far below their batch budget (a 512-token model would otherwise leave ~35% of a typical vault's chunks permanently unembeddable, re-failing every sweep).resolveMaxChunkTokens(env?)resolves the effective chunk-token cap for the ACTIVE embedding model so the chunkers SPLIT (never truncate) at a size the model accepts:GBRAIN_MAX_CHUNK_TOKENSenv (escape hatch for models not in a recipe; warn-once per distinct invalid value) → recipe per-modelmax_input_tokens×EMBED_INPUT_SAFETY(0.6 — covers the cl100k-basedestimateEmbedTokensoverestimate vs BERT-wordpiece tokenizer mismatch) →DEFAULT_MAX_CHUNK_TOKENS(2000 for every other provider).MIN_CHUNK_TOKENSfloor of 64 so a tiny declared limit can't produce confetti chunks; fail-open on any resolver error (a resolver bug must never change chunking for unaffected installs).maxInputTokensForModel(recipe, modelId)mirrorsmodel_dims' case-fold lookup rule (exact match first, then case-insensitive scan). Consumed atimport-file.ts's chunk step. Pinned bytest/embedding-input-limit.test.ts. -
src/core/embed-oversize-heal.ts— heals ALREADY-STORED chunks that exceed the active embedding model's per-input token cap (new imports respectresolveMaxChunkTokens(), but--stalere-embeds existingchunk_textrows chunked under looser caps, which fail every sweep forever).healOversizedChunkssplits oversized rows in place (never truncates; char-based hard split as the pathological fallback), re-indexes contiguously, carries each piece's modality/code-symbol metadata, and leaves embeddings unset soupsertChunkspreserves vectors for unchanged(chunk_index, chunk_text)pairs and NULLs the split/shifted ones;healOversizedPageChunksis the load-split-upsert-reload wrapper.healedChunksToStaleRows(chunks, slug, sourceId)remaps a post-heal chunk list back into the stale-row shape — the shared seam betweensrc/commands/embed.ts:embedAllStaleandsrc/core/embed-stale.ts:embedStaleForSourceso the two drains cannot drift;chunk_sourcepasses through UNCHANGED (coercingfenced_codetocompiled_truthwould makewrapChunkTextsForStoredModeprefix code chunks, violating the D20-T4 never-wrap convention). Pinned bytest/embed-oversize-heal.test.ts. -
src/core/stall-env.ts— shared env resolver for the stall-watchdog knobs (resolveStallAbortSecondsFromEnv(envVar, defaultSec)): one parameterized implementation behind BOTHGBRAIN_SYNC_STALL_ABORT_SECONDSandGBRAIN_EMBED_STALL_ABORT_SECONDSso the two surfaces' semantics cannot drift — unset/empty/garbage → the caller's default; any finite number returned as-is (<= 0disables the watchdog). Deliberately a leaf module (neither the sync nor the embed cluster imports the other's machinery just to parse an env var); env-only incident knobs by design, no config-dashboard surface. -
src/core/embed-stall.ts— progress-keyed stall watchdog for embed drains (mirror of sync's stall watchdog; bounds embed drains that hang without a pinned root cause). Two clocks, one trigger: LIVENESS ticks vianoteEmbedApiResponse()on EVERY settled embed API attempt — success, error, or retry (hooked inembed-retry.ts:embedBatchWithBackoff; diagnostic only, distinguishes "fully wedged" from "live but failing" in the abort message, scoped to the armed run); the TRIGGER is no SUCCESSFUL forward progress (readProgress(), wired to chunks embedded) forGBRAIN_EMBED_STALL_ABORT_SECONDS(env-only knob; default 900 viaDEFAULT_EMBED_STALL_ABORT_SEC,<= 0disables, garbage falls back to default) — a retry storm that never lands a chunk trips it BY DESIGN. A stall produces an ERROR RESULT (reason: 'stall_timeout'), never aprocess.exit: only the CLI wrapper (src/commands/embed.ts:runEmbed) maps it to a non-zero exit, and minion handlers convert it to a failed job viaassertEmbedNotStalled. The abort path runs an idempotent, deadline-bounded cleanup (EMBED_STALL_CLEANUP_DEADLINE_MS) that releases only the run's self-acquired single-flight locks and banks partial progress, so the next run resumes cleanly. The check interval is deliberately NOT unref'd — for non-single-flight callers it can be the only handle keeping a lost-promise hang loud;stop()clears it on every normal path. Pinned bytest/embed-stall.test.ts. -
src/core/sync-delta.ts— the single "what changed since last_commit" helper, consumed by BOTHperformSyncInner(sync executor) andestimateInlineNewTokens(cost estimator) so the gate's dollar figure can't drift from what the sync imports.computeSyncDelta(repoPath, fromCommit, toCommit, {detachedManifest?, detached?})→{status:'ok', manifest}|{status:'unavailable', reason:'anchor_missing'|'diff_failed'}. Anchor reachability viagit cat-file -t(a gc'd bookmark isanchor_missing, but a present-but-non-ancestor bookmark is still diffed tree-to-tree), thengit diff --name-status -M from..toparsed bybuildSyncManifest; merges the detached working-tree manifest when detached (buildDetachedWorkingTreeManifest). NO dirty/untracked probe in the estimator — pricing dirty files would create phantom costs on a busy brain. The executor imports only the commit diff on attached HEADs BY DEFAULT, but counts uncommitted working-tree files (untracked + dirty) asSyncResult.uncommitteddrift and, with--working-tree/ configsync.include_working_tree, merges the working-tree manifest into the delta so attached repos can import uncommitted state (the estimator does not price that opt-in).execFileSyncarray-args (shell-injection safe), 30s / 100 MiB budget. Test seam_setGitRunnerForTests. Pinned bytest/sync-delta.test.ts. -
src/core/spend-posture.ts— spend-control surface.resolveSpendPosture(engine): 'gated'|'tokenmax'(DB-planespend.posture, fail-opengated);tokenmaxmakes every cost gate informational across sync/reindex/enrich/onboard (spend still ledgered — removes the ceiling, not the accounting).parseUsdLimit(raw, def, {allowZero?})acceptsoff/unlimited/none→Infinity;formatUsdLimit(n)rendersInfinityas the string'unlimited'(never raw —JSON.stringify(Infinity)isnull);usdLimitToCap(n)mapsInfinity→undefinedat the BudgetTracker boundary so ledger rows never serialize null.normalizeSpendPosture/isValidSpendPostureback theconfig setvalidation. Doc:docs/operations/spend-controls.md. Pinned bytest/sync-cost-preview.test.ts+test/spend-off-switch.test.ts. -
src/core/ai/dims.ts— per-providerproviderOptionsresolver for embed-time dimension passthrough; the single source of truth for "which provider needs which knob to producevector(N)". ExportsdimsProviderOptions(implementation, modelId, dims)(called byembed()ingateway.ts),VOYAGE_OUTPUT_DIMENSION_MODELS(private const — the 7 hosted Voyage models that acceptoutput_dimension:voyage-4-large,voyage-4,voyage-4-lite,voyage-3-large,voyage-3.5,voyage-3.5-lite,voyage-code-3— nano deliberately excluded),VOYAGE_VALID_OUTPUT_DIMS = [256, 512, 1024, 2048] as const,supportsVoyageOutputDimension(modelId),isValidVoyageOutputDim(dims). Voyage path uses the SDK-supporteddimensionsfield ({ openaiCompatible: { dimensions: N } }), NOT Voyage'soutput_dimensionwire-key — thevoyageCompatFetchshim ingateway.ts:541translatesdimensions → output_dimensionbefore the HTTP body is built (the AI SDK's openai-compatible adapter doesn't recognize the wire-key, so sending it from here would be silently dropped and Voyage would return its default 1024-dim). Runtime guard: when a Voyage flexible-dim model is configured withdimsoutsideVOYAGE_VALID_OUTPUT_DIMS, throwsAIConfigErrorwith a paste-readygbrain config set embedding_dimensions <256|512|1024|2048>hint at the embed boundary (most common trigger:embedding_model: voyage:voyage-4-largewithoutembedding_dimensions, falling back toDEFAULT_EMBEDDING_DIMENSIONS=1536, an OpenAI default not a Voyage one). Every lookup in this module folds through the privatemodelMatchKey(trim + lowercase) so cased hub-form ids (Qwen/Qwen3-Embedding-4B) match the all-lowercase tables — the folded key is a MATCH key only, never sent to a provider (wire model ids are case-sensitive; error messages keep the original id so they stay paste-ready); the Qwen3-Embedding native-width table lives at module scope alongside the other dim tables. Consequence, deliberate: a cased Voyage/ZeroEntropy/Perplexity config matches and fails loudly at init when itsdimsis invalid, rather than skipping validation and silently producing wrong-width vectors. -
src/core/ai/types.ts— provider/recipe types.EmbeddingTouchpointhas optionalchars_per_token(default 4, matching OpenAI tiktoken on English) andsafety_factor(default 0.8, budget-utilization ceiling), both consulted only whenmax_batch_tokensis also set; Voyage declareschars_per_token=1+safety_factor=0.5to handle dense payloads (CJK/JSON/base64). Pre-split budget =max_batch_tokens × safety_factor / chars_per_token.EmbeddingTouchpoint.multimodal_models?: string[]model-level allow-list for recipes mixing text-only + multimodal models under one touchpoint (Voyage's 12 models sharesupports_multimodal: truebut onlyvoyage-multimodal-3accepts/multimodalembeddings); when omitted, recipe-levelsupports_multimodalis sufficient.AIGatewayConfig.embedding_multimodal_model?: stringletsembedMultimodal()route to a different model thanembedding_model(OpenAI text + Voyage images without flipping the primary pipeline).AIGatewayConfig.embedding_image_ocr_model?: stringis its OCR sibling:generateOcrText()routes to it instead of the expansion model; a directprovider:modelstring, nevermodels.tier-resolved.EmbeddingTouchpoint.trust_custom_dims?: true— passthrough tier for a user-declared--embedding-dimensionson local / bring-your-own-backend recipes (ollama, llama-server, litellm) where the model catalog can't be enumerated; consumed byisCustomDimValidForProviderinsrc/core/embedding-dim-check.tsAFTER Tier 1 (recipedims_options) and Tier 2 (provider Matryoshka allowlists), so a recipe that declares fixed options (openrouter) is still governed by those and fixed-dim hosted providers (openai/voyage/zeroentropy) stay fail-closed; the provider's/embeddingsresponse-dim validation catches a genuine mismatch pre-storage.Recipe.default_headers?: Record<string, string>(static) andRecipe.resolveDefaultHeaders?(env)(env-templated) seam for per-recipe headers riding alongside auth on every openai-compat touchpoint; mutually exclusive (declaring both throwsAIConfigErrorat gateway-configure time); keys conflicting with the resolved auth header (Authorization, the resolver's custom header) rejected atapplyResolveAuthcall time so defaults can't shadow auth. Used by OpenRouter for theHTTP-Referer+X-OpenRouter-Title+X-Titleattribution triple. -
src/core/ai/defaults.ts— leaf module holding the embedding/reranker default constants (no gateway import, so schema + registry helpers can read them without loading provider SDKs). Split-default:NEW_INSTALL_DEFAULT_EMBEDDING_MODEL(voyage:voyage-4) +NEW_INSTALL_DEFAULT_EMBEDDING_DIMENSIONS(1024) feed every new-install surface — init auto-pick canonical tiebreak, the interactive picker default, the no-keys hint, keyless fresh-install schema sizing (passed as an explicit init param), and all recommendation copy (playbook, banners, doctor fix-hints, advisor).DEFAULT_RERANKER_MODEL(=NEW_INSTALL_DEFAULT_RERANKER_MODEL=voyage:rerank-2.5) is the ONE runtime/mode-bundle reranker default: the threeMODE_BUNDLES.*.reranker_modelvalues and the gateway'srerank()fallback import it, so a brain with nosearch.reranker.modelrow reranks with Voyage onVOYAGE_API_KEY.LEGACY_DEFAULT_RERANKER_MODEL(zeroentropyai:zerank-2) is not a default anywhere; it names theRERANKER_SUNSETSrow (which STAYS while the recipe exists, so an explicitzeroentropyai:*config still short-circuits past the date instead of hanging 5s per query), the short-circuit tests and the migration copy. Init (init.ts:writeNewInstallRerankerDefault, shared by the PGLite + Postgres paths) writes NO reranker row when the default is ready (reranker-readiness.tsagainst the file-plane + process env — an explicit row equal to the bundle value would only earn doctor'ssearch_modereset nag), writes explicitsearch.reranker.enabled falsefor keyed installs WITHOUT a Voyage key (no key for the default; silence beats ano_keyaudit row per process), writes nothing for keyless installs (the documented keyless-recovery re-init must find virgin reranker config; keyless brains take the no-embedding search path, which never reachesapplyReranker), treats a ZeroEntropy embedding pick as any other keyed non-Voyage install (evaluated against env > file > DB-plane keys vialoadConfigWithEngine, so a--forcere-init sees a config-table key), and never clobbers an existing explicit choice.DEFAULT_EMBEDDING_MODEL(zeroentropyai:zembed-1) +DEFAULT_EMBEDDING_DIMENSIONS(1280) serve ONLY as the configless runtime fallback for brains with noembedding_modelin file config — their stored vectors live in ZE's 1280d space, so flipping the fallback under them would break retrieval before the provider dies; the fallback is slated for removal after the sunset, at which point unmigrated configless brains hard-error with the migrate command.ZEROENTROPY_SUNSET_DATE('2026-09-04') is the single source of truth for the upgrade banners, theprovider_sunsetdoctor check, and the ZE recipe'ssunsetmetadata. -
src/core/ai/reranker-readiness.ts— pure leaf answering "is the reranker actually going to run?" forgbrain search modes(buildModesReport.reranker_readiness), doctor'sreranker_health, and init's reranker-default write.rerankerReadiness(model, env, { now?, baseUrlOverrides? })→{model, provider, modelId (alias-canonical), recipeKnown, hasTouchpoint, modelListed, requiredKey (the first MISSING key, else the first required key — this is howReranker: … VOYAGE_API_KEY presentrenders; null only for keyless recipes), keyPresent, sunset, sunsetPassed, selfHosted, sunsetBlocks, ready}; the env snapshot comes from the CALLER — the leaf never readsprocess.envand never imports the gateway (init runs beforeconfigureGatewayand passesmergedProviderEnv(cfg, process.env)wherecfgisloadConfigFileOnly()merged with the brain's DB-plane provider keys vialoadConfigWithEngine— file plane alone if that read fails — so a--forcere-init sees a Voyage key that lives only in the config table). A recipe with a customresolveAuthneeds no env key (mirrors the gateway); aprovider_base_urlsoverride marks the provider self-hosted so a passed sunset does not block.describeRerankerFix(r)renders the paste-ready one-liner (sunset switch > unknown-model > missing key + disable command).src/core/ai/reranker-readiness-engine.tsis the engine-plane wrapper doctor andgbrain search modesshare:rerankerReadinessForEngine(engine, model, { now? })reads the LIVE gateway snapshot (env + base_urls — exactly whatrerank()consults, so the verdict cannot disagree with search and a mounted brain's DB plane cannot steer it) and falls back to env > file > DB-plane provider keys +provider_base_urlsvialoadConfigWithEngineonly when no gateway is configured;redactReadinessForRemote(modes-report.ts) stripsrequired_key/key_present/fixfrom thesearch_modesop for untrusted callers.test/ai/reranker-readiness.test.tspins agreement withisAvailable('reranker', m)on an env × model matrix (recipeKnown && hasTouchpoint && keyPresent) so the two predicates cannot drift. -
src/core/ai/gateway.ts— unified seam for every AI call.embedQuery(text, opts?)andisAvailable(touchpoint, modelOverride?)accept a model override so the resolved-column path embeds via the column's provider (Voyage / ZeroEntropy / OpenAI) instead of the global default; the hybrid path passes{embeddingModel: resolved.provider, dimensions: resolved.dimensions}and the gateway resolves the matching recipe viainstantiateEmbedding().isAvailable('embedding', 'voyage:voyage-3-large')checks the override's recipe (not the default) so hybrid skips vector search only when the active column's provider is actually down.zeroEntropyCompatFetchshim (sibling tovoyageCompatFetch) handles ZE's non-OpenAI-compatible wire shape — rewrites the request URL/embeddings → /models/embed, injectsinput_type(default'document'; the threaded'query'|'document'crosses the SDK boundary via the module-level__embedInputTypeStoreAsyncLocalStorage populated inembedSubBatch(), because the AI SDK's openai-compatible adapter stripsinput_typefromproviderOptionsbefore building the wire body;voyageCompatFetchinjects it opt-in the same way, andopenAICompatAsymmetricFetchis the fallthrough shim for every other openai-compat recipe — llama-server/litellm/ollama — a strict pass-through when nothing was threaded) and explicitencoding_format: 'float', and rewrites the response{results: [{embedding}], usage: {total_bytes, total_tokens}}→{data: [{embedding, index}], usage: {prompt_tokens, total_tokens}}so the SDK's openai-compatible Zod schema validates. Layer 1 (Content-Length) + Layer 2 (per-embedding) OOM caps via taggedZeroEntropyResponseTooLargeError(kept separate fromVoyageResponseTooLargeErrorbecausetest/voyage-response-cap.test.tsdoes structural source-text greps pinning the Voyage name). Wired ininstantiateEmbedding()via therecipe.id === 'zeroentropyai'branch.gateway.rerank()native HTTP path (no AI-SDK reranking abstraction): resolves the EFFECTIVE reranker asinput.model ?? getRerankerModel() ?? DEFAULT_RERANKER_MODEL(imported fromai/defaults.ts;voyage:rerank-2.5), posts to the recipe's reranker path (touchpoints.reranker.path, default/models/rerank; Voyage declares/rerank) with bearer auth — the request's top-N key is recipe-pluggable viatouchpoints.reranker.top_param(defaulttop_n; Voyagetop_k) — and returnsRerankResult[]sorted by relevance.warnSunsetOnce(recipe, touchpoint): once-per-(recipe,touchpoint) stderr DEPRECATED warning for recipes carryingsunsetmetadata, fired on actual use (embedding resolution + the rerank path) so brains still riding a dying provider hear about it on every process, not only at upgrade time; prints the sunset date plus the migrate /search.reranker.modelfix, never throws,_resetSunsetWarningsForTest()is the test seam. Past aRERANKER_SUNSETSdate (rerankerSunset()fromai/defaults.ts),rerank()short-circuits BEFORE the HTTP call: the check runs where the EFFECTIVE model is resolved (input.model ?? getRerankerModel() ?? DEFAULT_RERANKER_MODEL— the main case is an absent per-call model landing on the configured/legacy default) and throwsRerankError('sunset_short_circuit')soapplyRerankerfails open at once instead of burning the 5s timeout per query; suppressed under abase_urlsrecipe override (self-hosted wire-compatible endpoints outlive the hosted shutdown, same rule aswarnSunsetOnce); traceability is ONEsunset_short_circuitaudit row per process per model (written by the gateway to the rerank-failures JSONL, feeding doctor's reranker_health) plus one stderr line with the replacement-model switch command — per-query rows would flood the audit file until the user migrates;__setSunsetClockForTestsis the injected-clock seam for date-matrix tests, reset alongside_resetSunsetWarningsForTest().no_keypreflight: after the sunset check andrequireConfig(), a recipe without a customresolveAuthwhoseauth_env.requiredkey is absent fromcfg.envthrowsRerankError('no_key')BEFORE any HTTP —noKeyOnce()(mirror ofsunsetShortCircuitOnceMINUS the stderr line; memo_noKeyNoticed, cleared by_resetSunsetWarningsForTest()) writes ONEno_keyaudit row per process per model and nothing is printed (shell-per-query agents would otherwise see a line per search).auththerefore means "key present but rejected" (HTTP 401/403).RerankError.reasonclassifier:auth | no_key | rate_limit | network | timeout | payload_too_large | sunset_short_circuit | unknown. 5s default timeout (search hot path). Pre-flight payload guard rejects bodies overrecipe.touchpoints.reranker.max_payload_byteswithreason: 'payload_too_large'._rerankTransporttest seam mirrors_embedTransport.embedQuery(text)threadsinputType: 'query'throughdimsProviderOptions()(4-arg).getRerankerModel()returns only the EXPLICITLY configured model (callers need not pre-check availability —rerank()failsno_keyitself; the sync readiness predicate for dashboards isreranker-readiness.ts) +isAvailable('reranker')branch;configureGateway+reconfigureGatewayWithEnginethreadreranker_model;applyResolveAuth+defaultResolveAuthwiden touchpoint param to include'reranker'.embedMultimodalOpenAICompat()routes recipes withimplementation: 'openai-compatible'(LiteLLM, Anyscale, vLLM, Gemini multimodal via proxy) through the standard/embeddingsendpoint with content arrays carryingimage_urlentries; the Voyage/multimodalembeddingspath is unchanged (gateway selects by recipeimplementationtag). Runtime dimension validation throwsAIConfigError(with model id + observed + expected) before the vector reaches storage when the provider returns a width that doesn't match the recipe'sdefault_dimsor the brain'sembedding_dimensions. Pinned bytest/openai-compat-multimodal.test.ts. Module-scoped_embedTransportdefaults to AI SDKembedMany, with__setEmbedTransportForTests(fn)test seam so tests driveembed()with a stubbed transport.splitByTokenBudgetandisTokenLimitErrorexported@internal(pure functions reused by the test file). Module-level_shrinkState: Map<recipeId, {factor, consecutiveSuccesses}>halves the recipe's effectivesafety_factoron token-limit miss (floor 0.05) and heals back ×1.5 afterSHRINK_HEAL_AFTER=10consecutive successes.configureGateway()walks every registered recipe at construction and emits a once-per-process stderr warning for any embedding touchpoint missingmax_batch_tokens(excluding the canonical OpenAI fast-path).resetGateway()clears_shrinkState, the warned-set, and restores the real transport.embedMultimodal()readscfg.embedding_multimodal_modelfirst (falls back tocfg.embedding_model); after the recipe-levelsupports_multimodalfast-fail, validates the resolved model againsttouchpoint.multimodal_modelswhen declared (closes the Voyage-text-only-into-multimodal-endpoint footgun before any HTTP call).getMultimodalModel()accessor mirrorsgetEmbeddingModel/getChatModel. ExportedVoyageResponseTooLargeErrortagged class:voyageCompatFetch's two OOM-defense caps (Layer 1 Content-Length, Layer 2 per-embedding base64) throw it; the inbound response-rewriter's try/catch (which swallows parse failures so misshaped responses fall through to the SDK parser) checksinstanceof VoyageResponseTooLargeErrorand rethrows so the cap is actually effective (an assertion intest/voyage-response-cap.test.tspins theinstanceof ⇒ throw errline). AI SDK v6 toolLoop compat (gbrain skilloptrollouts AND production backgroundsubagentjobs both route throughchat()/toolLoop): inchat(), tool defs wrap the raw JSON Schema with the SDK'sjsonSchema()helper (inputSchema: jsonSchema(t.inputSchema)) — v6'sasSchema()treats a bare{jsonSchema: ...}object as a thunk and throws "schema is not a function"; exported puretoModelMessages(messages: ChatMessage[]): unknown[]converts gbrain's provider-neutralChatMessage[]into v6ModelMessage[]— tool results (pushed bytoolLoopasrole:'user'with bare-value tool-result blocks) become a dedicatedrole:'tool'message with structuredoutput:{type:'json'|'text'|'error-text', value}parts;nulloutput preserved as{type:'json', value:null}(not dropped); text/tool-call blocks pass through with v6 field names (toolCallId/toolName/input); applied at thegenerateTextcall (messages: toModelMessages(opts.messages)). The converter is load-bearing for the production subagent path, not just skillopt. Pinned bytest/gateway-model-messages.test.ts. Companion:src/core/skillopt/rollout.tsbuilds tool schemas through the sharedparamDefToSchemafromsrc/mcp/tool-defs.ts(single source of truth, recursive on items/enum/default), never an inline converter. Provider-agnostic plumbing:resolveNativeBaseUrl(provider, cfg)normalizes a configuredANTHROPIC_BASE_URL/OPENAI_BASE_URLto carry the/v1suffix and is passed explicitly at every nativecreateAnthropic/createOpenAIsite (chat/expansion/embedding), so an env-injected bare host doesn't 404; returnsundefinedwhen unset so the SDK default is preserved (Google deferred until its native suffix is verified).diagnoseEmbeddingfails closed withuser_provided_dims_unsetwhen a user-provided / zero-default recipe (litellm/llama-server) has no configuredembedding_dimensions.configureGatewaydoes not backfillembedding_dimensions(readers default it themselves), keeping the "no dims set" signal honest for that guard and the multimodal skip.withBudgetTracker: gateway-layer enforcement viaAsyncLocalStorage<BudgetTracker>.withBudgetTracker(tracker, fn)installs the tracker on the module-internal store; everygateway.chat / embed / rerankcall inside the scope auto-composes (reserve before, record in try/finally). Outside-scope calls are budget no-ops. Nested scopes restore the outer tracker on exit.getCurrentBudgetTracker()is the test seam. The chat path uses the pessimistic fallback on error paths; the embed path estimates input tokens from char count × recipe'schars_per_tokenbecause the AI SDK doesn't surface per-batch embed token usage; the rerank path estimates char count of query+docs. Pinned by 6 unit cases.reconfigureGatewayWithEngine(engine)(async, called fromcli.tsafterengine.connect(), before every command exceptCLI_ONLYno-DB commands) re-resolves expansion + chat defaults throughresolveModel()somodels.tier.*andmodels.defaultoverrides apply to both.DEFAULT_CHAT_MODELisanthropic:claude-sonnet-4-6.ChatOpts.temperatureis threaded verbatim to the AI SDKgenerateTextcall (unset → the provider's default; the LongMemEval judge pins0, the officialevaluate_qa.pysetting).ChatResult.responseModelcarries the model id the PROVIDER reported (response.modelId, e.g. a dated snapshot) when the SDK surfaced one and is absent otherwise, whilemodelstays the requestedprovider:modelId; eval receipts pin the two side by side (reader_model_snapshot,judge_model_snapshot).__setChatTransportForTestsmirrors__setEmbedTransportForTestsso tests drivechat()with a stubbed transport.toolLoopper-turn permit hook: optionalacquireTurnPermit()acquires a provider permit before EVERY round-trip and releases in finally; a lease-full throw propagates without consuming the turn (the subagent path wires it to the rate leases with provider-derived keys — Anthropic models shareanthropic:messages, others get<recipeId>:chat).ToolLoopStopReasonincludes'length': a zero-tool-call turn that hit the output cap propagates 'length' instead of folding into 'end', so truncation is never reported as a clean finish. Per-part provider state:ChatBlockvariants carry optionalproviderMetadatacaptured from SDK parts inchat()and re-emitted asproviderOptionson the rebuilt parts intoModelMessages()(attached only when present — metadata-free blocks stay byte-identical); this is the Gemini 3.x thoughtSignature echo.isThinkingByDefaultModel(modelStr)(exported) matches Claude 5-family ids behind any provider-prefix chain with a letters-only family segment (neverclaude-3-5-*);isThinkingModel(modelStr)(exported) is that regex OR the recipe's chatthinking_by_defaultcapability (fail-closed: unknown/chat-less recipes are non-thinking) and drivesdefaultMaxOutputTokens's 32k thinking headroom for chat()/toolLoop() callers that omit maxTokens (e.g.gbrain skilloptrollouts) and the subagent handler'sresolveMaxOutputTokens; think/index.ts shares the regex and makes the same capability check. Pinned bytest/ai/gateway-thinking-headroom.test.ts.expand()andgenerateOcrText()also record on the ambient tracker (they call generateObject/generateText directly and never pass throughchat()'s_recordBudget): record-only, no reserve — a breach surfaces on the NEXT reserving call, matchingchat()'s swallow ofBudgetExhaustedfromrecord(). Successes record normalized SDK usage vianormalizeSdkUsage(the ONE home for the v6/legacy usage shapes, used by chat's success path too; first FINITE field wins so a NaN v6 field can't shadow a real legacy value or poison the running total); failures record pessimistically undergateway.expand.failed/gateway.ocr.failed(a rejected attempt still billed provider tokens — one expand() can legitimately produce two records when the structured-output attempt fails and the text fallback runs). A recipe whose declared structured-output support is rejected at call time is remembered for the process lifetime (_structuredOutputRejectedRecipes) so the rejected attempt isn't re-paid on every call. OCR's input estimate is prompt text + a fixed per-image token constant, never base64 length (bytes are not tokens).generateOcrText()routes togetImageOcrModel()—embedding_image_ocr_modelwhen set, else the expansion model — gated byisAvailable('expansion', <ocr model>)so a misconfigured OCR model (provider without an expansion touchpoint, or unkeyed) fails closed to''instead of silently OCRing with the expansion model; the accessor mirrorsgetMultimodalModel(),configureGateway/buildGatewayConfigthread the field, and an unconfigured gateway stays a silent''no-op. Pinned bytest/ai/ocr-model-routing.test.ts.__setGenerateObjectTransportForTestsmirrors the generateText seam. +test/core/budget/expand-records-budget.test.ts+test/ai/gateway-ocr-budget.test.ts -
src/core/ai/recipes/zeroentropyai.ts— ZeroEntropy openai-compatible recipe declaring BOTHembedding(zembed-1, 7 Matryoshka dims: 2560/1280/640/320/160/80/40) ANDreranker(zerank-2flagship +zerank-1+zerank-1-small, 5MB payload cap) touchpoints.implementation: 'openai-compatible'(pinned bytest/ai/zeroentropy-recipe.test.ts).base_url_default: 'https://api.zeroentropy.dev/v1'already ends with/v1, so thezeroEntropyCompatFetchURL rewrite/embeddings → /models/embedproduces…/v1/models/embed(NOT…/v1/v1/…— pinned there too).chars_per_token: 1+safety_factor: 0.5match Voyage's dense-content hedge. Carriessunsetmetadata (ZEROENTROPY_SUNSET_DATE+ replacement models fromai/defaults.ts) that drives init picker/auto-pick exclusion, the gateway's once-per-process warn-on-use, and everygbrain providersrendering via the sharedsunsetMarkerinsrc/commands/providers.ts(liststatus cell, ⚠explainrows, and theenvdeprecation block that replaces the signup funnel); the recipe itself is slated for removal after the sunset date. -
src/core/ai/recipes/llama-server-reranker.ts— sibling ofllama-server(the embedding recipe) for llama.cpp in--rerankingmode. Distinct recipe rather than dual-touchpoint extension because--rerankingand--embeddingsare mutually exclusive at server-launch time, so the two backends need independent base URLs (default 8081 here vs 8080 there). Declaresrerankertouchpoint withmodels: [](user-provided id matching the--aliasthe user launched with),path: '/rerank'(leaf-only; consumesRerankerTouchpoint.pathoverride; gateway concatenates withbase_url_defaultwhich ends in/v1, producing…/v1/rerank),default_timeout_ms: 30_000(consumed bysrc/core/search/mode.ts's reranker timeout chain — CPU-only first-call warmup headroom; the 5s mode-bundle default would fail-open astimeout),cost_per_1m_tokens_usd: 0(recognized byFREE_LOCAL_RERANK_PROVIDERSinsrc/core/budget/budget-tracker.tsso--max-costcallers don't hard-fail on local rerank). Setup hint emphasizes--aliasbecause llama-server's/v1/modelsdefaults model id to the gguf file path without it. Covers Qwen3-Reranker via llama.cpp AND self-hosted ZE weights via llama.cpp — same recipe, different--modelat launch. Pinned bytest/ai/recipe-llama-server-reranker.test.ts. Voyage / Cohere / vLLM rerankers stay out of scope (different wire shapes). Companion surfaces:path?: string+default_timeout_ms?: numberonRerankerTouchpointinsrc/core/ai/types.ts; consumed by the URL build atsrc/core/ai/gateway.ts:rerank()and by mode-resolution atsrc/core/search/mode.ts:resolveSearchMode(precedence: per-call > config-key > recipe touchpoint default > mode bundle);LLAMA_SERVER_RERANKER_BASE_URLenv passthrough insrc/cli.ts:buildGatewayConfig;FREE_LOCAL_RERANK_PROVIDERSset insrc/core/budget/budget-tracker.ts:lookupPricing(rerank-kind-only zero-pricing for the local provider prefix); doctor-fix atsrc/commands/models.ts:probeRerankerConfigreadssearch.reranker.modelvialoadSearchModeConfig+resolveSearchMode(so doctor and live search read the same resolution and the file plane / DB plane cannot diverge — the field-planegetRerankerModel()reads nothing writes);probeRerankerReachabilityreads the recipe'sdefault_timeout_msso CPU-only cold-start doesn't false-fail. -
src/core/ai/recipes/nan.ts— nan.builders openai-compatible reranker recipe.POST {base}/v1/rerank→results[{index, relevance_score}], leaf-only (base_url_defaultends/v1; bare/rerankand/compatible-api/v1/reranksboth 404). Model id is the literalrerank(a Qwen3-Reranker-8B deployment).auth_env.required: ['NAN_API_KEY']. -
src/core/ai/recipes/openrouter.ts— OpenRouter openai-compatible recipe: single key, many providers viaopenrouter:<provider>/<model>strings.base_url_default: 'https://openrouter.ai/api/v1'. Embedding touchpoint: default modelopenai/text-embedding-3-small; per-modelmodel_dimscarries verified native widths (text-embedding-3-small 1536, text-embedding-3-large 3072, qwen/qwen3-embedding-8b 4096, bge-m3 + baai/bge-m3 1024) withdefault_dims: 0so an UNLISTED proxied id has NO silent default — it errors until the user supplies explicit dims (--embedding-dimensions/embedding_dimensions), whichtrust_custom_dims: trueaccepts (gemini-embedding-2-preview is deliberately unlisted — width unverified). Matryoshkadims_options: [512, 768, 1024, 1536]still governs the default model's shrink steps;max_batch_tokens: 300_000= OpenAI's aggregate-per-request token cap (NOT per-input). Chat touchpoint declares 8 curated entry points (gpt-5.2, gpt-5.2-chat, gpt-5.5, claude-haiku-4.5, claude-sonnet-4.6, claude-opus-4.7, gemini-3-flash-preview, deepseek-chat) but openai-compat tier accepts any model ID; deliberately nomax_context_tokensbecause OR's catalog spans 128K to 1M+.supports_subagent_loop: falseis enforced byclassifyCapabilities()insrc/core/ai/capabilities.ts(verdictunusable:no_subagent_loop—enforceSubagentCapable()insrc/core/model-config.tsconsumes the verdict: tool-less/unknown models fall back toTIER_DEFAULTS.subagentwith a warn; tool-capable providers without prompt caching run with a once-per-model cost warn); the legacy Anthropic-direct path additionally gates onisAnthropicProvider()insrc/core/model-config.tswhenagent.use_gateway_loopis off. DeclaresresolveDefaultHeaders(env)returning OR's three attribution headers:HTTP-Referer(required for OR app-attribution),X-OpenRouter-Title(preferred),X-Title(back-compat alias); defaults tohttps://gbrain.ai/gbrain; forks override viaOPENROUTER_REFERER/OPENROUTER_TITLEenv vars. Smoke-tested bytest/ai/recipe-openrouter.test.ts(incl. the shape guard: every model in the chat list matches^[a-z0-9-]+\/[a-z0-9._-]+$). -
src/core/ai/openrouter-families.ts— single source of truth for which OpenRouter model families may drive the subagent (tool) loop:OPENROUTER_SUBAGENT_FAMILIES(currentlyanthropic/+deepseek/, each backed by a live abort/retry replay pin undertest/e2e/) +openrouterModelSupportsSubagentLoop(modelId)(takes the bare OpenRouter id, noopenrouter:prefix). Shared by the openrouter recipe's predicate and the subagent handler's auto-route so the two can never disagree; a new family gets its own live pin before joining the list. -
src/core/ai/recipes/lmstudio.ts— LM Studio local OpenAI-compatible recipe (embedding touchpoint only; default base URLhttp://localhost:1234/v1, envLMSTUDIO_BASE_URL). Shipsmodels: []like llama-server — the model identity is whatever the user loaded in the app, so--embedding-model lmstudio:<id>+--embedding-dimensions <N>are required (the wizard refuses implicit defaults). Its own provider id keepsopenai:pointed at OpenAI (complement of the keylessOPENAI_BASE_URLroute). Documented operational failure mode: a degraded loaded instance passes the reachability probe but misses the query-embed deadline — queries degrade to keyword-only with theembed_timeoutstamp, reload the model to recover. -
src/core/ai/recipes/ollama.ts— the Ollama local recipe.thinking_by_defaultis a per-family predicate over the model id (qwen3 with a boundary soqwen2.5-*is never swallowed and a-coderexclusion — the instruct-only variant has no thinking mode; deepseek-r*, gpt-oss, magistral, phi*-reasoning including the-mini-reasoningtags), not a recipe-wide boolean, so non-reasoning local models keep the conservative default. -
src/core/rerank-audit.ts— failure-only JSONL audit at~/.gbrain/audit/rerank-failures-YYYY-Www.jsonl(ISO-week rotation, mirrorssrc/core/audit-slug-fallback.ts). ExportslogRerankFailure({reason, model, query_hash, doc_count, error_summary})+readRecentRerankFailures(days). Thesunset_short_circuitandno_keyreasons are written ONCE per process per model by the gateway itself (not per query byapplyReranker): the reranker's hosted API passed its announced shutdown date / the provider key is absent, so calls are skipped without HTTP and results pass through unreranked;authmeans key present but rejected. Deliberately nologRerankSuccess: writing once per tokenmax search is hot-path I/O churn AND success events leak query volume + timing into a local audit file.gbrain doctor'sreranker_healthcheck (doctor/checks/calibration.ts) resolves enablement + model throughresolveSearchMode(loadSearchModeConfig)— the plane search reranks with — andrerankerReadinessbefore reading the audit: readiness is evaluated against the DB-merged config plane (loadConfigWithEngine, so DB-plane provider keys andprovider_base_urlsself-host overrides count, exactly like the gateway the CLI configures): a brain with NO embedding provider (isAvailable('embedding')false on a configured gateway) isokeven when not ready — search runs keyword-only there and never reachesapplyReranker, so there is nothing to fix; otherwise enabled-but-not-ready (key absent / sunset passed without a self-host override / unknown model) warns with the paste-ready fix; only audit rows for the RESOLVED model count (rows for any other model never make the live default warn or send the operator to verify the wrong key);no_key/sunset_short_circuitskip rows are informational once ready (those processes ran unreranked; a long-lived worker started before the key was added keeps skipping until restarted) and never outlive the fix as a warn; then the auth / payload / budget / transient / unknown ladders on the non-skip rows; disabled → ok (with an enable hint when the default is ready). Query text SHA-256-prefix-hashed (8 hex chars) for privacy.GBRAIN_AUDIT_DIRenv override honored via the sharedresolveAuditDir(). -
src/core/search/embedding-column.ts— single source of truth for "whichcontent_chunks.*column does this query rank against?" Pure functions, no engine I/O:loadRegistry(cfg)walks theembedding_columnsconfig (DB plane, JSON map keyed by column name with{provider, dimensions, type}entries), seeds the OpenAIembeddingbuiltin when unset, validates everything before it lands (column-name regex, type ∈vector | halfvec, dims in [1, 8192], provider format) usingObject.create(null)+Object.hasOwnso a key likeconstructorrejects instead of resolving toObject.prototype.constructor.resolveColumn(registry, override?, cfg)is the boundary call: returns a frozenResolvedColumndescriptor ({name, provider, dimensions, type}) honoring per-call override →search_embedding_columnconfig →'embedding'default; throwsUnknownEmbeddingColumnErrorwith the list of registered names on miss.isCacheSafe(resolved, cfg)compares the full embedding SPACE (provider + dimensions + name) against cfg's default so a repointedembeddingbuiltin doesn't serve OpenAI-shaped cache rows.validateResolvedColumn(descriptor)re-validates hand-rolled descriptors that bypass the registry (internal-SDK passthrough) so the SQL-injection escape hatch through the descriptor field is closed. Consumed byhybridSearch,gateway.embedQuery(text, {embeddingModel, dimensions}),cosineReScore, and thequeryMCP op (per-callembedding_columnparam). Pinned bytest/search/embedding-column.test.ts(prototype-pollution, descriptor passthrough, env-only Postgres install, empty-brain coverage gate, cache-space comparison). Write side:resolveWriteColumnFromConfigRows({searchEmbeddingColumn, embeddingColumnsJson})resolves upsertChunks's active write target from the raw DB-plane config rows through the same registry/validation chain (legacyembeddingdescriptor when neither row routes elsewhere;embedding_imagefalls back to the legacy text column since image vectors travel in their own INSERT column; malformed registry JSON is forgiven like loadConfigWithEngine; unregistered names throw the paste-ready error), andvectorCastSuffix(resolved)is the placeholder-free'::vector' | '::halfvec(N)'cast shared by the read fragment builder and the write path so the two casts can't drift. Pinned bytest/e2e/upsert-chunks-registry-column.test.ts. -
src/core/search/rerank.ts— the call-site abstraction.applyReranker(query, results, opts)slots betweendedupResults()andenforceTokenBudget()insrc/core/search/hybrid.ts. Slicesopts.topNIn(default 30) by current RRF order, caps each document viacapRerankDoc(RERANK_MAX_DOC_CHARS=6000then a measured-ratio shrink toRERANK_MAX_DOC_TOKENS=1400— a 2048-ubatch llama-server minus query/template headroom minus a Qwen-tokenizer margin; every cut throughtruncateUtf8so no lone surrogate reaches the JSON body; docs under 1400 chars skip the tokenizer; applies to hosted providers too, so code/CJK chunks at the chunker ceiling lose part of their tail while prose is untouched), sends togateway.rerank(), reorders byrelevanceScoredesc, appends the un-reranked tail unchanged (recall protection). Fail-open on everyRerankError.reason: any error logs vialogRerankFailureand returns the input array unchanged — except the two SKIP classessunset_short_circuitandno_key(RerankSkipReason), which return the input immediately WITHOUT a per-query audit row (the gateway already wrote the one per-process-per-model row) and fireopts.onSkip(reason);hybrid.tspasses anonSkipthat stamps{stage: 'reranker_skipped', reason}intoHybridSearchMeta.degraded(closed vocabulary intypes.ts), which--explainrenders asdegraded: reranker_skipped (no_key)viaformatDegradedSummary— the only place a silently skipped reranker is visible from the CLI.types.tsclassifies the stage as ranking-only:RANKING_ONLY_DEGRADED_STAGES(={reranker_skipped}) +affectsRecall(entry)— consumers that ask "was recall impaired?" filter through it (the CLINo results.line incli.ts:describeEmptyRetrievaland the MCP empty-result block indispatch.ts:buildEmptyRetrievalBlockkeep saying "clean miss"; the degraded cache-TTL rule below uses the same predicate), while consumers that report what did not run (--explain, telemetry, eval rows) keep it; pinned bytest/degraded-stages-recall.test.ts.hybridSearchCachedexcludesreranker_skippedfrom the short degraded-TTL rule (it is a config state, not a transient limp: a keyless balanced brain keeps the fullcache.ttl_seconds, and the stored meta still carries the stamp). Stampsrerank_scoreonto reordered items so downstream telemetry sees the new ordering signal.topNOut: nullis the explicit "don't truncate" signal — semantically distinct fromundefined("fall through to mode bundle"). Test seam:opts.rerankerFnstubsgateway.rerankwithout the network. Document cap pinned bytest/rerank-doc-cap.test.ts. -
src/core/search/return-policy.ts(default OFF) — intent-aware adaptive return-sizing. Pure, dependency-light module that trims the final ranked candidate set to an intent-driven cap instead of returning the full top-K.entityintent gets a tight cap;temporal/event/generalget a recall-preserving cap (conceptis coerced togeneralbyhybrid.tsbefore the call — concept queries want breadth). AminKeepfailsafe (≥1) guarantees a human never gets a silent blank when candidates exist. WHY a cap, not a score-cliff detector: PrecisionMemBench instrumentation (gbrain-evals) measured the rank1→rank2 RRF gap is ~identical whether rank-1 is correct (0.602) or wrong (0.569) — mechanical decay, not a separatrix; rank-1 is right in 94% of single-answer cases, so "return a tight set" is the whole win and cliff-cutting just adds noise. ExportsAdaptiveReturnConfig,DEFAULT_ADAPTIVE_RETURN(frozen:enabled=false,entityMax=2,otherMax=6,minKeep=1),AdaptiveReturnDecision({applied, intent, cap, kept, total}),AdaptiveReturnInput(boolean | Partial<AdaptiveReturnConfig> | undefined),adaptiveReturnFromConfig(cfg),resolveAdaptiveReturn(perCall, fromConfig)(defaults → config → per-call merge),adaptiveReturnEnabled(...)(cache-skip gate check),applyAdaptiveReturn(results, intent, cfg)(the trim). Config knobs (DB or file plane):search.adaptive_return(master switch),search.adaptive_return_entity_max,search.adaptive_return_other_max,search.adaptive_return_min_keep(each clamped ≥1). Wired intohybridSearchAFTERapplyReranker, BEFORE thelimitslice, and ONLY on the first page (offset===0) — paginating a confidence-gated set is incoherent, so paginated calls fall through to the fixed limit. Stamps the decision ontoHybridSearchMeta.adaptive_returnforgbrain search --explain.hybridSearchCachedSKIPS the cache when the gate is on (a trimmed set must not be served to a gate-off lookup and vice versa).SearchOpts.adaptiveReturn+HybridSearchMeta.adaptive_returndeclared insrc/core/types.ts. Agent-facing: thequeryop (src/core/operations.ts) exposes anadaptive_returnboolean param whose description instructs the agent WHEN to set it (single-answer → on; breadth/exploration → off; passlimit:1for a hard single-answer cap), threaded intohybridSearchCached— end users never touch the config knob; their agent decides per query (same pattern assalience/recency). Pinned bytest/search/return-policy.test.ts(mechanism) +test/search/query-op-adaptive-return.test.ts(agent surface: param exists + description teaches both directions + the never-empty contract). -
src/core/search/autocut.ts(OFF in every bundle since rule R2;search.autocut truere-enables it where the reranker runs) — Weaviate-style autocut: score-discontinuity result-sizing on the cross-encoder rerank separatrix.applyAutocut(results, scoreOf, cfg)normalizes the reranker scores, finds the largest consecutive gap, and cuts there when it clearsjumpRatio(default 0.20); robust to unsorted provider output (cuts on a sorted copy, keeps items in INPUT order via a score threshold), guardstop<=0/non-finite, never returns empty, and no-ops when <2 results carry a finitererank_score(covers the reranker fail-open path). WHY rerank_score and NOT RRF/cosine: gbrain measured (seereturn-policy.ts) that the RRF rank1→rank2 gap is ~flat whether rank-1 is right or wrong — not a separatrix; the cross-encoder score IS. So autocut, when enabled, runs ONLY where the reranker ran (balanced+tokenmax;conservativeis a documented no-op). Weak-top floor: when the top rerank score is belowminTopScore(default 0.35; configsearch.autocut_min_top; scale-calibrated to the default reranker; a default-reranker change must re-tune it), cliff trimming is skipped entirely so a low-confidence list returns the full cluster instead of collapsing to one result. ExportsAutocutConfig,DEFAULT_AUTOCUT(frozen:enabled=true, jumpRatio=0.20, minKeep=1, minTopScore=0.35),AutocutDecision({applied, signal:'rerank'|'none', cut, kept, total, gapRatio}),AutocutInput,autocutFromConfig,resolveAutocut,applyAutocut. Cache-key integration (clean path, not the adaptive-return cache-skip hack): enable+sensitivity flow throughModeBundle→ResolvedSearchKnobs→knobsHashexactly likegraph_signals.mode.tsaddsautocut/autocut_jump/autocut_min_keep(autocutfalse in every bundle since rule R2 — the LongMemEval replay from the captured post-rerank pool found no floor that beat autocut off; jump 0.20 stays the sensitivity when re-enabled; floor 1 in every mode —search.autocut_min_keepsets the minimum result count a cut may leave, resolved through the same bundle → config → per-call chain) AND setsreranker_top_n_in = searchLimitfor reranked modes (so the reranker scores the full returned set; there is no un-scored tail for autocut to wrongly drop). Autocut folds intoknobsHashas its own parts entry (mode.ts:KNOBS_HASH_VERSIONis the single source of truth for the current hash version; every bump is a one-time global cache cold-miss on upgrade). Wired intohybridSearchAFTER adaptive-return, BEFORE the limit slice, first page only; emitsHybridSearchMeta.autocut. BOTH the cache-missfinalMetaand cache-HITcachedMetarebuilds carryautocut+adaptive_return+mode+embedding_column. Preserves alias-hop exact matches:applyAutocuttakes an optionalpreservepredicate; hybrid passesr => r.alias_hit === true || r.exact_lookup !== undefined || r.relational_pinned === trueso a canonical page injected byapplyAliasHopafter reranking (norerank_score), an exact-lookup tier hit, or a relational row re-pinned bypinRelationalRows(low cross-encoder score by construction; ALSO excluded from the cliff computation throughscoreOf, so text-row autocut is unchanged by the pin) is never cut. Agent surface:queryopautocutboolean (ceiling override —falseforces full top-K);SearchOpts.autocut;--explainshows per-resultrerank_score,formatAutocutSummaryrenders the decision when search meta is threaded;gbrain search modesattribution; metric glossaryautocut.signal/autocut.gap_ratio. Config:search.autocut,search.autocut_jump,search.autocut_min_keep. TheDEFAULT_AUTOCUTmodule constant is unchanged (enabled=true) for operators who re-enable it; the cliff logic is backed by an in-repo eval gate —test/search/autocut-eval.test.ts(alsobun run eval:autocut) measures precision-lift-without-recall-regression over labeled qrels fixtures with modeled cross-encoder distributions (no API key, no sibling repo; runs in CI): mean precision 0.33→0.94, recall 1.00→0.95, ZERO recall loss on enumeration queries. Env-overridable floors. Pinned bytest/search/autocut.test.ts(pure-fn),test/search/query-op-autocut.test.ts(agent surface),test/search/autocut-integration.serial.test.ts(IRON-RULE behavioral viarerankerFnDI seam: cliff trims, flat doesn't, no-reranker no-ops,autocut:falseceiling, composes with adaptive-return),test/search/autocut-eval.test.ts(the precision/recall gate), and the knobsHash assertions intest/search-mode.test.ts. -
src/core/ai/recipes/voyage.ts— Voyage AI openai-compatible recipe, home of the new-install default stack. Embedding touchpoint declaresdefault_model: 'voyage-4'+default_dims: 1024— the canonical pick for every "choose a model for the user" surface (models[0]staysvoyage-4-largein quality order; the new-install default isvoyage-4for price/quality balance and the shared v4 embedding space — seeNEW_INSTALL_DEFAULT_EMBEDDING_MODELinai/defaults.ts). Reranker touchpoint (the mode-bundle default viaDEFAULT_RERANKER_MODEL, sameVOYAGE_API_KEYas embeddings) allowlistsrerank-2.5($0.05/M) +rerank-2.5-lite($0.02/M) and Voyage's previewrerank-3($0.05/M) +rerank-3-lite($0.02/M) (2.5 pair verified 2026-08-15, rerank-3 pair 2026-09-06);default_modelstaysrerank-2.5deliberately — the array is the opt-in surface for every enforcement point (gateway.rerank()'stp.models.includesguard,gbrain models doctor'sreranker_configprobe,rerankerReadiness), and flipping the default would migrate every install that resolves it instead of an explicit config row. All four share one wire:path: '/rerank',top_param: 'top_k'(Voyage's response wire matches ZE's{results: [{index, relevance_score}]}; only the request's top-N key differs), 5MB byte-proxy payload cap. Declareschars_per_token=1+safety_factor=0.5so the gateway pre-splits Voyage batches at a 60K-character budget (50% of 120K-token cap with the dense-tokenizer ratio), since tiktoken-grounded budgeting undercounts Voyage's actual token usage. Declaresmultimodal_models: ['voyage-multimodal-3']so the gateway rejects text-only Voyage models pointed at the multimodal endpoint with a clearAIConfigErrorinstead of waiting for Voyage's HTTP 400. The hosted flexible-dim models that acceptoutput_dimensionlive inVOYAGE_OUTPUT_DIMENSION_MODELSinsrc/core/ai/dims.ts(v4 trio,voyage-code-4,voyage-3-large,voyage-3.5,voyage-3.5-lite,voyage-code-3; valid widths 256/512/1024/2048);voyage-4-nanois the open-weight variant fixed at 1024-dim that does NOT accept the parameter (negative assertion intest/ai/gateway.test.ts:dimsProviderOptionsreturnsundefinedforvoyage-4-nano).voyage-code-3is the recommended embedding model for gstack per-worktree code brains (Topology 3 indocs/architecture/topologies.md;voyage-code-4is the flexible-dim hosted code model, $0.12/M); discoverability surfaces: decision-tree branch indocs/integrations/embedding-providers.md, Topology 3 "Recommended embedding model" subsection, runtime nudge fromgbrain reindex --codeagainst non-code-tuned models. Recipe shape pinned bytest/ai/voyage-code-3-recipe.test.ts. -
src/core/ai/recipes/anthropic.ts— Anthropic recipe (chat + expansion touchpoints). Canonical id isclaude-sonnet-4-6(no date suffix); a reverse aliasclaude-sonnet-4-6-20250929 → claude-sonnet-4-6keeps stale user configs working (rescuesfacts.extraction_modelandmodels.dream.synthesize). Recipe shape pinned bytest/anthropic-model-ids.test.ts. -
src/core/ai/providers/claude-cli-language-model.ts(+ recipesrc/core/ai/recipes/claude-cli.ts) — ai-sdk LanguageModel adapter that shells out to the locally-installedclaudeCLI in print mode (OAuth-subscription lane, no API key;claude-cli:<model>is config-portable withanthropic:<model>). Tool use is system-prompt-instructed JSON emission: the recipe teaches the model the<use_tools>[{name,input}]</use_tools>format and the adapter parses those blocks back into ai-sdktool-callparts. An entry that carries its arguments flat besidename(noinputwrapper, the Anthropic tool_use shape with the wrapper dropped) is accepted: every key other thanname/inputand the Anthropic tool_use leftovers (type: "tool_use", atoolu_*id) becomes the input — any othertype/idvalue is a real argument (e.g.list_pages'typefilter) and stays — and a presentinputwins verbatim over stray sibling keys. Tool-call ids are ALWAYS gbrain-minted (toolu_claude_cli_<uuidv7>) — never model-authored: each doGenerate is a fresh subprocess replayed from an id-stripped transcript, so the model structurally cannot keep ids unique across turns (a model-authored id echoes the prompt's example entropy-free and collides); the prompt does not ask for an id, and a strayidfield is deliberately ignored (nothing round-trips it — the loop pairs results in-memory within one turn). doStream not implemented; callers (gateway.toolLoop) use doGenerate. The recipe declareschat+expansiontouchpoints (no embedding; the expansion list leads with the cheap haiku id and carries the same 30sdefault_timeout_msas chat for the subprocess cold start);gateway.expand()routesclaude-clithrough the schemalessviaTextpath because the adapter ignoresresponseFormat(generateObject would throw NoObjectGeneratedError on the fenced-JSON text). Pinned bytest/claude-cli-recipe.test.ts+test/ai/claude-cli-expansion.test.ts. -
src/core/ai/providers/claude-cli-scratch.ts— the claude-cli provider's per-PID scratch dirs (claudeCliCwdDir/claudeCliConfigDiroff theCLAUDE_CLI_CWD_PREFIX/CLAUDE_CLI_CONFIG_PREFIXbasename constants, plussweepDeadClaudeCliScratchDirs), split into a dependency-light module so transcript discovery (src/core/transcripts/discover.ts) can callisClaudeCliSelfTranscriptPath(path)WITHOUT importing the provider's @ai-sdk surface. Eachclaude --printsubprocess runs with cwd = a per-PID scratch dir (context isolation — no local CLAUDE.md auto-discovery), and Claude Code records a session transcript for that cwd under~/.claude/projects/<slugified-cwd>/; those sessions are gbrain's OWN internal LLM calls, and importing them back is a self-ingestion feedback loop (prompt scaffolding + page content re-entering as "conversations"). The prefix is lowercase[a-z-]only, so it survives Claude Code's cwd slugification verbatim and a substring match on the discovered path is a reliable self-exclusion fingerprint. Pinned bytest/transcripts-self-exclusion.test.ts. -
src/core/model-pricing.ts— single source of truth for paid-cloud CHAT/completion model pricing (USD per 1M tokens, input | output).CANONICAL_PRICINGis aprovider:model-keyed table (Anthropic Opus 5/4.8/4.7/4.6$5/\$25, Sonnet 4.6$3/\$15, Haiku 4.5$1/\$5both dateless + dated, plus OpenAI / Google / Together / DeepSeek panel models).canonicalLookup(modelId)resolves bare (claude-opus-4-8), colon (anthropic:claude-opus-4-8), and slash (anthropic/...) forms — bare ids default to theanthropic:provider; nested OpenRouter ids (openrouter:anthropic/...) intentionally MISS so OpenRouter markup isn't repriced as the inner vendor; after the exact match, a case-insensitive fallback folds BOTH sides (some canonical keys carry cased model tails verbatim), safe only while no two canonical keys collide case-insensitively (pinned by the drift guard). Every other chat-pricing table is a DERIVED view of this one (NOT a hand-copied duplicate), so cross-table price drift is structurally impossible. Embeddings live separately inembedding-pricing.ts(different unit). Pinned bytest/model-pricing.test.tswhose drift guard asserts each derived view equals canonical and that the cross-modal panel models are all present. -
src/core/ai/chat-usage.ts+src/core/ops/usage.ts— durable per-call chat usage accounting.gateway.chat()callsrecordChatUsageat its SUCCESS boundary (production provider path AND test-transport path) with the answering model + token usage; the record lands in thechat_usage_logtable (migration v140) via an injected sink (setChatUsageSink— no engine import here, so the gateway imports this leaf statically without deepening a cycle). Phase attribution is best-effort via AsyncLocalStorage (withChatPhase): the minion worker AND the cycle's inline drain (inline-drain.ts) both wrap handler execution injob:<name>, a cycle phase that meters its own spend wraps itself inphase:<name>(dream synthesize does), and direct callers record phase NULL. The innermost wrap wins, which is the point — a child drained INSIDE a wrapped phase must carry its ownjob:tag or the phase tag absorbs every child's spend and the ledger double-counts (one ledger per surface:minion_jobsstays the child-spend authority, the phase row is the orchestrator's own calls). Pricing resolves throughCANONICAL_PRICING(estimateChatCostUsd, cache_read/cache_write at provider cache rates when the table carries them); unknown models recordcost_usd = NULL, never a fake 0. Accounting is strictly fail-open + fire-and-forget: a sink error must never break a chat call. Theget_usageop (admin scope, NOT read —chat_usage_loghas no source dimension, so a source-restricted/federated token must not see brain-wide spend telemetry) returns per-model/per-phase aggregates only, plus an explicitcoverageblock stating what the ledger does NOT capture (subagent raw-SDK path, embeddings, pre-sink/pre-v140 calls, failed calls);budget-tracker.tsremains the pessimistic in-flight spend gate — this table is the after-the-fact ledger. -
src/core/anthropic-pricing.ts— bare-keyed Anthropic VIEW ofmodel-pricing.ts(theanthropic:canonical entries with the prefix stripped). Kept distinct because many callers look up by bare Claude id and becauseestimateMaxCostUsd(modelId, inTokens, maxOutTokens)carries the null-on-miss contract the dream-cycle budget gate depends on (non-Anthropic ids return null;BudgetMetertriescanonicalLookupfirst and only falls back here, so it warnsBUDGET_METER_NO_PRICINGand runs unbounded only when canonical has no rates either).estimateMaxCostUsdroutes bare/colon/slash ids throughsplitProviderModelId. Do NOT hand-edit prices here — the map is derived from canonical, so it cannot drift.ANTHROPIC_PRICINGis consumed bybudget/budget-tracker.ts,minions/batch-projection.ts, andcycle/budget-meter.ts. -
src/core/takes-quality-eval/pricing.ts— fail-closed budget pricing foreval takes-quality run --budget-usd N.MODEL_PRICINGis a curatedprovider:modelallowlist (default panel + likely overrides) whose VALUES are derived frommodel-pricing.tsviacanonicalLookup; an allowlisted id missing from canonical throws at module load. Schema is{input_per_1m, output_per_1m}. A model NOT on the allowlist aborts the run with an actionable error rather than guessing (distinct fromcross-modal-eval/runner.ts, which silently estimates zero on unknown models — both now source numbers from canonical). -
src/core/budget/budget-tracker.ts— the keystone budget primitive. One typed error (BudgetExhaustedwithreason: 'cost' | 'runtime' | 'no_pricing'), one schema-stable audit JSONL at~/.gbrain/audit/budget-YYYY-Www.jsonl. Contracts:record()throws when cumulative spend exceeds cap (the cap is a real ceiling, not a suggestion);reserve()admission checks cumulative + OUTSTANDING spend — each admitted projection is held until itsrecord()settles it, so N concurrent callers (skillopt's validation gate) can't all pass against the same cumulative and breach--max-cost-usdby (N-1)×per-call cost;reserve()hard-fails withreason: 'no_pricing'whenmaxCostUsdis set AND the model is missing from pricing maps (warn-once preserved when cap is unset);lookupPricingnormalizes recipe aliases throughresolveRecipebefore consulting the pricing tables, soclaude-cli:haikuprices exactly likeclaude-cli:claude-haiku-4-5-20251001atreserve(),record()andisModelPriceable()alike (the gateway reserves with the pre-resolution string the user configured);extractUsageFromError(err, fallback)returnserr.usagewhen the SDK provides it, else the pessimistic fallback (caller passesmaxOutputTokens, not the optimistic pre-call estimate).onExhausted(cb)fires once synchronously BEFORE the throw propagates so callers can persist checkpoints. Shared by brainstorm, cycle/budget-meter, and eval-contradictions;BudgetMeterkeeps its public shape over it (schema_version: 1stamped on every dream-budget audit line). Pinned by 18 unit cases. -
src/core/audit-week-file.ts— single source of truth for ISO-week audit JSONL filename math. ExportsisoWeek(d),isoWeekFilename(prefix, now?),resolveAuditDir()(honorsGBRAIN_AUDIT_DIR). Year-boundary correctness pinned by tests at 2020-W53 (the 53-week year), 2025-W01 rolling in from 2024-12-30 (Monday), 2026-W01. Four call sites:src/core/minions/handlers/shell-audit.ts,src/core/facts/phantom-audit.ts,src/core/audit-slug-fallback.ts,src/core/cycle/budget-meter.ts. Each keeps itscompute<X>AuditFilenamethin wrapper. -
src/core/diarize/payload-fitter.ts— generic fit-arbitrarily-large-items-into-per-call-token-budget utility.'batch'strategy is deterministic token-budgeted chunking with no LLM calls.'summarize'strategy embed-clusters into ceil(items/4) groups via cheap deterministic nearest-neighbor on cosine, Haiku-summarizes each cluster viaPromise.allSettledat parallelism=4. Each Haiku call composes the active BudgetTracker via the AsyncLocalStorage. Quality gate: whensuccess_ratio < min_success_ratio(default 0.75), result is flaggeddegraded: true— the fitter preserves the successful subset; the caller decides whether to surface a partial result or abort. -
src/core/brainstorm/checkpoint.ts— crash-resilient checkpoint forgbrain brainstormandgbrain lsd. Persists FULL idea bodies (~50KB/run) so resume MERGES pre-crash ideas with post-resume ideas before the judge runs (a resume that produces only second-run output is silent partial output).run_id = sha256(question + profile + sort(close_slugs) + sort(far_slugs)).slice(0,16)— NO embedding bits, stable across embedding-model swaps. Atomic write via.tmp + rename. ONE resume flag (--resume <run_id>covers both failed AND never-attempted crosses);--list-runsprints run_ids mtime-newest-first;--force-resumebypasses the 7-day staleness gate. Cycle purge phase (gbrain dream --phase purge) GCs checkpoints older than 7 days viagcStaleCheckpoints(7). Pinned bytest/e2e/brainstorm-resume.test.ts(20 unit + 3 E2E cases incl. the merge contract). -
src/core/remediation-checkpoint.ts—doctor --remediatecheckpoint at~/.gbrain/remediation/<plan_hash>.json.plan_hash = sha256(JSON.stringify(sorted recommendation ids)).slice(0,16). Schema-versioned, atomic.tmp + rename.gbrain doctor --remediate --resume <plan_hash>(no arg picks newest matching) loads it and skips completed steps. Mismatched plan_hash refuses with a paste-ready message. Cleared on clean completion. Pinned by 13 unit cases. -
src/core/model-config.ts— Model-string resolution (the seam every internal LLM call walks through). Four-tier system (ModelTier = 'utility' | 'reasoning' | 'deep' | 'subagent') withTIER_DEFAULTS(utility→haiku-4-5, reasoning→sonnet-4-6, deep→opus-4-7, subagent→sonnet-4-6) andtier?: ModelTieronResolveModelOpts.resolveModelDetailed()runs the 8-step chain and reports WHICH step won (ResolveSource): cliFlag → config key → deprecated key →models.tier.<tier>→models.default→ env var (GBRAIN_MODEL) → key-aware tier default → caller fallback (the tier key sits ABOVEmodels.default— tier-specific beats generic, so setting a cheap utility tier is honored even whenmodels.defaultis also set);resolveModel()is the thin wrapper for callers that only want the string. Step 7 is KEY-AWARE:resolveTierDefault(tier, env?)walksPROVIDER_TIER_DEFAULTS(anthropic first — zero change for keyed installs; openai second so an OPENAI_API_KEY-only install resolves servable defaults) over the merged provider env (mergedProviderEnv, config-file keys folded, env wins, empty strings dropped); injectedenvis used EXCLUSIVELY (no config read — hermetic for tests); when neither anthropic nor openai key is present, a SERVABLE file-plane pin for the tier (utility→expansion_model, other tiers→chat_model, checked viaproviderKeyReady,PIN_KEY_BY_TIERis the single tier→key map) wins over the floor so a single-provider install (DeepSeek-only, OpenRouter-only, ...) routes every bare-default caller — extract_atoms, facts classify, page-summary — to the provider it actually has a key for; no key and no servable pin →TIER_DEFAULTSunchanged. The openai entry carries NO literal model pins: it resolves per call through the latest-model discovery cache (src/core/ai/openai-latest.ts, account-discovered, priced-only) withopenaiStaticTierFallback()— the openai recipe's chat list ranked by the same grammar — as the offline floor, so the recipe is the single human-updated source and the account is the runtime source. Thegptalias resolves dynamically through the same path (the map entry is a documentation floor).resolveEffectiveChatModel(fileCfg, env)/resolveEffectiveExpansionModelare the ENGINE-FREE shared effective-model resolvers (GBRAIN_MODEL> servable file pin perproviderKeyReady(recipeauth_env.required) > key-aware tier default; unservable pins warn once and fall through) — used by BOTHreconfigureGatewayWithEngine's fallback layer anddetectCapabilities' extraction probe so runtime routing and the capability report cannot diverge; they read RAWloadConfig()output, never gateway state (the boot fold stamps defaults, making explicit pins indistinguishable there).isAnthropicProvider(modelString)checksprovider:modelprefix ORclaude-bare-id pattern (routes throughsplitProviderModelIdfromsrc/core/model-id.tsso slash-form ids likeanthropic/claude-sonnet-4-6classify correctly).enforceSubagentCapable()is the layer-2 runtime guard:tier === 'subagent'resolutions are classified viaclassifyCapabilities()—unusable:no_tools/unknownwarn once and fall back toTIER_DEFAULTS.subagent;degraded:no_caching(e.g. OpenAI) passes with a once-per-(source, model)cost warn._resetDeprecationWarningsForTest()clears all three warn memos (deprecation, subagent, unservable-pin). Pinned bytest/model-config.serial.test.ts. -
src/core/ai/model-resolver.ts— Recipe-touchpoint validator.assertTouchpoint(recipe, touchpoint, modelId)checks the PROVIDER's capability only (anthropic has no embedding touchpoint; voyage/ollama have no chat) and never gates on the model id. Recipemodels:arrays are informational — default-model selection (models[0]for--model <provider>shorthand and env-ready pickers), guard-test fixtures for the repo's own hardcoded defaults, andgbrain providers listdisplay — NOT a runtime allowlist, so frontier models newer than a recipe work without a recipe PR. A nonexistent id surfaces as the provider's ownmodel_not_foundat call time;gbrain models doctorlive-probes the configured models for a pre-flight check. Exception:gateway.rerank()keeps its own model-list check because each listed reranker id maps to a known request/response wire shape.embeddingDimsForModelmatches recipemodel_dimskeys exactly first, then case-insensitively with BOTH sides folded (configured ids arrive cased, and user-editable recipe tables can carry cased keys; without the fold a cased id would fall through todefault_dimsandgbrain initwould build a wrong-width column), before falling back todefault_dims. -
src/commands/models.ts—gbrain models [--json]read-only routing dashboard: prints tier defaults (utility/reasoning/deep/subagent), the resolved value for each (re-walking the resolution chain), every per-task override (14PER_TASK_KEYS, including provider-neutralmodels.contextual_synopsiswith legacy-key/env attribution andmodels.dream.extract_atomswhich reports via its own caller-specific resolver —resolveExtractAtomsModel()inextract-atoms.ts— rather than the generic chain, so the report can't diverge from what the phase actually calls), the alias map, and a source-of-truth column (default/config: <key>/env: <VAR>).gbrain models doctor [--skip=<provider>] [--json]fires a 1-tokengateway.chat()probe against each configured chat + expansion model and classifies failures into{model_not_found, auth, rate_limit, network, unknown}. The probe timeout resolves per model viaresolveChatProbeTimeoutMs— the recipe touchpoint'sdefault_timeout_mswhen declared, else the flat 5000ms default (mirrors the reranker probe's recipe-default fallback;claude-clideclares 30s because itsclaude -psubprocess cold start routinely outruns 5s and would false-fail every run asunknown). Wired intocli.tsdispatch +CLI_ONLYset. A zero-tokenembedding_configprobe runs FIRST, before any chat/expansion probes spend money:probeEmbeddingConfig()readsgetEmbeddingModel()+getEmbeddingDimensions()and (for Voyage flexible-dim models) checksisValidVoyageOutputDim(dims)againstVOYAGE_VALID_OUTPUT_DIMS.ProbeStatusvariant'config'+ optionalfix?: stringonProbeResultsurface a paste-readygbrain config set ...line in human + JSON output; touchpoint label'embedding_config'joins'chat'and'expansion'. -
src/core/init-embed-check.ts— embedding-key validation atgbrain init.runInitEmbedCheck(opts)runs a config-onlydiagnoseEmbedding(catches a missing key for ANY provider) plus a best-effortliveTestEmbed(1-tokengateway.embed(['probe'], {inputType:'query', abortSignal}), 5sAbortControllertimeout, never throws — catches an invalid/expired key). Loud warning to stderr; init still exits 0 (--no-embeddingis the deferred-setup escape;--skip-embed-check/GBRAIN_INIT_SKIP_EMBED_CHECK=1skip the check). Builds the effective env (process.env+ every file-plane provider keybuildGatewayConfigfolds — openai/anthropic/voyage/zeroentropy/dashscope/google — fromloadConfigFileOnly()+opts.apiKey) and configures the gateway viabuildGatewayConfigbefore diagnose/probe, so the check sees the same keys AND provider base URLs runtime will (no false "missing key" for config.json-keyed users; the probe hits the right endpoint). Init-specific warning text names--no-embedding/--skip-embed-check, not the sync-flavored--no-embed. Wired intoinitPGLite+initPostgresinsrc/commands/init.ts, with the result added to the--jsonenvelope asembedding_check {ok, reason?, live_ok?}. Pinned bytest/init-embed-check.test.ts(hermetic via the gateway embed-transport seam +withEnv). -
src/commands/jobs.ts:refreshGatewayForJob+src/core/ai/gateway.ts:refreshGatewayEnvFromFilePlane— long-lived-worker staleness boundary, three tiers: (1) DB-plane model config re-resolves per gateway-refresh job (reconfigureGatewayWithEngine); (2) FILE-plane config (~/.gbrain/config.json, incl. provider API keys) re-folds per job via the env-ONLYrefreshGatewayEnvFromFilePlane— never a fullconfigureGateway(buildGatewayConfig(loadConfig())), which would clobber DB-plane-merged fields (base_urls, chat options) with file-plane-only values; (3) true process env vars are fixed at worker start and need a restart.gbrain config set *_api_keywrites the DB plane, whichloadConfigWithEngine()deliberately never merges for key fields — those writes do NOT reach workers (TODO filed to reroute them to the file plane).facts-absorbsits inGATEWAY_REFRESH_JOB_NAMES; its handler converts execution-timechat_unavailablein a KEYED worker into a typed retryable failure (factsAbsorbShouldRetry) while a keyless worker completes the job as a calm skip. Pinned bytest/jobs-gateway-refresh.serial.test.ts. -
src/core/ai/openai-latest.ts— latest-model discovery: OpenAI defaults are NEVER pinned.refreshLatestOpenAIModels()(called fromreconfigureGatewayWithEngine, TTL 24h, 3s-bounded, fail-open,GBRAIN_MODEL_DISCOVERY=off|0kill switch — the test preload sets it) fetches the account's ownGET /v1/models, ranks ids through a conservative grammar (parseOpenAIChatId: bare family aliasesgpt-N.M+ known tier suffixes sol/pro/terra/luna/nano/mini; dated snapshots,-chatInstant-class, realtime/image and UNKNOWN future suffixes are ignored — rot degrades to newest-known-shape, never a wrong pick), and maps the newest family onto the tier ladder (utility→cheap, reasoning/subagent→mid, deep→top). ONLY ids with a canonical pricing row are eligible (rankOpenAIChatModels's priced filter): BudgetTracker fails closed (no_pricing) under a cost cap, so an unpriced discovered default would brick budget-capped backfills — a newer-unpriced family warns once naming the missingmodel-pricing.tsrow. Result lands in<configDir()>/model-cache.json(atomic tmp+rename);latestOpenAITiers()is the SYNC read (stale cache beats static fallback — TTL gates refresh, never use) consumed byresolveTierDefault's openai entry and the dynamicgptalias. Pinned bytest/openai-latest.serial.test.ts. -
src/core/ai/provider-env.ts—mergedProviderEnv(cfg, env): THE canonical provider-key/env fold. Maps file-plane config keys (openai/anthropic/zeroentropy/openrouter/voyage/dashscope/deepseek/litellm/together/google/azure_openai + azure endpoint/deployment/entra) to the env names recipes read, merges env on top (env wins ONLY for keys carrying a real value — '' and undefined dropped), then applies theGEMINI_API_KEY → GOOGLE_GENERATIVE_AI_API_KEYalias (canonical env name > alias > config fallback). Three consumers:buildGatewayConfig(gateway env),detectCapabilities(capability probe),resolveTierDefault/resolveEffectiveChatModel(key-aware model resolution) — so gateway env and the capability probe cannot drift. -
src/core/ai/build-gateway-config.ts—buildGatewayConfig(c: GBrainConfig): AIGatewayConfig, re-exported bysrc/cli.ts. Lets core modules (init-embed-check.ts) reuse it without importing the CLI entrypoint. Delegates the file-plane API-key fold + env merge tomergedProviderEnv(src/core/ai/provider-env.ts); keeps ownership of threading local-server*_BASE_URLenv vars into base_urls.process.envwins EXCEPT empty-string / undefined values are dropped before the merge, so an injected emptyANTHROPIC_API_KEY=''(Claude Code neuters subprocess LLM calls this way) can't clobber a valid config-plane key;'0'/'false'are preserved. Pinned bytest/ai/build-gateway-config.test.ts. -
src/core/skill-trigger-index.ts— Shared loader that unions per-skill SKILL.md frontmattertriggers:with curated RESOLVER.md / AGENTS.md rows fromskillsDirAND the parent dir (preserves the OpenClaw workspace-root layout). UNION semantics: explicit RESOLVER.md rows ADD to frontmatter triggers (don't replace). Dedup keyed on(skillPath, trigger.trim().toLowerCase()). Three consumers fold through this primitive —checkResolvable,runRoutingEvalCli,mounts-cache.composeResolvers— so fixing frontmatter reaches all of them. ExportsloadSkillTriggerIndex(skillsDir): SkillTriggerEntry[],entriesToResolverContent(entries): string(synthesizes a markdown-table resolver string forrunRoutingEval's string-content API),findPrimaryResolverPath(skillsDir): string | null, theFRONTMATTER_SECTIONconstant, and_resetWarnedSkillsForTests. Skip rules: non-directory entries (a symlink dirent is followed viastatSync—readdirSyncdirents report the link's OWN type without following, so skill dirs symlinked in from a shared store, e.g.~/.agents/skills/<name>, still load),_*/.*prefixes,conventions/+migrations/subdirs, skills with noSKILL.md(deprecatedinstall/graceful-skipped), notriggers:array, or malformed YAML (warn-once + skip). ReusesparseSkillFrontmatterfromsrc/core/skill-frontmatter.ts, including block and wrapped flow-sequence arrays. Pinned bytest/skill-trigger-index.test.ts(18 hermetic cases). CI gatebun run check:resolver(=bun src/cli.ts check-resolvable --strict --skills-dir skills/) wired intobun run verify. -
src/core/skill-catalog.ts— host-repo skill catalog backing the MCPlist_skills/get_skillops. Lets a thin MCP client (Codex desktop, Claude Code, Claude Cowork, Perplexity) DISCOVER + FOLLOW the agent repo's fat-markdown skills overgbrain serve— a skill is prose, so "using" one = fetching its body then calling the gbrain MCP tools the server already exposes. Read-scope, NOT localOnly (defensible only via the full mitigation stack): (1) publish gate —assertPublishEnabled(ctx, publishSkills); remote callers requiremcp.publish_skills === true, default-OFF so an upgrade never silently grants existing read tokens host-skill read; local callers (ctx.remote === false) always pass. (2) path confinement —assertSkillNameShaperejects separators/../null/space before any FS access; the clientnameis a manifest LOOKUP KEY (vialoadOrDeriveManifest), never a raw path segment;confineManifestPathdoes realpath + relative-containment +SKILL.md-regular-file check on EVERY entry (defeats poisoned manifest.jsonpath, symlink/..escape). (3) frontmatter allowlist —GetSkillResult.frontmatterprojects a safe subset; privatewrites_to+sourcesdropped. (4) prose-only + 256KB cap (MAX_SKILL_MD_BYTES, envGBRAIN_MAX_SKILL_MD_BYTES), size-checked twice (statSync + UTF-8 byte length). (5) no install_path serve for remote — remote callers useautoDetectSkillsDir(no install-path tier) so a hosted gbrain with no agent repo returnsstorage_error; local callers useautoDetectSkillsDirReadOnly. (6) MCP rate-limiter caps call rate. Config reads honor BOTH planes:readMcpPublishSkills/readMcpSkillsDirprefer the DB plane (engine.getConfig) over the file plane (ctx.config.mcp). Tool-honesty:crossReferenceTools(declared, ctx)splits a skill's declaredtools:intousable_toolsvsunavailable_tools;buildSkillCatalog'sinstructionsenvelope (SKILL_CATALOG_INSTRUCTIONS) carries the "these are prose, follow-then-call-tools" protocol. Skills are host-filesystem repo-global —sourceScopeOpts(ctx)/ctx.brainIddeliberately do NOT apply.buildSkillCatalogis resilient (one malformed/escaping skill is skipped, never throws). Config keys insrc/core/config.ts:GBrainConfig.mcp?: { publish_skills?, skills_dir? }+KNOWN_CONFIG_KEYSentriesmcp.publish_skills/mcp.publish_skills_prompted/mcp.skills_dir+mcp.prefix inKNOWN_CONFIG_KEY_PREFIXES.src/commands/init.tswritesconfig.mcp = { publish_skills: true, ... }for new installs (existing config wins on re-init).src/commands/upgrade.ts:runPostUpgradeadds a one-time consent prompt (gated bymcp.publish_skills_prompted; existing installs stay OFF until owner opts in). Three ops register insrc/core/ops/skills-catalog.ts(spread into theoperations.tsfaçade):list_skillswith optionalsectionfilter +cliHints:{name:'skills'};get_skilltakingname(+source_idfor brain-resident packs) +cliHints:{name:'skill', positional:['name']};list_brain_skillpack. They dynamically import this module to avoid the import cycle (skill-catalog statically imports theoperationsarray). Descriptions insrc/core/operations-descriptions.ts(LIST_SKILLS_DESCRIPTION,GET_SKILL_DESCRIPTION,SKILL_CATALOG_INSTRUCTIONS,SKILL_CLIENT_GUIDANCE), pinned bytest/operations-descriptions.test.ts. CLI:gbrain skills/gbrain skill <name>. Pinned bytest/skill-catalog.test.ts,test/skill-catalog-security.test.ts(path-confinement / poisoned-manifest / symlink-escape),test/skill-catalog-transports.test.ts(publish-gate + remote-vs-local) overtest/fixtures/skill-catalog/. -
src/core/check-resolvable.ts— Resolver validation: reachability, MECE overlap, DRY checks, structured fix objects.CROSS_CUTTING_PATTERNS.conventionsis an array (notability gate acceptsconventions/quality.mdand_brain-filing-rules.md).extractTriggers()delegates to the shared SKILL.md parser, so MECE gap detection and the trigger index agree on block lists, single-line flow sequences, wrapped flow sequences, and CRLF input.extractDelegationTargets()parses> **Convention:**,> **Filing rule:**, and inline backtick references. DRY suppression is proximity-based viaDRY_PROXIMITY_LINES = 40.parseResolverEntriesaccepts BOTH the markdown table AND a compact list format (- **skill-name**: trigger1 | trigger2 | trigger3or- skill-name: trigger1 | trigger2); shapes can mix in one file, folded by the multi-resolver merge. Skill name MUST be kebab-lowercase (regex[a-z][a-z0-9-]+) so prose bullets like- **Note**:/- **Convention**:/- **TODO**:don't false-match as skill rows.skillPathis ALWAYS derived asskills/<name>/SKILL.md: an optional→ \skills/path`(or ASCII->) suffix is stripped from the trigger but NOT honored as the path — two consumers (routing-eval.ts:skillSlugFromPath, the manifest lookup) assume the convention; use the table format for non-conventional paths. Multi-trigger rows fan out to one entry per trigger sharing the sameskillPath;checkResolvablededupes so the reachability count counts each skill once. Pinned bytest/check-resolvable.test.ts(resolver shapes plus trigger array syntax cases) +test/check-resolvable-openclaw-compact.test.ts(8 cases overtest/fixtures/openclaw-compact-resolver/andtest/fixtures/openclaw-mixed-merge/). Tutorial:docs/guides/scaling-skills.md(three-tier scaling: ~300-skill agent to ~4K tokens/turn from ~25K).checkResolvable(skillsDir, opts?)takes an optionalskillsDirSource(the detection tier fromautoDetectSkillsDir, threaded by both callers: doctor.ts and the check-resolvable command; an explicit--skills-dirpassesnull). Anunreachableissue downgrades fromerrortowarningonly when ALL hold: the directory was found via the ungatedcwd_walk_uptier, no resolver file contributes rows (noRESOLVER.md; a genericAGENTS.mdwith zero table rows counts as absent), unreachable skills outnumber reachable ones, and nomanifest.jsonexists on disk. This keeps a foreign tool'sskills/dir walked up to from cwd from being scored as a broken gbrain skillpack, while a real skillpack (RESOLVER.md present, higher-confidence tier, dense trigger coverage, or any manifest.json, even a corrupted one) stays strict. A foreign dir with zerotriggers:and no resolver file still hard-fails via themissing_file` branch (separate follow-up). -
src/core/repo-root.ts— SharedfindRepoRoot(startDir?): walks up fromstartDir(defaultprocess.cwd()) looking forskills/RESOLVER.md. Zero-dependency, imported bydoctor.tsandcheck-resolvable.ts; parameterizedstartDirmakes tests hermetic. Read-path / write-path split:autoDetectSkillsDir(shared, read+write-safe) has tier-0$GBRAIN_SKILLS_DIRoperator override ahead of the 4-tier chain.autoDetectSkillsDirReadOnlywraps it with a tier-5 install-path fallback that walks up fromfileURLToPath(import.meta.url)and gates onisGbrainRepoRootso unrelated repos can't false-positive. Read-path callers (doctor,check-resolvable,routing-eval) use the read-only variant; write-path callers (skillpack install,skillify scaffold,post-install-advisory) stay on the shared function so install-from-~can't retarget the bundled gbrainskills/instead of the user's workspace.SkillsDirSourcevariants'env_explicit','install_path';AUTO_DETECT_HINT_READ_ONLYdocuments the extra tier. The--fixsafety gate indoctor.ts+check-resolvable.tsrefuses auto-repair whendetected.source === 'install_path'. -
src/core/skills-integrity.ts— Tamper-evidence manifest for the bundledskills/tree; NOT a signature system. Pure functions overnode:cryptosha256:computeSkillsManifest(dir)(recursive, sorted '/'-relative paths, excludes the manifest itself, skips symlinks),renderSkillsManifest(dir)(2-space JSON + trailing newline, deterministic),verifySkillsManifest(dir, manifest)→{modified, missing, extra}. Committed manifest lives atskills/skills.lock.json(SKILLS_MANIFEST_FILENAME); regenerate viabun run scripts/generate-skills-manifest.ts. Consumers: the warn-onlyskills_manifest_integritydoctor check insrc/commands/doctor.ts(ok/skip when no manifest is present — user workspaces and compiled-binary installs are not drift) and the CI freshness guardscripts/check-skills-manifest-fresh.sh(bun run check:skills-manifest, inbun run verify). Pinned bytest/skills-integrity.test.ts. -
src/commands/check-resolvable.ts— Standalone CLI wrapper overcheckResolvable(). ExportsparseFlags,resolveSkillsDir,DEFERRED,runCheckResolvable. Exit rule: 1 on any issue (warnings OR errors), stricter than doctor'sokflag. Stable JSON envelope{ok, skillsDir, report, autoFix, deferred, error, message}— same shape on success and error.--fixrunsautoFixDryViolationsBEFOREcheckResolvable(same ordering as doctor).scripts/skillify-check.tssubprocess-callsgbrain check-resolvable --json(cached per process) and fails loud on binary-missing. AGENTS.md workspaces resolve natively (seesrc/core/resolver-filenames.ts).DEFERRED[]is empty. Resolver lookup is the multi-file merge insrc/core/check-resolvable.ts— entries collected from everyRESOLVER.md/AGENTS.mdacross the skills dir AND its parent, deduped byskillPath(first occurrence wins). UsesautoDetectSkillsDirReadOnlysocd ~ && gbrain check-resolvablefinds bundled skills via the install-path fallback;--fixcarries the same install-path safety gate (refuses to write whendetected.source === 'install_path'). -
src/core/resolver-filenames.ts— central list of accepted routing filenames (RESOLVER.md,AGENTS.md). Shared byfindRepoRoot,check-resolvable, and skillpack install so every code path walks the same fallback chain. -
src/commands/skillify.ts+src/core/skillify/{generator,templates}.ts—gbrain skillify scaffold <name>creates all stubs for a new skill: SKILL.md, script, tests, routing-eval.jsonl, resolver entry, filing-rules pointer.gbrain skillify check <script>runs the 10-step checklist (LLM evals, routing evals, check-resolvable gate, filing audit) against a candidate skill before it lands. -
src/commands/skillify-check.ts—gbrain skillpack-checkagent-readable health report. Exit 0/1/2 for CI gating; JSON for debugging. Wrapscheck-resolvable --json,doctor --json, and migration ledger into one payload. Required item 12 (brain_first_compliance) callsanalyzeSkillBrainFirst()on the candidate SKILL.md; exits 1 when the verdict ismissing_brain_first(external-lookup pattern present, no callout, nobrain_first: exempt). The scaffold path insrc/core/skillify/templates.tspre-inserts the canonical Convention callout into new SKILL.md files so freshly-scaffolded skills pass item 12. -
src/commands/book-mirror.ts—gbrain book-mirror --chapters-dir <path> --slug <slug> [flags]. Submits N read-only subagent jobs (one per chapter;allowed_tools: ['get_page', 'search']), waits for all viawaitForCompletion, reads each child'sjob.result, assembles two-column markdown CLI-side, writes a single operator-trustput_pagetomedia/books/<slug>-personalized.md. Trust narrowing happens at the tool-allowlist layer (subagents can't call put_page) so untrusted EPUB content can't prompt-inject any people page. Cost-estimate prompt before launching; refuses to spend in non-TTY without--yes. Per-chapter idempotency keys (book-mirror:<slug>:ch-<N>) for retry-friendly re-runs. Partial-failure: assembles completed chapters + a## Failed chapterssection. Pinned bytest/book-mirror.test.ts(9 cases). -
src/commands/skillpack.ts+src/core/skillpack/{bundle,scaffold,reference,migrate-fence,scrub-legacy,harvest,harvest-lint,copy,apply-hunks,diff-text,installer}.ts—install/uninstallare not supported (they exit non-zero with a hint to the replacement). Surface:scaffold(one-time additive copy viacopyArtifactsincopy.ts; refuses to overwrite; partial-state fills missing paired sources declared in SKILL.md frontmattersources:),reference(read-only diff lens +--apply-clean-hunkstwo-way auto-apply via pure-JS unified-diff parser/applier inapply-hunks.ts+diff-text.ts),migrate-fence(one-shot strip of legacy fence; cumulative-slugs receipt → row-parsing fallback; preserves rows verbatim as user-owned routing),scrub-legacy-fence-rows(opt-in row cleanup with skill-present + non-empty-triggers gate),harvest(host→gbrain inverse with symlink-reject + canonical-path containment via avalidateUploadPath-style gate + default-on privacy linter inharvest-lint.tsagainst~/.gbrain/harvest-private-patterns.txtplus built-in a built-in fork-name pattern + email + Slack-channel patterns; rollback on match). Paired-source declarations live in each SKILL.md's frontmattersources:array (validated byloadSkillSourcesinbundle.ts).autoDetectSkillsDir(insrc/core/repo-root.ts) has acwd_walk_uptier ahead of~/.openclaw/workspace($OPENCLAW_WORKSPACEprecedence preserved).gbrain skillpack check --strictexits non-zero on drift (CI gate); top-levelgbrain skillpack-checkkeeps exit-1-on-issues for cron. Companion editorial skillskills/skillpack-harvest/SKILL.mddrives the genericization checklist. Doc:docs/guides/skillpacks-as-scaffolding.md. Test coverage acrosstest/skillpack-{copy,scaffold,reference,reference-apply,apply-hunks,migrate-fence,scrub-legacy,harvest,harvest-lint,frontmatter-sources}.test.ts+ 9-case E2E intest/e2e/skillpack-flow.test.ts.installer.ts+test/skillpack-install.test.tsremain becausegbrain skillpack diffusesdiffSkillfrom there. -
src/core/skillpack/{personas,bridge-state,harness-bridge}.ts+src/commands/skillpack/{shared,scaffold,reference,harness}.ts— the harness skill bridge.src/commands/skillpack.tsis a peeled FAÇADE (dispatch + HELP_TOP + the install/uninstall removal errors; module-size ratchet); per-subcommand handlers live insrc/commands/skillpack/and the flag registry scans the dir via the façade'sfacadeExpansionentry — spell foreign CLI flags dash-less in these modules' comments/strings (prose-bleed class).scaffold --harness <claude-code|openclaw|codex|opencode>installs a persona-curated set into the harness's native skills dir: personas live inskills/plugin-lanes.json#personas(personas.tsis the SINGLE validation implementation —scripts/generate-plugin-tree.tsimports it; membership ⊆ the plugin lane set, so lane-excluded slugs are refused with their recorded reason); slugs path-resolve viabundle.ts'suniverse:'manifest'(the 8 lane additions are all absent from openclaw.plugin.json#skills). Safety: frontmatter fail-loud gate pre-write (a frontmatterless SKILL.md bricks Codex sessions), target-side confinement inassertTargetsConfined(deepest-existing-ancestor realpath — copy.ts confines SOURCES only), refuse-overwrite, and written-only ownership inbridge-state.ts(~/.gbrain/skillpack-bridge-state.json, schemagbrain-skillpack-bridge-v1, fail-open load, install-time sha256 per written file — never touchesskillpack-state.json, whose loader drops unknown keys).--stubrenders cold-pull pointers (frontmatter verbatim +<!-- gbrain-skill-stub v1 -->marker; ships the shared-dep closure AND sibling aux files —get_skillserves only the SKILL.md body) behind a three-check preflight insrc/commands/skillpack/harness.ts(publish gate dual-plane, per-slug servability viaverifySlugsServable, best-effort local surface warn — the module deliberately does NOT importsrc/mcp/surface.ts, whose comments would bleed serve flags into the allowlist;get_skill ∉ STARTER_OPSis pinned by a contract test).reference --harnessis a stub-aware three-way lens (local_editvsupstream_driftvsunknown— no install-time hash, never auto-applied — via the hash ledger; marker fallback survives state loss both directions) +--apply-clean-hunks(refuses stub files);remove --harnessdeletes ledger-owned files only;skillpack statusrenders an installed-bridges section fromcollectBridgesStatus. openclaw delegates torunScaffold({skillSlugs}); codex/opencode require an explicit dest until observation runs. Claude-code dirs come fromhost-specs.ts(claudeUserSkillsDir/claudeProjectSkillsDir, HOME-env-first). Tests:test/skillpack-{personas,bridge-state,harness-bridge,reference-harness,scaffold-harness}.test.ts. Doc: the harness-bridge section ofdocs/guides/skillpacks-as-scaffolding.md. -
src/core/skillpack/{manifest-v1,tarball,state,remote-source,trust-prompt,bootstrap-display,scaffold-third-party,registry-schema,registry-client,rubric,doctor,init-scaffold,pack-publish,endorse,audit}.ts+examples/skillpack-reference/+docs/skillpack-anatomy.md+scripts/build-skillpack-anatomy.ts— third-party skillpack ecosystem.gbrain skillpack scaffold <owner/repo|https-url|./tgz|./local-dir>resolves the spec viaclassifySpec, fetches through SSRF-hardenedgit-remote.ts(git) or extracts the tarball into~/.gbrain/skillpack-cache/<host>/<owner>/<repo>/<sha>/, validatesskillpack.json(api_versiongbrain-skillpack-v1), checksgbrain_min_version, surfaces a TOFU first-install identity-confirm prompt (author + source + pinned commit + tarball SHA + tier; non-TTY requires--trust), records the pin in machine-owned~/.gbrain/skillpack-state.json(schemagbrain-skillpack-state-v1, atomic.tmp + rename,isAlreadyTrustedskips re-prompt on author+pin match), runs throughenumerateScaffoldEntries→copyArtifacts(one-time additive, refuses to overwrite), then DISPLAYSrunbooks/bootstrap.mdWITHOUT executing (deliberately does not auto-execute). Registry catalog atgarrytan/gbrain-skillpack-registrysplit intoregistry.json(PR-able,gbrain-registry-v1) +endorsements.json(maintainer-only overlay,gbrain-endorsements-v1);effectiveTiermerges.registry-client.tsfetches both viaIf-None-Matchetag with 1h soft-TTL + stale-fallback (originsfresh_fetch | cache_warm | cache_soft_stale | cache_hard_stale); hard-fail only on no-cache + no-network. CLI:gbrain skillpack {search,info,registry,doctor,init,pack,endorse}. Doctor walksSKILLPACK_RUBRIC_V1(10 binary dimensions: 5 required CORE — manifest_valid, skills_have_skill_md, routing_evals_present ≥5 intents, skills_have_unique_triggers MECE, changelog_present_and_current — and 5 quality BADGES — unit_tests_present, e2e_tests_present, llm_eval_present ≥3 cases, bootstrap_runbook_present, license_present); tier eligibility:endorsedneeds all 10,communityneeds core + ≥3 badges,experimentalneeds core only,blockedwhen any core fails.--quick~5s structural sweep;--fix --yesauto-scaffoldsauto_fixable: truedimensions and refuses to overwrite files whose mtime is newer thanskillpack.json.gbrain skillpack init <name>lands 11 files (skillpack.json, SKILL.md, routing-eval.jsonl, test/example.test.ts, e2e/example.e2e.test.ts, evals/example.judge.json, runbooks/{bootstrap,uninstall,upgrade-template}.md, CHANGELOG, README, LICENSE); freshly-init'd scores 10/10;--minimalskips test/e2e/evals.gbrain skillpack packpacks a deterministic tarball via GNU tar (--sort=name --mtime=@0 --owner=0 --group=0 --numeric-owner+GZIP=-n+TZ=UTC); refuses ontier_eligibility === 'blocked'. Extract caps (5000 files / 100MB total / 1MB per file / 255-char paths / 100:1 compression ratio); rejects symlinks/hardlinks/devices/FIFOs.gbrain skillpack endorse <name> [--tier ...] [--push] [--dry-run]runs in a clone of the registry repo: validates the pack inregistry.json, mutatesendorsements.jsonvia pureapplyEndorsement, stable-key-orders the write, commitsendorse: <name> -> <tier>, optionally pushes. JSONL audit at~/.gbrain/audit/skillpack-YYYY-Www.jsonl(ISO-week rotated, honorsGBRAIN_AUDIT_DIR).examples/skillpack-reference/is a 10/10 reference pack pinned bytest/e2e/skillpack-third-party.test.ts.docs/skillpack-anatomy.mdauto-generated viascripts/build-skillpack-anatomy.ts(--checkfor CI drift). CLI dispatch insrc/commands/skillpack.tsdisambiguates third-party (contains/,://,.tgz) from bundled-skill kebab; kebab routes bundled-first, registry-fallback. Tests:test/skillpack-{manifest-v1,tarball,state,remote-source,trust-prompt,registry-schema,registry-client,rubric,doctor,init-scaffold,pack-publish,endorse,audit,scaffold-third-party}.test.ts+test/e2e/skillpack-third-party.test.ts. Spec atdocs/designs/SKILLPACK_REGISTRY_V1_SPEC.md. -
src/core/archive-crawler-config.ts— safety gate for thearchive-crawlerskill. Refuses to run unlessarchive-crawler.scan_paths:is explicitly set in the brain repo'sgbrain.yml. Mirrors the storage-config.ts parsing pattern (sibling file, separate concern from storage tiering).loadArchiveCrawlerConfig(repoPath)throwsArchiveCrawlerConfigError(missing_section | empty_scan_paths | invalid_path | parse_error).normalizeAndValidateArchiveCrawlerConfigrejects relative paths and..traversal;~is expanded; paths are stored resolved and terminated with the PLATFORM separator (path.sep) so error output reads natively on each OS.isPathAllowed(candidate, config)is the runtime per-file gate (scan_paths prefix-match with directory-boundary correctness; deny_paths overrides). Candidate, scan_paths and deny_paths all funnel through the privatetoComparablePrefix()before the prefix test — on Windows it folds\→/and lowercases (NTFS is case-insensitive, so a deny_path spelledPrivatemust still matchprivate, else the gate fails OPEN); on POSIX it is identity apart from the trailing separator, deliberately NOT folding, since\is a legal filename character and paths are case-sensitive. Storing a native separator while appending a hardcoded/is the mixed-separator bug that madeisPathAlloweddeny every real path on Windows; the two functions must stay symmetric or the prefix test is meaningless. Pinned bytest/archive-crawler-config.test.ts(26 cases, platform-selected fixtures +it.if-gated win32/POSIX comparison semantics). -
test/helpers/tty-harness.ts+scripts/dx-explore.ts— the repo's single real-PTY layer, on pureBun.spawn({terminal:})(Bun 1.3.10+; engines.bun pin in package.json; no node-pty).launchTtyspawns any CLI under a true pseudo-terminal with the hermetic env contract fromtest/helpers/agent-harness.ts(hermeticChildEnv;dropEnvstrips pass-through auth keys), records timestamped output frames, and exposeswaitFor/waitForAny/mark/sendKey/waitForQuiet/waitForExit/close. Lifecycle rule: onlyclose()clears the wall-clock kill timer — always call it in afinally. Pure helpers (stripAnsi,computeStalls,renderStallsReport,parseDriveCommand,buildClaudeTuiSeed) are unit-tested intest/tty-harness.test.ts; that file's live-PTY smokes aredescribe.skipIf(!ptySupported())-gated. Transcript writes are structurally redacted:redactSecrets(secrets ≥MIN_REDACT_SECRET_LEN→[REDACTED:<name>]) runs at every write site,coalesceSecretStraddlesmerges frames so a secret split across a frame boundary can't bypass redaction, andsaveTranscripttakes an explicitredactmap (seam — dx-explore builds it, tty-harness stays import-free of it).scripts/dx-explore.tsis the DX-exploration driver built on it — a developer instrument, not a test: nothing asserts, transcripts land gitignored under.context/dx-runs/(doc:docs/guides/bootstrap.md). The interactivegbrain initpickers are asserted for real intest/init-picker-pty.serial.test.ts(serial lane, so it runs in required CI). -
src/core/skillpack/{init-brain-pack,brain-pack-advisory,brain-pack-lint,brain-resident-locate,nag-state}.ts— brain-resident skillpacks.manifest-v1.tsgains optionalbrain_resident+schema_pack(additive).runInitBrainPackscaffolds a pack beside brain content (brain_resident:true, exactgbrain_min_version, 5-section machine-parseable README);applyWritePlanis factored out ofinit-scaffold.tsfor the shared refuse-overwrite loop.brain-pack-lint.lintBrainPackToolsvalidates each skill'stools:against the serving op set (E6 version-skew). Topology A:src/commands/sources.tsrunAddprintsbrain-pack-advisoryto stderr afteropsAddSource, fail-open;nag-state.ts(~/.gbrain/skillpack-nag-state.json) keys declines by(source-repo brain_id, source_id, pack_name)with puredecideNagAction(first/reminder/version-bump/ceiling) — declines count ONLY on CLI-interactive displays. Topology B:brain-resident-locate.loadResidentPacksForServer(source-scoped viasourceScopeOpts) backs thelist_brain_skillpackop;getResidentSkillDetailbacksget_skillsource_id;scaffold_specis the git source, never a server FS path. Tests:test/skillpack-{init-brain-pack,nag-state,brain-resident-locate}.test.ts+ the brain-resident cases intest/skillpack-manifest-v1.test.ts. -
src/core/advisor/{types,run,render,recommended-set,history,apply,collect-*}.ts+src/commands/advisor.ts—gbrain advisor: read-only ranked actions from brain state.run.runAdvisorexecutes the hardcoded COLLECTORS array (version [cache-only], migration, schema-pack, stalled-jobs [absent-table tolerant], usage-shape, setup-smells, uninstalled-brain-pack, uninstalled-bundled, chronicle, mcp-client-fit), each in its own try/catch;collect-mcp-client-fit.ts(E3) flags full-surface MCP clients whose 30d usage fits STARTER_OPS (exactrescope-client --surface starterfix; ≥10-call threshold; automation-shaped clients excluded per D12) plus STARTER_OPS drift (top-used ops missing; starter members unused 90d) via the sharedsrc/core/mcp-usage.tsreader — starter membership is judged against the exportedALWAYS_INCLUDED_STARTER_OPS(surface.ts) so the always-included lane never reads as unused, and the missing-from-starter arm excludeslocalOnlyops (never proposable for a network surface, mirroring derive-starter-ops) — REMOTE runs redact client identifiers to aggregate counts (amendment 29), and its dismiss/snooze rides the nag-state engine with its own~/.gbrain/advisor-usage-nag-state.json(local runs only);rankFindingsorders critical>warn>info then collector order, caps the info tail, and dropsworkspace_dependentfindings whenremote(A1).render.tsis the shared=-bar renderer used by the advisor ANDpost-install-advisory.ts(generalized to a single current-staterecommended-set.RECOMMENDED,install→scaffold).history.tsappends bounded~/.gbrain/advisor-history.jsonl(no DB migration) for since-last-run deltas; local-only.apply.resolveApplyTargetis the allowlist+injection guard forcommands/advisor.ts --apply <id>(structured argv, never a shell; local-only). Theadvisorop (operations.ts) is read-scoped, NOT localOnly, gated bymcp.publish_advisor(config.ts; default off) and strictly read-only on remote. CLI wired incli.ts(CLI_ONLY+ dispatch). Bundled skillskills/gbrain-advisor/+ weekly cron recipe. Tests:test/advisor-{core,apply,op-gate,ranking-eval}.test.ts. -
src/core/chronicle/{eligibility,config,backstop,extract-events,ontology,narrative}.ts+src/eval/chronicle/harness.ts+src/commands/eval-chronicle.ts— Life Chronicle: the temporal spine.eligibility.isChronicleEligibledecides which pages auto-emit events (meeting/conversation/calendar-event + directory rescue; diary and event pages NEVER eligible — privacy + anti-loop).backstop.runChronicleBackstopis the put_page hook body (fires ONLY onstatus==='imported'+ the auto-link trust gate + the default-OFFauto_chronicleflag; enqueues achronicle_extractminion job — LLM never runs on the write path).extract-events.runChronicleExtractis the job body: deterministic when/who, injectable judge (default = chat gateway; output cap 4000 tokens by default, operator overridechronicle.judge_max_tokens), an ALL-or-nothing parse barrier (isValidProposalrequires a real parseable date — a malformed batch writes NOTHING), then content-addressedlife/events/pages + atimeline_entriesprojection viaengine.upsertEventProjection(dedup(event_page_id, date); idempotent re-runs). An unusable judge response is never recorded asno_events: astopReason: 'length'truncation or a no-JSON-array response (parseJudgeJsonreturnsnullon parse failure;[]only for a legitimate empty array) surfaces asstatus: 'skipped'with reasonjudge_truncated/judge_parse_failed.ontology.tscarries the deterministic pieces of the bi-temporal per-entity ontology that RIDES THEfactsTABLE (migration v122 addsdimension/value/value_hash/dim_status):valueHash(normalized, timestamp-free → crash-retry idempotent),normalizeDimension(seed alias lexicon),isNovelDimension(novel → quarantined, excluded from resolution/context until confirmed). The engine methods (mergeOntologyFact— corroborate on same value, forward-supersede viavalid_until+superseded_byon a new value, backdated conflicts kept + flagged;getOntologywith--asofvalid-time travel;discoverOntologyDimensions;findOntologyConflicts— currently-open rows only) live in BOTH engines; both engines are on the R8valid_untilwrite allow-list (engine-layer,dimension IS NOT NULLrows only). Chronicle reads (getTimelineForDate/getSince/getLastSeen/getOnThisDay) JOIN the depth page (deleted_at IS NULL), hide soft-deleted event projections at READ time, and order by eventeffective_datefor intra-day sequence. Ops:chronicle_day/chronicle_since/chronicle_last_seen/chronicle_on_this_day/ontology_*/volunteer_chronicle(agent orientation viasrc/core/context/chronicle-context.ts)/chronicle_backfill(admin, localOnly). Diary privacy: fail-closed forctx.remote !== falsecallers — diary-sourced ontology + conflict values are redacted, the four timeline read ops (chronicle_day/chronicle_since/chronicle_on_this_day+volunteer_chronicle'srecent_timeline) drop rows whose depth page, event page, orsourceprovenance lives underlife/diary/, andchronicle_last_seenanswers the never-seen shape (last_date/last_event_slug/days_agoall null) when the latest sighting is diary-sourced or a diary page is queried as the entity. Page visibility rides the same policy:ontology_get/ontology_conflicts/volunteer_chronicle's ontologies resolve throughreadPolicyOpts, and both engines'getOntology/findOntologyConflictsapplyprivateProvenanceFilterFragment(an observation whose provenance page, looked up in the fact's own source, isvisibility: privateis dropped BEFORE per-dimension resolution and before the conflict HAVING, so an untrusted caller resolves the newest value they may see rather than a private value or a hole). The chronicle reads' event-page LEFT JOIN carries the caller's source scope in BOTH engines, so a scoped read can't surface another source's event pages through the join. Search:applyChronicleTypeBoostinsearch/hybrid.ts(bounded [1.0,1.25], fires only inside therecency !== 'off'post-fusion branch → non-temporal search bit-for-bit unchanged). Advisor collectorcollect-chronicle.ts(conflicts + coverage gap); doctorchronicle_projection_health(BRAIN category). Eval:gbrain eval chronicle— deterministic, own in-memory PGLite, 6 gold tasks (day order, last-seen, supersession, asof, conflict, isolation), exit 0 iff 6/6. Tests:test/chronicle-*.test.ts,test/eval-chronicle.test.ts. -
src/core/skill-manifest.ts— parser forskill-manifest.jsonrecords. Used by skillpack installer to detect drift between the shipped bundle and the user's local edits, so updates merge instead of overwriting. -
src/commands/routing-eval.ts+src/core/routing-eval.ts—gbrain routing-evalcatches user phrasings that route to the wrong skill. Readsskills/<name>/routing-eval.jsonlfixtures ({intent, expected_skill, ambiguous_with?}). Structural layer runs incheck-resolvableby default (zero API cost).--llmis a placeholder for a future LLM tie-break layer; today it emits a stderr notice and runs structural only. UsesautoDetectSkillsDirReadOnlyand the same multi-file resolver merge ascheck-resolvable, so on OpenClaw layouts (skills/RESOLVER.md+../AGENTS.md) all three commands see the same trigger index. RESOLVER.md rows carry the full frontmattertriggers:arrays so the structural matcher sees realistic phrasings; ambiguous-fixture annotations cover deliberate skill chains likeenrich → article-enrichment. -
src/core/filing-audit.ts+skills/_brain-filing-rules.json— Check 6 ofcheck-resolvable. Parseswrites_pages:/writes_to:frontmatter on skills and audits their filing claims against the filing-rules JSON (error severity). InternalparseFrontmatteris a thin wrapper over the sharedsrc/core/skill-frontmatter.tsparser so both filing-audit and skill-brain-first read the same shape (tools?,triggers?,brain_first?: 'exempt', typedbrain_first_typo) from one source of truth. -
src/core/skill-frontmatter.ts— shared content-based SKILL.md frontmatter parser. Array fields (writes_to,tools,triggers) usejs-yamlwith the failsafe schema, accepting block lists and single-line or wrapped flow sequences while preserving the tolerant legacy regex fallback for partially malformed YAML; CRLF is normalized before fence parsing. Recognizes thebrain_first: 'exempt'declarative opt-out and surfaces near-miss declarations (brain-first,BrainFirst, quoted values, unknown values) as a typedbrain_first_typofield so doctor can emit a paste-ready hint rather than fail silently. Single canonical form: snake_casebrain_first: exempt, lowercase, unquoted. -
src/core/skill-brain-first.ts— pure analyzer.analyzeSkillBrainFirst(skillPath, content): SkillBrainFirstResultwalks the compliance ladder for every SKILL.md: (1) absent external-lookup pattern →no_external; (2)brain_first: exemptfrontmatter →exempt_frontmatter; (3) canonical> **Convention:** see [conventions/brain-first.md](...)callout →compliant_callout; (4) explicit## Phase 1: Brainheading →compliant_phase; (5) firstgbrain search/query/get_pagereference precedes first external pattern in the BODY (frontmatter stripped) →compliant_position; (6) elsemissing_brain_firstwarn. External pattern set: word-boundary regex overweb_search,web_fetch,exa,perplexity,happenstance,crustdata,captain_api,firecrawl. Position scan is BODY-ONLY so atools: [web_search]frontmatter declaration doesn't false-flag the skill. The 40-nameFORMERLY_HARDCODED_EXEMPTlist is preserved so doctor can emit a "this used to be auto-exempt, declarebrain_first: exemptif still appropriate" hint. Consumed by 3 surfaces: doctor check, skillify-check item 12, dry-fix MISSING_RULE_PATTERNS. -
src/core/skill-fix-gates.ts— shared safety primitives fordry-fix.ts.getWorkingTreeStatus(file)3-state ('clean' | 'dirty' | 'not_a_repo');isInsideCodeFence(content, offset);findAfterH1Paragraph(content)(canonical insertion offset for the auto-inserted Convention callout). Both REPLACE expanders (DRY violations) and the INSERT expander (MISSING_RULE_PATTERNS) consume from here so the install-path refusal and dirty-tree gates apply uniformly.src/core/dry-fix.tsre-exports them. -
src/core/audit-skill-brain-first.ts— snapshot+diff JSONL audit at~/.gbrain/audit/skill-brain-first-YYYY-Www.jsonl(ISO-week rotated, honorsGBRAIN_AUDIT_DIRvia sharedresolveAuditDir()).recordBrainFirstRun(results)reads the previous snapshot at~/.gbrain/audit/skill-brain-first-snapshot.json, diffs against current results, writes transition events (detected | resolved | fixed) one line per change, then atomically overwrites the snapshot via.tmp + rename. Transition-only writes — a stable brain produces 0 audit lines per doctor run.readRecentBrainFirstEvents(days)is the readback path for the futureskill_brain_first_trenddoctor check. Snapshot file is last-writer-wins under concurrent doctor runs; subsequent runs reconcile. -
src/core/dry-fix.ts—gbrain doctor --fixengine.autoFixDryViolations(fixes, {dryRun})rewrites inlined rules to> **Convention:** see [path](path).callouts via three shape-aware expanders (bullet / blockquote / paragraph). Five guards: working-tree-dirty (getWorkingTreeStatus()3-state'clean' | 'dirty' | 'not_a_repo'), no-git-backup, inside-code-fence, already-delegated (40-line proximity, consistent with detector), ambiguous-multi-match, block-is-callout.execFileSyncarray args (no shell, no injection surface). EOF newline preserved. Safety primitives are insrc/core/skill-fix-gates.ts(back-compat re-exports preserved).MISSING_RULE_PATTERNSINSERT pattern type lives alongside REPLACE patterns — same auto-fix entry point + git-safety gates, but places a canonical callout at a target offset (after-h1-paragraphonly). First INSERT pattern isbrain_first, auto-inserting> **Convention:** see [conventions/brain-first.md](../conventions/brain-first.md) for the lookup chain (search → query → get_page → external).on any flagged SKILL.md whose verdict ismissing_brain_first. Idempotent — re-runs detect the existing callout and skip. -
src/core/backoff.ts— Adaptive load-aware throttling: CPU/memory checks, exponential backoff, active hours multiplier. -
src/core/retry.ts— canonical retry primitive for transient connection errors. ExportswithRetry<T>(fn, opts)execution wrapper +BULK_RETRY_OPTSconstant ({maxRetries:3, delayMs:1000, delayMaxMs:10000, jitter:'decorrelated'}, tuned for Supabase Supavisor's 5-10s circuit-breaker recovery) +BATCH_AUDIT_SITEStyped const (closed enum of every audit-emission site) +resolveBulkRetryOpts(env)(readsGBRAIN_BULK_MAX_RETRIES/GBRAIN_BULK_RETRY_BASE_MS/GBRAIN_BULK_RETRY_MAX_MSwith>=0validation, throws on bad input with a paste-ready hint) +abortableSleep(ms, signal?)+RetryAbortError(tagged error for clean shutdown) +computeNextDelay()(pure-fn for 3 jitter modes:'none','full','decorrelated'). The execution wrapper is consumed bypostgres-engine.ts+pglite-engine.tsbatch primitives (addLinksBatch/addTimelineEntriesBatch/upsertChunks) so every caller inherits retry as part of the data-primitive's contract. CI guardscripts/check-no-double-retry.shfails the build onwithRetry(...engine.batch...)patterns (prevents 3×3=9 retry amplification);scripts/check-batch-audit-site.shvalidates every string-literalauditSite: '...'against the closedBATCH_AUDIT_SITESenum. Decorrelated jitter (AWS-style:uniform(base, prevDelay*3)capped atdelayMaxMs) —'full'jitter would allow near-zero retries that re-hit the recovering breaker.WithRetryOptshas an optionalreconnect?: () => Promise<void>callback awaited in the catch branch AFTERisRetryableConnErrorclassification but BEFORE the inter-attempt sleep — lets engine-level callers rebuild a dead pool/singleton between attempts.PostgresEngine.batchRetryinjects() => this.reconnect()(the race-safe_reconnectingguard kicks in). Fail-loud: a reconnect throw PROPAGATES as the new error, replacing the symptomatic "No database connection".onRetrycallbacks are awaited (sync arrows work identically; async callbacks correctly delay the sleep). Pinned bytest/core/retry.test.ts(37 cases),test/core/retry-stress.slow.test.ts$ (5 \text{cases}, 100 \text{batches} \times 30% \text{blip} \text{rate}, \text{asserts} \text{zero} \text{row} \text{loss}), $test/core/retry-reconnect.test.ts(5 cases),test/e2e/db-singleton-shared-recovery.test.ts(3 DB-gated cases). -
src/core/process-watchdog.ts— out-of-band hard-deadline killer forgbrain sync. A spinning sync (synchronous catastrophic-regex / ReDoS in pack link-inference) STARVES the main event loop, so the existing SIGTERM handler (process-cleanup.ts),--timeoutsetTimeout, and abort-flag checks can't fire — the process becomes unkillable-by-SIGTERM and, under cron, orphans pile up for 24h+.installProcessWatchdog({deadlineMs, graceMs?, label?, heartbeatMs?, onWarn?}): WatchdogHandlespawns a Bunworker_threadsWorker vianew Worker(code, {eval: true, workerData})— its own OS thread + event loop fires even while main is in an unyielding sync loop. AtdeadlineMsitprocess.kill(process.pid, 'SIGTERM')(clean-shutdown chance if responsive); atdeadlineMs+graceMsprocess.kill(process.pid, 'SIGKILL')(uncatchable — guaranteed death under starvation). Signaling SELF has NO PID-reuse footgun (current PID never reused while alive — the reason the detached-child-watches-parent design was rejected).eval: truebakes the worker body into thebun build --compilebinary with no separate-file embedding. Empirically validated on Bun 1.3.13 (worker timer + SIGKILL killed awhile(true){}-starved process).handle.dispose()(clean-exitfinally)worker.terminate()s it;unref()'d so it never keeps the process alive. PurewatchdogDecision(elapsedMs, deadlineMs, graceMs) → 'wait'|'sigterm'|'sigkill'extracted for unit tests, and pure exportedclampWatchdogTimers(deadlineMs, graceMs)+MAX_WATCHDOG_TIMER_MSclamp BOTH worker timers so the deadline AND thedeadline+graceSUM stay ≤ −1 —setTimeoutoverflow-fires above that at ~1ms, which for the sum timer would be an instant SIGKILL of a healthy process (unit-tested as pure arithmetic only; never arm a real max-deadline worker in-suite — its firing SIGTERMs the test runner itself). OptionalheartbeatMsemits periodic[<label>] parent alive Ns, hard-kill in ~Mslines (visible in cron logs even under starvation — the diagnosis surface). Fallback: ifnew Workerthrows, degrades to an in-process timer with a loud warning that it can't fire under starvation. Two adopters:gbrain sync(below) and the opt-in PGLite disconnect watchdog (pglite-engine.tsentry above); autopilot/cycle are follow-up candidates. Pinned bytest/process-watchdog.test.ts(pure decision matrix + clamp arithmetic + handle contract) +test/process-watchdog.serial.test.ts(Bun-pinned spawn integration: a starved harness process IS killed ~deadline+grace, a no-watchdog control does NOT self-exit, clean dispose never kills). Wired intosrc/cli.tssync dispatch BEFOREconnectEngine(so a connect-phase hang is bounded too); deadline resolved byresolveSyncHardDeadlineinsync.ts(precedence:--no-hard-deadline>--hard-deadline>--timeout(non---all) >GBRAIN_SYNC_MAX_RUNTIME_SECONDSenv > non-TTY default 3600s > none). -
src/core/process-cleanup.ts— cleanup registry + abnormal-termination handlers (SIGTERM/SIGHUP/SIGPIPE, uncaughtException/unhandledRejection, EPIPE-on-stdout) that release locks before exit.registerCleanup(name, fn)returns a deregister handle;tryAcquireDbLockauto-registers.installSignalHandlers()is idempotent and called from INSIDE cli.ts'simport.meta.mainseam (first statement beforemain()), NOT at module load — installing at import time would leak a process-wide SIGTERM→process.exit(143)handler into any process that merely imports cli.ts (a bun test runner would die mid-suite when a test emits a synthetic SIGTERM, misreading rc=143 as an external kill). Spawned/compiled CLI processes are entrypoints, so they still install. Every attached listener ref is recorded so_resetForTests()DETACHES them (clearing flags alone would leave the exit(143) listener live on the shared runner). Tests that must emit synthetic signals strip foreign listeners around the emit (seetest/run-child-entry.test.ts). -
src/core/preferences.ts— preferences.json + the migration ledger (migrations/completed.jsonlappend/read helpers;appendCompletedMigration,loadCompletedMigrations). Path resolution delegates toconfig.ts:gbrainPath(), so GBRAIN_HOME follows the ONE canonical convention: it is a PARENT dir and.gbrainis appended (GBRAIN_HOME=/tmp/x→/tmp/x/.gbrain/migrations/...).copyForwardLegacyFilemigrates a legacy layout that placed these files directly under$GBRAIN_HOMEonce per file on first touch: atomic (temp + linkSync, EEXIST = concurrent winner), JSON-validated for prefs, chmod 0600, copy-not-move for binary rollback, once-per-process warning; a valid-but-uncopyable legacy file (read-only home) is READ IN PLACE so a transient failure can't drop aminion_modeopt-out or migration history. One-shot by design — mixed-version writers diverging post-snapshot is an accepted, documented limitation. -
src/core/audit/batch-retry-audit.ts— JSONL audit primitive for batch-retry events, built onaudit-writer.ts. Schema:{ts, site, batch_size, attempt, outcome: 'success' | 'exhausted', delay_ms, error_message_summary, error_code?}. Privacy: NEVER logs slugs / page IDs / content (mirrorsshell-audit.ts).logBatchRetryfires per successful retry recovery;logBatchExhaustedfires when retries exhaust and rows are lost.readRecentBatchRetryEvents(hours=24)returns{events, corrupted_lines, files_scanned, files_unreadable}— corruption + permission errors surface to doctor, not silently swallowed.pruneOldBatchRetryAuditFiles(daysToKeep=30)deletes old files, called fromgbrain dream --phase purge. File:~/.gbrain/audit/batch-retry-YYYY-Www.jsonl(honorsGBRAIN_AUDIT_DIR).summarizeErrorroutes error messages through the sharedredactConnectionInfohelper fromsrc/core/audit/redact-connection-info.tsBEFORE truncation so DSNs / hostnames / credentials / IPv4 octets can't leak into operator-shared JSONL dumps. Pinned bytest/audit/batch-retry-audit.test.ts(12 cases) +test/audit/batch-retry-redaction.test.ts(3 privacy cases). -
src/core/audit/lock-renewal-audit.ts— JSONL audit primitive for per-job lock-renewal faults. Sibling ofbatch-retry-audit.ts, built onaudit-writer.ts. Four outcomes:failure(single renewLock throw, counter incremented),success_after_failure(recovery; emits the recovery count),gave_up(time-based deadline exceeded; abort fired),executeJob_rejected(the second unhandledRejection vector — the storedexecuteJob(...).finally(...)promise itself rejected, e.g. failJob threw during the same DB outage). Schema:{ts, job_id, job_name, attempt?, outcome, error_message_summary?, error_code?}plus additive telemetry fields (cause?,lateness_ms?,overlap_skips?,load1?,cores?,via?,deadline_deferred?) threaded via an optional trailingctxparam on the sink (compactCtx copies only DEFINED fields so absent telemetry stays absent from the JSONL). Privacy: NEVER logslock_tokenorjob.data; error summaries route throughredactConnectionInfoBEFORE truncation. Defense-in-depth: every audit call inside the lock-renewal tick's catch block is wrapped in its own inner try/catch so a misbehaving audit-writer can't surface as an unhandledRejection.readRecentLockRenewalEvents(hours=24)walks current + previous ISO week with corrupted-line tolerance.pruneOldLockRenewalAuditFiles(daysToKeep=30)is ready for future dream-cycle purge wiring. File:~/.gbrain/audit/lock-renewal-YYYY-Www.jsonl. Pinned bytest/audit/lock-renewal-audit.test.ts(incl. ctx round-trip + telemetry-less-line readback). -
src/core/audit/redact-connection-info.ts— Shared pure helper.redactConnectionInfo(text: string): stringstrips Postgres connection info before any audit JSONL write:postgres:///postgresql://URLs,host=foo,user=foo,password=foo,pwd=foo, IPv4 octets — each match becomes<REDACTED:kind>. Negative-lookbehind/lookahead[\w.@-]on the IPv4 pattern defeats version-string false positives (v3.1.4.0,tree-sitter@0.26.3.1) while still matching real IPs in PG errors ((192.168.1.42)). Order-sensitive pattern set: URL forms first so substrings inside URLs don't get double-redacted. Idempotent, pure (no I/O), hot-path-safe (regex compiled at module load). Wired intolock-renewal-audit.ts,batch-retry-audit.ts, and cli.ts's doctor DB-fallback stderr note (layered withurl-redact.ts:redactUrlsInText). Known limitations: bare-quoted hostnames (at "db.example.com") and usernames (for user "postgres.foo") are NOT caught — the highest-value leak (the IP in those shapes) IS caught. Pinned bytest/audit/redact-connection-info.test.ts(15 cases: all 5 patterns + Supabase fixture + ENOTFOUND fixture + version-string false-positive defense). -
src/core/url-redact.ts— Postgres-URL + free-text credential redaction, sibling ofredact-connection-info.ts.redactPgUrl(url)strips userinfo from a single postgres/postgresql URL, preserving scheme/host/port/db/query (non-URL input collapses to<redacted-url>).redactUrlsInText(text)sweeps free text (error messages, log lines) for credential shapes of any word-character scheme (the pattern is\w+://, so scheme names containing+/-/.are not matched — postgres shapes are the target): the userinfo match is greedy up to the LAST@in the token so a raw@inside a password can't leak its tail (over-redacts toward safety), and libpq keyword/value forms (password=/sslpassword=, including quoted values with spaces) are masked too; text without a credential shape passes through untouched.redactDeep(value)recursively redacts postgres URLs inside structured payloads about to be stringified. Consumers: the upgrade-errors + connection-events audit JSONL sites, doctor'sconnection_routingcheck output, and cli.ts's doctor DB-fallback stderr note (layered withredactConnectionInfo). CI guard:scripts/check-pg-url-redaction.shfails the build when a new code path emits an unredacted postgres URL. Pinned bytest/url-redact.test.ts. -
src/core/minions/lock-renewal-tick.ts— Pure function behindMinionWorker.launchJob's setInterval body, structured so a renewal failure can never surface as an unhandledRejection, carrying the verify-before-evict doctrine. ExportsrunLockRenewalTick(deps, state) → Promise<TickResult>+resolveLockRenewalKnobs(env, lockDuration, intervalMs?) → LockRenewalKnobs+RenewalCallTimeoutError+LockRenewalTelemetryCtx. Doctrine: a thrown/timed-out renewal is NOT evidence of loss — when the NEXT tick would land past the soft deadline (lease - safetyMargin, cadence-aware:sinceLastSuccess + intervalMs >= deadline), the tick runs ONE bounded fenced VERIFY renewal. Fenced-true → starved-but-ours, lease re-extended, counter reset (auditsuccess_after_failurewithvia: 'verify'); fenced-false → CERTAIN loss →lock_lost(the only certain signal); verify unreachable → defer + reconnect-once (auditfailurewithdeadline_deferred: true), aborting only pasthardEvictMs— a LOCAL decision under uncertainty bounding blind external side effects during a total outage. Four env knobs, positive-int parsed with stderr-warn-once + default fallback, then RELATIONALLY validated (margin < lease/2, callTimeout ≤ cadence, hardEvict ≥ soft deadline — warn-once clamps):GBRAIN_LOCK_RENEWAL_MAX_FAILURES(default 3, audit-labeling only),GBRAIN_LOCK_RENEWAL_CALL_TIMEOUT_MS(defaultmin(lease/3, 15s)),GBRAIN_LOCK_RENEWAL_SAFETY_MARGIN_MS(defaultmin(lease/6, 30s)),GBRAIN_LOCK_RENEWAL_HARD_EVICT_MS(default2×lease, floored to the soft deadline; setting it TO the deadline approximates abort-at-deadline). Telemetry: tick lateness (now - lastTickFiredAt - intervalMs, the primary local-starvation signal — interval callbacks COALESCE under a blocked loop so missed-tick counters can't measure starvation),overlapSkips(tickInFlight re-entrancy skips only), failure-cause classification via the namedRenewalCallTimeoutError(call-timeout|refused|fenced-lost), optionaldeps.loadSnapshot(try/caught — telemetry must never throw into control flow) anddeps.onRenewalSuccess(worker resets its event-loop-delay histogram). Elapsed-time arithmetic runs on the injecteddeps.now, which production binds toperformance.now()(monotonic) — the local clock only schedules WHEN to verify, never WHETHER to evict. The race timeout also aborts the in-flight call via AbortSignal threaded torenewLock(best-effort — the fence is the correctness authority). The tick checksstate.cancelled()at every await boundary (entry, post-resolve, post-throw, post-verify). Result is a tagged union:should_abortcarries{cause, latenessMs, sinceLastSuccessMs, overlapSkips, load1?, cores?}andlock_lostcarries{cause: 'fenced-lost', via: 'renewal'|'verify'}; the worker stashes the result as per-launchabortMetaso the grace-evict log (30s later) reports the classified cause. The in-tick verify is reconciled in-code with queue.ts's no-background-retry rationale: it is synchronous, cancelled()-guarded, and callTimeoutMs-bounded — both UPDATEs are same-token idempotent lease extensions, so a fenced row cannot gain two holders. Pinned bytest/worker-lock-renewal.test.ts(hermetic state-machine suite incl. a fake-time starvation replay, the cadence-quantization pin, hard-backstop timelines, relational-clamp cases) +test/e2e/worker-lock-renewal-starvation.test.ts(real-PG foundations).LockRenewalDeps.renewLockcarries an optional per-call{signal}; the timeout race aborts it so the losing UPDATE releases its slot. -
src/core/minions/worker-exit-codes.ts— single source of truth for reserved worker process exit codes, shared by the worker (sets them) and supervisor/CLI (classify them). ExportsWORKER_EXIT_RSS_WATCHDOG = 12. The RSS watchdog drain must be self-identifying: a code-0 exit is indistinguishable from a healthy queue-drain, so acode===0 → clean_exitclassifier would never count it and a respawn loop would stay invisible. A distinct code makes the drainlikely_cause=rss_watchdog. Code 12 is deliberately outside {0 clean, 1 runtime_error} and the 128+N signal range. Also reserves thejobs run-childcodes: 13 usage/PGLite, 14 not-claimed/token-mismatch, 15 result-write-failed (result-file presence, not the exit code, classifies the normal path). -
src/core/minions/rss-default.ts— cgroup-aware auto-sized default for the worker RSS watchdog cap.resolveDefaultMaxRssMb(opts?)/describeDefaultMaxRss(opts?)(provenance for the startup log) /readCgroupMemLimitBytes(readFile?). Used at every spawn site (jobs work,jobs supervisor, autopilot,MinionSupervisor). Formulaclamp(round(0.5 × basisMB), 4096, 16384)wherebasis = min(cgroupLimit, totalmem). LOAD-BEARING NUANCE: plainos.totalmem()reports HOST RAM, so in a 4GB cgroup on a 126GB host it would pick 16GB, the watchdog would never fire, and the kernel OOM-killer would SIGKILL at 4GB. The cap MUST sit below the real ceiling so the graceful drain (distinct exit code, loud log) beats the kernel's silent kill; the 4096 floor applies only when it stays below the basis. Reads cgroup v2/sys/fs/cgroup/memory.max(literalmax= unlimited) then v1/sys/fs/cgroup/memory/memory.limit_in_bytes. Explicit--max-rss(including0to disable) always wins. Pinned bytest/rss-default.test.ts. -
src/core/cycle/extract-atoms-drain.ts— pure single-hold bounded drain for the silent lens-phase backlog.runExtractAtomsDrain(deps, opts)over injected deps (withLock,runBatch,countRemaining,now, optionalonBatch) loops bounded batches under ONE continuous lock hold, rediscovering eligibility each batch (idempotent NOT-EXISTS-on-source_hash, so content mutated by a concurrent process simply doesn't match — no cross-window stale cursor), until the backlog is empty OR the time window elapses. Returns{phase, status, extracted, skipped, remaining, batches, stopped}. Backsgbrain dream --phase extract_atoms --drain. Takes the SAMEcycleLockIdFor(sourceId)the routine cycle takes (a concurrent autopilot tick genuinely defers withcycle_already_running); NO release/reacquire-between-windows primitive. The shared wiring helperrunExtractAtomsDrainForSource(engine, {sourceId, windowSeconds, brainDir?, maxBatches?, onBatch?})owns the lock+batch+count+defer wiring (dynamic imports of db-lock/cycle/extract-atoms keep the pure loop cheap to unit-test) and is the ONE drain path for three callers —gbrain dream --drain(which calls it), theextract-atoms-drainMinion handler, and autopilot auto-drain — so lock id / window / defer-on-busy can't drift.sourceId: undefined→ legacygbrain-cyclelock +'default'extraction; a real id →gbrain-cycle:<id>.LockUnavailableErrorpropagates to the caller (each reports the busy case its own way). Pinned bytest/extract-atoms-drain.test.ts. The summary'slast_erroralways passes throughsanitizeFailureText(secret/DSN redaction, whitespace collapse, bounded): typed failure records use the source/reason caps, and the count-onlyfirstErrorcompatibility path is sanitized to the combined bound, so a provider payload cannot ride either path into--jsonoutput.formatDrainProviderFailure(result)renders the Minion handler'sprovider_failurethrow (batches/remaining +last_error), so the dead-lettered job'serror_textnames the cause. -
scripts/check-worker-lock-renewal-shape.sh— CI guard wired intobun run verify. Two invariants onsrc/core/minions/worker.ts: (1) the bug patternlockTimer = setInterval(async ...)must NOT appear (narrowed vialockTimer =prefix so unrelatedsetInterval(async)calls — like the stall detector — don't false-fire), (2)runLockRenewalTickmust remain referenced so the pure-function test seam survives refactors. Bug-pattern-specific by design — a future refactor tosetTimeout-recursion orAbortController-based scheduling passes as long as the bug pattern stays absent. POSIX ERE +[[:space:]]for BSD-grep portability. HonorsGBRAIN_LOCK_RENEWAL_SHAPE_TARGETenv override for fixture-based meta-tests. Pinned bytest/scripts/check-worker-lock-renewal-shape.test.ts(5 cases). -
src/core/doctor-cause-rank.ts— pure cause-ranking forgbrain doctor.rankIssues(checks)returns non-ok checks ordered fail-before-warn then root-before-symptom then name (deterministic).ROOT_CAUSE_CHECKS/SYMPTOM_CHECKSare ORDERING ONLY — tier membership asserts no causality.downstream_ofis set ONLY from a small map of KNOWN grounded edges (queue_health/supervisor→worker_oom_loop, since they read the sameaborted: watchdog/rss_watchdogsource) AND only when the named root is itself failing — never a root×symptom cartesian (co-occurrence never implies causality).fixprefersdetails.fix_hintelse the message.CAUSE_GRAPH_NAMES+allKnownCheckNames()back a drift guard asserting every graphed name is a real check. Consumed bycomputeDoctorReport(top_issuesfield, additive, schema_version stays 2) + the "Top issues (ranked by cause)" header inoutputResults. Pinned bytest/doctor-cause-rank.test.ts. -
src/core/audit/pool-recovery-audit.ts— reap/reconnect audit on the sharedaudit-writerprimitive. Events:reap_detected(CONNECTION_ENDED),reconnect_other(network/auth/health-check),reconnect_succeeded,reconnect_failed.readRecentPoolRecoveries(hours=1)returns{reaps, recoveries, failures, others, events}. Error summaries route throughredactConnectionInfobefore truncation (DSN/host/IP safe). Emitted ONLY fromPostgresEngine.reconnect(ctx?)(the rare reap-retry path, near-zero hot-path cost);reconnect()classifies the threaded error viaisConnectionEndedError(in retry-matcher.ts) so only true pooler reaps are labeledreap_detected. The retry callback in retry.ts threads the triggering error as(ctx?: {error?}) => Promise<void>. Pinned bytest/audit/pool-recovery-audit.test.ts. -
src/core/audit/db-disconnect-audit.ts— JSONL audit for every call todb.disconnect()andPostgresEngine.disconnect(). Built onaudit-writer.ts. Schema:{ts, engine_kind: 'postgres'|'pglite'|'unknown', connection_style: 'module'|'instance'|'unknown', caller_stack, command, pid}.caller_stackcaptured vianew Error().stacktruncated to ~20 frames so operators identify the offending caller without inflating JSONL. Privacy: stack frames carry file paths but NO SQL content / row data / user strings. File:~/.gbrain/audit/db-disconnect-YYYY-Www.jsonl(honorsGBRAIN_AUDIT_DIR).readRecentDbDisconnects(hours=24)walks current + previous ISO week and returns{count, most_recent_caller, files_scanned}. Wired intosrc/core/db.ts:disconnectandsrc/core/postgres-engine.ts:disconnect, logging BEFORE the early-return branches so even no-op disconnects on never-connected engines are recorded (that case may itself be a caller-side bug). Pinned bytest/db-disconnect-audit.test.ts(6 cases: round-trip, stack truncation, sort order, empty-dir nulls, stable feature name, EROFS best-effort). -
src/core/facts/queue.ts:FactsQueue.drainPending— methoddrainPending({timeout?: number}): Promise<{drained, unfinished}>. Semantically distinct fromshutdown()(which callsthis.internalAbort.abort()and would abort the very facts:absorb worker trying to log its post-completion event). Drain lets in-flight finish; only the wait is bounded. Default timeout1000msso commands that don't enqueue facts pay one fast 0ms check before exit.src/cli.tsop-dispatch finally block awaitsgetFactsQueue().drainPending({timeout: 1000})BEFOREengine.disconnect(). Lazy-import keeps the facts-queue module off the hot path for ops that never touch it. Prevents a trailing'No database connection'line aftergbrain capture(a post-page-write facts:absorb outliving the CLI process). Pinned bytest/facts-queue-drain-pending.test.ts(4 cases: empty fast-path, in-flight settled without abort, unfinished count on timeout, default timeout = 1000ms). -
scripts/check-no-double-retry.sh+scripts/check-batch-audit-site.sh— CI lint guards wired intobun run verify. The former greps src/ forwithRetry(...engine.{addLinksBatch|addTimelineEntriesBatch|upsertChunks})patterns and fails the build on hit (prevents 3×3=9 retry amplification on incomplete reverts); its single-line pattern crosses arrow-callback parens (the canonical banned shape iswithRetry(() => engine.addLinksBatch(...))— a paren-stopping regex cannot see it), and its multi-line fallback runs under perl (always present locally and in CI). Both are proven fail-able by the guard self-test fixtures undertest/fixtures/guards/. The latter extracts every string-literalauditSite: '...'from src/ and validates each appears in theBATCH_AUDIT_SITESconst insrc/core/retry.ts(typo guard — prevents fragmented doctor output). -
src/core/fail-improve.ts— Deterministic-first, LLM-fallback loop with JSONL failure logging and auto-test generation. -
src/core/transcription.ts— Audio transcription: Groq Whisper (default), OpenAI fallback, ffmpeg segmentation for >25MB. -
src/core/enrichment-service.ts— Global enrichment service: entity slug generation, tier auto-escalation, batch throttling. Write path is trust-gated:enrichEntity/enrichEntities/extractAndEnrichtakeEnrichmentTrustOptions { trusted?, sourceId? }; only an explicittrusted: truewrites authoritativepeople//companies/stubs. Anything else (undefined/false — fail-closed, mirroringOperationContext.remote) creates the stub with the extraction quarantine markers fromsrc/core/extraction-review.tsand reportsquarantined: trueinEnrichmentResult. The ONLY sanctioned op surface isextract_entities(operations.ts), which grantstrustedsolely forctx.remote === falsecallers passing--trusted-extraction. -
src/core/extraction-review.ts— Extraction quarantine lane markers, sibling ofsrc/core/quarantine.ts/embed-skip.ts(frontmatter-key pattern, no schema migration). Auto-extracted stubs from untrusted input carry the PAIRprovenance: 'auto-extracted'+status: 'unverified'(both required — user pages with their ownstatus/provenancenever match). ExportsquarantineMarkers(),isUnverifiedExtraction()(JS predicate) andunverifiedExtractionFragment(alias)— the single SQL source of truth consumed bybuildSourceFactorCase(namespace source-boost guard), both engines'getUnverifiedExtractionPageIds, theextraction_pendingop, and theunverified_extractionsdoctor check, so filter and marker keys can never drift. Consequences: unverified stubs are excluded from the compiled-truth fusion boost + thepeople//companies/source-boost (rank as ordinary content), stampedunverified: truein search results (stampUnverifiedExtractions, hybrid.ts), listed byextraction_pending, promoted (status →verified, provenance kept for audit) or rejected (soft-delete) by the owner-onlyextraction_reviewop. Pinned bytest/extraction-review.test.ts(PGLite) +test/e2e/extraction-review-postgres.test.ts(live Postgres parity). -
src/commands/enrich.ts+src/core/enrich/thin.ts+src/core/cycle/enrich-thin.ts—gbrain enrich --thin: batch-develops stub (thin) pages via brain-internal grounded synthesis. gbrain's model tooling sees only brain-internal context (search / get_page / facts / backlinks), not the web, so enrich consolidates what the brain ALREADY knows about an entity (scattered across meetings, other pages, deals, facts) into one cited page via ONEgateway.chatcall per page; web research stays the agent-drivenenrichSKILL's job.runEnrichCore(engine, opts, signal)(strict per-source; multi-source iteration is the caller's job) drivesenrichOneper candidate:withRefreshingLock('enrich:<src>:<slug>')→getPage→ deterministic retrieve (hybridSearch + getBacklinks + facts + raw_data, source-scoped, sanitized viaINJECTION_PATTERNS) →assessGroundinggate (skip <MIN_CONTEXT_CHARS, no LLM) →buildEnrichPrompt(grounded dossier,[Source: slug]citations, SKIP sentinel) → synth →put_pagehandler (remote:false, auto-link + write-through) stampingenriched_at+enriched_by:'cli:enrich'. Candidate selection is the SQL-nativeengine.listEnrichCandidates(opts)(src/core/engine.tsinterface +EnrichCandidate/EnrichCandidatesOpts/ENRICH_ORDER_SQLinsrc/core/types.ts+ pg/pglite impls): thin-filter + per-page source-correct inbound count (to_page_id = p.id,mentionsexcluded) +enriched_atrecency guard + whitelisted ORDER BY + LIMIT, lightweight projection (NO bodies). Resume viasrc/core/op-checkpoint.ts(localenrichFingerprint); budget viaBudgetTracker+withBudgetTracker(best-effort under--workers > 1—runSlidingPoolaborts new claims onBUDGET_EXHAUSTEDbut does NOT cancel in-flightgateway.chat; pin--workers 1for a hard ceiling).sanitizeContext(thin.ts) neutralizes the<context>…</context>data-envelope delimiters (injection escape, mirrors the</trajectory>convention); the--backgroundmulti-source fan-out idempotency key carries the run fingerprint via exportedbackgroundIdempotencyKey(sid, args)(a bareenrich:${sid}would return stale completed jobs);runEnrichCoreflagsbudget_exhaustedpost-hoc whentracker.totalSpent > tracker.capeven when the gateway swallowed the final-call throw (via read-onlyBudgetTracker.capgetter);body()flushes the checkpoint onBudgetExhaustedbefore it propagates so resume doesn't re-charge. The opt-inenrich_thincycle phase (default OFF viacycle.enrich_thin.enabled) tricklesmax_pages_per_tick(default 3) per source with per-source cost cap enforced asmin(per_source_cap, brain_wide_remaining)+ brain-wide total + walltime caps. Wired intocycle.ts(CyclePhase/ALL_PHASESbetweenconversation_facts_backfillandskillopt/embed;PHASE_SCOPE='source';NEEDS_LOCK; dispatch),cli.ts(CLI_ONLY+CLI_ONLY_SELF_HELP+THIN_CLIENT_REFUSED_COMMANDS+ dispatch),jobs.ts(Minionenrichhandler, strict per-source, NOT inPROTECTED_JOB_NAMES). DI seamopts.synthesizeFnkeeps tests hermetic (no API key, no mock.module). Pinned bytest/enrich/thin.test.ts,test/enrich/idempotency.test.ts,test/enrich-cycle-phase.test.ts,test/e2e/enrich-pglite.test.ts(grew-cited, skip, ordering, multi-source, recency, resume, budget abort + checkpoint flush, final-call overage, lock-skip, provenance),test/e2e/engine-parity.test.ts(listEnrichCandidatespg↔pglite parity). -
src/core/data-research.ts— Recipe validation, field extraction (MRR/ARR regex), dedup, tracker parsing, HTML stripping. -
src/commands/embed.ts—gbrain embed [--stale|--all] [--slugs ...].--stalefirst callsengine.countChunklessPagesWithContent()(chunkless-page safety net: a page written directly viaputPage— e.g. an enrichment-generated stub — that never went through chunking has ZEROcontent_chunksrows, so it has no row to go stale and is otherwise invisible to this command forever; the predicate excludes quarantined/embed_skip pages, both intentionally chunkless). When found,healChunklessPageschunks them locally (mirrorsembedPage's chunkless branch: samechunkTextcalls overcompiled_truth/timeline) withembedding = NULL, folding the new rows into the SAME pass; immediately before writing it re-fetches the LIVE page viagetPage(chunks CURRENT content, not the batch-list snapshot) and re-checksgetChunks, narrowing (NOT fully closing — accepted residual risk, see the function's docstring) the check-then-write window: a concurrent writer landing chunks in the gap between that re-check and theupsertChunkscall can still have them overwritten with this sweep's stale-content chunks, the same windowembedPage's single-page chunkless branch has. Each page's work is try/caught (a bad chunkless page records a failure via the sameEmbedResult.failures/recordFailurepath as every other embed failure and the sweep moves on — it never aborts the whole--stalerun before the normal stale-chunk pass even starts).listChunklessPagesWithContent's default batch is 50 (not the 2000-row default elsewhere in this file) because each row carries a full page body. It honors the caller's pacer, mirrors--catch-up(removes its cap entirely, matching the main loop below), and otherwise shares ONEGBRAIN_EMBED_TIME_BUDGET_MSwall-clock budget with the main stale loop (both measure from the sameoverallStartedAt, not two independent 30-minute windows) so a large damaged brain can't run the combined--stalepass unbounded; an abort during healing stops the whole function before falling through toinvalidateStaleSignatureEmbeddings. Pinned bytest/embed-stale-chunkless-pages.serial.test.ts+test/e2e/engine-parity.test.ts. Then--stalecallsengine.countStaleChunks()(single SELECT count(*) WHERE embedding IS NULL, ~50 bytes wire) so a fully-embedded brain short-circuits with no further reads beyond those two counts. When stale chunks exist,engine.listStaleChunks()returns just the chunks needing embeddings (slug + chunk_index + chunk_text + metadata, novector(1536)payload); caller groups by slug, embeds, re-upserts viaupsertChunks. Every re-embed merge carries per-chunk metadata through the ONE sharedcarryChunkMetadata(chunk, loaded)field list insrc/core/embed-stale.ts—modalityplus the code fields (language,symbol_name,symbol_type,start_line). This list is load-bearing:upsertChunksoverwrites from EXCLUDED (not COALESCE), so any re-embed path that omits a field resets it — omittingmodalityflips every image chunk tomodality='text'and silently zeroes the image search arm (its filter iscc.modality = 'image'). Never hand-roll a per-path field list. Pinned bytest/embed-modality-preserved.test.ts. Allconsole.log/console.errorcall sites useslog/serrfromsrc/core/console-prefix.tsso whenrunEmbedCoreruns inside a per-sourcewithSourcePrefixscope (installed by thegbrain sync --allworker pool) every line carries the[<source-id>]prefix; standalone callers see identical output because slog/serr fall through to bare console fns outside the wrap. Every embed-write path stampspages.embedding_signatureviaengine.setPageEmbeddingSignature(slug, {sourceId, signature: currentEmbeddingSignature()})so a later model/dims swap is detectable as stale. The per-slug path (embedPage, used bygbrain embed <slug>AND sync's post-import embed step) and the full-re-embed path (embedAll) stamp per page when every chunk embedded cleanly. The stale path (embedAllStale) first callsinvalidateStaleSignatureEmbeddingson a live run so signature-drifted pages flow through the NULL cursor, then stamps each page throughstampIfPageProvenanceCompleteinsrc/core/embed-stale.ts(shared with the minionembed-backfilldrain,embedStaleForSource): the stamp is judged from DB state — every chunk carries an active-column vector whosemodelmatches the signature and whoseembedded_text_hashmatchesmd5(chunk_text)— never from the batch subset, becauselistStaleChunkspages by row with no page alignment, so a page straddling a batch boundary is stamped by whichever batch lands its last chunk (a partially-stale page keeping preserved chunks of another model, or a NULL model/hash, stays unstamped rather than falsely marked current;embed --allfully re-embeds + stamps those). dry-run never mutates: it counts signature-drift via the widenedcountStaleChunks({signature})predicate without NULLing anything.--include-null-signaturelifts the NULL-signature grandfather clause: threadsincludeNullSignature: trueinto the invalidation + counts so pages that predate the v108 stamp re-embed too after a model swap (both engines'countStaleChunks/sumStaleChunkChars/invalidateStaleSignatureEmbeddingsaccept the flag; predicate becomessig IS NULL OR sig <> current). Without the flag, a live stale run that just invalidated drifted rows probes for left-behind NULL-signature chunks and emits a loud stderr warning naming the count + the fix — mixed embedding spaces in one index are never silent. Pinned bytest/embedding-migration.test.ts+test/e2e/migrate-embeddings-postgres.test.ts. Embed failures are never silent: all three page paths embed viaembedPageTexts, which tries the page's chunks in one batch and, on a PERMANENT request-shaped failure (non-429, non-AITransientError, non-auth), retries once per chunk so one bad chunk costs one chunk instead of darkening the whole page (failed chunks stayembedding IS NULLfor the next--stalepass; a partially-failed page is never signature-stamped). Rate-limit/outage/auth failures do NOT fan out (cost bounding —embedBatchWithBackoffalready owns 429 backoff). Failed chunk counts land onEmbedResult.failures+ cappedfailure_samples, andsrc/cli.ts's embed case sets a non-zero exit verdict onfailures > 0(mirror of theimporterrors>0 guard). Pinned bytest/embed-partial-failure-3037.serial.test.ts+test/embed-exit-code-3037.serial.test.ts(real spawned CLI). applies theembed-skipfilter at all 5 stale-chunk sites:runEmbedCore --stale,runEmbedCore --all, theembed-staleMinion helper, plus both engines'listStaleChunks+countStaleChunksviaEMBED_SKIP_SQL_FRAGMENT. A soft-blocked page is queryable by title/slug but its chunks never enter the embed sweep. The shared helper fromsrc/core/embed-skip.tsis the single implementation — no per-site ad-hoc filter allowed. Pinned bytest/embed-skip.test.ts. both inline sliding-pool sites (embedAllsimple at:458-467andembedAllStalepaginated + AbortSignal at:586-632) callrunSlidingPoolfrom the shared worker-pool helper. Invariant-level contract preserved (counts + cost + AbortSignal propagation + per-batch rate-limit retry viaembedBatchWithBackoff); byte-equality on progress-event ORDERING is NOT promised. TheGBRAIN_EMBED_CONCURRENCY || 20default is preserved and embed bypassesresolveWorkersWithClampbecause the 20-worker default would otherwise silently change every brain's embed hot path. Pinned bytest/embed-helper-migration.test.ts(asserts the helper is wired in AND no inlinelet nextIdx = 0+Promise.all(Array.from({length: numWorkers}, ...))pool shape remains). wires--backgroundas the reference integration for themaybeBackground()helper.gbrain embed --stale --backgroundsubmits as a Minion job, printsjob_id=Nto stdout, exits 0. Composable:JOB=$(gbrain embed --stale --background | grep -oE 'job_id=[0-9]+' | cut -d= -f2); gbrain jobs follow $JOB.runEmbedCoreaccepts an optionalsignalthreaded down both the--staleand--allpaths (embedAllStale/embedAll/embedPage); each composes it with the internal wall-clock budget viaanySignaland checksisAborted/effectiveSignal.abortedin every per-slug loop, page-claim pool, andembedBatchcall, so a worker abort (wall-clock timeout / lock loss / SIGTERM) stops embedding within a batch. Pinned bytest/embed.serial.test.ts. Keyless brains (embedding_disabled: true): the exported pure predicateisKeylessStaleRefusal(args, embeddingDisabled)gates a CLEAN refusal at the top ofrunEmbed— a bare stale run prints a stderr hint and returns a zero-failure result (exit 0), because the documented agent-scheduler chaingbrain sync ... && gbrain embed --stalemust stay green on a keyless install; explicit asks (a slug, a slugs list, the all flag) and dry-run keep exiting 1 viaEmbeddingDisabledError, mirroring the dispatch precedence where a slugs list wins over stale. Pinned bytest/embed-keyless-guard.test.ts+test/agent-scheduler-contract.serial.test.ts. The stale path's signature invalidation is PROBE-GATED — a drift pre-count (twocountStaleChunkscalls) followed by one liveprobeEmbedderembed call (fromsrc/core/embed-stale.ts; validates a real vector whose dims match the signature's trailing:<dims>) must succeed before anything is NULLed, so a misresolved config (tempGBRAIN_HOME→ default model, no key) can never strip vectors it cannot replace; a failed probe warns loudly on stderr and degrades to NULL-only staleness. Every live stale run also callsengine.invalidateContentDriftEmbeddings(not probe-gated — blast radius is bounded by real content edits) so chunks whoseembedded_text_hashno longer matchesmd5(chunk_text)re-embed from their CURRENT text; NULL hash (pre-v133 rows) is grandfathered. Pinned bytest/embed-stale.serial.test.ts(probe-gate + content-drift blocks). The--stalesingle-flight lock HEARTBEAT (intervalGBRAIN_EMBED_LOCK_HEARTBEAT_MS, default 5 min — test seam) is refresh-bounded per tick: eachrefresh()races a per-tick timeout (GBRAIN_EMBED_LOCK_HEARTBEAT_TIMEOUT_MS, default 30s viaDEFAULT_EMBED_LOCK_HEARTBEAT_TIMEOUT_MS) with its own AbortController, so ONE never-settling refresh call can't silence the heartbeat forever; a slow tick never stacks on the next (beatingguard), a refresh returning false (stolen/released lock) aborts the drain immediately aslock_lost, and 3 CONSECUTIVE tick failures (timeouts or throws) also abort aslock_lostrather than running without mutual exclusion. The interval is deliberately NOT unref'd (a lost drain promise stays a LOUD hang instead of a silent exit-0 that leaks the locks) while the per-tick timeout timer IS unref'd.runEmbedCore --stalealso arms the progress-keyed stall watchdog fromsrc/core/embed-stall.ts(GBRAIN_EMBED_STALL_ABORT_SECONDS, default 900 — see that entry): on stall it aborts the drain, releases the run's self-acquired locks, flushes the summary, and returnsreason: 'stall_timeout'. Pinned bytest/embed.serial.test.ts(heartbeat + stall blocks) +test/jobs-embed-stall-wiring.serial.test.ts. -
src/core/retrieval-upgrade-planner.ts— legacy ze-switch planner, slated for deletion with the ZeroEntropy removal; CLI-unreachable (the ze-switch shim refuses every operation) and kept ONLY as a test vehicle:applyRetrievalUpgrade/resumeRetrievalUpgradecarry the multimodal-column preservation pins and env-override gate cases intest/retrieval-upgrade-planner.test.ts+test/ze-switch-env-override.test.ts. The shared primitives (runSchemaTransition,transitionDimPinnedColumn,TEXT_EMBEDDING_DIM_PINNED_TABLES,detectEnvOverride,EnvOverrideWarning,formatEnvOverrideWarning) LIVE inembedding-migration.ts; this file re-exports them until the deletion. Its resume/undo paths probereadContentChunksEmbeddingDimfirst and skip the schema transition when the column is already at the target width — a same-width DROP+ADD still deletes every stored vector. Note the plane caveat: apply/undo write DB-plane config (engine.setConfig) that the file-plane-canonical embed pipeline does not read. -
src/core/embedding-migration.ts— provider-agnostic embedding migration core; owns the schema-transition + env-gate primitives.runSchemaTransition(engine, targetDim): ONE transaction rebuilds all three dim-pinned text-embedding-space columns (content_chunks.embedding,query_cache.embedding,facts.embedding) preserving each column'svector/halfvectype, HNSW gated onhnswIndexExpected(dims > 2000 skip the index; exact scans stay correct — 2048d targets work); image/multimodal columns deliberately untouched; AFTER commit it clearsembedded_atin yielding 50k batches (belt-and-braces hygiene — read surfaces key on the vector itself).planEmbeddingMigration: workload via widened stale predicates (absent-column fallback counts every chunk), cost,signature_census(top-5 page signatures — DB-reality corroboration of From),synopsis_tier_pages(context-tier downgrade consent), bundle-awarereranker_warning,dim_changevia the sharedschemaRebuildNeeded(null ⇒ rebuild, same computation apply uses).applyEmbeddingMigration: env≠target refusal (detectEnvOverride;detectEnvPresencedrives the ==target notice) → marker v2 write (same-target re-apply preservesstarted_at; retarget recordssupersededhistory +force_sunset_target) → full rebuild when the chunks column is off-width, else INDEPENDENT repair of stale dim-pinned columns (readDimPinnedWidths) → false-target-stamp clearing (chunk-model truth) + guarded NULL-signature-inclusive invalidation (embed_skip pages' retained vectors preserved; both viaembedding-invalidation.ts) BEFORE config writes (same-dim-swap crash safety) → DB plane → file-plane callback → cache purge.verifyMigrationComplete: the ONLY basis for "nothing to migrate" — column + pinned widths, wide stale census, false-stamp census blocker (embedded chunks whosemodelcontradicts a page already stamped with the target signature),missing_embeddingsresidue, chunkless-pages census, embed_skip-NULL visibility (report-only detail, never a blocker), marker state, un-merged file plane (or env-canonical), env-contradiction blocker; never trustsfrom_model.completeEmbeddingMigration: marker delete + completion stamp in ONE transaction (no lost-receipt crash window); accepts content-freeextra(smoke-check outcome).readMigrationState(corrupt-safe),readMigrationStatus(read-only, spend-free, everything-degrades-to-null),verifySearchRoundTrip(completion smoke check: query-sideembedQuery+searchVector, hit identity by page_id, NEVER throws, warn-don't-block, content-free samples),resolveRerankerExposure/resolveRerankerPlan/applyRerankerAction(D8 companion switch: bundle-resolved exposure; auto→target-provider default reranker, never a silent third provider; config write + cache purge in one tx),reconcilePageSignatures(batch-boundary stamp repair). Engine-pure; every step idempotent under crash + re-run — the NULL-embedding column is the checkpoint. Pinned bytest/embedding-migration.test.ts,test/migrate-embeddings-hardening.serial.test.ts,test/e2e/migrate-embeddings-postgres.test.ts. -
src/core/embedding-invalidation.ts— signature-truth guards shared by the migration + embed paths, engine-pure viaexecuteRaw(identical SQL on both engines).invalidateStaleSignatureEmbeddingsGuarded(engine, {signature, sourceId?, includeNullSignature?}): same semantics as the engines'invalidateStaleSignatureEmbeddingsPLUS theNOT (frontmatter ? 'embed_skip')predicate every stale selector applies — never NULL what nothing will re-embed (without it a migration would destroy vectors retained on embed_skip pages permanently and silently). ALL invalidation call sites (applyEmbeddingMigration,runEmbedCore,embedStaleChunks) route through it.countFalseStampedChunks/clearFalseStampedSignatures: the chunk-model truth cross-check — pages stamped WITH the target signature whose embedded chunks'modelnames another provider (bare provider-less model tails exempt, no surprise paid re-embed); count feeds plan/verify/--statushonesty, clear runs in apply so the NULL-signature-inclusive invalidation re-embeds them. Pinned bytest/embedding-migration.test.ts(#4305 false target stamps + #4306 embed_skip-safe invalidation). -
src/core/ze-exposure.ts— ZeroEntropy sunset exposure detection: the one shared answer to "is this brain still depending on ZeroEntropy?", consumed by thev0_46_3version migration (src/commands/migrations/v0_46_3.ts) and the stage-2 upgrade banner. Standalone module by design — it must outlive the ze-switch/retrieval-upgrade subsystem.detectZeExposure(engine, fileCfg?, env?)resolves exposure from the EFFECTIVE embedding model (envGBRAIN_EMBEDDING_MODEL→ fileembedding_model→ the legacy configless runtime fallback — resolution-based, NOT vector evidence, so a configless brain is exposed even with zero vectors), the resolved reranker (throughresolveSearchMode, the same plane search actually reranks with — the bundle default is Voyage, so only an explicitzeroentropyai:*search.reranker.modelrow exposes), and ZE-backed customembedding_columns(file + DB planes). Tri-statestatus: a failed DB probe downgrades to'unknown', never to'clear'— callers nag on'unknown'(fail-safe). Blast-radius counts (ZE-stamped pages, embedded chunks, est. re-embed cost at the recommended target) are LIMIT-capped atBLAST_RADIUS_CAP(100K) so a million-chunk brain can't stall apply-migrations on a full-table COUNT; blast-radius probe failures are informational and never flip status.renderZeActionRequired(exposure)renders the shared ACTION REQUIRED body both consumers wrap (migrate command with--dim 1024+ the 1280-not-a-valid-Voyage-width note, thevoyage:rerank-2.5reranker fix, the no-automated-custom-column-off-ramp honesty, and the env-override callout whenGBRAIN_EMBEDDING_MODELitself forces ZE). -
src/commands/migrate-embeddings.ts—gbrain migrate embeddings --to <provider:model> [--dim N] [--dry-run] [--yes] [--json] [--no-embed] [--status] [--reranker auto|off|keep|<model>] [--retarget] [--batch-size N] [--pace[=mode]] [--ignore-env-override] [--force-sunset-target](alias:gbrain retrieval-upgrade, slated for removal). ONE shared orchestrator for CLI and themigrate_embeddingsop:planMigrationFlow(plan +verifyMigrationComplete+ env presence + un-merged file plane + brain/DB identity viaredactPgUrl+ concurrent-writer census + reranker plan + in-flight-other marker) andexecuteMigrationFlow(globalgbrain-embedding-migrationDbLock → retarget gate under it → all-source embed locks sorted withincludeArchived→ live embed probe → apply → reranker probe + switch → drain viarunEmbedCore({heldLocks, …})so the migration never lock_skips itself, with a 5-min heartbeat whose refresh-false/3-errors ABORTS aslock_lost→ reconcile → completion smoke check stamped into the marker → transactional complete; locks released in finally). The skip path exits 0 ONLY onverify.completewith no pending retarget decision and no pending reranker action (a resolved switch/disable runs as a config-only completion).--statusis read-only/spend-free and never refuses on env (it REPORTS planes, key PRESENCE booleans only, censuses, markers verbatim incl. corrupt, the exact resume command, and the last completion + smoke-check outcome).persistEmbeddingFileConfigwrites throughloadConfigFileOnly(never persists env-sourced keys) and supports env-canonical no-file deployments (env pins target ⇒ proceed with notice). Exit codes: 0 completed/verified-no-work, 1 locked/refused/failed/incomplete (message names which; lock_skipped and lock_lost get their own copy), 2 non-TTY without--yes. Pinned bytest/migrate-embeddings-flow.serial.test.ts,test/migrate-embeddings-boundary.serial.test.ts,test/migrate-embeddings-hardening.serial.test.ts. Discovery surfaces all render the canonical command fromrenderCanonicalMigrationCommands(src/core/ai/defaults.ts): gateway deprecation line, init warnings, upgrade ACTION REQUIRED banner, doctorprovider_sunset, ze-switch refusal, advisor — drift-guarded bytest/canonical-migration-command.test.ts; the shutdown date lives once asZEROENTROPY_SUNSET_DATEindefaults.ts. -
src/commands/ze-switch.ts— pure refusal/redirect shim for the sunset ZeroEntropy switch. Every invocation refuses or redirects with exit 1 andreason: 'provider_sunset', printing the off-ramp (gbrain migrate embeddings --to voyage:voyage-4 --dim 1024 --dry-run); the--jsonrefusal envelope is{status:'refused', reason:'provider_sunset', migrate, migrate_preview, message}(live command + cost preview, each carrying an explicit--brainsuffix);--undoREDIRECTS (reads theze_switch_previous_snapshotconfig row and prints the exact migrate command that returns the brain to its pre-switch provider —{status:'redirected', reason:'provider_sunset', undo_command, undo_preview, message}in--json; missing/corrupt snapshot degrades to the refusal);--helpanswers engine-free viaCLI_ONLY_SELF_HELP+ theSELF_HELP_WITHOUT_ENGINEwrapper in cli.ts (exit 0, truthful sunset copy). Its flags stay REGISTERED (quoted literals feed the generated registry row) so existing scripts reach the refusal instead of a pre-dispatch unknown-flag error — the shim never consults them. Nothing here mutates the brain. The whole command is slated for removal after the sunset. Pinned bytest/ze-switch-cli.test.ts+test/cli-help-without-brain.serial.test.ts. -
src/commands/providers.ts—gbrain providers list | test [--touchpoint T] [--model ID] | env <id> | explain [--json]: provider-recipe discovery + smoke-testing over the recipe registry.listrendersformatRecipeTableagainst the SAME env the gateway actually sees (buildGatewayConfig(cfg).env, file-plane keys folded in) so the STATUS column matches whatproviders testand init would report. Home of the ONE shared sunset-marker primitive (sunsetMarkerText/sunsetMarker, generic onrecipe.sunset— any future provider sunset inherits it) consumed by all three human-facing renderings so they can't drift: theliststatus cell,explainrows (lead marker is ⚠ regardless of key readiness — never a green ready-check on a sunsetting provider), and theenvblock, where the pureformatEnvOutput(recipe, env)(testable without spawning the CLI) replaces the signup funnel (setup_url/setup_hint) with the deprecation notice, replacement models, and the canonical migration command fromrenderCanonicalMigrationCommands— key STATUS still renders for existing users. Pinned bytest/providers.test.ts. -
src/core/conversation-parser/— 19-pattern built-in chat-format registry + opt-in LLM polish/fallback. Modules:types.ts(PatternEntry + ParseResult + DateContext + CaptureMap + TimezonePolicy),builtins.ts(19 hand-vetted patterns sourced from public format docs — iMessage/Slack, Telegram bracket + text-export, bold-paren-time, bold-time-dash, bold-name-no-time, chatgpt-export-you-chatgpt, Discord classic + export, WhatsApp ISO + US, Signal, Matrix/Element, IRC classic + weechat, Teams export, markdown-heading-turn; module-load validation runs everytest_positive[]+test_negative[]sample at startup so a typo in any built-in regex makes gbrain refuse to start;DEFAULT_SPEAKER_CLEANexported as a module-level default),parse.ts(orchestrator with pattern-priority scoring across the first 10 lines + date derivation chainexplicit > frontmatter.date > effective_date > '1970-01-01'+ multi-line continuation + timezone warning; also populatesParseResult.unrecognized_headings— when the WINNING pattern is heading-anchored, heading-shaped lines whose label is outside the pattern's speaker set FOLD into the previous turn's body (or drop before the first anchor) while the parse still returnsregex_match, silently crediting one speaker with another's words; detection is diagnostic-only and unconditional (not behindopts.diagnostic— the extractor's decline gate depends on it), fence-aware (a fence closes only on ITS OWN marker; an unclosed fence suppresses detection below it), labels deduped/capped at 10 entries ≤48 chars, undefined when empty so healthy-page JSON stays byte-identical;gbrain conversation-parser scansurfaces the field in both human and JSON output),llm-base.ts(sharedrunLlmCall<T>with content-hash cache in-process + DB-persistent via migration v97 + 4-strategy JSON repair + Anthropic-key probe; polish and fallback are thin wrappers),llm-polish.ts(opt-IN; headroom guard skips when tracker within $0.10 of cap; pureapplyPolishfor merge/drop/edit ops),llm-fallback.ts(opt-IN; NO regex inference + NO persistence),eval.ts(scoreFixture+aggregateScores+parseFixtureJsonlfor the fixture-corpus CI gate),nightly-probe.ts(DI-stubbed; mode-gated default tokenmax=ON, conservative/balanced opt-in; adversarial false-positive detection). Patternbold-name-no-time(regex/^\*\*(?!\[)(.+?):\*\*\s*(.*)$/, ordered after the time-bearing bold patterns) parses**Speaker:** textwith NO per-line timestamp (Circleback/Granola/Zoom), anchoring every message atT00:00:00Zof the frontmatter date (line order preserves sequence, same no-time convention asirc-classic); the(?!\[)lookahead rejects telegram-bracket**[18:37] Name:**; non-shadow is the colon-INSIDE-bold regex (NOT declaration order —parse.tsscores every candidate independently, order is only the tie-break). Because**Label:** textis a common prose idiom, the pattern sets optionalPatternEntry.score_full_body: truesoparse.tsrecomputes the winner's acceptance score over the FULL body before theSCORING_MIN_ACCEPTANCEfloor, keeping a bold-label notes page atno_match. Patternchatgpt-export-you-chatgpt(regex/^\*\*(You|ChatGPT):\*\*\s*(.*)$/, declared just beforebold-name-no-timeso it wins the score tie on the two literal ChatGPT-export labels) parses ChatGPT's web-export → Markdown shape, where each turn's**You:**/**ChatGPT:**anchor is followed by a blank line and a multi-paragraph reply;bold-name-no-timealone cannot absorb that shape (multi_line: falsescores a long reply's density near zero and falls belowSCORING_MIN_ACCEPTANCE), so this pattern setsmulti_line: true+score_continuations_as_body: true(excludes non-**-prefixed reply lines from the density denominator) +score_full_body: true(belt-and-suspenders full-body recompute of the winner). The speaker capture is a closed two-value enumeration (YouorChatGPTexactly, never an arbitrary(.+?)label), so it cannot reopenbold-name-no-time's BROAD-REGEX GUARD against notes pages. Because a bare**You:**/**ChatGPT:**heading (unlikebold-time-dash's bold-name+time+dash anchor) is a plausible label in ordinary prose ABOUT ChatGPT, the pattern also sets the optionalPatternEntry.score_continuations_min_distinct_speakers: 2:scoreFromLinesonly grants the continuation-density-exclusion score when the anchor lines that fully matchregexcollectively capture at least that many DISTINCTspeaker_groupvalues, so a page with only a solitary**You:**heading (or several repeats of the SAME role) falls back to the ordinary flat-density score instead of getting the same acceptance immunity a genuine two-party exchange gets. It ALSO setsPatternEntry.score_continuations_max_preamble_lines: 5, because distinct-speaker count alone still lets ONE genuine**You:**/**ChatGPT:**pair embedded anywhere inside an otherwise unrelated long document (both roles present) through:scoreFromLinestracks the index of the FIRST fully-matching anchor line and only grants the immunity when that index is at or before the bound (tolerating a short title/heading preamble, not an arbitrary amount of unrelated prose before the transcript). Patternbold-paren-timeparses**Speaker** (HH:MM): textand(HH:MM:SS)(date_source: frontmatter). Fallback gates:SCORING_HEAD_TRIGGER_THRESHOLD = 0.3triggers a full-body re-score when the head pass scores below that;SCORING_MIN_ACCEPTANCE = 0.05blocks essay false-positives. ExportedscorePatternFull(body, entry); privategetNonBlankLines(body, headCap?)+scoreFromLines(lines, entry)DRY the quick_reject+regex loop. CLI surfaces atsrc/commands/eval-conversation-parser.ts(gbrain eval conversation-parser <fixture.jsonl>exit 0/1/2, wired intobun run verifyviacheck:conversation-parser) andsrc/commands/conversation-parser.ts(scan <slug>debug,list-builtins,validate <file>). Doctor checks:conversation_format_coverage,progressive_batch_audit_health,conversation_parser_probe_health. Pinned bytest/conversation-parser/{parse,llm-base,llm-fallback,llm-polish,nightly-probe}.test.ts+ the 27-case baseline attest/extract-conversation-facts.test.ts(back-compat invariant). Migration v97 (conversation_parser_llm_cache_table). Fixtures attest/fixtures/conversation-formats/{imessage,imessage-time-only-12h,telegram-bracket,whatsapp-iso,whatsapp-us,signal-export,irc-classic,irc-weechat,matrix-element,teams-export,all,adversarial,bold-name-no-time,bold-time-dash,chatgpt-export-you-chatgpt}.jsonlwithscripts/check-fixture-privacy.shbanning real-name leaks. Maintainer guidance: conversation parser patterns. -
src/core/progressive-batch/— shared ramp-up + cost-cap + verification primitive (trial 10 → ramp 100 → ramp 500 → full, with verification at each stage), with verifier+policy injection (callers describe HOW TO MEASURE SUCCESS, not WHEN TO WAIT FOR CTRL-C). Modules:types.ts(Stage, StageVerdict, AbortReason, discriminatedVerifierunionOutputCountVerifier | IdempotentMutationVerifier | NoopVerifier, Policy, StageReport),orchestrator.ts(runProgressiveBatch(items, verifier, policy, runner)— readsgetCurrentBudgetTracker()ahead ofPolicy.maxCostUsdfail-closed; null both ways triggersabort_cost_cap reason='no_budget_safety_net'),audit.ts(ISO-week JSONL at~/.gbrain/audit/progressive-batch-YYYY-Www.jsonlvia the sharedaudit-writerprimitive),stage-report.ts(ASCII formatter for the defaultPolicy.onStageReport). Env knobs:GBRAIN_PROGRESSIVE_BATCH_DISABLED=1,GBRAIN_PROGRESSIVE_BATCH_AUTO=1(skip Ctrl-C grace),GBRAIN_PROGRESSIVE_BATCH_STAGES=10,100,500. Sites that "jump straight to full" stay that way by default; ramp is opt-in per-site viaPolicy.interactiveAbortMs > 0. Pinned bytest/progressive-batch/orchestrator.test.ts(35 cases, every verdict path). -
src/commands/extract-conversation-facts.ts+src/core/cycle/conversation-facts-backfill.ts— bulk fact extraction for long-form conversation pages. Walks conversation/meeting/slack/email/imessage/imessage-daily pages, splits them into time-windowed segments (30-min gap or 30-msg cap), prepends a topical/temporal header, and uses the strictextractFactsFromTurnWithOutcome()path so provider and output failures remain retryable instead of becoming successful empty pages. Invariants: strict per-source core (runExtractConversationFactsCore({sourceId, ...})always takes one sourceId; CLI + cycle phase each do their own multi-source iteration becausePHASE_SCOPE='source'is taxonomy-only); bounded two-phase enumeration (paginatedlistPages({type, sourceId, limit:10}); per-page body capMAX_PAGE_BODY_BYTES=25MB); page-globalrow_numaccumulator (the facts unique index is(source_id, source_markdown_slug, row_num)); versioned snapshot-bound outcomes (cli:extract-conversation-facts:terminal:v2for complete pages and a separatenon-extractable:v2source for recognized pages with no eligible segment); operation checkpoints are scheduling hints only and never suppress a replay without a matching v2 outcome; optionalopts.budgetTracker?is used as-is, while an absent tracker is created withmaxCostUsd; body reads cover compiled truth, timeline, and configured raw-transcript sidecars;facts.extraction_enabledkill-switch with--override-disabled;--types LISTallowlist (conversation,meeting,slack,email,imessage,imessage-daily);--backgroundviamaybeBackground; and speaker-shaped-fold decline — when the parse reportsunrecognized_headingscontaining a speaker-shaped label (1-2 title-cased words, not in the doc-heading stoplist — the stoplist gates the DECLINE only, so a miss is warn-noise, never data loss) AND the parse produced fewer than two distinct speakers, the page is declined instead of extracted under wrong attribution: counted inpages_skipped_unrecognized_speaker(Result/CLI/cycle surfaces), warned to stderr with the folding pattern id, and deliberately NON-terminal (no durable audit row and no orphan cleanup, so a future parser/pattern fix retries the page); multi-speaker pages with folds proceed warn-only. The companionconversation_facts_backfillcycle phase is default-off, iterates every source, and enforces per-source plus brain-wide cost and wall-time caps. Migration v94 provides the partial facts index used by outcome lookups.computeConversationFactsBacklogCheckreports fresh completed, scanned-not-extractable, and unfinished counts separately, warning when more than 10 eligible pages lack a fresh v2 outcome.sources auditexposesfacts_backfill_estimate: {pages, est_segments, est_cost_usd, types}. Pinned bytest/extract-conversation-facts.test.tsandtest/doctor-conversation-facts-backlog.test.ts.--workers Nfor LLM-bound fact extraction over conversation pages, with a per-page advisory lock viasrc/core/db-lock.ts:withRefreshingLock(lock idextract-conversation-facts:<source>:<slug>, TTLPER_PAGE_LOCK_TTL_MINUTES=2with 20s refresh viaMath.max(15s, 120s/6);LockUnavailableErrortriggers skip-and-continue with rate-limited log per (source, minute) +pages_lock_skippedcounter + CLI exits 3 when non-zero AND no hard failures).deleteOrphanFactsForPage(engine, sourceId, slug)provides delete-orphans-first replay safety — wipes facts from a prior crashed run for this (sourceId, slug) before re-extracting, so a terminal audit row can never certify a partial insertFacts failure.assertFactsEmbeddingDimMatchesConfig(engine)is the startup preflight (throwsFactsEmbeddingDimMismatchErrorwith paste-ready ALTER hint BEFORE the first insert; cached per engine via WeakMap). Result type carriespages_lock_skipped+orphan_facts_cleaned. Checkpoint state is a sharedcpMap: Map<slug, endIso>(NOT a per-page-mutatedcpEntries: string[]) so atomicMap.setsurvives parallel workers. Minion handlerextract-conversation-factsinsrc/commands/jobs.tsround-tripsworkersviajob.data.workersfor--background --workers 20. Cycle config keycycle.conversation_facts_backfill.workers(default 1; opt-in concurrency under brain-wide cost + walltime caps). Pinned bytest/extract-conversation-facts-workers.test.ts+ the existing extract-conversation-facts behavioral tests. withsrc/commands/doctor.tsdurable outcome authority: page completion survives operation-checkpoint GC through versioned terminal audit rows (cli:extract-conversation-facts:terminal:v2), while recognized pages with no eligible segment use the separatecli:extract-conversation-facts:non-extractable:v2source. Each outcome is bound to the exact parsed snapshot: regular pages usecontent_hashplus the UTC effective date; raw-conversation sidecars and legacy null-hash pages use a canonical SHA-256 over every parser-relevant input. Selection checks the token before locking, refetches under the lock, and verifies it again before writing the outcome, so an edit cannot be certified by stale work. The strict extraction path treats provider, refusal, truncation, malformed/schema-invalid output, segment-write, cleanup, and terminal-write failures as unfinished work; bulk failures incrementpages_failed, affect CLI/cycle receipts and exit status, and never advance the legacy checkpoint. Checkpoints are only a scheduling hint: a slug without a matching v2 outcome is replayed delete-first.no_match, errors, cancellation, and dry runs never become durable negatives. Result, CLI, cycle, and doctor surfaces keep completed, scanned-not-extractable, unfinished, failed, and lock-skipped counts separate. See Conversation backfill durable outcomes for the operator and maintainer contract. Pinned bytest/extract-conversation-facts.test.tsandtest/doctor-conversation-facts-backlog.test.ts. -
src/core/facts/conversation-types.ts— single-source conversation-type allowlist (ALLOWED_TYPES: conversation, meeting, slack, email, imessage, imessage-daily +AllowedType). Every consumer derives from this frozen leaf module —extract-conversation-facts.ts(which re-exports both names verbatim for its existing importers), the cycle backfill phase,doctor.ts+doctor/checks/search-eval.ts,sources.ts,jobs.ts— so no hand-copied list can drift. It lives undersrc/core/facts/(notsrc/commands/) becausescripts/generate-flag-registry.tsscans option-shaped literals one relative-import level deep — importing the constant straight fromextract-conversation-facts.tswould transitively attribute that command's whole option surface to doctor/jobs/sources in the generated CLI_ONLY registry. Drift-guarded bytest/conversation-facts-type-allowlist-drift.test.ts. -
src/core/link-extraction.ts— shared library for the graph layer.extractEntityRefs(canonical) matches[Name](people/slug)markdown links and Obsidian[[people/slug|Name]]wikilinks;extractPageLinks,inferLinkTypeheuristics (attended/works_at/invested_in/founded/advises/source/mentions),parseTimelineEntries,isAutoLinkEnabled. Markdown links, bare-slug prose refs, and slash-shaped wikilinks match ANY dir-shaped path (ANY_DIR_SEGMENT), not a directory whitelist — nonexistent targets are dropped by the persist paths' page-existence checks (resolveCandidateSources, put_page's allSlugs filter,addLinksBatchINNER JOINs) and counted asskippedMissingTargetin the extract summaries; theDIR_PATTERNwhitelist is only the typed fast-path for pass-2b wikilinks (non-whitelisted[[dir/...]]get an equivalent direct typed candidate in pass 2c, plus the flag-gated suffix rescue for non-exact matches). Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts. Opt-in global-basename wikilink resolution (default off):WIKILINK_GENERIC_REcatches bare[[name]]wikilinks outsideDIR_PATTERN(third pass2cinextractEntityRefs);EntityRef.needsResolution: truetags refs from this pass (the ref'sslugis the wikilink TARGET,namethe optional display alias).SlugResolvergains optionalresolveBasenameMatches(name): Promise<string[]>(multi-match by design — emits one edge per matching page). The single shared basename matcher isbuildBasenameIndex(slugs)+queryBasenameIndex(index, name)+normalizeBasename(keys raw/lower/slugified tail, stable-sorted shorter-first then lexical), used bymakeResolver, the FSresolveBasenameMatchesFromSlugs, AND the doctor check so they cannot drift.makeResolver(engine, {mode, sourceId})builds the index lazily viaengine.getAllSlugs({sourceId})— source-scoped so a bare[[name]]never resolves to a same-tail page in a different source.extractPageLinksgainsopts.globalBasename(routesneedsResolutionrefs throughresolveBasenameMatcheskeyed onref.slug, emits candidates taggedlinkType: 'wikilink_basename'+linkSource: 'wikilink-resolved', skips self-loops) andopts.skipFrontmatter. All three surfaces (FS extract, DB extract,put_pageauto-link) tag provenance withlink_source='wikilink-resolved';put_pageincludes it in its reconcilable-edge set so stale basename edges are removed when the wikilink or the flag goes away. ExportsWIKILINK_BASENAME_LINK_TYPE+isGlobalBasenameEnabled(engine)(resolution order: envGBRAIN_LINK_RESOLUTION_GLOBAL_BASENAME→ DB configlink_resolution.global_basename→ default false).gbrain doctor'slink_resolution_opportunitycheck surfaces a paste-ready enable hint when ≥5 bare wikilinks would resolve AND ≥20% match. Migration v113 widenslinks_link_source_checkto admit'wikilink-resolved'; v114 then opens it to any kebab-case provenance (^[a-z][a-z0-9]*(-[a-z0-9]+)*$, ≤64 chars) so external derivers register their own tag (e.g.citation-graph) without a migration.LINK_EXTRACTOR_VERSION_TSalso lives here (bump likeCHUNKER_VERSIONto invalidate prior extract-stale stamps). Pinned bytest/link-extraction.test.ts,test/extract-fs.test.ts,test/doctor.test.ts,test/e2e/global-basename-pglite.test.ts. -
src/commands/extract.ts—gbrain extract links|timeline|all [--source fs|db] [--source-id <id>]: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use for live brains with no local checkout). No in-memory dedup pre-load — candidates buffered 100 at a time and flushed viaaddLinksBatch/addTimelineEntriesBatch;ON CONFLICT DO NOTHINGenforces uniqueness at the DB layer,createdcounter returns real rows inserted.ExtractOpts.slugs?: string[]enables incremental extract viaextractForSlugs()(single combined links+timeline pass); the cycle path threads sync'spagesAffectedthrough.walkMarkdownFiles(brainDir)still runs to buildallSlugsfor link resolution.--source-id <id>scopes extraction to one source on federated brains (resolved viaresolveSourceWithTier()before any SQL; failures hintgbrain sources list).gbrain extract --stale [--source-id <id>] [--catch-up] [--dry-run] [--json]branch (extractStaleFromDB) — incremental DB-source link+timeline sweep over pages whosepages.links_extracted_atwatermark is stale. Embedded callers (the cycle) passquiet: trueso the helper writes nothing to stdout, including its JSON result; standalone CLI output is unchanged. InjsonModeevery fs path,extractForSlugsincluded, reports a dropped flush as a{event:'batch_error'}stderr line rather than going silent. Stale predicate (shared by both engines + the doctor check):links_extracted_at IS NULL OR links_extracted_at < LINK_EXTRACTOR_VERSION_TS::timestamptz OR updated_at > links_extracted_at(theupdated_atarm catches MCPput_page/sync --no-extractedited-since-extract). ThreeBrainEnginemethods (parity in postgres-engine.ts + pglite-engine.ts + bootstrap probes):countStalePagesForExtraction(opts?),listStalePagesForExtraction({batchSize, afterPageId?, sourceId?, versionTs?})(returns page CONTENT to avoid N+1getPage;rowToStalePagein utils.ts maps the row,StalePageRowin types.ts),markPagesExtractedBatch(refs, defaultExtractedAt)(3-array unnestslug[],source_id[],ts[]; each ref may carry its ownextractedAt).STALE_BATCH_SIZEdefault 25 (GBRAIN_EXTRACT_STALE_BATCH; small because page bodies are unbounded — the LIMIT is the only fetch-time memory bound);STALE_TIME_BUDGET_MS30min wall-clock (--catch-upremoves the cap). Non-swallowing flush: link/timeline flush throws propagate and abort the batch; stamp LAST so a crash leaves pages unstamped and they re-extract idempotently (addLinksBatchON CONFLICT DO NOTHING + timeline dedup).extractStaleFromDBstamps with each row's READupdated_at(notnow()), so a concurrent edit during the sweep keeps the page stale and it re-extracts next run rather than marked fresh-with-old-content. Source-correct stamping at DB-extract sites viastampExtracted(best-effort, never throws);extractLinksFromDBonly stamps the combined watermark whensubcommand === 'all'(a links-only run must not hide timeline staleness).LINK_EXTRACTOR_VERSION_TSlives insrc/core/link-extraction.ts(bump likeCHUNKER_VERSIONto invalidate all prior stamps). Migration v112 (pages_links_extracted_at) adds nullableTIMESTAMPTZ+ composite(source_id, links_extracted_at)index (CONCURRENTLY + invalid-remnant pre-drop on Postgres, plain on PGLite), NO backfill so the real backlog surfaces on firstgbrain doctor. Schema parity in schema.sql + pglite-schema.ts + schema-embedded.generated.ts +REQUIRED_BOOTSTRAP_COVERAGE.src/commands/doctor.ts:checkLinksExtractionLag(thelinks_extraction_lagcheck, also indoctorReportRemote) warn-only by default (>GBRAIN_EXTRACTION_LAG_WARN_PCT, default 20%; sharedEXTRACTION_LAG_WARN_PCT_DEFAULT+EXTRACTION_LAG_MIN_PAGES=100+ exported_resolveEnvNumber), hard-fails only whenGBRAIN_EXTRACTION_LAG_FAIL_PCTis set; vacuous-skips <100 pages (no--source); pre-v112 brains graceful-skip viaisUndefinedColumnError; strictly a SQL COUNT (safe on remote/thin-client).src/commands/sync.tscarries--no-extract(threaded through single-source +--all+syncOneSource), stampslinks_extracted_atforpagesAffectedat the inline-extract call site, andmaybeExtractionNudgeprints a one-line stderr nudge after asynced | first_sync | up_to_datesync that leaves a backlog (shouldNudgeAfterSyncpure predicate;GBRAIN_SYNC_NO_EXTRACT_NUDGEsuppresses).src/core/retry.tslists'extract.stale'inBATCH_AUDIT_SITES;src/core/doctor-categories.tslistslinks_extraction_laginBRAIN_CHECK_NAMES. Pinned bytest/extract-stale.test.ts(incl. edited-after-stamp + crash-contract),test/sync-inline-extract-stamps.serial.test.ts,test/sync-nudge-status-gate.test.ts,test/doctor-links-extraction-lag.test.ts, engine-parity (Postgres↔PGLite) for the 3 methods + v112 round-trip. The stale SELECT in both engines projects a deterministic full-µs UTC stringto_char(updated_at AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS.US"Z"') AS updated_at_iso(carried onStalePageRow.updated_at_iso, populated byrowToStalePagein utils.ts with an ISO-only fallback — neverString(Date), which::timestamptzmisparses);extractStaleFromDBstamps that exact-precision value, not a JSDate(which truncates to milliseconds), so on Postgreslinks_extracted_atequals the row'supdated_atto the microsecond andlinks_extraction_lagclears — a ms-truncated stamp stays strictly below the µsupdated_atand leaves every page perpetually stale, whichextract --stalecould never satisfy.to_char(not raw::text, which isDateStyle-fragile) keeps the projection deterministic. ThemarkPagesExtractedBatchSQL is unchanged, so callers passing an explicit (e.g. backdated)extractedAtstill control the stamp and the edited-since arm is exact. A deterministic PGLite case intest/extract-stale.test.tsinjects a µsupdated_at, runs--stale, and asserts the lag is 0 and stays 0. -
Extract CLI help —
EXTRACT_HELPinsrc/commands/extract.tsis the canonical detailed usage shared by--helpand invalid-subcommand errors.src/cli.tsroutesextract --helpbefore engine connection so help works on unconfigured installs; the top-level TOOLS block advertises every mode-specific flag. Pinned bytest/cli-help-discoverability.test.ts. -
src/core/extract/receipt-writer.ts+src/core/extract/rollup-writer.ts+src/commands/extract-status.ts+src/commands/extract-explain.ts+src/commands/extract-benchmark.ts+src/core/schema-pack/scaffold-extractable.ts— unified extract operator surface. Every shipped extractor (deterministicfacts.conversationinsrc/commands/extract-conversation-facts.ts+ three LLM-backed cycle phases atsrc/core/cycle/{extract-atoms,synthesize-concepts,propose-takes,extract-facts}.ts) writes ONE receipt page per run (writeReceipt) + UPSERTs a row toextract_rollup_7d(upsertExtractRollup). Receipt slugextracts/{date}/{kind}/{source_id}/{run_id_short}/round-{N}.md; frontmatter stamps BOTHtype: extract_receiptANDdream_generated: true(belt+suspenders against extraction-loop guard drift).extract_receiptjoinsALL_PAGE_TYPESinsrc/core/types.ts;extracts/prefix gets a 0.3x source-boost demote insrc/core/search/source-boost.ts. Migration v104 addsextract_rollup_7d (kind, source_id, day, cost_usd, halt_count, eval_pass_count, eval_fail_count, round_completed_count, rollup_write_failures, updated_at)with PK(kind, source_id, day)+idx_extract_rollup_7d_day. Rollup writes best-effort with process-scoped error-dedup so transient DB failures bumprollup_write_failuresinstead of crashing the cycle.extract_healthdoctor check reads last 7 days, warns at halt-rate > 10% AND when rollup_write_failures > 0; pre-v104 brains reportok. CLI:gbrain extract status [--source-id ID] [--kind X] [--verbose] [--json](7-day rollup, sorted halt_rate desc + cost desc, top-5 + "more rows" hint, stableschema_version: 1);gbrain extract --explain <kind>(resolution chain pack-declared vs built-in cycle phase, prompt_template + fixture_corpus paths with✓/(missing), last 7d rollup);gbrain extract benchmark --pack X --kind Y(loads pack fixture corpus through strict path validation — rejects absolute paths,..traversal, null bytes, AND symlinks resolving outside pack root; ships as a stub-reporter).src/core/schema-pack/manifest-v1.tswidensextractablefromz.boolean()toz.union([z.boolean(), ExtractableSpecSchema])(carriesprompt_template,fixture_corpus,eval_dimensions,benchmark_min_recall, plus reservedverifier_path— parses but refuses at runtime);extractableSpecsFromPack+getExtractableSpec+refuseVerifierPathInV042insrc/core/schema-pack/extractable.ts;gbrain schema scaffold-extractable <type> --pack <pack>declares the type extractable, generates 5 placeholder fixtures + a prompt template stub underpacks/<pack>/{fixtures,prompts}/extract/, refuses to overwrite without--force. Pinned bytest/extractable-spec-widening.test.ts(22),test/extract/receipt-writer.test.ts(12, canonical PGLite block R3+R4),test/extract/benchmark.test.ts(17),test/extract/status.test.ts(15),test/schema-pack/scaffold-extractable.test.ts(15, privacy guards),test/doctor-extract-health.test.ts(8). -
src/commands/import.ts—gbrain import <path> [--source-id <id> | --source <id>]: page import with the path-set checkpoint.runImportcontains NOprocess.exit: all five preflight/argv failure sites (deferred-setup embedding sentinel, missing embedding credentials, invalid--workers, missing dir arg, unreadable target) throw the exported typedImportAbortError(carriesexitCode; the user-facing message is printed at the throw site), so in-process callers — thesync_brainMCP op, autopilot, the Minionsimporthandler — survive a failed preflight as a normal tool/job error instead of the whole serving process dying mid-call. The CLI'simportcase catches it and exitse.exitCode. Pinned bytest/import-abort-error.test.ts. Human-only output (theinfo()lines and the end-of-runImport completesummary) goes throughslog()fromconsole-prefix.ts— identical toconsole.logoutside a wrap, but an in-process caller in its own--jsonmode (sync --json→performFullSync→runImport, wrapped inwithHumanLogsToStderr) keeps stdout pure JSON;import --jsonitself sends them to stderr directly. Pinned bytest/sync-json-stdout-clean.test.ts(first-sync case) +test/import-json-stdout.serial.test.ts.--source-id <id>(alias--source <id>; passing both with different values aborts) routes pages to the named source (resolved viaresolveSourceWithTier()at the boundary; consistent acrossimport,extract,graph-query,sources current;graph-querytakes--source <id>). Pinned bytest/import-source-id.test.ts. An unscoped import that resolves to tierseed_defaultruns the default-write assessment throughassessDefaultWriteGuardOnce(engine)(memoized per engine — in-process callers such as thesync_brainop, autopilot, and minion sync invokerunImportrepeatedly on one engine) and WARNS viaformatDefaultWriteWarning(a, '--source-id')when the brain's pages overwhelmingly live outsidedefault— never aborts, since aborting would take an in-process host down mid-call;GBRAIN_ALLOW_DEFAULT_WRITE=1skips it. Pinned bytest/import-default-write-guard-once.test.ts.gbrain importCLI +runImportlibrary entrypoint. Uses a path-set checkpoint viasrc/core/import-checkpoint.ts(the walk still appliessortNewestFirst()for embed-cost ordering, but checkpoint correctness does not depend on sort order). A file enterscompleted: Set<relativePath>only when itsprocessFilereturns success (including content-hash short-circuit no-ops); failed files never enter the set so the next run retries them automatically with no manual~/.gbrain/import-checkpoint.jsondelete. This rules out three failure classes: parallel-import-with-slow-worker dropping the slow file on crash-resume (the slow file isn't incompleteduntil its ownprocessFileresolves), failed-file-bumps-counter-past-itself (failures don't add tocompleted), and sort-flip-drops-newest-N-on-cross-version-resume (order is not part of the checkpoint). Old positional checkpoints are detected and discarded with a stderr line on first resume (re-walking is cheap becausecontent_hashshort-circuits unchanged files). Checkpoint persists every 100 successful adds, not every 100 processed files. ThemanagedBookmarkopt (set byperformFullSyncwhenrunImportis the full-sync engine) suppressesrunImport's ownsync.last_commitadvance so the sharedapplySyncFailureGate(src/core/sync-failure-ledger.ts) owns the bookmark + failure-ledger gating on that path — one gate decides advance/block/auto-skip across both sync paths. Pinned bytest/import-checkpoint.test.ts+test/import-resume.test.ts(incl. the SLUG_MISMATCH retry case).collectSyncableFiles' shared emit filterisCollectibleForWalkerapplies the SAME segment-levelpruneDirgate as incremental sync'sclassifySync— load-bearing for thegit ls-filesfast path, which enumerates tracked files under dot-dirs/vendored trees that the FS walk never descends into; without itsync --fullwould import (and resurrect soft-deleted) pages incremental sync excludes. Pinned bytest/import-git-fastpath-prune.test.ts.runImportopts also carryexclude(glob filter over dir-relative paths, threaded byperformFullSyncforsync --exclude; warns when every file is excluded — NAV-4) andslugRoot(slug/source_pathbase for monorepo subdir syncs; the resume checkpoint stays dir-relative perresumeFilter's contract). The--jsonpayload carries, beside the counts,unchanged(content-hash no-ops),malformed_skipped, andfailures: [{path, error}]— returned per-file failures (invalid frontmatter, oversize, symlink, slug mismatch) count under bothskippedanderrors, are excluded fromunchanged, and producestatus: "partial"with a nonzero CLI exit. Failed imports retain their completed-path checkpoint; queued imports report failure instead of successful completion. The failure ledger is written only for Git repositories, so the JSON failure list remains a non-Git caller's per-file channel. Pinned bytest/import-json-stdout.serial.test.ts. -
src/core/import-checkpoint.ts—loadCheckpoint(brainDir),saveCheckpoint(brainDir, completed),resumeFilter(files, completed, brainDir),clearCheckpoint(), plus theImportCheckpointtype. Path-set format{schema_version, brainDir, completed: string[]}. Atomic write via.tmp+rename()so a mid-write crash never leaves a partial JSON.loadCheckpointreturnsnullon: missing file, malformed JSON, brainDir mismatch (ran against a different brain), and the old positional format (logged to stderr before discard).resumeFilterreturns{toProcess, skippedCount}— pure, no I/O, deterministic.clearCheckpointis no-op-on-missing for clean-exit cleanup. HonorsGBRAIN_HOMEviagbrainPath()sowithEnv({GBRAIN_HOME: tmpdir})test isolation works without monkey-patching fs. Best-effort persistence —saveCheckpointlogs warnings on write errors but never throws. -
src/commands/graph-query.ts—gbrain graph-query <slug> [--type T] [--depth N] [--direction in|out|both] [--source <id>] [--include-foreign]: typed-edge relationship traversal (renders indented tree). The local walk is scoped to the resolved source (--source/GBRAIN_SOURCE/.gbrain-source/ path match / brain default viaresolveSourceId, the same scalar scope thetraverse_graphop gets fromsourceScopeOpts); an explicit--sourcethat fails to resolve throws.--include-foreign(or--source __all__) drops the scope so the walk spans every source. The foreign-edge footer (N edges to foreign-source pages hidden; pass --include-foreign) prints only when the walk was actually scoped, so cross-source edges never disappear silently and the footer never describes a filter that did not apply. The local path usesengine.traversePathsDetailedand prints a stderr note when the walk hitTRAVERSE_PATH_ROW_CAP(shallowest edges kept; lower--depthor narrow with--type/--direction); the thin-client path goes through thetraverse_graphop, which has nosource_idparam (the server scopes the walk to the caller's grant): an explicit--sourcethere is rejected with exit 1 (same policy + wording asapplyThinClientSourceScope, never silently dropped) and--include-foreignprints a stderr note that it is not forwarded. Pinned bytest/graph-query.test.ts(local) +test/graph-query-thin-client-source-flag.serial.test.ts(thin client; mocksisThinClient+callRemoteTool). -
src/commands/sources.ts—gbrain sources {list,add,remove,archive,restore,archived,purge,current,status,audit}.current [--json]callsresolveSourceWithTier()and printssource_id,tier(flag | env | dotfile | local_path | brain_default | seed_default), and optionaldetail(decision table inskills/conventions/brain-routing.md).status [--json]— read-only per-source dashboard (last sync, staleness, page count, embedding coverage, unacked failures); thin wrapper aroundbuildSyncStatusReport+printSyncStatusReportfromsrc/commands/sync.ts;--jsonemits stable{schema_version: 1, sources, ...}on stdout; filters input tolocal_path IS NOT NULL AND archived IS NOT TRUE.audit <id> [--json]— read-only dry-run disk scan for size distribution + would-blocks + junk-pattern hits WITHOUT touching the DB; walkssources.local_path, reads each markdown file, runsassessContent()fromsrc/core/content-sanity.ts, aggregates by verdict (ok | warn_oversize | hard_block_junk_pattern). The liverunStatushealth table gains aBACKFILLcolumn betweenEMBEDandFAILS(active(N)beatsqueued(N)beatsidle, fromSourceMetrics.backfill_active/backfill_queuedinsrc/core/source-health.ts) so operators see deferredembed-backfillminion work aftersync --allexits 0;jobCountsBySourceinsource-health.tswidens itsminion_jobsSQL with twoCOUNT(*) FILTER (WHERE name = 'embed-backfill' AND ...)aggregates (best-effort, all-0 on pre-minions brains).removepre-checks OAuth-client referents (guided refusal viaformatClientReferentsBlock— the raw FK violation never reaches the operator), then commits the row DELETE atomically with an in-tx referents re-check (a registration racing the pre-check maps to the same guided refusal); durability-scaffolding teardown (unhardenBrainRepo: hook/cron/credential) runs only AFTER the commit, so a refused delete leaves the scaffolding intact.purgeruns the same referents pre-check before its hard DELETE. Pinned bytest/content-sanity.test.ts,test/import-file-content-sanity.test.ts,test/source-health.test.ts. -
src/commands/sources-set-path.ts—gbrain sources set-path <id> <path> [--force]: non-destructivelocal_pathrepair (a DB pointer update, never touches files on disk). Loud rejection on a missing source (never a silent 0-row UPDATE); prints the prior value before changing it so the change is visible/reversible; absolutizes relative paths + normalizes MSYS/Git-Bash drive spellings BEFORE the existence check and the UPDATE (storing.verbatim would plant the phantom-path class this command exists to fix). Enforces the sharedassertNoOverlappingPathguard (src/core/sources-ops.ts) before the UPDATE — a repointed source nesting inside or swallowing another source's tree exits 6 with the sameoverlapping_pathwordingsources adduses;--forcebypasses it. Lives in its own module (like sources-demo.ts) so sources.ts stays under its ratchet ceiling. Thedefault_source_local_pathdoctor check names this as the repair. -
src/commands/reindex-frontmatter.ts—gbrain reindex-frontmatter.reindexFrontmatterCli(engine, args)takes the ALREADY-CONNECTED engine from cli.ts's dispatch; it must never build/connect its own engine — a second connect on the same PGLite data dir self-deadlocks on the data-dir lock (this process already holds it). Same rule applies torunBackfillCommand(engine, args)insrc/commands/backfill.tsand any future command dispatched from cli.ts's engine-connected switch. Pinned bytest/reindex-frontmatter-connect.test.ts(library path) andtest/reindex-frontmatter-pglite-spawn.serial.test.ts(CLI dispatch seam, both commands). -
src/core/source-config-sql.ts+src/core/sources-load.ts— canonical recovery for non-objectsources.configvalues. The application reader unwraps nested JSON strings and merges recoverable array fragments left-to-right; the shared SQL expression mirrors that policy atomically for both engines, source config updates, archive/restore, and the paste-readysource_config_shapedoctor repair.localFederatedSourceIdsreads config through the same parser so stdio/CLI federation cannot silently disagree withsources list.sourceConfigHasRemoteUrluses that parser for autopilot pull policy, including PGLite's JSON-string config shape. Invalid fragments degrade to{}rather than throwing. Pinned bytest/sources-load.test.ts,test/job-pull-policy.test.ts,test/list-all-sources.test.ts,test/local-federated-search-scope.test.ts,test/destructive-guard.test.ts, andtest/doctor-source-config-shape.test.ts. -
src/core/source-resolver.ts— 6-tier source resolution.resolveSourceWithTier(engine, explicit, cwd)returns{ source_id, tier: SourceTier, detail? }alongsideresolveSourceId()(unchanged).SOURCE_TIER_NAMES = ['flag', 'env', 'dotfile', 'local_path', 'sole_non_default', 'brain_default', 'seed_default'](7 entries; order matches priority). Tiersole_non_defaultslots betweenlocal_pathandbrain_default: when NOsources.defaultconfig is set AND exactly one registered source haslocal_pathAND isn't'default'AND the'default'source holds no active pages (the emptiness guard — the tier's charter is rescuing brains whose'default'is empty; when'default'carries an established corpus, auto-routing would hijack every bare write into the side-source, so the resolver falls through toseed_defaultand prints a one-line stderr notice naming both sides so the suppressed flip is diagnosable, sameGBRAIN_NO_SOLE_NON_DEFAULT_NUDGE=1suppression knob as the routing nudge; a failed pages probe on an exotic/legacy schema keeps the auto-route), auto-route to it; archived sources excluded (try/catch for pre-v34 brains); privatepickSoleNonDefaultSource(engine)shared by both resolver entry points so they cannot drift. ExportedformatSoleNonDefaultNudge(sourceId): string | nullbuilds the user-facing stderr nudge (null whenGBRAIN_NO_SOLE_NON_DEFAULT_NUDGE=1).src/commands/sync.ts:1497-1519callsresolveSourceWithTierunconditionally so the tier fires;src/commands/import.ts:96-128mirrors with the tier-gated nudge. Consumed bygbrain sources current,import --source-id,extract --source-id, and thesource_routing_healthdoctor check. Pinned bytest/source-resolver-with-tier.test.ts(withEnv()per test-isolation lint),test/source-resolver-sole-non-default.test.ts(20 cases incl. both-ways emptiness-guard + probe-failure fallback),test/sync-sole-non-default-routing.test.ts(3 PGLite cases driving realrunSync),test/pages-source-scoping-4329.test.ts(real-engine both-ways guard coverage). Also exportsnoGrantFederatedScope(engine, hasSourceGrant, sourceId), the widening decision transports share: returnslocalFederatedSourceIds(..., 'seed_default')ONLY whenhasSourceGrant === false(the legacy-bearer no-grant floor),undefinedfor a granted token, for an OAuth client (flagundefined— a falsy gate here would widen every OAuth client), and when the resolver throws (best-effort; the scalar scope stands rather than failing the request). Called bysrc/commands/serve-http.ts; pinned bytest/no-grant-federated-scope.test.ts(6 cases). Unscoped default-write guard family (theseed_defaulttier is the only one that actually lands a write in'default'):assessDefaultWriteGuard(engine)→DefaultWriteAssessment { shouldGuard, defaultPages, nonDefaultPages, nonDefaultSources, failed? }— an unindexed full-pagesaggregate;shouldGuardwhen ≥1 non-default source holds pages AND non-default pages outnumber default's; a query failure returns the fail-open no-guard verdict stampedfailed: true, which cachers/latchers must not treat as settled.assessDefaultWriteGuardOnce(engine)is the per-engine WeakMap memo (arms on the first SUCCESSFUL assessment whatever its verdict, forgets a failed attempt so the next caller retries, coalesces concurrent callers;__resetDefaultWriteGuardMemois the test seam).assessUnscopedDefaultWrite(engine, tier, mutating)→{ warning, assessed }(assessed: falseonly when the aggregate failed), withmaybeWarnUnscopedDefaultWriteas its warning-only projection;formatDefaultWriteWarning(a, sourceFlag?)/formatDefaultWriteRefusal(cmd, a)render the two surfaces;defaultWriteAllowedByEnv()is theGBRAIN_ALLOW_DEFAULT_WRITE=1escape hatch. Consumers:syncrefuses (dry-run warns),importwarns via the memo, MCP stdio'screateDefaultWriteAdvisoryprints once. Also exportsisResolverUserError(e)— the ONE predicate for this module's user-facing throws (Source "…" not found./not found or is archived.,Invalid --source value,Invalid GBRAIN_SOURCE value), consumed by dream.ts, agent.ts and code-scope.ts so a CLI command turns an unknown/archived source into a clean stderr line + exit 1 while anything else keeps propagating. -
src/core/migrate.ts— schema-migration runner. Owns theMIGRATIONSarray (source of truth for schema DDL).Migrationinterface carriessqlFor?: { postgres?, pglite? }(engine-specific SQL overridessql) andtransaction?: boolean(false forCREATE INDEX CONCURRENTLY, which Postgres refuses in a transaction; ignored on PGLite). Key migrations: v14 (handler branches onengine.kindfor CONCURRENTLY-on-Postgres with invalid-remnant pre-drop viapg_index.indisvalid, plainCREATE INDEXon PGLite); v15 (minion_jobs.max_stalleddefault 1→5 + backfill non-terminal rows); v24rls_backfill_missing_tables(sqlFor: { pglite: '' }no-op — PGLite has no RLS engine, targets subagent tables absent from pglite-schema.ts); v30dream_verdicts (file_path TEXT, content_hash TEXT, worth_processing BOOL, reasons JSONB, judged_at TIMESTAMPTZ, PK(file_path, content_hash))(RLS-enabled under BYPASSRLS; synthesize reads/writes to avoid re-judging); v35 auto-RLS event triggerauto_rls_on_create_tablefires onddl_command_endforWHEN TAG IN ('CREATE TABLE','CREATE TABLE AS','SELECT INTO')runningALTER TABLE … ENABLE ROW LEVEL SECURITYon newpublic.*tables (no FORCE) + one-time backfill on every existingpublic.*base table whose comment doesn't match^GBRAIN:RLS_EXEMPT\s+reason=\S.{3,}(per-table failure aborts the offending CREATE TABLE; no EXCEPTION wrap; PGLite no-op viasqlFor.pglite: ''; breaking change: intentionally-RLS-off public tables need the GBRAIN:RLS_EXEMPT comment before upgrade); v40pages_emotional_weight(pages.emotional_weight REAL NOT NULL DEFAULT 0.0, column-only metadata-only); v46mcp_request_log_params_jsonb_normalize(UPDATE ... SET params = params::text::jsonb WHERE jsonb_typeof(params) = 'string', idempotent); v60-v65 six-migration chain wiring source-scoping intooauth_clients— v60 (oauth_clients_source_id_fk:source_id TEXTNULL→'default'backfill + FK tosources(id) ON DELETE SET NULL), v61 (federated_read TEXT[] NOT NULL DEFAULT '{}'), v62 (explicit-CASE backfill sosource_id IS NULL→'{}'), v63 (fail-loud check every row's source_id is in its federated_read array), v64 (FK flipped toON DELETE RESTRICT), v65 (GIN index for array-containment); v68eval_candidates_embedding_column(eval_candidates.embedding_column TEXT NULLper-row provenance forgbrain eval replayto reproduce the same retrieval space; NULL-tolerant); v108pages_embedding_signature(pages.embedding_signature TEXT NULL=<provider:model>:<dims>stamped viasetPageEmbeddingSignature; GRANDFATHER — stale predicate isembedding_signature IS NOT NULL AND embedding_signature <> $currentso NULL is NEVER stale and upgrade never re-embeds the whole corpus; no index; metadata-only); v109sources_newest_content_at(sources.newest_content_at TIMESTAMPTZdurable newest-COMMIT HEAD committer time written bywriteSyncAnchor, read by the REMOTE staleness path instead of shelling to git; mirror in pglite-schema.ts + schema.sql + bootstrap probe); v110page_aliases((id, source_id, alias_norm, slug, ...)withUNIQUE (source_id, alias_norm, slug)+ lookup indexes on(source_id, alias_norm)and(source_id, slug);alias_normisnormalizeAlias()output so WRITE/READ key on the same form; also insrc/core/pglite-schema.ts); v111search_telemetry_rank1_columns(ADD COLUMN IF NOT EXISTSon both engines:sum_rank1_score,count_rank1, three bucketsrank1_lt_solid/rank1_solid/rank1_highonsearch_telemetry— aggregate not per-query rows so rank-1 median drift is bounded-growth; ALTERs right after v57 which created the table); v114links_link_source_check_kebab_regex(openslink_sourcefrom the closed allowlist to a kebab-case format gate^[a-z][a-z0-9]*(-[a-z0-9]+)*$+char_length<=64; Postgres branch usesNOT VALID+VALIDATE CONSTRAINTwithtransaction:false, PGLite plain DROP+ADD; existing built-ins all satisfy the regex so VALIDATE never fails on existing data); v116code_edges_source_backfill_and_callee_index(idempotent: backfills NULLcode_edges_symbol/code_edges_chunksource_idfrom each edge'sfrom_chunkpage — a NULL never matches a scopedAND source_id = …filter, so scopedcode-callers/code-calleeswould return 0 rows on multi-source brains — plus plainCREATE INDEXonfrom_symbol_qualifiedfor both edge tables, which had no index and seq-scanned per BFS node); v129dream_verdicts_triage_v1_columns(additiveADD COLUMN IF NOT EXISTSwideningdream_verdictsinto a scored triage record —score,content_type,segments,entities,model,triage_version; legacy rows keepscoreNULL and read as cache misses, re-judged once; no backfill, no index; same SQL on both engines sincedream_verdictsis migration-created on PGLite too, so the columns take the COLUMN_EXEMPTIONS route intest/schema-bootstrap-coverage.test.tsrather than bootstrap probes); v143dream_verdicts_ttl(30-dayexpires_atTTL: nullableADD COLUMN+SET DEFAULTrun BEFORE thejudged_at-derived backfill — statement order is load-bearing so a legacy writer racing the upgrade window picks up the default instead of inserting a NULL that would failSET NOT NULLon every retry; on Postgres the schema blob'sdream_verdicts_expires_idxforward-references the column, soforward-reference-bootstrap.tsprobes + ALTERs it before the blob replays, and both engines'getDreamVerdictread predicate is NULL-tolerant —expires_at IS NULL OR expires_at > now()— so a pre-backfill row reads as a hit instead of silently re-judging the corpus). The dedup-index self-heal (timeline_dedup_index, seetimeline-dedup-repair.ts) is NOT version-gated:runMigrationsinvokesrepairTimelineDedupIndexon every pass (including the no-pending early-return path) because a merge-renumbered migration can leave the version counter past the index change while the index stays the old shape.retry-matcher.tsandtimeline-dedup-repair.tsare static dependencies becauserunMigrations()executes from live engine initialization; the engine dynamic-import guard scans this file with both engine implementations. v95:pages_dedup_partial_indexaddsCREATE INDEX pages_dedup_idx ON pages (source_id, content_hash) WHERE deleted_at IS NULL. Postgres usesCREATE INDEX CONCURRENTLYwithtransaction: false+ pre-drops any invalid remnant; PGLite uses plainCREATE INDEX. PowersfindDuplicatePagehot path (O(log n) instead of O(n)). v74mcp_spend_loguses BTREE on(client_id, created_at)+(token_name, created_at)—date_trunc('day', TIMESTAMPTZ)is NOT IMMUTABLE so can't appear in index expressions; acreated_atrange scan covers the per-day rollup. v75embedding_multimodal_columnis column-only (no HNSW index — deferred to post-reindex per pgvector best practice). -
src/cli.ts— no-DB fallbacks announce themselves on stderr instead of degrading silently.dreamdispatch binds the caught engine-connect error and emits[dream] WARNING: could not connect to DB (...)before falling through to filesystem-only phases; therunDream(null, ...)no-DB fallback is preserved (pinned bytest/cli-dream-engine-warn.test.ts, 2 subprocess cases against good + bad DATABASE_URL).doctordispatch does the same: whenconnectEngineOR the DB-backedrunDoctorrun throws, it emits[doctor] DB-backed doctor run failed (...) — falling back to filesystem-only checks, scrubbing the error message through BOTHurl-redact.ts:redactUrlsInTextandredact-connection-info.ts:redactConnectionInfofirst, because doctor output is exactly what users paste into issues and CI logs (pinned by the fallback case intest/doctor-minions-check.test.ts: stderr note present, credentials absent, stdout stays parseable--json). -
skills/conventions/brain-routing.md— agent-facing convention skill documenting the canonical 6-tier source resolution chain (flag → env → dotfile → local_path → brain_default → seed_default) with paste-ready decision tables. Linked from CLAUDE.md's "Two organizational axes" section and fromgbrain sources current's hint output. -
test/operations-trust-boundary.test.ts+scripts/check-operations-filter-bypass.sh— operations trust-boundary contract coverage. Pure assertions over all 74 ops (every op has a scope annotation; every mutating op has a non-read scope;localOnly: trueops are excluded fromoperations.filter(op => !op.localOnly); the seven sensitive localOnly ops snapshot-pinned by name) plus targeted handler-invocation guards for the HTTP-callable classes:submit_jobwithname='shell'+ctx.remote=trueMUST reject (shell-job RCE);search_by_imagewithimage_path+ctx.remote=trueMUST reject (image leak); and the list-op read-leak class —find_orphans/get_recent_salience/find_anomalies/find_expertsinvoked withctx.remote=trueover a seededvisibility: privatepage MUST NOT return its slug/title, with a local positive control proving the assertions non-vacuous, count self-consistency assertions pinning the no-count-oracle posture (both orphan denominators shrink with hidden rows;excludeduntouched; anomalycountmirrors the visible slug list), and a soft-delete re-probe pinning the includeDeleted fail-closed path.file_uploadandsync_brainomitted from handler-invocation tests because they'relocalOnly: true(that path would test an impossible production scenario). The shell guard grepssrc/for any module importing theoperationsvalue outside the canonical filter site atsrc/commands/serve-http.ts(three import shapes: destructured, aliased, namespace), with an explicit 10-entry allow-list + a literal-string check thatserve-http.tsstill containsoperations.filter(op => !op.localOnly). Wired intobun run verify. Dynamic sibling:test/remote-privacy-sweep.test.ts(next entry) — same doctrine, corpus-seeded, catches leaks in ops neither curated list has heard of yet. -
test/remote-privacy-sweep.test.ts— registry-driven, corpus-seeded leak detection across the ENTIRE dispatch surface (catches a new remote surface that forgets the world-only filter). Seeds a hermetic PGLite corpus carrying high-entropy PRIVATE sentinels (private page title/body, a private fact, a non-world take, fence-row content — never slugs, which are legitimately echoed in "Page not found" errors and would false-positive on correct behavior) plus WORLD markers as positive controls; enumerates every non-localOnly op from the registry; dispatches each one remote-shaped throughdispatchToolCall(the exact layer both MCP transports share) in BOTH caller shapes — scalar source-scoped (like stdio) and federated with an auth-array read grant; and asserts no private sentinel appears anywhere in the serialized response envelope: structured fields, rendered text, error messages, and the_meta.brain_hot_memorychannel alike. Fail-closed maintenance contract:EXPECTED_OUTCOME's key set must be set-equal to the enumerated registry, so a NEW op FAILS the suite until classified (data= must prove corpus contact via a WORLD marker /ok/error— envelope still sentinel-checked /denied= publish-gated, must error naming itsmcp.*gate; plus aPARAM_FACTORYentry if it can return corpus data — the failure message prints exactly what to add). A separate arm asserts everylocalOnlyop is denied fail-closed over non-stdio transports;KNOWN_EXPOSEDop-wide exemptions must stay empty absent written justification + review. Phase W covers mutating ops against fresh slugs (destructive ops last, corpus-intact re-checks between phases). Honest scope: covers the dispatch surface for transport-floor callers; does NOT cover the IPC context-pack handler (src/mcp/context-pack-handler.tsbypassesdispatchToolCall) or write-triggered restoration echoes — the write-side sweep is a filed TODO. Static siblings:test/operations-trust-boundary.test.ts(previous entry) +scripts/check-operations-filter-bypass.sh. -
src/core/content-sanity.ts— pure assessor for the content-sanity defense.assessContent(content, opts): SanityVerdictreturns one ofok | warn_oversize | hard_block_junk_pattern | soft_block_oversizewith{reason, bytes, matched_pattern_name?}. Six built-in junk patterns (Cloudflare challenge dumps, CAPTCHAs, 403 dumps, bare error-page titles) compiled at module load; operator literal substrings vialoadOperatorLiterals()fromsrc/core/content-sanity-literals.ts.ContentSanityBlockErrortagged class is the typed throw shape every wrapper site (gbrain import,put_pageMCP op,gbrain sync,/ingestwebhook) catches via the existing exception flow. The bytes-parity contract pinsBuffer.byteLength(content, 'utf8')against the embedder's actual byte count so a 499K-byte page can't be soft-blocked on assessment then overflow on embed. Knob resolution chain env > file (~/.gbrain/config.json) > DB > defaults. Four knobs:content_sanity.bytes_warn(50_000),content_sanity.bytes_block(500_000),content_sanity.junk_patterns_enabled(true),content_sanity.disabled(false;GBRAIN_NO_SANITY=1is the loud-stderr kill-switch).assessContentSanity(opts): SanityAssessmentreturns the three-tier disposition (shouldQuarantine/shouldFlag+ reason/detail) consumed byimportFromContentandgbrain quarantine scan; adds the fuzzy prose-vs-markup ratio pass (markup chars / total abovemax_markup_ratio; code pages exempt; gated byprose_check_enabled) on top of the byte + junk-pattern passes. Three more knobs:content_sanity.junk_disposition(quarantinedefault |reject; no env override — a destructive flip belongs in explicit config),content_sanity.max_markup_ratio(0.85, envGBRAIN_MAX_MARKUP_RATIO, clamped(0,1]),content_sanity.prose_check_enabled(true). Pinned bytest/content-sanity.test.ts. Per-pattern opt-out without the kill-switch:content_sanity.disabled_patternsnames built-in junk patterns to skip (a JSON array["access_denied"]or a comma listaccess_denied,error_title; malformed JSON falls back to the comma parse so a hand-typed value still lands) — the file plane takes the array, the DB plane the string. -
src/core/content-sanity-literals.ts— operator literal-substring loader. Reads~/.gbrain/junk-substrings.txt, one literal per non-comment non-blank line; optional# name=<id>header pairs an identifier with the following literal so audit JSONL groups by site (linkedin_auth_wall,reddit_blocked, etc.). Fail-soft on ENOENT (missing file = empty array). Loaded on every ingest. Deliberately literal substrings (NOT regex) to defeat ReDoS. Pinned bytest/content-sanity-literals.test.ts. -
src/core/embed-skip.ts— 5-site shared predicate for the soft-block embed-skip filter. ExportsshouldSkipEmbedding(frontmatter): boolean(JS predicate for callers holding the page in memory),EMBED_SKIP_SQL_FRAGMENT(parameterized SQL clause shared by Postgres + PGLite viaexecuteRaw), andbuildEmbedSkipMarker(reason: string)(writesfrontmatter.embed_skip = {at: ISO_TIMESTAMP, reason}so the JSONB shape stays uniform). The 5 sites:embed.ts --stale,embed.ts --all, theembed-staleMinion helper, plus both engines'listStaleChunks+countStaleChunks. Single source of truth so the filter cannot drift. Pinned bytest/embed-skip.test.ts(cross-site invariant + JSONB shape). -
src/core/audit/content-sanity-audit.ts— ISO-week JSONL audit at~/.gbrain/audit/content-sanity-YYYY-Www.jsonlbuilt on theaudit-writer.tsprimitive. Records every hard-block, soft-block, warn-trip, and quarantine/flag event with{kind, source_id, slug, bytes, matched_pattern_name?, reason, ts}. Doctor reads the last 7 days, aggregates by(matched_pattern_name, source_id)so operators see which scraper is the problem. HonorsGBRAIN_AUDIT_DIRfor shared-filesystem multi-host setups. Pinned bytest/audit/content-sanity-audit.test.ts. -
src/core/quarantine.ts— the two frontmatter markers the content-quality gate writes, sibling ofsrc/core/embed-skip.ts(same marker-as-JSONB-object pattern, same JSONB?existence check that works on Postgres AND PGLite; no schema migration — both are frontmatter JSONB keys).quarantine(keyQUARANTINE_KEY) HIDES: set ONLY for high-confidence junk, writes zero chunks, excluded from search viaquarantineFilterFragment(pageAlias)/QUARANTINE_FILTER_FRAGMENT(thep-aliased constant), the single source of truthbuildVisibilityClausecalls so the search filter and marker key can't drift.content_flag(keyCONTENT_FLAG_KEY) WARNS, does NOT hide: set for fuzzy markup-heavy / oversize, page stays searchable, marker is READ INTO search/get_page output — deliberately NO SQL filter fragment. Three distinct markers, three reasons (never overloaded):embed_skip= oversized-but-clean,quarantine= junk hidden,content_flag= odd-examine-still-here; a page can carry more than one (oversize → embed_skip + content_flag:oversized) and each clears independently. ExportsbuildQuarantineMarker/isQuarantined/filterOutQuarantined,buildContentFlagMarker/getContentFlag/hasContentFlag, plus the two key constants. Pinned bytest/quarantine.test.ts. -
src/commands/quarantine.ts—gbrain quarantine <list|clear|scan>operator surface for the content-quality gate.list [--json] [--include-flagged]paginateslistPagesand reports quarantined (HIDDEN) pages, optionally alsocontent_flag(FLAGGED, searchable) pages.clear <slug> [--force] [--no-embed] [--json]drops both markers and re-imports through the normal pipeline so the page re-chunks + re-embeds and becomes searchable; the gate re-runs on import so genuinely-junk pages re-quarantine (exit 1) unless--forcesetsGBRAIN_NO_SANITY=1for that one import.scan [--limit N] [--apply] [--no-embed] [--json]re-assesses already-ingested pages so junk predating the gate gets marked (unchanged content short-circuits normal sync, so it never re-assesses otherwise); dry-run uses the SAME effectivecontent_sanityconfig thresholds--applywill use, idempotent (skips already-marked pages),--applyre-imports withforceRechunkto set markers + (for quarantine) drop chunks. Dispatched incli.ts. Pinned bytest/quarantine-cli.test.ts. -
src/core/zombie-reap.ts— idempotentinstallSigchldHandler()so JS-spawned children get reaped via Bun's internalwaitpid(). Bun (like Node) only auto-reaps when a SIGCHLD listener is registered; without it, every child the worker spawns (shell jobs, embed batches, sub-agents) becomes a zombie on exit and holds connection slots. Called once at module load fromsrc/cli.ts(Windows platform guard — SIGCHLD doesn't exist on Windows). Cross-file leak guard via_uninstallSigchldHandlerForTests(). Layer 1 of the three-layer zombie defense; Layer 2 is tini-as-PID-1 wrapping the worker subtree (viasrc/core/minions/spawn-helpers.ts); Layer 3 is the container's own tini for hard Bun crashes. -
src/core/minions/— Minions job queue: BullMQ-inspired, Postgres-native (queue, worker, backoff, types, protected-names, quiet-hours, stagger, handlers/shell). Private dream-inline-* queues carry an explicit lifecycle: owner job id + owner token + a renewable lease (columns on minion_jobs;queue.tsreconcilePrivateQueueterminalizes one queue through the normal cancelJobs bookkeeping and stamps every cancellation with the machine-readableprivate_queue_reconciled:reason family;renewPrivateQueueLeaseis MONOTONIC — GREATEST — so a default-horizon renewal can never shrink a creation-time lease;reconcileOrphanedPrivateQueuescancels only provably-orphaned queues: no healthy child lock, owner terminal/missing or lease expired; legacy unowned rows are left to Doctor/retriage;classifyPrivateQueueForRecoveryis PUBLIC so the doctor orphan check buckets through the same verdict). Recovery runs on every lane that can strand a queue: supervisor beforeSpawn, baregbrain jobs workstartup (skipped under GBRAIN_SUPERVISED=1 — the supervisor already ran it; autopilot children never set the flag, so autopilot spawns/respawns recover too), and dream-cycle start (cycle.ts, next to the cycle lock reap — the ONLY lane on PGLite, which inlines every child).deriveWedgeSignalis queue-aware: a dream-inline queue is never 'wedged' (no shared worker can claim it) — it reportsprivate_queue: trueso jobs stats / get_job_stats / doctor all point at reconciliation instead of the impossible worker restart. -
src/core/minions/queue.ts— MinionQueue class (submit, claim, complete, fail, stall detection, parent-child, depth/child-cap, per-job timeouts, cascade-kill, attachments, idempotency keys, child_done inbox, removeOnComplete/Fail).add()takes a 4thtrustedarg (separate fromoptsto prevent spread leakage); protected names inPROTECTED_JOB_NAMESrequire{allowProtectedSubmit: true}and the check runs trim-normalized (whitespace-bypass safe). The same trusted arg carries{allowPgliteInlineWorker: true}, a narrowly scoped exception used only whengbrain jobs submit embed-backfill --followstarts and awaits a worker in that process. Before config, schema, policy, SQL, or transaction access,add()invokes the centralized embed-backfill admission gate; PGLite and unknown engine kinds refuse, while explicit Postgres preserves queued submission.add()plumbsmax_stalledthrough with a[1, 100]clamp; omitted values let the schema DEFAULT (5) kick in.handleWallClockTimeouts(lockDurationMs)is Layer 3 kill shot for jobs whereFOR UPDATE SKIP LOCKEDstall detection and the timeout sweep both fail to evict (wedged worker holding a row lock via a pending transaction). The submission backpressure guard covers two options sharing onepg_advisory_xact_locknamespace keyed on(name, queue, source):maxWaiting(rate cap, countswaitingonly, NULL-source-as-wildcard scope) andmaxPending(single-flight, countswaiting+ live-lockactiverows wherelock_until > now(), EXACT source scope viaCOALESCE(data->>'sourceId', data->>'source_id')— an expired-lock active never suppresses, keeping the waitingClaimable wedge detectors fed). All three coalesce return paths (idempotency fast-path, cap-hit, ON CONFLICT race fallback) stamp non-persistedcoalesced: truemetadata on the returned job. Both filter onqueuein addition tonameso cross-queue same-name jobs don't suppress each other.claimandrenewLockissue their UPDATE viaengine.executeRawDirect(notexecuteRaw) so the lock heartbeat runs on the direct session-mode pool that the transaction pooler won't recycle mid-hold; on PGLite this is identical toexecuteRaw. The two terminal dead-letter paths (handleWallClockTimeoutswall-clock kill and the stall dead-letter CTE in the stall sweep) BOTH incrementattempts_madeso a long job killed there reads as an honest attempt instead ofattempts 0 / started N; the stall path also bumpsstalled_counter, surfaced bygbrain jobs getasAttempts: M/N (started: X, stalled: S/MaxS). At submit,add()stamps a defaulttimeout_msviadefaultTimeoutMsFor(jobName)(fromhandler-timeouts.ts) when the caller passed none, andclaim()COALESCEs a still-NULLtimeout_msfromHANDLER_DEFAULT_TIMEOUT_MS(raw-object jsonb bind) derivingtimeout_atfrom the coalesced value — the durable invariant covering rows that predate submit-time stamping; an explicitopts.timeout_msalways wins. The lock lease follows the identical three-layer shape:add()stampsopts.lock_duration_ms(clamped [5s,1h] via the sharedclampLockDurationMs; INSERT-only — an idempotency-key re-submit never mutates the first submitter's lease) elsedefaultLockDurationMsFor(jobName);claim()deriveslock_untilfromCOALESCE(lock_duration_ms, map, worker default)and stamps the resolved lease, SQL-clamping the row/map-resolved value to [5s,1h] (the worker-default fallback passes through unclamped — operator-configured, and tests legitimately use sub-5s leases) so a bypass-written out-of-range row can't produce a pathological lease;handleWallClockTimeouts' null-timeout fallback usesCOALESCE(lock_duration_ms, worker default).handleStalled(graceMsOverride?)applies the stall-sweep reclaim grace (GBRAIN_MINION_STALL_RECLAIM_GRACE_MS, default 15s, 0 = legacy predicate, capped at 600s with a warn-once clamp so an oversized value can't disable stalled-job recovery) to all three sweep predicates — a lease that lapsed within the grace is a starved owner's head start, not a steal candidate. Guarded bytest/queue-lock-retry.test.ts(claim never falls back toexecuteRaw),test/postgres-execute-raw-direct.test.ts(routing decision matrix), andtest/minions.test.ts(attempt accounting + default-timeout stamping).MinionQueue.add()gatessubagentjobs on capability, not provider:data.modelis classified viaclassifyCapabilities()(lazy-imported fromsrc/core/ai/capabilities.tsto keep queue's eager-load surface small) and rejected only when the model cannot run a tool loop (unusable:no_tools) or names an unknown provider; degraded verdicts pass through with a gateway cost warning. Layer 1 of the three-layer subagent capability enforcement (layers 2+3:model-config.ts:enforceSubagentCapableruntime fallback +src/commands/doctor.tssubagent_providercheck). Pinned bytest/agent-cli.test.ts. All three terminal reaper paths (stall dead-letter, wall-clock kill, cascade) route through the ONE privatekillJobs(tx, rows, cause, message)tail: it emitschild_done(outcome)to each non-terminal parent's inbox and flips anywaiting-childrenparent whose last open child just died back towaiting— so a dead-lettered child can never strand its aggregator parent. Callers lock parents FIRST vialockParentsOrdered(ascending-id row locks, matchingfailJob's parent-before-child order) so the reapers andfailJobcan never deadlock on parent/child lock acquisition. An idempotent stranded-parent sweep runs once per stall tick (~30s): anywaiting-childrenparent with zero non-terminal children flips back towaiting(single UPDATE, NOT EXISTS on an indexed FK) — self-heals every stranding class, including parents stranded before the sweep existed. Every automatic re-run path clearsstarted_at(failJob's delayed retry branch, the stall requeue, lease release,promoteDelayed, and the parent-unblock flips) so the per-attempt wall clock measures execution, not backoff/queue wait — a retried job can't be dead-lettered by the wall-clock sweep before executing a line. Pinned bytest/queue-stall-parent-unblock.test.ts+test/queue-started-at-retry.test.ts.renewLockaccepts optional{signal}forwarded to executeRawDirect so the renewal tick's timeout race CANCELS a hung UPDATE instead of orphaning a checked-out pool slot (best-effort — the token fence is the correctness authority; a late-landing renewal can only extend a lease nobody else claimed). -
src/core/minions/embed-backfill-admission.ts— Centralized, dependency-light embed-backfill admission contract shared by the publicMinionQueue.addboundary, CLI, MCP operations, automatic submitters, sync cost reporting, and doctor remediation.embedBackfillWorkerSurfacerecognizes only explicitpostgresas worker-backed; the declared engine-kind switch is compile-time exhaustive, while untyped/cast unknown runtime values still fail closed asno_worker_surface.assertEmbedBackfillQueueAdmissionrefuses before any database/config access unless the explicit PGLite inline-worker trust is present, validates payload source IDs through the canonicalsource-id.tscontract before rendering any remedy, andembedBackfillManualDrainCommand(sourceId)is the single injection-safe exactgbrain embed --stale --source <id>recovery command. -
src/core/embed-backfill-submit.ts— Automatic per-source embed-backfill submitter with a true discriminated result union:submittedrequiresjobIdplus an explicit spend-bypass payload,cooldownrequires a reason plus a numeric remaining duration or explicitnullfor active work,spend_cappedrequires cap/spend payloads, andno_worker_surfacerequires the refused engine kind. It queries cooldown/spend state only after the shared admission classification accepts a worker-backed engine, preserving Postgres behavior while guaranteeing PGLite/unknown refusal before config, schema, or SQL access; callers handle every status exhaustively. -
src/core/minions/worker.ts— MinionWorker class (handler registry, lock renewal, graceful shutdown, timeout safety net). Aborted jobs callfailJobwith reason (timeout/cancel/lock-lost/shutdown);shutdownAbort(instance field) fires on SIGTERM/SIGINT and propagates toctx.shutdownSignal(shell handler listens; non-shell handlers don't). Per-job timeout firesabort.abort(new Error('timeout'))then a 30s grace-then-evict safety net force-evicts the job frominFlightand marks it dead if the handler ignores the abort signal (the generic abort listener fires for ANY abort reason). ThelaunchJoblock-renewal block is a thin sync wrapper around the purerunLockRenewalTickfromsrc/core/minions/lock-renewal-tick.ts(NEVERsetInterval(async () => await renewLock(...))— that shape surfaces an unhandledRejection during PgBouncer rotation). Guarantees: (1)cancelledflag captured in the timer closure stops in-flight IIFEs writing misleading audit events after the job ended; (2) re-entrancy guardtickInFlight(skips counted asoverlapSkipstelemetry) + per-callPromise.racetimeout with best-effort AbortSignal cancellation; (3) verify-before-evict: eviction requires a fenced-false (certain loss) or thehardEvictMsbackstop — never bare local arithmetic; the per-job lease (job.lock_duration_ms ?? opts.lockDuration) drives the renewal state, and the cadence clamps tomin(lease/2, 60s); (4) explicit.catch()on the storedexecuteJob(...).finally(...)promise closes the second unhandledRejection vector; (5) exportedINFRASTRUCTURE_ABORT_REASONS = new Set(['lock-renewal-failed', 'lock-lost'])so executeJob's catch skipsfailJobfor these (PgBouncer blips don't dead-letter healthy jobs; the stall detector reclaims); (6) inFlight generation-safety — force-evict and the handler's finally delete the inFlight entry ONLY when it still carries their own lockToken (the token is the generation; an evicted execution's late delete cannot remove a same-worker re-claim's entry); (7) eviction observability — the worker keeps aperf_hooks.monitorEventLoopDelayhistogram (reset on every successful renewal, sampled ns→ms at eviction) plus raw loadavg, and both the abort and grace-evict log lines carry cause / since-last-success / tick-lateness / overlap-skips / load / event-loop-delay from the stashedabortMeta. CI guardscripts/check-worker-lock-renewal-shape.sh(inbun run verify) asserts the bug pattern stays absent ANDlaunchJobkeeps callingrunLockRenewalTick. Engine-ownership invariant:start()does NOT callengine.disconnect()on shutdown — the CLI handler insrc/commands/jobs.ts case 'work'owns engine lifecycle via try/finally with loud error logging. RSS watchdog uses non-file-backed pages on Linux: exportedparseRssFromProcStatus(status)(pure parser; field-presence regex soRssAnon: 0 + RssShmem: 512parses correctly) andgetAccurateRss(readStatus?)(reads/proc/self/statusforRssAnon + RssShmem, falls back toprocess.memoryUsage().rsson macOS / restricted containers / kernel <4.5); the defaultgetRssinWorkerOptsisgetAccurateRss.checkMemoryLimittracks peak RSS, fires an 80%-of-cap soft-warn (once per crossing, carrying peak + in-flight job kinds), and on exceed sets_rssWatchdogTriggered=true(exposed viaget rssWatchdogTriggered()) sojobs work's finally canprocess.exit(WORKER_EXIT_RSS_WATCHDOG)after disconnect (drain self-identifying instead of an opaque code-0 exit). The poll loop wrapsclaimin try/catch: on a retryable conn error it reconnects ONCE and continues to the next tick rather than blind-retrying (a retry afterUPDATE...RETURNINGcommitted but the socket died would double-claim).LockRenewalDepsis wired withreconnectwhen the engine supports it. The self-health DB-liveness probe runs EVEN under a supervisor (GBRAIN_SUPERVISED=1): the outer guard isif (this.opts.healthCheckInterval > 0)and only the STALL-detection block is wrapped inif (!isSupervisedChild)— so a supervised worker whose own pool dies self-exitsunhealthy(db_dead)afterdbFailExitAfterprobes (the supervisor watches a different connection and can't see this worker's dead pool), while the supervisor's progress watchdog owns forward-progress. Pinned by thetest/worker-lock-renewal.test.tshermetic suite,test/audit/lock-renewal-audit.test.ts,test/scripts/check-worker-lock-renewal-shape.test.ts,test/worker-shutdown-disconnect.test.ts(assertsdisconnectSpy).not.toHaveBeenCalled()),test/worker-rss.test.ts(11),test/worker-supervised-db-probe.test.ts(3). Per-job process isolation:MinionWorkerOpts.jobIsolation(inline|process) +childCliInvocation/childTiniPathswaphandler(context)forrunJobInChild(...)in executeJob — every reporting branch (completeJob, failJob, lease release, infra no-burn) is reused on the child outcome; when isolated the parent-side context is not built at all. Three extra no-burn child classes in the catch:ChildSpawnInfraError+ChildWorkerShutdownError(released, no attempt burned) andChildNotClaimedError(the child proved the claim was already gone — reclaimed/cancelled before the handler ran — so nothing is recorded against it). The health probe delegates torunDbProbe(db-probe.ts) with a cancellation signal on every probe and emits averdict(pool_starved/server_unreachable/unknown) on thedb_deadunhealthy payload.getHandler(name)is the read-only registry accessor run-child uses. -
src/core/minions/supervisor.ts— MinionSupervisor process manager. Spawnsgbrain jobs workas a child, restarts on crash with exponential backoff, periodic health check.consecutiveHealthFailurescounter; on 3 consecutive failures emitshealth_warnwithreason: 'db_connection_degraded'and callsengine.reconnect()to swap in a fresh pool, then resets. Worker exit classifier emitslikely_causeonworker_exitedevents:oom_or_external_kill(SIGKILL),graceful_shutdown(SIGTERM),runtime_error(code 1),clean_exit(code 0),unknown. ConsumesdetectTini()+buildSpawnInvocation()fromsrc/core/minions/spawn-helpers.tsto wrap the worker subtree in tini-as-PID-1 when tini is onPATH(handles native-addon zombie reaping the in-process SIGCHLD reaper can't reach); exposesisTiniDetectedread-only accessor. The spawn-and-respawn loop is the sharedChildWorkerSupervisorcore: MinionSupervisor composes it viarunSuperviseLoop()→new ChildWorkerSupervisor({...})and mapsChildSupervisorEventback throughemit()SupervisorEvent (JSONL audit consumers see byte-compatible output). PID lock, signal handlers, health check, andprocess.exiton the HARD crash ceiling stay in MinionSupervisor. Crossing the SOFT budget (maxCrashes) does not permanently give up: the core drops into degraded retry (capped backoff + acrash_budget_degradedhealth_warn) and self-heals when a respawn runs stably; permanentprocess.exit(MAX_CRASHES)fires only at the hard ceilingresolveHardStopMaxCrashes(maxCrashes)(defaultmaxCrashes × 10, envGBRAIN_SUPERVISOR_HARD_STOP_CRASHES,0= never). Separately,gbrain jobs supervisor status+gbrain doctordetect a live supervisor through this queue lock (inspectLock+isLockHolderLive, freshness-keyed so PID reuse can't false-positive) when the$HOME-derived pidfile is absent, so a split-$HOMEdeployment does not read a healthy supervisor as "not running".code=0leavescrashCountuntouched (so a worker alternating real crashes + watchdog drains still tripsmax_crashes);cleanRestartBudget(default 10 restarts per 60s) caps the macOS/non-Linux-fallback tight-loop viahealth_warn { reason: 'clean_restart_budget_exceeded' }+ backoff.shutdown()drains viachildSupervisor.killChild('SIGTERM')+awaitChildExit(35_000). Progress watchdog:healthCheck()restarts an alive-but-wedged child viachildSupervisor.restartCurrentChild(35_000)when a queue has claimable work, 0 live-lock active jobs, and stale completions acrosswedgeRestartChecks(default 3) consecutive checks pastwedgeRestartMinutes(default 15, 0 disables) + astartupGraceMswindow; bounded bywedgeRestartLoopBudget(default 3 /wedgeRestartLoopWindowMs) which switches to a one-shotwedge_restart_loopalert. The wedge query is the exportedqueryWedgeSignals(engine, queue, handlerNames)— name+queue-scoped,active_healthy= live-lock only (an expired-lock active row does NOT mask the wedge), due-delayed counted. Claimable names are derived at start via a throwawayregisterBuiltinHandlersworker (its newquietopt). Flags--wedge-restart-minutes/--wedge-restart-checks+ envGBRAIN_WEDGE_RESTART_MINUTES/GBRAIN_WEDGE_RESTART_CHECKS. The worker argv is built by the exported purebuildWorkerArgs(opts)(appends--nice Nwhenopts.nice_requestedis set, alongside--concurrency/--queue/--max-rss); the niceness apply RESULT (nice_requested/nice_effective/nice_error, computed by the CLI in jobs.ts — the supervisor doesn't call setPriority) rides on thestarted/worker_spawnedaudit emissions. Queue-scoped singleton: the real authority is a DB lock (tryAcquireDbLockfromsrc/core/db-lock.ts) keyed onsupervisorLockId(queue)=gbrain-supervisor:<queue>— keyed on the QUEUE ALONE because the lock row lives inside the target database, so the (database) half of the mutex is physical, not part of the key (mixing in a config-derived DB identity would let two supervisors on the same physical DB via different-but-equivalent URLs compute different ids and both acquire). Two supervisors with different$HOME/--pid-fileagainst the same(database, queue)cannot both run with conflicting--max-rss: the second exitsLOCK_HELD. The pidfile-cleanupprocess.on('exit')listener is installed BEFORE the DB-lock acquisition so theLOCK_HELDearly-exit can't strand the pidfile this process just created. The default pidfile is brain-scoped (supervisor-<brainId>.pid) so different brains under one HOME don't false-block. The lock refreshes on its ownsetInterval(TTL 5min, refresh 60s, max 3 failures = 180s < TTL); a refresh that THROWS past the threshold exitsLOCK_LOST(code 4) rather than risk a split-brain, while a fenced refresh returningfalse(0 rows matched — the lock was stolen or force-cleared) is treated as CERTAIN loss, not a blip: the supervisor emitshealth_error { reason: 'supervisor_lock_lost' }and exitsLOCK_LOSTimmediately (counting it toward the failure threshold would let two supervisors drain the same queue for up to two more refresh windows).shutdown()releases the lock so a clean restart re-acquires immediately. Thestartedaudit recordsmax_rss_mbsogbrain doctor'ssupervisor_singletoncheck can surface the effective cap. ExportssupervisorLockId()and the pureclassifySupervisorSingleton({lockLive, lockHolderHost, lockHolderPid, localHost, localPid}) → 'no_lock'|'single'|'mismatch'(host+pid compare, bare pid meaningless cross-host) that doctor consumes. Pinned bytest/supervisor.test.ts(16 cases),test/supervisor-tini.test.ts,test/supervisor-wedge.test.ts,test/supervisor-build-worker-args.test.ts, andtest/supervisor-db-lock.test.ts.SupervisorOpts.jobIsolationpasses--job-isolation processto the spawned worker via a CONDITIONALbuildWorkerArgspush (inline argv stays byte-identical).queryWedgeSignals/probeQueueStatethread a per-probe AbortSignal; the probe timeout cancels the losing query (pool-slot release under exhaustion). -
src/core/minions/detached-stderr.ts— durable stderr sink + spawn helper forgbrain jobs supervisor start --detach.openDetachedStderrSink()opens an append-mode log in the audit dir (${GBRAIN_AUDIT_DIR:-~/.gbrain/audit}/supervisor-stderr.log, next to the JSONL lifecycle audit), falling back to the null device, then'ignore'; never throws (a sink failure must not block the supervisor start). The detached supervisor gets the fd asstdio[2]and its worker inherits it, so ONE durable descriptor covers both processes — not a per-call EPIPE swallow. Prevents the SIGPIPE failure of inheriting the INVOKER's stderr: a short-lived automation runner closing its capture pipe after the start payload would make the worker exit 141 on its next stderr write (and could take the supervisor down writing its own crash event), leaving active jobs and DB locks stale while producers keep enqueueing. Pinned bytest/detached-stderr.test.ts. -
src/core/minions/child-worker-supervisor.ts— shared spawn-and-respawn core reused by bothMinionSupervisor(standalonegbrain jobs supervisordaemon) andsrc/commands/autopilot.ts(autopilot daemon), so the two consumers can't drift into parallel-loop bugs. Pure class: NO PID file, NO signal handlers, NOprocess.exit, NO health check. Lifecycle events fire via injectedonEvent: (ChildSupervisorEvent) => void. Exit classifier:code === 0leavescrashCountUNCHANGED (preserves flap detection across mixed exit sequences);code != 0followsrunDuration > stableRunResetMs ? 1 : ++crashCount. Clean-restart budget: sliding window of code=0 exits; when count exceedscleanRestartBudget(default 10) insidecleanRestartWindowMs(default 60s), emitshealth_warn { reason: 'clean_restart_budget_exceeded' }and appliescleanRestartBudgetBackoffMs(default 1s). The exit classifier special-casesWORKER_EXIT_RSS_WATCHDOG—likely_cause='rss_watchdog', and that exit does NOT bumpcrashCount(routes to its own breaker); a dedicated_watchdogExitTimestampssliding window trips a loudrss_watchdog_loophealth_warn naming the cap when N watchdog exits land inside the window, INDEPENDENT of the stable-run reset (which would otherwise hide a >5-min-run watchdog drain loop). OptswatchdogLoopBudget(3),watchdogLoopWindowMs(600000),watchdogBackoffMs(30000);ChildSupervisorEventextended. Public read-only accessorschildAlive,inBackoff,crashCount;killChild(signal)gates on liveness (exitCode === null && signalCode === null), NOT.killed—.killedflips true once a signal is sent, so a!this._child.killedguard would make a follow-up SIGKILL after an ignored SIGTERM a silent no-op.restartCurrentChild(graceMs)(wedge self-heal) captures the CURRENT child ref, SIGTERM→grace→SIGKILLs THAT ref (never the respawn — closes the timer-kills-fresh-worker race), and flags_intentionalRestartso the exit islikelyCause='wedge_restart', leavescrashCountUNTOUCHED (never tripsmax_crashes; likerss_watchdog), and respawns immediately (backoff ms:0 reason='wedge_restart').awaitChildExit(timeoutMs)short-circuits whenchild.exitCode !== null || child.signalCode !== nullso fast-SIGTERM responders don't cause a 35s shutdown hang. Degraded-retry: therun()loop does not fireonMaxCrashesExceededat the softmaxCrashes; it announceshealth_warn { reason: 'crash_budget_degraded' }once per episode and keeps respawning with capped backoff (the 60s cap makes it a paced retry, not a hot loop), re-arming after a stable-run reset drops the count. Permanent give-up fires only athardStopMaxCrashes(defaultmaxCrashes × HARD_STOP_CRASH_MULTIPLIER= 10×;0disables). Test hooks_backoffFloorMs,_now.supervisor-audit.tscarriesrss_watchdogas a non-clean cause + its ownCrashSummary.by_causebucket, andwedge_restarttoCLEAN_EXIT_CAUSES(a self-heal, not a crash; denylist preserved so future causes route tolegacy). Pinned bytest/child-worker-supervisor.test.ts(12 cases). -
src/core/minions/spawn-helpers.ts— puredetectTini()+buildSpawnInvocation()consumed by bothsupervisor.tsandautopilot.ts(one implementation for both spawn sites; tini wrapping is testable withoutmock.module(), rule R2 ofscripts/check-test-isolation.sh).detectTini()callsexecFileSync('which', ['tini'])with explicitenv: process.envso Bun sees runtime PATH mutations.buildSpawnInvocation(tiniPath, cmd, args)returns{cmd, args}with tini prepended when present, or the bare invocation otherwise. Pinned bytest/spawn-helpers.test.ts(5) andtest/supervisor-tini.test.ts(4). -
src/core/minions/job-isolation.ts— per-job process-isolation protocol: atomic outcome-file codec (writeChildOutcomeFiletmp+rename,decodeChildOutcomeFilewith a 32MiB cap that throwsUnrecoverableError— oversize results die loudly on attempt 1, decode errors report byte counts never content),encodeHandlerError/reconstructHandlerErrorpreserving the twoinstanceofclasses executeJob branches on (UnrecoverableError, RateLeaseUnavailableError), the child argv/env contract (CHILD_ENV,buildChildArgs),resolveChildCliInvocation(env override → compiled binary → bun-dev fallback → null for fail-fast), andkillProcessGroup(pid, sig)— children are spawned detached (own group) because SIGKILL on a tini pid alone orphans the handler grandchild, and Bun rejects negative pids in process.kill (oven-sh/bun#15791) so group signaling falls back to POSIX /bin/kill. Pinned bytest/job-isolation-protocol.test.ts(incl. real-process grandchild-death). -
src/core/minions/child-job-runner.ts— parent-side runner:runJobInChild(opts)spawns the child (detached + tini when available, stdio inherit for handler logs, per-job lifecycle log lines), maps per-job abort → group SIGTERM now + group SIGKILL at +25s (CHILD_KILL_GRACE_MS, inside the 30s force-evict backstop), gives worker-shutdown children the drain window to finish AND report (a non-reporting shutdown kill throwsChildWorkerShutdownError→ released, no attempt burned), classifies pre-exec spawn failure asChildSpawnInfraError(released, no attempt burned), and bounds child pools via env (GBRAIN_POOL_SIZEdefault 3,GBRAIN_DIRECT_POOL_SIZE=1). Pinned bytest/child-job-runner.test.ts(real children) +test/worker-job-isolation.test.ts(full parent path on PGLite). -
src/core/minions/run-child.ts—runChildJobEntry(engine, opts, injectables): thejobs run-childcore. Re-reads the job row and validates status+token (exit 14 on mismatch, handler never runs), builds the shared token-fenced context against the CHILD's engine, runs the handler, writes ONE atomic outcome file (handler failure = reported outcome = exit 0; only write-failure exits 15). Installs a SIGTERM→shutdownSignal-ONLY handler (inline signal-separation parity: ctx.signal stays live so cooperative handlers finish + report inside the drain window; the parent's group SIGKILL at drain end is the backstop) and a parent-liveness watchdog pollingprocess.kill(parentPid, 0)(ppid checks are dead code under tini) that aborts BOTH signals on parent death with a hard exit after grace. No worker machinery (parent owns liveness). Pinned bytest/run-child-entry.test.ts. -
src/core/minions/job-context.ts— sharedbuildJobContext(engine, queue, job, lockToken, signal, shutdownSignal)(extracted verbatim from executeJob) so inline mode and the run-child child wire identical token-fenced DB callbacks. -
src/core/minions/db-probe.ts— hermetic DB-liveness probe with pool-starvation disambiguation:runDbProbe(deps)probes the read pool (signal-cancelled attimeoutMs), on failure probes the DIRECT lane (DIRECT_PROBE_TIMEOUT_MS, only when dual-pool is genuinely active) and returns a verdict —pool_starved("server IS reachable; fault is in the transaction-pooler path — client pool exhaustion or a pooler-layer fault", deliberately an honest disjunction),server_unreachable, orunknown. Gauge counts render as a labeled tracked SUBSET; no waiter/available arithmetic. Pinned bytest/db-probe.test.ts. -
src/core/minions/niceness.ts— OS scheduling-priority (niceness) primitives for the--niceflag.parseNiceValue(raw)whole-string parses + range-validates to POSIX[-20, 19](rejects"3.5"/"10abc"thatparseIntwould silently truncate).applyNiceness(nice, setPriority?, getPriority?)callsos.setPriority(0, n)and ALWAYS re-readsos.getPriority(0)afterwards — in both the success and the catch paths — so a denied renice (EPERM) or anRLIMIT_NICEclamp records the real effective value (e.g.0), not null; returns{applied, requested, effective, error?}.getEffectiveNiceness(pid, getPriority?)reads an arbitrary pid's niceness (null on dead/unreadable).formatNice(n)→+10/0/-5. Applied only at the CLI layer (jobs.ts) so worker.ts/supervisor.ts stay embeddable. Pinned bytest/niceness.test.ts. -
src/core/minions/worker-registry.ts— live worker registry backing niceness observability. Each runninggbrain jobs workself-registersworker-<pid>.jsonundergbrainPath('workers')(brain-isolated viaGBRAIN_HOME; entries tagged withcurrentBrainId()so multiple DBs under one home don't cross-report).registerWorker(info)is best-effort (never blocks the worker) and returns a cleanup fn the caller wires to BOTH the shutdownfinallyANDprocess.on('exit')(the unhealthyprocess.exit(1)bypasses the awaited finally).readWorkers(getNice?)enumerates the dir, drops confirmed-dead pids (classifyLiveness: ESRCH = dead/prune, EPERM = alive/keep), applies a pid-reuse start-time guard (zone-freeps -o etime=parsed byparseEtimeToMs, start = now - elapsed; rejects a pid that started >5s after the entry was written), and re-measures each live worker's niceness now. Reports the worker's REAL pid, sidestepping the tini-wrapper-pid problem. Pinned bytest/worker-registry.serial.test.ts. -
src/core/minions/supervisor-pid.ts—readSupervisorPid(pidFile) → {pid, running}: the sharedexistsSync → readFileSync → parseInt → process.kill(pid,0)PID-file + liveness reader shared byjobs.ts(supervisor status),jobs.ts(stats), anddoctor.ts. EPERM from the liveness probe counts as running. Pinned bytest/supervisor-pid.test.ts. -
src/core/minions/handler-timeouts.ts— per-handler-type defaults for BOTH per-job time knobs, co-located so they can't drift apart unseen (they are different quantities: the budget bounds total runtime, the lease bounds dead-worker reclaim — never derive one from the other).HANDLER_DEFAULT_TIMEOUT_MS: 30 min forsubagent,subagent_aggregator,embed-backfill,autopilot-cycle,autopilot-global-maintenance; 10 min forchronicle_extract+facts-absorb; 60 min forcontextual_reindex_per_chunk.HANDLER_DEFAULT_LOCK_DURATION_MS: 300 s for the long LLM/loop handlers, 120 s for the single-LLM-call handlers,shelldeliberately absent (fast dead-worker reclaim; verify-before-evict protects it anyway).defaultTimeoutMsFor/defaultLockDurationMsForreturn the mapped default ornull(short handlers keep the tight null-default wall-clock / the 30 s worker lease).clampLockDurationMs+LOCK_DURATION_MS_MIN/MAX([5s, 1h]) is the ONE clamp shared byqueue.add, the CLI--lock-duration-msflag (--dry-runechoes the clamped value), and the MCPsubmit_jobparam; the same bound is mirrored in SQL at claim time and by theminion_jobs.lock_duration_msrange CHECK, so every layer agrees. Layers (explicit value always wins):MinionQueue.add()stamps at submit;MinionQueue.claim()COALESCEs NULL columns from the maps (durable invariant); migration v128 one-shot backfilledtimeout_mswith authoring-time snapshot values — do NOT sync v128 when editing the maps (lock_duration_mshas no backfill: NULL = worker default). Pinned bytest/minions.test.ts+test/migrations-v128.test.ts+test/migrations-v130.test.ts. -
src/core/minions/admission.ts— submit-side queue admission control (the drain-side pool-starvation half is the job-isolation work; claim fairness is a filed TODO, deliberately out of scope). Three primitives resolved per name viaresolveAdmissionPolicy(configminions.*> per-name defaults tables, 60s in-process cache, fail-open with a once-per-process stderr warn; env kill-switchGBRAIN_MINIONS_ADMISSION=0disables all three): PARAM-COALESCING (PARAM_COALESCE_DEFAULT— on forsubagent;computeParamHash= sha256 of stable-stringified payload excluding only__param_hashitself —__owner_client_idis deliberately INCLUDED so owner lanes never cross; parentless + waiting-only + age-bounded to ttl/2), WAITING-TTL (WAITING_TTL_DEFAULT_HOURS— 48h forsubagent; swept byMinionQueue.handleWaitingTTLthroughcancelJobs(ids, {reason, rootStatuses:['waiting']})so descendants cancel, child_done lands, aggregator parents resolve, and the reason stamps ROOT ids only; ≤500/tick oldest-first; warn-before-act isrunWaitingTtlTick— first tick counts affected + stampsTTL_NOTICE_SHOWN_KEYwith an ISO timestamp, sweeping starts only afterttlNoticeGraceMs()(1h default, envGBRAIN_MINIONS_TTL_NOTICE_GRACE_MS) elapses;gbrain upgradeprints the same one-shot notice and starts the same clock; legacy'true'flag values sweep immediately), and NAME-GLOBAL QUOTA (QUOTA_MAX_WAITING_DEFAULTEMPTY by operator decision — activates only viaminions.quota_max_waiting.<name>; counts the name across ALL queues so per-rundream-inline-*fanout queues can't dodge it, EXACT under concurrency via aminion_quota:<name>advisory xact lock taken only when a quota is configured; throws typedQueueQuotaExceededError, checked everywhere viaisQueueQuotaExceededError— dream submitters record a phase skip, synthesize rolls back the current transcript's fresh chunks, agent fanout cancels the whole tree,submit_agentmaps to a structuredrate_limitedOperationError).TTL_REASON_PREFIXis the single source for the sweep's error_text prefix and the stats/doctor LIKE patterns;safeConfigSegmentgates untrusted job names out of copy-pasteable config hints. Alerting ridesgetStats(drained_completed/failed/dead/cancelledkeyed on finished_at +waiting_now+oldest_waiting_minutes), thejobs statsDIVERGENT-QUEUE / waiting-TTL screams (GBRAIN_QUEUE_DIVERGENCE_RATIO=2,GBRAIN_QUEUE_DIVERGENCE_MIN_WAITING=50; divergence compares intake vs COMPLETED so TTL-cancel storms can't masquerade as throughput) +--json, and doctorcheckQueueHealth. Pinned bytest/minions-admission.test.ts+test/jobs-stats-divergence.serial.test.ts. -
src/core/minions/types.ts—MinionJobInput+MinionJobStatus+ handler context types.MinionJobInput.max_stalledis optional; omitted values let the schema DEFAULT (5) kick in, provided values are clamped to[1, 100]. -
src/core/minions/errors.ts— dependency-freeUnrecoverableError, re-exported throughtypes.tsfor compatibility. Delegated policy imports this leaf directly, avoiding initialization cycles through durable authority. Fresh-process import tests preserve the shared error identity and isolated-worker reconstruction. -
src/core/minions/protected-names.ts— side-effect-free constant module exportingPROTECTED_JOB_NAMES+isProtectedJobName(). Kept pure so queue core can import without loading handler modules.PROTECTED_JOB_NAMESincludessynthesize,patterns,consolidate. These phases internally submitsubagentchildren withallowProtectedSubmit=trueand can spend Anthropic credits. Only trusted local callers (CLI, autopilot,doctor --remediate) can submit them; MCP requests are rejected bysubmit_job's protected-name guard. -
src/core/minions/handlers/shell.ts—shelljob handler. Spawns/bin/sh -c cmd(absolute path, PATH-override-safe) orargv[0] argv[1..](no shell). Env allowlistPATH, HOME, USER, LANG, TZ, NODE_ENV+ callerenv:overrides +inherit:-resolved keys. UTF-8-safe stdout/stderr tail viastring_decoder.StringDecoder. Abort (eitherctx.signalorctx.shutdownSignal) fires SIGTERM → 5s grace → SIGKILL on child. RequiresGBRAIN_ALLOW_SHELL_JOBS=1on worker (gated byregisterBuiltinHandlers).ShellJobParams.inherit?: string[]is a free-form list of snake_case config-key names; the worker resolves each vialoadConfig()and injects the value under the derived env key (database_url→GBRAIN_DATABASE_URL; else uppercased). Names persist inminion_jobs.data(and the shell-audit JSONL); values never do. The canonical validatorvalidateShellJobParams(siblingshell-validate.ts) runs PRE-ENQUEUE in both submit surfaces —gbrain jobs submit shell(jobs.ts:271) AND thesubmit_jobop forname='shell'(operations.ts:2085); the handler-entry re-validation here is defense-in-depth (so validation can never run only AFTERqueue.add()has persisted the row). The validator does NOT police which config keys the agent inherits — same-uid trust model treats the agent as a peer of the worker. -
src/core/minions/handlers/shell-inherit.ts— three helpers.INHERIT_NAME_RE(/^[a-z][a-z0-9_]*$/) is the snake_case shape guard used by the validator; rejects__proto__, leading-underscore, uppercase, and path-traversal shapes so audit logs stay readable and prototype-pollution lookups can't smuggle through.deriveEnvKey(name)maps config-key → child-env-key (name.toUpperCase()with one override:database_url→GBRAIN_DATABASE_URLbecause plainDATABASE_URLis ambiguous).resolveInheritValue(cfg, name)is the value lookup; usesObject.hasOwnto defeat prototype-pollution lookups, returns undefined for missing / non-string / empty-string values. No closed enum — agent and worker share a uid, so refusing arbitrary config keys defends nothing in that trust model. -
src/core/minions/handlers/shell-validate.ts—validateShellJobParams(data, opts?)shared pre-enqueue validator. ThrowsUnrecoverableErrorwith paste-ready operator hints on every failure. Three rules: (1) cmd/argv/cwd/env shape, (2) inherit array shape + snake_case regex per element (prototype-pollution defense), (3) fail-fast on missing config value withgbrain config set <key>hint. Optionalredact_secrets?: booleanfor output-side scrubbing. Deliberately does NOT police WHICH secrets the agent passes — single-uid trust model. Test seam:opts.configdrives the validator hermetically without mocking. Re-called atshell.tshandler entry for defense-in-depth (catches rows that bypassed the pre-enqueue validator). -
src/core/minions/handlers/shell-redact.ts— opt-in output-side scrubbing for shell-job stdout/stderr. PureredactSecretsInText(text, secrets): string-modereplaceAllso regex metacharacters in values stay literal. When the caller passesredact_secrets: true(or--redact-secrets), the handler builds a Map of inherit-name → resolved-value and post-processes both tails before throw/return so persistedresult.stdout_tail/result.stderr_tail/error_textcarry<REDACTED:name>. Onlyinherit:-resolved values are scrubbed; caller-suppliedenv:values pass through. Heuristic — defeatsecho "$GBRAIN_DATABASE_URL", not adversarial encode-then-print. Defaultfalse. -
src/core/config.ts:ensureGitignore— idempotent retroactive writer of~/.gbrain/.gitignore(single line*). Called fromsaveConfig()so every config-writing path lays it down, AND fromrunPostUpgrade()so existing users pick it up ongbrain upgrade. Never clobbers a user-customized.gitignore(checks file exists + content non-empty before writing). Scope: blocks casualgit add ~/.gbrainfrom inside an enclosing worktree, but does NOT cover already-tracked files, screenshots, backups (Time Machine / iCloud / Dropbox), orgit add -f. The doctor checkhome_dir_in_worktreesurfaces what.gitignorecan't. -
src/core/minions/handlers/shell-audit.ts— per-submission JSONL audit trail at~/.gbrain/audit/shell-jobs-YYYY-Www.jsonl(ISO-week rotation; override viaGBRAIN_AUDIT_DIR). Best-effort:mkdirSync(recursive)+appendFileSync; failures logged to stderr, submission not blocked. Logs cmd (first 80 chars) or argv (JSON array). Never logs env values. -
src/core/minions/handlers/supervisor-audit.ts— supervisor lifecycle JSONL audit at~/.gbrain/audit/supervisor-YYYY-Www.jsonl(ISO-week rotation; sharescomputeIsoWeekName()withshell-audit.ts).writeSupervisorEvent(emission, supervisorPid)appends one line per event (started,worker_spawned,worker_exited,backoff,health_warn,health_error,max_crashes_exceeded,shutting_down,stopped,worker_spawn_failed).readSupervisorEvents({sinceMs})is the readback forgbrain doctor. ExportsisCrashExit(event),summarizeCrashes(events),CrashSummarytype, andCLEAN_EXIT_CAUSESdenylist ('clean_exit' | 'graceful_shutdown'). Single shared point — bothgbrain doctor(supervisor check) andgbrain jobs supervisor statusimport from here so the two surfaces can't drift.isCrashExitclassifies a singleworker_exitedagainst the denylist: clean/graceful are NON-crashes; everything else (incl. any futurelikely_causefromchild-worker-supervisor.ts) is a crash; audit lines lackinglikely_causefall back tocode !== 0.summarizeCrashesreturns{total, by_cause: {runtime_error, oom_or_external_kill, unknown, legacy}, clean_exits}— thelegacybucket catches both old fallback entries AND unrecognized future causes (fail-loud, not silent underreport); denylist-over-allowlist is deliberate. Pinned bytest/supervisor-audit.test.ts(14 cases) and 4 source-grep wiring assertions intest/doctor.test.ts. -
src/core/minions/backpressure-audit.ts— sibling of shell-audit.ts formaxWaitingANDmaxPendingcoalesce events. JSONL at~/.gbrain/audit/backpressure-YYYY-Www.jsonl. One line per coalesce with(queue, name, waiting_count/max_waiting OR pending_count/max_pending, returned_job_id, ts).readRecentCoalesceCountsfeeds thejobs statsBackpressure line (reads current + previous ISO-week files so a 24h window survives week boundaries, filtered per queue). Makes the backpressure guards' coalesces visible instead of silent drops. Pinned bytest/backpressure-audit-read.test.ts. -
src/core/minions/handlers/subagent.ts— LLM-loop handler. Two-phase tool persistence (pending → complete/failed), replay reconciliation for mid-dispatch crashes, dual-signal abort (ctx.signal+ctx.shutdownSignal), Anthropic prompt caching on system + tool defs.makeSubagentHandler({engine, client?, ...})factory;MessagesClientis an injectable interface the real SDK implements structurally. Per-turn output cap resolves viaresolveMaxOutputTokens(data.max_tokens→agent.max_output_tokensconfig → 8192 default); astop_reason: 'max_tokens'final turn surfaces asSubagentStopReason 'max_tokens'(not a silentend_turn), and a max_tokens stop mid-tool-round injects a truncation note into the tool-result turn so the model re-issues the dropped call. ThrowsRateLeaseUnavailableError(renewable) when rate-lease capacity is full. Both loop paths (the direct Anthropic SDK turn loop and the gateway toolLoop'sacquireTurnPermit) heartbeat the held lease at ttl/3 for the duration of each provider call — single-flight renewals so a stalled renewal never stacks behind a starved pool; a FALSE renewal means the lease row was pruned/stolen (the slot is already re-admitted), so the in-flight call aborts and converts to a lease-full requeue instead of running abovemaxConcurrent. Anthropic 400prompt is too longresponses (status 400 + body matches/prompt is too long|prompt_too_long|context.*length/i) classify asUnrecoverableErrorso the job goes straight todeadon first attempt instead of stalling three times. Catches both initial-prompt overflow and turn-N tool-loop accumulation thatsynthesize.ts's chunker can't bound ahead of time. terminal-state short-circuit on resume. When a stored message thread already ends instop_reason: 'end_turn', the handler returns{ ok: true }immediately instead of issuing anothermessages.createcall (re-prompting pastend_turnwould get a 400 and dead-letter an already-successful job). Pinned bytest/subagent-handler.test.ts. Oneshot dispatch:data.mode === 'oneshot'on a FRESH job (zero persisted messages) routes torunSubagentOneshot(subagent-oneshot.ts) before either loop; a validation fallback re-enters the loops in the SAME job, stampedsynth_mode_used: 'agentic_fallback'+fallback_reason. Write accounting (finalizeWriteAccountingin subagent-persistence.ts): every job's result carries pages_attempted/written/failed derived from the tool-execution ledger (settled rows only);data.require_writesjobs (dream synthesize + patterns fan-outs) throw UnrecoverableError → dead when attempted>0 with zero successes.resolveMaxOutputTokenstakes the model: thinking-by-default models (Claude 5 by name or recipe-declaredthinking_by_defaultsuch as DeepSeek v4, via the gateway's sharedisThinkingModel) default to 32000 when neither per-job nor config caps are set. Gateway-pathonToolCallStarthas a uniq_subagent_tools_use_id backstop: a provider repeating a tool id across turns persists under a#m<idx>o<ordinal>-suffixed debug id (second violation reconciles to the existing row's gbrain_tool_use_id) instead of dead-lettering the job. Persistence helpers live in subagent-persistence.ts (pure peel;__testingunchanged). Tool-execution rows carry NO job-wide unique on the raw providertool_use_id(migration v131 dropsuniq_subagent_tools_use_id: providers like claude-cli legitimately re-mint the same short id every turn, and a collision would dead-letter the job); row identity is the stable(job_id, message_idx, ordinal)unique, and readers/settlement resolve an execution by(message_idx, tool_use_id). Settle writes (complete/failed) target exactly ONE row — the call's own ordinal first, then a legacy ordinal=NULL row, never a row that already settled complete (a broader status disjunct could only capture a same-id sibling's row); pending inserts guard legacy NULL-ordinal rows via NOT EXISTS and use the stable-idON CONFLICTas the zombie-worker backstop. Replay/reconcile resolve an execution by persisted ordinal first (validated against the raw id), then by(message_idx, tool_use_id)only when the id is unique within the turn, then the same-tool legacy positional fallback. Mixed-version window: a still-running pre-v131gbrain jobs workdaemon errors on tool persistence after the migration until it restarts (its two-phase writes targeted the dropped constraint) -
src/core/minions/handlers/subagent-oneshot.ts— oneshot synthesis runner. ONE rate-leased tool-lessgateway.chatcall (staticONESHOT_SYSTEMJSON contract rides the prompt-cache prefix; sub-budget min(5 min, timeout/4) →fallback_reason: 'oneshot_timeout'), then all-or-nothing validation BEFORE any write: JSON contract (parseOneshotResponse, ≤12 pages), slug grammar +allowed_slug_prefixes+ reflections/originals task shape + the exactoneshot_slug_suffix(the idempotency boundary, enforced structurally), exact-match wikilink rule against existing ∪ in-batch slugs (extractWikilinkTargets; cold-brain relaxation: <5 pages AND no manifest accepts syntactic presence). Writes go through the SAMEbrain_put_pageToolDef the loop uses (fences/side-effects/provenance identical) withdeferEmbeds, bracketed by standard ledger rows under invocation-scoped idsoneshot-<inv8>-p<i>; a post-batchautoLinkWrittenPagepass materializes in-batch forward wikilink edges. Ledger-first crash recovery: any prior oneshot rows → finalize from the ledger, never re-call the nondeterministic model. Transcript rows persist ONLY after success (a failed attempt is never replayable as a completed result). Every failure shape returns{kind:'fallback', reason}and the same job falls through to the agentic loop. Pinned bytest/minions/subagent-oneshot.test.ts+ the oneshot describes intest/subagent-handler.test.tsandtest/e2e/dream-synthesize-pglite.test.ts.ONESHOT_SYSTEMspells out JSON string escaping (quotes, backslashes, line breaks). A parse failure whose reported output usage reaches the requested cap falls back aslengtheven when the provider normalized the stop toend/other; below-cap malformed output staysunparseable, and valid JSON at the cap proceeds normally. -
src/core/minions/handlers/subagent-persistence.ts— subagent transcript + tool-execution persistence (pure peel from subagent.ts): loadPriorMessages/Tools/ToolsV2, persistMessage, persistToolExecPending/Complete/Failed (all$N::text::jsonbdiscipline), plusfinalizeWriteAccounting: derives pages_attempted/written/failed from the job's put_page ledger rows (settled only — pending counts toward neither), merges them into every SubagentResult, and throws UnrecoverableError on require_writes jobs whose every attempted write failed;scopeToolUseIdPrefixnarrows the scan to the oneshot invocation family. -
src/core/minions/handlers/subagent-aggregator.ts—subagent_aggregatorhandler. Claims AFTER all children resolve (queue guarantees every terminal child posts achild_doneinbox message with outcome). Reads inbox viactx.readInbox(), builds a deterministic mixed-outcome markdown summary. No LLM call. -
src/core/minions/handlers/subagent-audit.ts— JSONL audit + heartbeat writer at~/.gbrain/audit/subagent-jobs-YYYY-Www.jsonl. Events:submission(one per submit) +heartbeat(per turn boundary:llm_call_started | llm_call_completed | tool_called | tool_result | tool_failed, plus the oneshot markersoneshot_fallback | oneshot_timeout; oneshot-path events carrymode: 'oneshot'and fallback events areason— both rendered bygbrain agent logs). Never logs prompts or tool inputs.readSubagentAuditForJob(jobId, {sinceIso})is the readback forgbrain agent logs. -
src/core/minions/rate-leases.ts— lease-based concurrency cap for outbound providers (default keyanthropic:messages, max viaGBRAIN_ANTHROPIC_MAX_INFLIGHT). Owner-tagged rows withexpires_atauto-prune on acquire;pg_advisory_xact_lockguards check-then-insert; CASCADE on owning job deletion.renewLeaseWithBackoffretries 3x (250/500/1000ms). Canonical home ofRateLeaseUnavailableError(thrown when acquire finds no slot; the worker and the inline dream drain both recognize it and requeue WITHOUT burning an attempt — lease-full is a scheduling condition, not a failure;subagent.tsre-exports it for compatibility) andleaseFullBackoffMs()(the shared 1–3s jittered lease-full requeue backoffworker.tsandinline-drain.tsboth use, so the two curves cannot silently desync). -
src/core/minions/handlers/contextual-reindex-per-chunk.ts— per-page contextual re-embed handler. Resolvesmodels.contextual_synopsisonce and isolates cross-worker leases by the full resolved model id.GBRAIN_CONTEXTUAL_SYNOPSIS_RPMcontrols the cap;GBRAIN_CONTEXTUAL_HAIKU_RPMis the compatibility alias. -
src/core/minions/handlers/embed-backfill.ts—embed-backfilljob handler (the deferred lane behind sync's cost gate). Cap ladder forembed.backfill_max_usd: a present-but-invalid value (0, negative, garbage) ismisconfiguredand FAILS CLOSED to the $10 default, which is never dropped; only the IMPLICIT default cap is lifted (uncapped-with-warning to stderr) when the embedding model is unpriceable viaisModelPriceable+pricing.overrides; the off tokens (off/unlimited/none) remove the ceiling. Single-flights via the same per-source lock key as CLIembed --stale(embed-backfill-lock.ts). Carries the progress-keyed stall watchdog arm: progress = bankedembedded + chunksProcessed; a stall aborts the drain and throwsstall_timeoutso the queue fails the job and the resumable cursor re-runs. -
src/core/minions/wait-for-completion.ts— poll-until-terminal helper for CLI callers.TimeoutErrordoes NOT cancel the job;AbortSignalexits without throwing. DefaultpollMs: 1000 on Postgres, 250 on PGLite inline. -
src/core/minions/transcript.ts— renderssubagent_messages+subagent_tool_executionsto markdown. Tool rows splice under their owning assistanttool_useby(message_idx, tool_use_id)— raw provider tool ids may repeat across turns, sotool_use_idalone is not an identity; echoedtool_resultblocks check ownership against the nearest preceding assistant turn's key. UTF-8-safe truncation; unknown block types fall through to fenced JSON. -
src/core/minions/plugin-loader.ts—GBRAIN_PLUGIN_PATHdiscovery. Absolute paths only, left-wins collision,gbrain.plugin.jsonwithplugin_version: "gbrain-plugin-v1", plugins ship DEFS only (no new tools),allowed_tools:validated at load time against the derived registry. -
src/core/minions/tools/brain-allowlist.ts— derives the subagent tool registry fromsrc/core/operations.ts(13-name allow-list, size pinned bytest/brain-allowlist.serial.test.ts). Attachment toolsfile_listandfile_urlare excluded. Registry construction and execution reject everylocalOnlyoperation;selectAllowedToolsrejects malformed bindings and preserves explicit empty lists as no tools. Includesadd_timeline_entry(the canonical timeline write), fenced server-side by the sameenforceSubagentSlugFencepolicy asput_page. By defaultput_pageschema is namespace-wrapped per subagent (^wiki/agents/<subagentId>/.+). WhenBuildBrainToolsOpts.allowedSlugPrefixesis set, the put_page schema describes the prefix list to the model and the OperationContext is threaded withallowedSlugPrefixes— trusted local jobs receive these from the submitter; remote-owned jobs receive the grant intersection plusdelegatedAuth. Their per-call operation contexts carry current source grants and retain remote visibility policy, without requiring direct read/write scopes for independently bound delegated tools.BuildBrainToolsOpts.deferEmbeds(server-side-only, set by the oneshot runner for its programmatic writes; never hydrated from any wire payload) threadsOperationContext.deferEmbedsso put_page defers chunk embeddings for the phase-end backfill. Allow-list includesget_recent_salience+find_anomaliesbut deliberately NOTget_recent_transcripts(all subagent calls runctx.remote === trueand the trust gate rejects remote callers, so it would always reject; the cycle synthesize phase callsdiscoverTranscriptsdirectly instead).paramsToInputSchema()consumesparamDefToSchemafromsrc/mcp/tool-defs.ts; required-aggregation at the tool-def level stays here (the shared helper is per-param).execute()runs the samevalidateParamsthe MCP dispatchers use before the op handler, so a call that omits a required parameter (or sends the wrong type / an unknown enum value) fails with<tool>: Missing required parameter: <name>the model can act on, instead of a handler crash. -
src/mcp/tool-defs.ts—buildToolDefs(ops, opts?)helper; the stdio MCP server, the OAuth HTTPtools/list, and the subagent tool registry all consume it, byte-for-byte equivalence pinned bytest/mcp-tool-defs.test.ts.opts.strictParams: true(whenmcp.strict_paramsresolves'reject') additionally declares the_meta/dry_runpassthrough keys inpropertiesand closes each schema withadditionalProperties: false(schema-validating clients must not strip_meta.session_id); both emission states pinned. Exports the recursiveparamDefToSchema(p: ParamDef)— single source of truth for ParamDef→JSON Schema mapping shared bybuildToolDefsandsrc/core/minions/tools/brain-allowlist.ts(subagent registry). Recursive onitemsso nested array-of-arrays preserves inner shape on the wire. Key ordering (type, description, enum, default, items) is intentional soJSON.stringifyoutput stays byte-stable.test/mcp-tool-defs.test.tshas afindArrayWithoutItemswalker that fails on anytype: 'array'lackingitems.type. -
src/core/verbs.ts— MEMORY_VERBS v1: the four frozen protocol verbs (remember,entity,synthesize,forget) as first-class Operations, plusMEMORY_VERBS_VERSION(single source of truth, =1),VERB_NAMES, the hand-authoredRESPONSE_SCHEMASregistry (Operation carries input params only; response shapes live here and conformance validates LIVE responses against them), andERROR_SCHEMA. The fifth verb is the extendedrecallop in operations.ts. RUNTIME LEAF invariant: operations.ts spreadsverbOperationsinto its array at module-eval time, so this file must never statically import operations.ts VALUES (type-only imports fine; handlers use dynamic import) — violating it causes a TDZ crash on whichever module evaluates second. Every verb error carries a populatedsuggestion+protocol_version(viaverbErrorin operations.ts). Theforgetverb deliberately has NO cliHints (CLI_ONLYforgetdispatches first and would shadow it). Frozen contract: docs/protocol/MEMORY_VERBS_v1.md; pinned by test/memory-verbs-conformance.test.ts. -
src/core/verbs/entity-card.ts—buildEntityCard(engine, sourceId, name, {remote}): the zero-LLM sub-100ms card behind theentityverb. Resolution reuses the Retrieval Reflex precision arms (alias > exact slug > exact title > slug-suffix; exact-slug candidates include the RAW input because slugify flattens slashes in namespaced slugs). Within exact-title collisions, canonical entity types (person,company,organization,entity) outrank note/conversation containers, then GREATEST(updated_at, last_retrieved_at) breaks same-shape ties; a lone non-entity exact-title page remains a valid fallback. Per-arm degradation: a pre-page_aliases brain still resolves via arm 2 (aka returns empty, never throws). Card assembly is a parallel Promise.all of depth-1 indexed reads (page row, alias reverse lookup, getLinks+getBacklinks mentions-excluded cap 10, getBacklinkCounts, getTimeline(5), listFactsByEntity world-only-when-remote); deliberately NOT the recursive-CTE traversePaths — the card is a latency contract (CI gate: test/entity-card-perf.slow.test.ts, p99 < 100ms × GBRAIN_PERF_BUDGET_MULTIPLIER + 50× getPage-p50 ratio guard on a 20K corpus).summaryruns through the exportedsafeSynopsis(the get_page fence boundary). Miss → keyword near-miss suggestions with create_safety hints. -
src/core/verbs/usage-log.ts— observability sidecar: one JSONL line per verb call at~/.gbrain/integrations/memory-verbs/usage.jsonl(gbrainPath — GBRAIN_HOME honored;brainId()= the resolved gbrain home). LOCAL ONLY, never uploaded, stats-only (lock-free 10MB rotation may drop lines; O_APPEND line-atomic, best-effort on Windows).logVerbUsageis fire-and-forget (never awaited, never throws); written from the DISPATCH layer so param-validation failures count.readVerbUsage/earliestVerbUsageTsfeedgbrain protocol stats(incl. measured TTHW vs the init-stampedprotocol_installed_at) and the doctormemory_verbs_usagecheck. -
src/core/verbs/conformance.ts+src/core/verbs/conformance-fixtures.ts— the conformance runner core (transport-agnostic: minimalConformanceClient= list_tools + call_tool) and the embedded fixture set. Deterministic by construction: shape/enum/behavior/round-trip checks only, never ranking quality. Validation is NON-STRICT on extra fields (additive-forever means unknown fields are always legal). Entity-card cases seed via put_page when the target exposes it and skip honestly on verbs-only targets; synthesize is cost-gated behind --synthesize.validateAgainstSchemais a minimal JSON-Schema-subset validator (type unions, required, properties, enum, const, items). Fixtures mirror totest/fixtures/memory-verbs/cases.json(BrainBench seeds; drift-guarded by the conformance test). The negative self-test (test/memory-verbs-conformance.test.ts) proves the runner FAILS a lying server. -
src/core/facts/write-single.ts—writeSingleFact(fact, ctx): the zero-LLM single-fact seam behindremember.runFactsPipelineis extraction-first (LLM-gated) and cannot back a pre-formed fact; this reuses the pipeline's post-extraction stages directly: resolve → embedding-cosine dedup (same 0.95 threshold) → fence-first write with the same legacy DB-only fallbacks (thin-client, unparented, stub-guard). Supersession: deterministic rule — same entity_slug + same kind + similarity ≥ threshold + DIFFERENT text ⇒ the new fact supersedes (fence path: append new + forgetFactInFence(old) + superseded_by link; DB path: engine insertFact supersedeId). Provenance lands on NewFact.source verbatim (no FactsBackstopCtx). No embedding provider ⇒degraded_dedup: true(near-duplicates may insert — documented).isNullLikeEntity(entity)(exported) normalizes null-like entity tokens ("null","undefined","none","n/a","nil","-", empty/whitespace) to ABSENT before resolution — applied here rather than only at the verb boundary so EVERY writeSingleFact caller gets the guard and theresolved?.slug ?? entityReffallback can never adopt"null"as a slug. -
src/core/facts/forget.ts—forgetFactInFence(engine, id, {reason?, sourceId?, worldOnly?})checks source and visibility before inspecting state, then commits a durable withdrawal throughfacts/withdrawal.ts. A writable source fence is atomically struck with a date and reason; otherwise the DB-body mirror is best-effort. Both paths remain withdrawn on stale reimport while the withdrawal database is preserved. The file resolver and per-page lock match fence writes. The body-only mirror deliberately leaves a hash mismatch so the next import re-chunks. Forget is retraction, not deletion of original prose, files or backups. Pinned bytest/forget-reconcile-durability.test.ts,test/privacy-strip-and-forget.test.ts, andtest/e2e/delegated-grants-withdrawal.test.ts. -
src/core/facts/withdrawal.ts+withdrawal-schema.ts— durable withdrawal key is(source_id, visibility, SHA256(normalized claim)); entity/page names are excluded so rename and index recreation do not restore it. Normalization folds case and whitespace, not meaning. The source row lock serializes withdrawal with the facts trigger; the trigger prevents insert/update from reactivating a matching claim. Import overlays valid matching fence rows as struck before hashing/chunking; malformed fences retain their existing diagnostics and the DB trigger still protects derived facts.writeSingleFactrefuses an identical withdrawn claim before embedding, using the protocol error envelope. Ordinary TTL/supersession expiry creates no withdrawal. Migration backfill only trusts explicitforgotten:markers; old unmarked DB-only intent cannot be inferred. Markdown-only copies do not preserve DB-only withdrawals. -
src/core/minions/delegated-policy.ts— immutable submitted grant ceiling intersected with the current OAuth grant. Tools, explicit operation authority, source-only reads, write prefixes, concurrency and finite budget may narrow but never widen during execution; changed source/brain, revoked clients, archived sources or unusable intersections fail closed. Explicit job namespaces resolve towiki/agents/<job-id>/*. Removingsubmit_agentstops the next execution boundary; explicit operation allowlists also constrain delegated tools, while legacy null operation lists preserve separately bound agent-only tools. Surface controls transport discovery. Empty tools never mean all tools. Legacy running jobs adopt the ceiling frozen in their persisted payload by grant rescope; host/current brain aliases mean the serving brain. -
src/core/minions/delegated-admission.ts+delegated-tools.ts— queue admission locks the client row through validation, idempotency/coalescing, counting all nonterminal owned jobs across queues, and insertion. Returning an identical existing job adds no slot. Every delegated tool execution, including replay and one-shot writes, rechecks the current intersection and rebuilds the operation context with its current source/path bounds and authenticated read-source ceiling, including explicit per-call source overrides. Remote-owned writes cannot overwrite private existing pages (including dedup redirects), append their timeline, or run trusted-workspace auto-link/timeline hooks. One-shot remote-owned writes keep links as literal text without probing hidden target existence or counts; trusted local behavior is preserved. Trusted owner fields are separate from user parameters. Replay cannot replace identity or original bounds; valid owned replays and terminal-job retries consume a slot under the same client lock. Legacy terminal jobs missing a stored ceiling require fresh authenticated submission. -
src/core/minions/delegated-spend.ts+src/core/ai/invocation-guard.ts+guarded-generation.ts— async-local admission around each owned provider attempt, including nested paid tool work; unrelated local calls retain their transport behavior. Finite caps require canonical pricing and a declared input ceiling plus enforced output maximum before IO; unknown pricing/bounds refuse. Hidden SDK retries are disabled for owned attempts. Actual reported usage is settled with cache categories normalized across SDK/raw responses; absent usage remains explicit unresolved liability. The current grant is checked again under the reservation's client lock. -
src/core/minions/budget-meter.ts— atomic reserve/settle accounting. Per-client advisory lock plus client row lock protects current cap and outstanding holds. NULL cap means unlimited; a caller's existing finite ceiling still applies. Pending and expired unresolved holds count across UTC dates; TTL marks overdue rather than releasing liability. Unknown-bound holds prevent subsequent finite admission until reconciled. Settlement is idempotent and writes reservation state plus spend log in one transaction. Tests cover races, changed caps, unknown usage, late settlement and both engines. -
src/core/facts/extract.ts— the LLM facts extractor (extractFactsFromTurnWithOutcomeand friends; the output cap, truncation retry, and salvage behavior are described under the trajectory entry). Deterministic junk gate:isJunkFact(text, kind?)testsJUNK_FACT_PATTERNS— assistant plan/offer narration, meta-narration about the conversation itself, provider billing/rate-limit strings captured verbatim — and is kind-aware: a candidate the extractor classifiedcommitmentis exempt from the plan-narration arm ONLY ("I'll send the deck by Friday" is the surface shape of a genuine commitment, the one kind the loop engine exists to capture); meta-narration and provider-error strings are junk for every kind.isJunkFilterEnabled(engine)is the config kill switch (fail-open on a config read error);getFactsExtractionPromptAppendixsupplies the config-driven prompt appendix. Entity gate at the candidate loop (the ONE seam where LLM-emitted entity tokens becomeentity_slug):isUnknownSpeakerLabel(anonymous diarizer labels —Speaker A,SPEAKER_00,spk_0,other|unknown|guest) andisNullLikeEntity(placeholder STRINGS"null"/"None"/"n/a"where the prompt asked for JSON null; shared withwrite-single.ts) both null the attribution and KEEP the fact, so no consumer (runFactsPipeline,extract-conversation-facts) can mintentity_slug='null'through the resolver's fallback-slugify floor. The provider-error arm is start-anchored (^\W*, optional error/status token or a<step> stopped|failed because …prefix, then(spend|rate) (limit|cap) … (hit|exceeded|reached)), so a genuine fact that merely mentions a spend limit or rate limit mid-sentence survives; the verbatim provider strings still match. Two config keys:facts.extraction_prompt_appendix(non-empty text is appended to the extraction prompt, so an operator can steer what counts as a fact without a release) andfacts.extraction_junk_filter(falsedisables the deterministic junk gate; default on). -
src/mcp/surface.ts— MCP tool-surface modes:'verbs'(exactly the ops markedverb: true) |'starter'(the ~27-op daily-driver set) |'full'(default — identity; existing installs unchanged).STARTER_OPSis composed PROGRAMMATICALLY: a spread ofVERB_NAMES(never a hand-count) + the fallback daily slice (BRAIN_TOOL_ALLOWLIST+ the agent lanesubmit_agent/get_agent_job) +whoami+request_tools+capture(a DIRECT literal, deliberately NOT via the allowlist); re-derived from production usage viascripts/derive-starter-ops.ts(paste-in proposal, never an auto-edit).ALWAYS_INCLUDED_STARTER_OPS(verbs +whoami+request_tools+ the agent lane +capture) is the exported always-set consumed by BOTHderive-starter-ops.tsand the advisor starter-fit collector, so neither re-types the composition.parseSurfaceFlag(strict, loud reject),resolveSurface(flag > configmcp_surface> full),filterOpsForSurface,allowedOpNames. Enforcement is TWO-layer and fail-closed: the advertised list ANDdispatchToolCall'sallowedOpsset (a hidden op returnsunknown_tooleven when called by name) — applied on stdio (server.ts) and BOTH HTTP paths (serve-http.ts after!localOnly, and the second transport http-transport.ts). Per-client ceiling machinery (OAuth transport only):effectiveSurfaceForClientis the pure compositionclamp(min(server ceiling, client row surface ?? mcp.default_surface_dcr ?? ceiling)); the per-REQUEST application lives in serve-http.ts'sresolveEffectiveSurface, which short-circuits averbsceiling (min() cannot go lower, so the config read is skipped) and, on a default-surface read failure, applies the LAST successfully read default (held per process, fail-closed) so a transient config outage cannot silently widen a NULL-surface client to the ceiling;resolveClientRowSurfaceignores unknown row values with a bounded warn-once (the column's value space is documented OPEN for future tier names);resolveDefaultClientSurfacereads the DCR default dual-plane (DB > file > null) and never throws;clampSurface/readForceSurfaceEnvfold in theGBRAIN_MCP_FORCE_SURFACEkill switch — NARROW-ONLY by construction (min(); can never widen past the ceiling).minSurface/surfaceWiderThanown the verbs < starter < full rank math. Pinned by test/mcp-surface.test.ts (membership, monotonicity verbs ⊆ starter ⊆ full, ceiling + force-narrow cases). -
src/commands/protocol.ts—gbrain protocol [--json] | conformance [--target <http-url|stdio-cmd>] [--token] [--synthesize] | stats [--days N].--jsonemits input schemas from the LIVE Operation defs + RESPONSE_SCHEMAS (doc/code can't drift). Conformance default target self-spawns gbrain's own stdio server (dev.tsentry vs compiled binary both handled); CI certifies stdio with --synthesize (no key ⇒ asserts the cleanunavailableerror). Stats aggregates the usage sidecar + the measured TTHW; output states "local JSONL only — never uploaded". CLI_ONLY + SELF_HELP wired in cli.ts; no pre-bound engine. -
src/core/minions/attachments.ts— Attachment validation (path traversal, null byte, oversize, base64, duplicate detection). -
src/commands/agent.ts—gbrain agent run|logs|registerdispatcher.runsubmitssubagent(or N children + 1 aggregator) under{allowProtectedSubmit: true}; single-entry--fanout-manifestshort-circuits; children geton_child_fail: 'continue'+max_stalled: 3;--followis the default on TTY (streams logs + pollswaitForCompletionin parallel; Ctrl-C detaches, does not cancel).logsdelegates toagent-logs.ts;registerlazy-importsagent-register.ts. Subcommand-aware help is answered BEFORE any engine or queue work and STOPS at the--terminator (agent run -- --helpsubmits the LITERAL prompt); on a brainless machine (null engine) help still prints while realrun/logsinvocations refuse with an init hint. -
src/commands/agent-logs.ts—gbrain agent logs <job> [--follow] [--since]. Merges JSONL heartbeat audit +subagent_messagesinto a chronological timeline.parseSinceaccepts ISO-8601 or relative (5m,1h,2d). Transcript tail renders only for terminal jobs. -
src/commands/agent-register.ts—gbrain agent register <name> --harness claude-code|codex|opencode|openclaw: mints a scoped OAuth client + access token and prints the exact harness wiring in one step (CLI-only, never an MCP op). Composes existing parts —registerScopedClient(auth.ts),exchangeClientCredentials, the mcp-registration argv builders,renderCodexHttpServerBlock,openclawThinClientBlock— no new auth machinery. Order is load-bearing: cli.ts pre-connect guards (thin-client refusal + PGLite live-serve probe) → pure parse (exit 2) → preset resolve → source validation (existence + not-archived, engine laneANY(\$1::text[])) → column pre-flight OUTSIDE any tx (25P02 forbids in-tx degrade) → ONEengine.transaction(name-scopedpg_advisory_xact_lock→ duplicate-name pre-check →ensureWorkspaceSourcecreate-or-reuse-only-when-truly-empty (refuses page-, fact-, or file-bearing and archived sources) →registerScopedClientwith ttl + surface) → COMMIT → post-commit fail-open surface audit → token exchange on the OUTER engine (the tx sql is dead) → serve probe (probeServeHealth+ the unconditionalSCOPES_MIN_SERVE_VERSIONfloor line): a probe-PROVEN pre-scopes serve REFUSES registration (serve_too_old) unless--allow-old-serve— such a serve verifies scoped tokens as full access; an unreachable serve stays a warning, and the reissue lane warns instead of refusing (the secret is already rotated by that point) → render + print (human, or ONE JSON doc carryingprobe_note+serve_warningfrom the probe; secrets redacted unless--show-token). Presets:daily-driver(write one source; federated reads = SNAPSHOT of non-archived sources at registration, EXCLUDING other agents'*-workspacescratch sources — a workspace named explicitly in--federated-readis still granted, and the print counts the exclusions) andcoding-agent(write-isolated derived<name>-workspaceDB-only source; requires--federated-read); both default the client to thestartersurface — override with--surfaceat registration or widen per client viagbrain auth rescope-client. Always writestoken_ttl(default 30 days — the server default is 1 hour and would kill a pasted config).--url|--portrequired (every block embeds the brain URL). A scope blocklist keeps operator-grade scopes onauth register-client.--reissue <client-id>rotates the secret under the same advisory lock and reprints the block (headed by the stored client_name); rotation is not revocation — outstanding tokens live until expiry. Failure after COMMIT prints the client_id + the exact revoke command (never a false "nothing was created"). openclaw renders the honest thin-client CLI block (no native remote-MCP client upstream yet). Pinned bytest/agent-register.test.ts. -
src/commands/jobs.ts—gbrain jobsCLI subcommands +gbrain jobs workdaemon. Help is real and guarded:JOBS_HELP(full block incl. watch/stats/smoke flags + a footer naming exactly the five subcommands with dedicated help) andJOBS_SUBCOMMAND_HELP(work/supervisor/submit/watch/prune) print from a guard at the TOP ofrunJobs, BEFORE the thin-client refusal and the subcommand switch —jobs work --helpcan never start a daemon; only--help/-hare help tokens (barehelpcan be a job name); cli.ts routes it engine-free viaSELF_HELP_WITHOUT_ENGINE+CLI_ONLY_SELF_HELP.formatJobDetailprints the effective wall-clock budget (Timeout:/Deadline:— 1x deadline kill when claimed, 2x wall-clock backstop, or which default applies) with a Date|string-defensive renderer;timeout_atis inJOB_DATE_FIELDSfor thin-client rehydration.jobs statsprints aBackpressure (24h)line (per-name coalesce counts from the backpressure audit, current+previous ISO-week files, queue-filtered, best-effort) plus a suppressed-by hint naming the in-flight live-lock job when waiting=0 past the sharedGBRAIN_WEDGED_QUEUE_WARN_MINUTESthreshold — the read-only visibility for maxPending single-flight suppression. Pinned bytest/jobs-subcommand-help.serial.test.ts,test/jobs-format-detail.test.ts,test/jobs-stats-backpressure.serial.test.ts.case 'work'wrapsworker.start()in try/finally and owns engine lifecycle — callsengine.disconnect()on shutdown with loud error logging (the worker must not disconnect an engine it doesn't own; pool slots free immediately on shutdown rather than waiting for TCP keepalive).jobs submitsurfaces theMinionJobInputretry/backoff/timeout/idempotency surface as flags:--max-stalled,--backoff-type fixed|exponential,--backoff-delay,--backoff-jitter,--timeout-ms,--idempotency-key,--max-waiting(maxPendingis deliberately internal-only — no flag; see TODOS).jobs smoke --sigkill-rescueis the SIGKILL-rescue guard.registerBuiltinHandlersalways registerssubagent+subagent_aggregator(no env flag —ANTHROPIC_API_KEYis the cost gate, trust is viaPROTECTED_JOB_NAMES) and loadsGBRAIN_PLUGIN_PATHplugins at startup with a loud per-plugin line;shellhandler still gated byGBRAIN_ALLOW_SHELL_JOBS=1(RCE surface). Theautopilot-cyclehandler forwardsjob.data.phasestorunCycle, validated againstALL_PHASESfromsrc/core/cycle.ts(invalid names filtered; empty/missing falls back to the default cycle); whensource_idis set it bindsbrainDirto that source'slocal_path(null for a pure-DB source, never the global repo) and checksisSourceInCooldownbeforerunCycle, returning a no-opskipped(not a failure) for a source still in its failure cooldown. The siblingautopilot-global-maintenancehandler runsMAINTENANCE_PHASES(mixed ∪ global) once (nosourceId,pull:false) and stampsautopilot.last_global_aton success.resolveJobPullgives both cycle and standalone sync jobs one positive-polaritypullcontract while preserving queued payloads that still carry the inverse legacynoPullkey; explicitpullwins. Thesynchandler resolvessourceIdat entry fromsources.local_path(mirrorscycle.ts:480) so multi-source brains read the per-sourcelast_commitanchor; concurrency routes throughautoConcurrency()insrc/core/sync-concurrency.ts(PGLite stays serial);noEmbeddefault istrue.gbrain jobs supervisor statusconsumessummarizeCrashes()fromsrc/core/minions/handlers/supervisor-audit.tsfor parity withgbrain doctor: JSON addscrashes_by_cause: {runtime_error, oom_or_external_kill, unknown, legacy}+clean_exits_24h; human output includes per-cause + clean-exits lines. Pinned bytest/job-pull-policy.test.tsand 4 source-grep wiring assertions intest/doctor.test.tsrequiringcrashes_by_cause+clean_exits_24h=in bothdoctor.tsandjobs.ts.gbrain jobs watchdecouples its two output axes:--jsonpicks FORMAT (human default, never gated on isTTY),--followpicks LOOP (defaultisTTY && !json). Non-TTY with no flags prints ONE human snapshot then exits (clean for subagent/pipe/cron);--followopts into a continuous stream (human plain per tick, or JSONL with--json); a TTY with no flags keeps the live ANSI dashboard. Resolution is the pureresolveWatchMode(opts, isTTY): {json, follow, useAnsiDashboard}insrc/commands/jobs-watch.ts; the dispatch wires--follow. Pinned bytest/jobs-watch-mode.test.ts(format×loop matrix incl. the TTY+--json-one-shot case) +test/e2e/non-tty-output.serial.test.ts(thecmd </dev/nullnon-empty-stdout contract). registers 11 Minion handlers:reindex,repair-jsonb,orphans,integrity,purge,synthesize(PROTECTED),patterns(PROTECTED),consolidate(PROTECTED),extract_facts,resolve_symbol_edges,recompute_emotional_weight. Phase wrappers delegate torunCycle({phases:[name]})sosrc/core/cycle.tsstays the single source of truth for phase semantics. The standalonesynchandler passesnoExtract: trueto matchrunPhaseSync's contract (doctor's remediation plan emitting[sync, extract]would otherwise double-extract). Theextracthandler routes{stale: true}jobs (submitted by performSync's size-gate defer branch) throughextractStaleFromDBscoped todata.sourceId, chaining a continuation job (no maxWaiting — same NULL-sourceId coalesce hazard; timeout derived fromSTALE_TIME_BUDGET_MS) when the sweep's budget leavesstaleRemaining > 0with forward progress.parseJobIsolationFlag(args, env?)(--job-isolation, space/= forms,GBRAIN_JOB_ISOLATIONfallback, default inline);case 'work'resolves + fail-fast validates the child CLI invocation and warns when--max-rssis combined with isolation (watchdog covers the worker only);case 'run-child'is the INTERNAL child entrypoint (quiet registerBuiltinHandlers incl. plugin discovery, CLI owns disconnect+exit); startup warning when a Supabase-shaped engine runs single-pool (kill-switch collapse made loud); thedb_deadfatal text is verdict-aware (pool starved vs server unreachable). Genericjobs submitnormalizes the job name once before trust, queueing, handler lookup, and audit; after the admission check and dry-run return, its explicitensureSchema()preflight still runs before shell/handler validation so stale brains retain the canonical init/migration error precedence. On PGLite,jobs submit embed-backfillrefuses withno_worker_surfaceand a paste-ready inline embed command before any queue access;--dry-runevaluates the same read-only feasibility gate and never says “Would submit” for an impossible job.--followis the sole CLI exception because it starts and awaits the inline worker; padded names follow the same normalized path and cannot strand a row. The MCPsubmit_joboperation applies the same gate (including dry-run), translates the expected refusal to branchableOperationError('no_worker_surface'), and never emitsinternal_errorfor this capability miss. Thesynchandler forwardsjob.data.github_item({repo, number, kind}) intoperformSyncfor github-kind single-item webhook refreshes. -
src/commands/features.ts—gbrain features --json --auto-fix: usage scan + feature adoption salesman. -
src/commands/autopilot.ts—gbrain autopilot --install: self-maintaining brain daemon (sync+extract+embed). Freshness sync jobs always send an explicit positive-polaritypullvalue derived from the source's parsedremote_url, so local-only sources skip pull and PGLite JSON-string configs behave like Postgres objects. ConsumesdetectTini()fromsrc/core/minions/spawn-helpers.ts, resolved once at startup. Composes aChildWorkerSupervisorinstance for spawn-and-respawn with--max-rss 2048andmaxCrashes: 5.onMaxCrashesExceededroutes through autopilot's ownshutdown('max_crashes')so the autopilot lockfile gets cleaned up.shutdown()drains viachildSupervisor.killChild('SIGTERM')+awaitChildExit(35_000). Pinned bytest/autopilot-fanout-wiring.test.tsandtest/autopilot-supervisor-wiring.test.ts(6 static-shape guards: composes ChildWorkerSupervisor not legacy names,--max-rss 2048in argv,maxCrashes: 5literal, shutdown-via-callback, no workerProc reference). tick body invokesrunNightlyQualityProbewhencfg.autopilot.nightly_quality_probe.enabled === true(default OFF — opt-in to protect API spend). NO scheduler-side rate-limit check —runNightlyQualityProbe's internalshouldRunNightly(reading the audit JSONL) is the single source of truth. Probe call wrapped in try/catch that logs vialogErrorand does NOT bumpconsecutiveErrors(probe failure is informational, never crashes the loop). Defaultmax_usdcap = 5. Pinned bytest/autopilot-nightly-probe-wiring.test.ts. per-sourceextract_atomsauto-drain. Postgres-only block after the freshness fan-out: gated onautopilot.auto_drain.enabled(default true) AND!packDeclaresPhase(engine,'extract_atoms')(the silent-backlog condition) AND per-sourcecountExtractAtomsBacklog > threshold(default 25) AND a daily capfloor(max_usd_per_day / ~\$0.30). EnumeratesloadAllSources. Submits the PROTECTEDextract-atoms-drainjob ({allowProtectedSubmit:true}) with a UTC-day time-sloted idempotency keyautopilot-extract-atoms-drain:<src.id>:<utcDay>(a static key would block the source after the first job completed).src/core/minions/protected-names.tslistsextract-atoms-drain;src/commands/jobs.tsregisters the handler (thin wrapper overrunExtractAtomsDrainForSource,LockUnavailableError→{deferred:true}; aprovider_failureresult throwsformatDrainProviderFailure(result)— batches/remaining plus the drain's sanitizedlast_error— so the dead-lettered job'serror_textnames the cause, e.g. a missing provider key);src/core/config.tscarries theautopilot.auto_drain.*config keys + theautopilot.key prefix. Pinned bytest/extract-atoms-drain-handler.test.ts,test/autopilot-auto-drain-wiring.test.ts. federated-brain co-existence + launchd hygiene. (1)LOCK_PATHresolves viagbrainPath('autopilot.lock')so it honorsGBRAIN_HOME(two brains can run autopilot simultaneously without lock-stealing); lock file stores PID, startup checkskill -0 <pid>before refusing to start (a stale lock from a crashed process does not block). (2) exportedclassifyReconnectError(err)returns'recoverable' | 'unrecoverable'; unrecoverable causesprocess.exit(0)so launchd backs off instead of loopingconfig.database_url undefined. (3) exported puregenerateLaunchdPlist(wrapperPath, home)setsThrottleInterval=300so launchd respects the exit-0 backoff. Pinned bytest/autopilot-lock-path.test.ts+test/autopilot-reconnect-classifier.test.ts. targeted-submit loop instead of blanketautopilot-cycledispatch. Each tick: cheapengine.getHealth()(single SQL count) +computeRecommendations(), then route by shape —score >= 95 AND no plan AND <60min since last full→ sleep;score >= 95 AND >=60min→ submitautopilot-cycle(60-min floor exercises phase-coupling invariants on healthy brains);plan <= 3 steps AND est <5min→ submit individual handlers;plan large OR score < 70→ submit fullautopilot-cycle. Thegbrain-cyclelock ensures targeted submissions and the full cycle can't run concurrently.maxWaiting: 1per submit closes the queue-fan-out vector. daemon env lane:writeWrapperScriptadditively sources<gbrainDir>/env(honorsGBRAIN_HOME) after the shell profiles with aset -awrap so dotenv-styleKEY=valuelines export too, and creates a fully-commented 0600 template on install (never overwritten, never chmod'd, never removed by uninstall; template excludesGBRAIN_HOME— the wrapper bakes it AFTER sourcing). Exported purechatBootWarning(chatAvailable, gbrainDir)derives both remediation paths from the passed dir and prints viaconsole.logat daemon boot (stdout is the autopilot.log sink on all four targets; stderr lands in the unsurfaced autopilot.err) using the bare global-modelisAvailable('chat')probe on purpose (mirrors the bare-gated phases; facts extraction is model-aware and doctor owns it). Install is reload-safe so the warning's re-run---installremediation is true: launchd unloads before load (bare load errors on a loaded agent), systemdtry-restarts afterenable --now, cron/container print a residual-process notice instead of auto-killing. Pinned bytest/autopilot-install.test.ts(16 cases incl. real-bash wrapper execution). -
src/mcp/instructions.ts— canonical MCP initialize-time agent operating contract. It is compile-time source text (no runtime filesystem dependency; static imports of pure core leaves are fine — the rule bans filesystem/dynamic loading) and is imported by the stdio SDK server, OAuth HTTP SDK server, and legacy raw HTTP initialize path so all transports return the same instructions without drift. The contract covers skill routing, untrusted-content posture, source scope, and theput_pagewhole-page replacement/read-before-write rule. Two append-only extensions compose on top of it, in a fixed order:buildMcpInstructions({writeback?})adds the opt-in ambient-writeback section (fromsrc/core/facts/writeback-instructions.ts) whenmemory.auto_writebackis enabled — absent/off is BYTE-IDENTICAL toGBRAIN_MCP_INSTRUCTIONS, and three exact-equality transport pins depend on that;resolveMcpInstructions(config, env?, { writeback? })takes that composed base and appends the operator-set deployment identity LAST under aDeployment identity:banner (no transport can weaken the contract):GBRAIN_MCP_INSTRUCTIONSenv wins when non-blank, elsemcp.instructionsfrom the FILE plane (~/.gbrain/config.json;gbrain config set|unset mcp.instructionsroute there viaFILE_PLANE_DOTTED_KEYSinsrc/commands/config.ts); an empty or whitespace-only env value is UNSET, not an override, and blank identity + writeback off returns the byte-identical canonical contract. Every transport callsresolveMcpInstructionswith its own writeback opts: stdio once at boot via the fail-closed resolver (restart-to-flip, the strict-params posture; read failure = OFF bundle, never a wrong visibility posture; a runninggbrain servealso needs a restart to pick up a new identity); OAuth per request (write-scope-gated, and the section renders only when THIS token's actual visible set — surface filter + bound-client fence — can callremember;extract_factsadvertised under the same predicates); legacy bearer per initialize (surface-clamped: a surface withoutremembergets no section). All three route availability throughambientOptsFrom(wb, {remember, extractFacts}). Pinned over a real SDKStdioClientTransportbytest/e2e/serve-stdio-roundtrip.test.ts, over raw HTTP bytest/http-transport.test.ts, the identity plane (incl. the three-way composition order) bytest/mcp-server-identity.test.ts+test/config-set-mcp-instructions.test.ts, and the enabled-state builder/parity matrix bytest/mcp-instructions-writeback.test.ts. -
src/mcp/server.ts— MCP stdio server (generated from operations). Its SDKServeris initialized with the shared contract fromsrc/mcp/instructions.ts, so a real client'sgetInstructions()is populated during the initialize handshake.createDefaultWriteAdvisory(engine, {enabled, write?})is the stdio lane's once-per-process unscoped-default-write advisory: on a mutating call whose source resolved to tierseed_defaultit runsassessUnscopedDefaultWrite(src/core/source-resolver.ts) and prints the warning to stderr; it latches on the first SUCCESSFUL assessment whatever the verdict (the aggregate is expensive and its inputs are process-stable), never latches on a failed one (fail-open for that write; the next mutating seed_default call retries), coalesces concurrent calls onto one in-flight assessment, and never blocks a write (a throwing writer still counts as assessed). Non-mutating calls and non-seed tiers return early without latching. Tool-call handler delegates todispatchToolCallfromsrc/mcp/dispatch.tsso stdio + HTTP transports share one validation, context-build, and error-format path. Stdin'end'/'close'shutdown hooks are skipped whenprocess.env.MCP_STDIO === '1'— gateway-piped stdio MCP wrappers (OpenClaw'sbundle-mcp) pipe the handshake then close their stdin half, which would otherwise kill the server before the first tool call; signal handlers (SIGTERM/SIGINT/SIGHUP) + the parent-process watchdog still cover legitimate disconnects.src/commands/serve.tsexposesServeOptions.mcpStdio?: booleanas a test seam so the guard is exercisable without process.env mutation. The resolve-IPC listener (retrieval-reflex resolve / turn_context / context_pack + the serve-delegated sync and sweep kinds) binds throughsrc/mcp/resolve-ipc-binding.ts(next entry) on BOTH serve transports — stdio here andgbrain serve --http; the shutdown chain awaitsshutdownDelegatedSync()beforeengine.disconnect(). Pinned bytest/serve-stdio-lifecycle.test.tsandtest/e2e/serve-stdio-roundtrip.test.ts. -
src/mcp/source-preflight.ts—assertStdioSourceBindable(engine, env?): stdio-lane boot preflight called first thing instartMcpServer, before any transport attaches. A well-formedGBRAIN_SOURCEthat names no ACTIVE source row (SELECT id FROM sources WHERE id = \$1 AND archived = false, the same predicate asassertSourceExistsinsrc/core/source-resolver.ts) throws a message naming the value and the fix (gbrain sources list), so serve exits 1 instead of binding every call to a phantom scope where reads return[]and writes die on the sources FK while health stays green. Exemptions mirrorresolveMcpStdioSourceScope: unset env,__all__, and malformed values are left to the resolver; an engine error fail-opens (guards config, not connectivity); a degraded engine (isEngineDegraded) is never touched so the boot does not spend the proxy's reconnect attempt. HTTP is out of scope (tokens carry their own source grant). Pinned bytest/mcp-stdio-source-preflight.test.ts(helper cases + a realstartMcpServerintegration case with a tmpGBRAIN_HOME). -
src/mcp/resolve-ipc-binding.ts— shared resolve-IPC listener wiring for BOTH serve transports.bindResolveIpcForServe(engine, defaultSource)binds the retrieval-reflexresolve,turn_context, andcontext_packkinds (socket + secret keyed offhash12(database_url)under~/.gbrain/runviaresolveSocketPathForConfig; the bound-source posture rejects requests naming any other source) plus the serve-delegated sync (sync_start/sync_status/sync_abort→src/core/serve-sync-runner.ts) and maintenance-sweep (sweep_start/sweep_status→src/core/serve-sweep-runner.ts) kinds — each delegation family in its OWN try/catch, so a runner failure logs[serve-sync]/[serve-sweep] handlers unavailableand the core kinds still start; both families share theGBRAIN_SERVE_SYNC_IPC=0kill switch. Engine-uniform: Postgres brains listen too (the hook lane is engine-free by design, so IPC through a serve is its only DB path there). Best-effort by contract — a bind failure never blocks the serve, and a socket already owned by a live listener (another serve) returns the null binding (that serve is the IPC provider). Reflex-channel event logging fires at DELIVERY (onDelivered/onTurnContextDelivered), never inside the resolver. Pinned bytest/resolve-ipc-binding.test.ts. -
src/mcp/dispatch.ts— shared tool-call dispatch consumed by both stdio (server.ts) and HTTP transports. ExportsdispatchToolCall(engine, name, params, opts),buildOperationContext(engine, params, opts), and re-exportsnormalizeOptionalParams/validateParamsfromsrc/mcp/validate-params.ts(the validation module — call order normalize → validate → findUnknownParams is load-bearing). Single source of truth for(ctx, params)handler arg order and theOperationContextshape. Defaultsremote: true(untrusted); local CLI callers passremote: false. Deny layers, in order:opts.allowedOpssurface enforcement (hidden op →unknownToolEnvelope, byte-identical to a nonexistent op; did-you-mean candidates drawn ONLY from the caller's visible surface minus localOnly minus gated names), unknown op, the localOnly transport backstop (op.localOnly && transport !== 'stdio'→ same envelope; stdio IS the local surface), param validation, strict/warn unknown-arg handling (reject mode returnsinvalid_paramswith suggestions; warn mode attaches_meta.warnings+ a model-visible notice block), thenenforceBoundClientOpAllowList(the bound-client fence) inside the handler try._metaassembly per docs/protocol/MCP_META_CHANNELS.md: handler-emitted keys viactx.emitResponseMeta(retrieval, warnings) attach first and independently of themetaHook(brain_hot_memory, built bysrc/core/facts/meta-hook.ts— its 30s cache is keyed by ENGINE IDENTITY + source + visibility tier + session + allow-list hash, so a process hosting multiple engines can never serve one brain's cached facts into another's envelope; each entry's deadline is additionally CLAMPED to the earliest retainedvalid_until, so a fact expiring inside the window never outlives its read-time TTL by cache staleness), each producer isolated so one failure never drops another's key; empty retrieval results get a SECOND model-visible content block viabuildEmptyRetrievalBlock. ExportsisListLevelDenialEnvelope(parsed)— the honest-catalog metric classifier: true for op-level denials the tools/list filter should have prevented (detail: 'config_key=...'publish-gate backstop,detail: 'fence=op'fence op-level deny), false for argument-level denials; serve-http logs matches asstatus='denied_after_list'. ExportsrequestLogStatusForResult(result)— the ONE request-log status decision (success/success_with_warnings/denied_after_list/error); BOTH HTTP transports (serve-http.ts and the legacy-bearer http-transport.ts) route every tools/call row through it so the honest-catalog metric sees all HTTP traffic (pinned by test/denied-after-list.test.ts + test/http-transport.test.ts). Also exportssummarizeMcpParams(opName, params)— privacy-preserving redactor formcp_request_logand the admin SSE feed, returns{redacted, kind, declared_keys, unknown_key_count, approx_bytes}. Intersects submitted top-level keys against the operation's declaredparamsallow-list (declared keys preserved sorted; unknown keys counted but never named, closing the attacker-controlled-key-name leak). Byte counts bucketed up to nearest 1KB so an attacker can't binary-search secret-content sizes by probing. Raw payload visibility is opt-in viagbrain serve --http --log-full-params(loud stderr warning). Logging paths route through this helper, notJSON.stringify(params). -
src/mcp/validate-params.ts— param normalization + validation for dispatch (direct unit surface).validateParams(required/type/enum — enum membership is a TYPE error in BOTH strict modes; the caller's raw value is never echoed into the message),normalizeOptionalParams(null/''optional-param idioms become truly absent, copy-on-write),findUnknownParams(unknown top-level keys on the NORMALIZED object;UNKNOWN_PARAM_ALLOWLIST=_meta+dry_run; per-op did-you-mean from the op's OWN declared params only),buildUnknownParamWarnBlock(the model-visible warn notice),resolveStrictParamsMode(engine, config)(dual-planemcp.strict_params, DB > file >'warn'; a failed DB read applies the LAST successfully read DB mode, held per process — fail-closed, so a transient config outage on a reject-mode server cannot re-open the warn grace period; test seamresetStrictParamsModeCache()). Privacy: raw unknown key names reach the CALLER only, nevermcp_request_log. Pinned by test/validate-params.test.ts. -
src/mcp/publish-gates.ts— publish-gate resolution for the honest tools/list.readPublishGate(engine, config, key): dual-plane (DB > file > false), never throws — a FAILED read resolves false (hide-on-doubt matches the default-off consent posture).disabledOpsForPublishGates(engine, config): the op-name set tools/list subtracts; one getConfig read per distinct gate key per call, deliberately NOT memoized sogbrain config set mcp.publish_skills truetakes effect on the next list without a restart. Call-time gates inside the handlers stay as the fail-closed backstop (their denials carrydetail: 'config_key=<key>'— the machine-readable denial grammar). Pinned by test/publish-gates.test.ts. -
src/mcp/tool-catalog.ts—renderToolCatalogMarkdown(): the docs/TOOL_CATALOG.md renderer. Config-independent + deterministic (no engine/config reads, no timestamps): non-localOnly ops grouped one section perOperation.area, per-op first-sentence description (frombuildToolDefs's non-strict shape), scope, STARTER_OPS membership, publish-gate key. Generated byscripts/generate-tool-catalog.ts; freshness-guarded byscripts/check-tool-catalog-fresh.shinbun run verify(the METRIC_GLOSSARY pattern). Pinned by test/tool-catalog.test.ts. -
src/core/surface-audit.ts—writeSurfaceChangeAudit(engine, audit): the surface-mutation audit trail. Every surface mutation (rescope CLI,POST /admin/api/rescope-client,request_toolspersist) writes one typedmcp_request_logrow —operation='surface_change', params a RAW object{actor, client_id, old, new, via}viaexecuteRawJsonb(neverJSON.stringifyinto::jsonb). Zero new DDL; ridesidx_mcp_log_time_agent+ the retention TODO. Best-effort: a failed audit write warns to stderr, never fails the committed mutation (callers needing fail-closed semantics inspect the returned boolean). CLI-actor rows are written even though stdio ops don't otherwise log (documented exception — audit outranks the transport-logging convention). The usage reader excludes these rows from op-call stats. -
src/core/mcp-usage.ts—readClientOpUsage(engine, {days}): the ONE shared reader overmcp_request_log, consumed bygbrain auth clients --usage, the advisormcp-client-fitcollector, andscripts/derive-starter-ops.ts. Encodes the row-hygiene rules once (normalizeLoggedOperation): JSON-RPC method rows (tools/list,initialize, …) andsurface_changeaudit rows drop; the legacytools/call:<name>prefix strips to the op name, and the hygiene check re-runs on the stripped name (tools/call:tools/listis still not an op call). Onlystatus IN ('success','success_with_warnings')rows count as usage — denied or erroring traffic cannot "use" its way into starter derivation or advisor fit findings. Windows oncreated_at(ridesidx_mcp_log_time_agent); plain SQL throughengine.executeRaw, both engines. Behavioral automation classification:likely_automation= >90% of calls arecontext_pack/deltaboundary verbs (the hook lane is stdio and never logs, so there is no name convention to key on). Sees HTTP clients ONLY — stdio never writes the log. Pinned by test/mcp-usage.test.ts. -
docs/protocol/MCP_META_CHANNELS.md— normative_metaconventions for MCP tool responses: one producer per top-level key, additive-forever within a key, producer isolation, and the registered-keys table (brain_hot_memory,retrieval,warnings). Anything the model must SEE rides a content block (mainstream harnesses don't feed_metato the model);_metaserves structured consumers. Add a key by registering it in the table — one producer, additive-forever. -
src/mcp/rate-limit.ts— Bounded-LRU token-bucket limiter.buildDefaultLimiters()returns the two-bucket pipeline: pre-auth IP (30/60s, fires BEFORE the DB lookup so brute-force load againstaccess_tokensis capped) + post-auth token-id (60/60s). TrackslastTouchedMsseparately fromlastRefillMsso an exhausted key can't be reset by hammering past the TTL. LRU cap bounds memory under attacker-controlled key growth.refund(key)returns one consumed token (capped at the limit) — therequest_toolspersist path refunds when a race-lost 0-row UPDATE (concurrent operator pin) means no write actually happened; dry-run previews never draw a token at all (the limiter meters actual writes only). -
src/commands/serve-http.ts— Express 5 HTTP MCP server with OAuth 2.1, admin dashboard, and SSE live activity feed. The legacyPOST /ingestqueue refuses operation-snapshot clients; they use approved MCP writes so execution stays inside the shared grant contract. Started viagbrain serve --http [--port N] [--token-ttl N] [--enable-dcr] [--public-url URL] [--bind HOST] [--log-full-params]. Combines MCP SDK'smcpAuthRouter(authorize/token/register/revoke), a customclient_credentialshandler running BEFORE the router (SDK's token endpoint throwsUnsupportedGrantTypeErrorfor CC; custom handler falls through forauth_code/refresh_token),requireBearerAuthmiddleware for/mcpwith scope enforcement +localOnlyrejection before op dispatch, andexpress-rate-limitat 50 req / 15 min on/token. Serves the built admin SPA fromadmin/dist/with SPA fallback./admin/eventsSSE broadcasts every MCP request.cookie-parserwired (Express 5 has no built-in). Startup logging prints port, engine, issuer URL (honors--public-url), client count, DCR status, and the admin bootstrap token line — but the generated token's raw value only prints when stderr is an interactive TTY (shouldSuppressBootstrapPrint): a non-TTY (containerized/piped) start hides it so the secret never lands in centralized log storage, env-sourced tokens ($GBRAIN_ADMIN_BOOTSTRAP_TOKEN) are always hidden,--print-admin-tokenforces the raw value on a trusted terminal, and--suppress-bootstrap-tokenhides everything. The/mcprequest handler's OperationContext literal setsremote: trueexplicitly (without itsubmit_job's protected-name guard would see a falsy undefined and aread+write-scoped OAuth token could submitshelljobs).summarizeMcpParamsfromsrc/mcp/dispatch.tsfeeds bothmcp_request_logwrites and the SSE feed by default (raw via--log-full-params). CookieSecureflag set behind HTTPS or a public-URL proxy; magic-link nonce store LRU-bounded; DCR disable routes through theGBrainOAuthProviderdcrDisabledconstructor option (not a router monkey-patch);transport.handleRequestwrapped in try/catch to return a JSON-RPC 500 envelope; OperationError + unexpected exceptions unified throughbuildError/serializeErrorso/mcpalways returns the same envelope./healthis liveness-only viaprobeLiveness(engine, engineName, version, timeoutMs)racingengine.executeRaw('SELECT 1', undefined, { signal })against the exportedHEALTH_TIMEOUT_MS = 3000; when the timeout wins, anAbortControllercancels the Postgres query (PGLite can only discard its eventual result) before the same taggedProbeHealthResult503 envelope is returned (single timer-cleanup site); body shape{status, version, engine}only. Full stats live at admin-only/admin/api/full-stats(gated byrequireAdmin, callsprobeHealth(engine, ...)) — keepsgetStats()'s 6× count(*) off the public route so a saturated pool doesn't trigger orchestrator restart cascades. OAuth/admin/audit SQL outside the abortable liveness probe routes throughsqlQueryForEngine(engine)fromsrc/core/sql-query.tsso it works against PGLite; the fourmcp_request_log.paramsINSERT sites (success / auth_failed / scope_denied / server-error) go throughexecuteRawJsonb(engine, ...)so the column stores real objects (params->>'op'returnssearch, not the quoted string).--bind HOSTdefaults127.0.0.1(self-hosters pass--bind 0.0.0.0); a stderr WARN fires when--public-urlis set without--bind; the banner prints aBind:line.AuthInfo.sourceId+AuthInfo.allowedSources+AuthInfo.takesHoldersAllowListare the typed source of truth, populated byoauth-provider.ts:verifyAccessToken(source scope from theoauth_clientsrow; takes-holders fromaccess_tokens.permissions.takes_holdersfor legacy bearer tokens). The/mcpdispatch site readsauthInfo.takesHoldersAllowList ?? ['world']— absent grants (OAuth-client tokens, pre-v29 brains) fail closed to world-only takes visibility, while an explicit[]grant is preserved as deny-all; pinned end-to-end bytest/e2e/serve-http-takes-holders.test.ts. The HTTP MCPtools/listhandler at:837-849usesparamDefToSchema(v)fromsrc/mcp/tool-defs.tsso array params keepitems(strict-mode OAuth clients otherwise reject the whole tool list).POST /ingestenforces the slug-prefix write fence at the ROUTE, not the op layer: the route hands its payload to theingest_captureminion handler, which deliberately bypassesput_page, so noOperationContextexists andenforceClientSlugFencenever runs — a slug-bound client must therefore supplyX-Gbrain-Slugand it must satisfyslugUnderBoundPrefixes, else 403 (without the check a bound client could overwrite any page inside its granted source). The write source is resolved server-side asauthInfo.sourceId ?? 'default'and travels on the job asjob.data.sourceId; the caller-suppliedX-Gbrain-Source-Idheader routes nothing and only names the emitter (webhook-<clientId>), which is what the event'ssource_idand the 202's back-compatsource_idfield carry. The 202 additionally reports the routed destination aswrite_source_id, and the resolved write source joins the queue idempotency key so a rescoped client lands a new capture instead of deduping against its old job. confidential revoke: a pre-router/revokehandler validates the RFC 7009 body, verifies hash-only secrets for bothclient_secret_postandclient_secret_basic, rejects mixed authentication, preserves the SDK path for public clients, and separates opaque client-auth failures from retryable/backend failures. OAuth metadata advertises both confidential methods. Pinned bytest/e2e/serve-http-oauth.test.ts. three admin routes:/admin/api/calibration/profile,/admin/api/calibration/charts/:type(image/svg+xml; type in {brier-trend, domain-bars, pattern-statements, abandoned-threads}),/admin/api/calibration/pattern/:id(drill-down). The manual mint route/admin/api/register-clientaccepts optionalsource+federatedReadbindings mirroring the CLI's--source/--federated-read(validated vianormalizeSourceInput/normalizeFederatedReadInputfromsrc/core/source-id.ts; omitting both preserves the default bindingsource_id='default'/federated_read=[source_id], invalid values return a structured 400invalid_source), so an admin SPA or provisioning proxy can mint a source-confined client over HTTP. The route mirrors the CLI lane end to end: structured 400s forunknown_source/archived_source(one batchedANY(\$1::text[])existence check),invalid_token_ttl(sharedTOKEN_TTL_MIN/MAX_SECONDSbounds, integer-validated BEFORE the tx), andbrain_too_old(column pre-flight OUTSIDE the tx — a pre-scoped-clients brain refuses up front instead of aborting mid-transaction); the duplicate-name pre-check + INSERT run in ONE transaction under the SAME name-scoped advisory lock the CLI takes (registerClientNameLockKey), returning 409duplicate_namewith the existingclient_id; the INSERT composesregisterScopedClient(the CLI's registration core) so the two paths cannot drift; any post-commit failure includes the createdclient_idso the operator can revoke (never a false "nothing was created"). Pinned bytest/register-client-source-normalize.test.ts. The/mcptools/list is the honest catalog: per-request filters (token scope incl. theagentCallablecarve-out, bound-client fence viaopAllowedForBoundClient, publish gates viadisabledOpsForPublishGates) over the surface-filtered op set, schemas viabuildToolDefs(strict emission whenmcp.strict_paramsresolves reject); the tools/listmcp_request_logrow records the listed size asparams.tool_count. Call-time op-level denials the list should have prevented — the inline scope deny, publish-gate backstop (config_key=...), fence op-level deny (fence=op, classified viaisListLevelDenialEnvelope) — logstatus='denied_after_list'instead of'error'(the trend-to-zero honest-catalog metric, see docs/operations/mcp-surface-runbook.md) — every tools/call row's status resolves throughrequestLogStatusForResult(dispatch.ts) on BOTH transports. The admin health-indicators error rate countsstatus NOT IN ('success','success_with_warnings')and excludesoperation='surface_change'audit rows from numerator AND denominator (audit rows record operator/self actions, not traffic). Per-request surface resolution lives inresolveEffectiveSurface: averbsceiling skips the default-surface config read, and a failed read applies the last successfully read default per process (fail-closed — a transient config outage cannot widen a NULL-surface client). OAuth protected-resource metadata (RFC 9728) names/mcpas the resource (resourceServerUrl), so the SDK mounts the PRM at/.well-known/oauth-protected-resource/mcpand the 401resource_metadataURL is derived from the same value; the bare root path is kept as an alias by rewritingreq.urlonto the SDK's handler (same document, same CORS). -
src/core/sql-query.ts— engine-aware tagged-template SQL adapter for OAuth/admin/auth infrastructure.sqlQueryForEngine(engine)returns aSqlQuery((strings, ...values) => Promise<rows[]>) that walks the template, builds$Npositional SQL, asserts every value is aSqlValue(string | number | bigint | boolean | Date | null), and routes throughengine.executeRaw(sql, params)(Postgres via postgres.jsunsafe(sql, params), PGLite viadb.query(sql, params)). Deliberately narrower than postgres.js'ssqltag: no nested fragments,sql.json(),sql.unsafe(),sql.begin(), or array binding — the narrow scalar-only surface is the feature (keeps it from drifting into a partial postgres.js clone). JSONB writes go throughexecuteRawJsonb(engine, sql, scalarParams, jsonbParams)which composes positional$N::jsonbcasts and passes JS objects through; an object reaches the wire with the correct type oid, so executeRawJsonb is safe (verified bytest/sql-query.test.tson PGLite,test/e2e/auth-permissions.test.ts:67on Postgres). Positional binding is NOT universally immune, though: binding aJSON.stringify(x)string to a bare$N::jsonbviaunsafe()double-encodes it into a jsonb string scalar on real Postgres (PGLite hides it). Fixes: pass a raw object (executeRawJsonb /sql.json), or cast through$N::text::jsonb.scripts/check-jsonb-pattern.sh(template grep) doesn't fire onexecuteRawJsonb(...)because it passes objects; the positional$N::jsonb+JSON.stringifyform is caught byscripts/check-jsonb-params.mjs. Consumed bysrc/commands/auth.ts,src/commands/serve-http.ts,src/core/oauth-provider.ts,src/commands/files.ts,src/mcp/http-transport.tsso all five work uniformly against PGLite and Postgres. -
src/commands/serve.ts—gbrain servestdio MCP entrypoint with idempotent shutdown across every parent-disconnect signal. Stdio EOF, SIGTERM, SIGINT, SIGHUP, and parent-process death (every reparent case — PID 1, launchd subreaper, systemd, tmux, or a parent shell withPR_SET_CHILD_SUBREAPER) all funnel into onecleanup(reason)that first awaitsshutdownDelegatedSync()(idempotent shared promise; a running delegated sync is aborted and settles its checkpoint against the live engine, with the cleanup deadline extended byGBRAIN_SERVE_SYNC_SETTLE_MSonly while a job runs) and then releases the engine and the PGLite write-lock dir within 5 seconds (otherwise the lock is held indefinitely after Claude Desktop / Cursor / launchd-managed gateways disconnect, forcing a 5-minute stale-lock wait on next start). Watchdog reparent check isgetParentPid() !== initialParentPid(the=== 1check missed the subreaper case under launchd/systemd). Bun'sprocess.ppidcache is stale across reparenting (oven-sh/bun#30305) sogetParentPid()runsspawnSync('ps', ['-o', 'ppid=', '-p', PID])per tick. Startup probe verifiespsis on PATH; if not (stripped containers, busybox), the watchdog skips installing AND emits a loud[gbrain serve] watchdog disabled: ps unavailable ...stderr line so operators see the degraded mode. The idle maintenance sweep skips ticks while a delegated sync runs and drains deferred embeds afterwards (maybeDrainDeferredEmbeds). Both stdin activity listeners (the default-on idle sweep and the opt-in--stdio-idle-timeout) attach only AFTER the MCP SDK transport's own stdin listener — the sweep on its first interval tick, the idle timeout through the activatorinstallStdioLifecyclereturns, whichrunServeinvokes oncestartMcpServerresolves (and which re-arms the countdown so boot time does not eat into the idle window) — so neither can flip stdin into flowing mode and consume a fast client's initialize frame before the SDK sees it. Pinned bytest/serve-stdio-lifecycle.test.ts(22 cases). -
src/core/grants/— canonical OAuth client grant model shared by CLI, admin HTTP, provisioning, and worker admission.model.tsvalidates active sources, independent direct/delegated fences, explicit reviewed tools, namespaces, concurrency, spend, and TTL;host/currentbrain aliases normalize to the serving brain (null), while other IDs need independently verified host identity.profiles.tssnapshots current eligible operation names for six explicit profiles; new defaults are renewable 1 hour (static 30 days), concurrency 1, and unlimited delegated spend.allowed_operations = NULLpreserves legacy scope-based behavior; a new profile missing its snapshot fails closed.service.tslocks the client, checksgrant_revision, and commits the grant, secret-free audit, and original ceilings for pending/active legacy jobs atomically; stale edits fail, repeated repair is a no-op, and repair preserves populated limits. Issued scopes intersect live scopes on access and refresh; source/path/operation/delegation changes are live, newly added scopes require new issuance, and TTL changes do not rewrite existing expiry.migration.tssplits legacy fences and removes only invalidagentgrants with repair reasons; it loads registry validation lazily only after locking existing delegated clients, so empty engine imports and cold migrations do not register unrelated background sinks. This justified engine-live import is line-marked; constraints still install when no clients exist.schema.tssupplies migration147/bootstrap columns and invariants for both engines. SQL JSON passes throughtext::jsonbfor driver parity. Tests:test/client-grants.test.ts, OAuth compatibility suites, andtest/e2e/client-grants.test.ts(Postgres CAS, job ceilings, and actual HTTP preview/edit). -
src/commands/serve-http-grants.ts+admin/src/components/ClientGrant.tsx— typed admin grant request validation and shared profile/advanced permission editor. Register/rescope/TTL routes inserve-http.tsuse canonical grant validation and audit; legacy response shapes remain available. New UI creation defaults to memory-writer and requires a preview, while edits carry the displayed revision and require a fresh preview after any field change or conflict. Detail/catalog endpoints expose non-secret grants and the shared harness registry. Operation authority snapshots, source sets, delegated tools/namespaces, spend, concurrency, and future TTL are independently reviewable; profile regrant is explicit.admin/src/pages/Agents.tsxdownloads the privateHarnessCredentialshandoff and links current harness guides instead of maintaining native config snippets. Rebuild committedadmin/distandsrc/admin-embedded.tsafter SPA edits. -
src/core/oauth-provider.ts—GBrainOAuthProviderimplementing the MCP SDK'sOAuthServerProvider+OAuthRegisteredClientsStore. Backed by raw SQL (works on both PGLite and Postgres — OAuth is infrastructure, not a BrainEngine concern). Full OAuth 2.1:authorize+exchangeAuthorizationCodewith PKCE,client_credentials,refresh_tokenwith rotation,revokeToken,registerClient(DCR validates redirect_uri ishttps://or loopback per RFC 6749 §3.1.2.1). All tokens + client secrets SHA-256 hashed before storage. Owner-approved authorization codes have a 10-minute TTL. Grant transactions lock the active client row before consuming codes or refresh tokens and creating replacements; revocation uses the same lock. Provider-side S256 PKCE validation is mandatory (skipLocalPkceValidation=trueprevents the SDK from discarding the verifier first). Existing active-client grants remain usable.pgArray()escapes commas/quotes/braces so a comma-bearing redirect_uri can't smuggle a second array element. Legacyaccess_tokensfallback inverifyAccessTokenhonors the original-schemascopes TEXT[]column vianormalizeTokenScopes(NULL = grandfatheredread+write+admin, so every pre-scopes token is byte-identical; an array is filtered to known scopes and honored as-is,[]/all-unknown preserved as deny — a dedicated column is structurally immune to the permissions-object-replacement wipe class), and threads BOTH stored grants off the token'spermissionsJSONB:source_idviaparseLegacyTokenScopeandtakes_holdersviaparseTakesHoldersAllowList(both insrc/core/legacy-token-scope.ts, shared with the legacy HTTP transport so the two transports cannot drift;[]takes-holders preserved as explicit deny-all, missing/non-array → undefined → the/mcpdispatch site's fail-closed['world']; OAuth-client tokens carry no takes-holders grant pending per-client storage — TODOS.md).sweepExpiredTokens()runs on startup in try/catch and returns the count viaRETURNING 1+ array length. RFC hardening:client_idfolded atomically into theDELETE WHEREfor both auth-code exchange and refresh rotation (wrong-client paths don't burn the row); refresh-scope-subset enforced against the original grant on the row (RFC 6749 §6, so revoking a scope shrinks existing refresh tokens);client_idbound onrevokeToken(RFC 7009 §2.1);/tokenredirect_urivalidated against the/authorizevalue (RFC 6749 §4.1.3, empty-string treated as missing not wildcard);verifyAccessToken/getClientcatch onlyisUndefinedColumnErrorfromsrc/core/utils.ts(only SQLSTATE 42703 falls through to legacy; lock timeouts/network blips throw);dcrDisabledconstructor option letsserve-http.tsdisable/registerwithout monkey-patching the router. Module-privatecoerceTimestamp()normalizes postgres-driver-as-string BIGINT columns to JS numbers at 5 read sites (getClientfor RFC 7591 §3.2.1 numeric timestamps,exchangeRefreshToken+verifyAccessTokenfor the SDK'stypeof === 'number'check); throws on NaN/Infinity (fail loud at boundary), returns undefined for SQL NULL (callers treat NULL as expired). Not promoted toutils.ts— generic BIGINT precision-loss risk.registerClienthonorstoken_endpoint_auth_method: "none"(RFC 7591 §3.2.1): public PKCE clients storeclient_secret_hash = NULLand the response omitsclient_secret; confidential clients (client_secret_post/client_secret_basic) keep their one-time-reveal shape;getClientnormalizes NULLclient_secret_hashto JSundefinedso the SDK's clientAuth path accepts public clients.verifyAccessTokenJOINsoauth_clients.source_id(write scope, scalar) +oauth_clients.federated_read(read scope, TEXT[]) +oauth_clients.bound_slug_prefixes(write fence, TEXT[] — consumed byenforceClientSlugFenceinoperations.ts) onto the returnedAuthInfo; legacy brains degrade viaisUndefinedColumnErrorfallback, dropping the newest projection first.rescopeClient(clientId, {sourceId?, federatedRead?, boundSlugPrefixes?})is the trusted-operator rescope (CLIgbrain auth rescope-client, adminPOST /admin/api/rescope-client);boundSlugPrefixesis tri-state — undefined leaves the binding untouched,nullclears it, a non-empty array replaces it (explicit empty array rejected as ambiguous deny-all) — so roster churn updates the write fence in place without rotating secrets. withsrc/commands/serve-http.ts: custom/tokenmiddleware that runs BEFORE the MCP SDK'sclientAuth. The SDK does plaintext compare against the request'sclient_secret; gbrain stores SHA-256 hashes only, so every confidential-client/tokenrequest would fail. The middleware detects confidential auth viaAuthorization: Basicheader ORclient_secret_postform body (both shapes per RFC 6749 §2.3.1), verifies viaverifyClient(client_id, presented_secret)(SHA-256 hash compare), and falls through to the SDK for public PKCE clients (which the SDK's clientAuth still accepts via NULL-client_secret_hashnormalization). Pinned bytest/oauth-confidential-client.test.ts(bothclient_secret_basicandclient_secret_post). -
admin/— React 19 + Vite + TypeScript admin SPA embedded in the binary viaadmin/dist/served byserve-http.ts. 7 screens: Login (bootstrap token → session cookie), Dashboard (metrics + SSE feed + token health), Agents (sortable table + sparklines + Register), Register (modal with scope checkboxes + grant type selector), Credentials reveal (Copy + Download JSON + one-time-only warning), Request Log (filterable paginated), Agent Detail drawer (Details / Activity / Config Export tabs + Revoke). Design tokens:#0a0a0fbg, Inter for UI, JetBrains Mono for data, 4-32px spacing scale, rounded pill badges. HTTP-only SameSite=Strict cookie auth. 65KB gzip. Build:cd admin && bun install && bun run build; output atadmin/dist/is committed for self-contained binaries. -
src/commands/auth.ts— token management.gbrain auth create/list/revoke/testfor legacy bearer tokens (create --scopes read,writenarrows a token via thescopes TEXT[]column with mint-time validation — a typo'd scope refuses loudly, never silently denies or widens;listshows id + scopes columns with grandfathered rows rendered honestly;revoke --id <uuid>revokes exactly one row since names are not unique, and bulk revoke-by-name says when it hit several;permissions set-takes-holdersMERGES into the permissions JSONB viaCOALESCE(permissions,'{}'::jsonb) || \$2::jsonb— a whole-object replace would silently wipe thesource_idfederation grant), plusgbrain auth register-clientandgbrain auth revoke-client <client_id>for OAuth 2.1 client lifecycle.revoke-clientruns an atomicDELETE...RETURNINGonoauth_clients; FKON DELETE CASCADEonoauth_tokens.client_idandoauth_codes.client_idpurges every active token + auth code in one transaction;process.exit(1)on no-such-client (idempotent). Legacy tokens stored as SHA-256 hashes inaccess_tokens; OAuth clients inoauth_clients; legacy tokens with no scopes grant grandfather toread+write+adminon the OAuth HTTP server (no migration); scoped tokens are honored at exactly their grant. Every SQL site routes throughsqlQueryForEngine(engine)fromsrc/core/sql-query.ts(andexecuteRawJsonbfor the takes-holderspermissionsJSONB column) sogbrain authworks against PGLite; the takes-holders write goes throughexecuteRawJsonb(engine, sql, [name, hash], [{takes_holders:[...]}])which round-trips withjsonb_typeof = 'object'.register-clientaccepts--source <id>(write authority, scalar),--federated-read <S1,S2,...>(read scope, array), and--token-ttl <seconds>(per-client access-token TTL persisted tooauth_clients.token_ttl, boundsTOKEN_TTL_MIN_SECONDS=60 toTOKEN_TTL_MAX_SECONDS=7,776,000/90d) and prints the resolvedWrite source+Federated reads; clients without asource_idbackfill to'default'via migration v60. The registration core is the exportedregisterScopedClient(sql, engine, name, parsed, opts)— exit-free, print-free, injected-handle (engine-bound callers likeagent registerpass the dispatcher's engine; a secondwithConfiguredSqlengine self-deadlocks PGLite's single-writer lock), returns aRegisteredClientdata object and throws on failure; the thin CLI wrapper owns exit/print. Its printerformatRegisterClientOutputis a BYTE-PINNED contract (connect.ts'sdefaultRegisterOAuthClientregex-scrapesClient ID:/Client Secret:from it in production) — pinned bytest/auth-register-client-output-pin.test.ts.preflightOauthClientColumns(sql)probesinformation_schema.columnsfor the optional columns (token_ttl,surface,federated_read,source_id,deleted_at) so statement shapes are decided BEFORE any transaction — Postgres/PGLite abort the whole tx on any statement error (25P02) and SqlQuery has no savepoint seam, so "catch 42703 and continue" inside a tx is impossible; skipped optional writes surface asRegisteredClient.skippedwith an apply-migrations hint.gbrain auth clients [--usage] [--days N] [--json]lists clients with scopes, per-client surface, write source (source_id), and federated reads (federated_read) — one projection-widened SELECT with a degrade ladder for pre-migration brains (drops the newest columns first, never errors);--usagejoins per-client op-call counts viasrc/core/mcp-usage.ts. The baregbrain auth create <name>form (no--takes-holders) mints a token via the exported pureparseAuthCreateArgs(rest). Pinned bytest/auth-create-args.test.ts+test/auth-register-client-args.test.ts. -
src/core/mcp-client.ts— the thin-client transport (trust boundary).callRemoteTool(config, toolName, args, opts)withCallRemoteToolOptions {timeoutMs, signal};buildAbortControllercomposes an external signal with the timeout. All transport errors normalize toRemoteMcpErrorvia thetoRemoteMcpErrorfunnel: stableRemoteMcpErrorReasonunion,RemoteMcpErrorDetail.kind('timeout' | 'aborted' | 'unreachable') sub-tag,RemoteMcpErrorDetail.codecarrying server-supplied error codes (e.g.missing_scope).extractToolErrorCodeparses the operation{error: string, message}shape as well as legacy envelopes. Only the locked SDK’s typedStreamableHTTPErrorwith numeric code 401 triggers one refresh and reconstructed client; application error text never causes authentication replay. The outer abort signal reaches discovery, minting, initialization, fetch and tool calls, with per-request limits retained and clients closed after failed initialization or abort.unpackToolResult<T>(res)parses tool-call JSON content._clearMcpClientTokenCache()test escape. The CLI routing seam that consumes this lives insrc/cli.ts(runThinClientRouted); seedocs/architecture/thin-client.md. -
src/commands/connect.ts+src/core/connect-probe.ts—gbrain connect <mcp-url> [--token <bearer>]one-command coding-agent onboarding from a bearer token. Turns an MCP URL + token into a paste-readyclaude mcp add ... -H "Authorization: Bearer ..."block (default) or, with--install, runs it directly and smoke-tests the token. Direct HTTP MCP — Claude Code talks straight to a remotegbrain serve --http, no local install needed. Token resolution:--token>$GBRAIN_REMOTE_TOKEN> placeholder (print) / error (install). The generated block tells the agent to callget_brain_identity+list_skills(theLEARN_INSTRUCTIONexport, which namescapture— a starter-surface MCP op — alongsideput_pagefor full-control writes) with a core-tools fallback for hosts without skill publishing. URL normalization appends/mcpto a bare host but REJECTS a scheme-less host; the pure registration helpers (normalizeMcpUrl,isLinkLocalOrMetadata,buildClaudeMcpAddArgvwith its optional scope param — claude's default islocal, so the harness lane passesuserexplicitly —buildCodexMcpAddArgv,validateToken,redactToken,shellQuote/cmdString,issuerFromMcpUrl, theOAUTH_SECRET_NOTEsecret-hygiene constant, andopenclawThinClientBlock— the honest openclaw wiring print: a scopedgbrain init --mcp-onlythin-client block, deliberately NOT a stdio mcpServers config since that grants full local DB access) live insrc/core/mcp-registration.ts(core must not import from commands; connect.ts re-exports them —OAUTH_SECRET_NOTEincluded — so its surface and tests are unchanged) and are unit-tested. Flags:--token,--name <id>(defaultgbrain, validated againstNAME_RE),--agent claude-code|codex|opencode|perplexity|generic,--install,--yes(required for--installin non-TTY),--force,--json(token redacted unless--show-token),--timeout-ms.connectis inCLI_ONLY+CLI_ONLY_SELF_HELP; dispatched incli.ts:handleCliOnlywith no local DB connect.AGENT_SPECSdrives per-agent rendering +--install:claude-code→buildClaudeMcpAddArgv(literal-H "Authorization: Bearer <tok>");codex→buildCodexMcpAddArgv=codex mcp add <name> --url <url> --bearer-token-env-var GBRAIN_REMOTE_TOKEN(on the CONNECT lane Codex reads the token from the env var at runtime, never written to config — the harness lane insrc/core/bootstrap/harness.tsis the deliberate exception, writing a managed block with the inlinehttp_headers = { Authorization = "Bearer <t>" }credential (codex-cli >=0.149 rejects inlinebearer_tokenfor streamable_http at config load) because framework-spawned codex inherits no shell profile;--installruns it and prints anexport GBRAIN_REMOTE_TOKENhint when missing);opencode→buildOpencodeMcpAddArgv=opencode mcp add <name> --url <url> --header "Authorization=Bearer {env:GBRAIN_REMOTE_TOKEN}"(the interpolation is stored LITERALLY and resolved by opencode at read time — token never in argv/config/--json;--installwrites the entry directly throughConnectDeps.writeOpencodeRemoteEntry→opencode-json.tsin env token mode, no binary required;--forcemaps to the writer'sallowReplaceOtherSourceso an OURS entry at an old url — a rotated serve — is replaceable, mirroring the exec lanes'--forcesemantics, while foreign same-name entries still refuse with url-appropriate copy: pick--name);perplexity+genericareinstallable:falseand reject--install.--oauth(supportsOAuth:true= perplexity/generic only) emits an OAuth 2.1 client-credentials connector block (Issuer URL viaissuerFromMcpUrl= mcp-url minus/mcp, Client ID, Client Secret) — least-privilege scopes + short-lived rotating tokens vs a long-lived full-access secret. Creds from--client-id/--client-secret(BYO) or--register(deps.registerOAuthClientshellsgbrain auth register-client <name> --grant-types client_credentials --scopes <DEFAULT_SCOPES="read write"> --token-endpoint-auth-method client_secret_postand parsesClient ID:/Client Secret:);--oauthrejected for claude-code/codex and incompatible with--install.buildJsonis a generic shape (agent,command/command_argvnull for perplexity/generic,header,env_var, oauth fields with redaction); the codexcommandcarries only the env-var name, never the token.cmdString(binary, argv)POSIX-single-quotes args.ConnectDeps={isTTY, promptYesNo, hasBinary(bin), runBinary(bin, argv), probe, env(name), registerOAuthClient, writeOpencodeRemoteEntry}— binary-generic soclaudeandcodexshare the exec path while opencode rides the writer member;envinjectable for tests. Security: rendered command single-quotes the token so shell metacharacters can't run code when pasted; token validated before it lands in an HTTP header; link-local / cloud-metadata addresses (incl. IPv4-mapped IPv6::ffff:169.254.x.xand AWS IMDSv2-over-IPv6fd00:ec2::254) refused as a token-exfil guard while localhost/RFC1918/LAN stay allowed; token redacted from all error output.src/core/connect-probe.tsis the raw-bearer MCP smoke probe backing--install: connects the official MCP SDKClientoverStreamableHTTPClientTransportwith a STATICAuthorizationheader (no OAuth/discovery — distinct frommcp-client.ts:callRemoteToolwhich is OAuth-only andremote-mcp-probe.ts:smokeTestMcpwhich only sendsinitialize), runs the fullinitializehandshake viaclient.connect(), then callsget_brain_identity(read-scope, non-localOnly) to prove a tool call round-trips. Never throws — every failure maps to{ ok: false, reason: 'auth' | 'unreachable' | 'timeout' | 'tool_error' | 'unknown', message }so a wrong/expired token fails at setup, not on the agent's first request.DEFAULT_PROBE_TIMEOUT_MS = 15_000shared withconnect.ts.serve-http.tsadds exported pureskillPublishStatus(publishSkills)for the startup bannerSkills: published / not publishedline + a one-linegbrain config set mcp.publish_skills truestderr nudge when publishing is OFF. Docs:docs/mcp/CODEX.md,docs/mcp/PERPLEXITY.md,docs/mcp/CLAUDE_CODE.md,docs/tutorials/connect-coding-agent.md. Pinned bytest/connect.test.ts(pure-helper + render, all five agents incl. the opencode writer-install lane) +test/e2e/connect-bearer.test.ts(raw-bearer probe + full OAuth chain register→connect→discovery→/tokenmint→get_brain_identity, client registered inbeforeAllbefore serve takes the PGLite single-writer lock; drives realclaude+codexbinaries throughconnect --installwith sandboxedHOME/CODEX_HOME, asserts registration + token never in Codex config, skips when a binary is absent) +test/e2e/serve-stdio-roundtrip.test.ts(spawns realgbrain servestdio against a freshinit --pglitebrain, drives the SDK client throughinitialize→tools/list→tools/call, asserts the advertised core-tool set includingcapture, a starter-surface MCP op) +test/serve-skills-publish-nudge.test.ts. -
src/commands/upgrade.ts— self-update CLI.runPostUpgrade()enumerates migrations from the TS registry (src/commands/migrations/index.ts) and tail-callsrunApplyMigrations(['--yes', '--non-interactive'])so the mechanical side of every outstanding migration runs unconditionally. -
src/commands/migrations/— TS migration registry (compiled into the binary; no runtime walk ofskills/migrations/*.md).index.tslists migrations in semver order.v0_11_0.ts= Minions adoption orchestrator (8 phases).v0_12_0.ts= Knowledge Graph auto-wire orchestrator (5 phases: schema → config check → backfill links → backfill timeline → verify);phaseASchemahas a 600s timeout for duplicate-heavy brains.v0_12_2.ts= JSONB double-encode repair orchestrator (4 phases: schema → repair-jsonb → verify → record).v0_14_0.ts= shell-jobs + autopilot cooperative (pending-host-work ping for skills/migrations/v0.14.0.md). All orchestrators are idempotent and resumable frompartialstatus. The RUNNER owns all ledger writes — orchestrators returnOrchestratorResultandapply-migrations.tspersists a canonical{version, status, phases}shape (orchestrators never callappendCompletedMigration).statusForVersionpreferscompleteoverpartial(never regresses); 3 consecutive partials → wedged →--force-retry <version>writes a'retry'reset marker. Schema-only migrations v14 (pages_updated_at_index) + v15 (minion_jobs_max_stalled_default_5with UPDATE backfill) live in theMIGRATIONSarray insrc/core/migrate.ts.in-process.tsexportsrunMigrateOnlyCore({timeoutMs?})— single source of truth for "bring schema to head" (configureGateway→createEngine→connect→initSchema→disconnect, idempotent, 600sMIGRATE_ONLY_TIMEOUT_MSguard, throwsMigrateOnlyErroron no-config / timeout); the orchestrators' 9 schema phases ANDinit.ts:initMigrateOnlyboth delegate to it so schema bring-up can't drift (in-process, so no spawn can die withgetaddrinfo ENOTFOUNDon Windows + bun + Supabase pooler).runGbrainSubprocessis the diagnostic wrapper for the remaining non-schema spawns (extract/repair/stats): captures child stderr (64MB buffer) into the thrown error.v0_13_1.ts:phaseCGrandfatheris a CHUNKED bulk SQL pass keyed onpages.id(globally unique PK, NOT slug — slug uniqueness is(source_id, slug)), filtersdeleted_at IS NULL(no tombstones), chunked inCHUNK_SIZEbatches (DELETE_BATCH_SIZE convention) for bounded lock-hold; the rollback log carries{id, slug, source_id, pre_frontmatter}so rollback is unambiguous across sources; idempotent + resumable (each UPDATE flips its rows out ofGRANDFATHER_WHERE). Pinned bytest/migration-in-process.serial.test.tsandtest/migrations-v0_13_1-grandfather.test.ts. -
src/commands/migrations/v0_46_3.ts— ZeroEntropy sunset notice migration, detect-and-notify ONLY. Detects the HOST brain's exposure viasrc/core/ze-exposure.ts(read-only); when exposed — or when exposure is UNKNOWN, fail-safe — prints the ACTION REQUIRED banner and appends one idempotent entry to~/.gbrain/migrations/pending-host-work.jsonlpointing the host agent atskills/migrations/v0.46.3.0.md. Performs NO config writes, NO pinning, and never invokesmigrate embeddings(the migration costs money and needs a target key — that decision belongs to the user/agent via the playbook). UNKNOWN returnscompletewith anexposure_unknowndetail rather thanpartial(three consecutive partials would wedge the whole migration chain behind--force-retry); the stage-2 upgrade banner andgbrain doctorcarry the ongoing nag instead. Host-scoped (apply-migrations runs once per host with a global completed.jsonl); mounted/team brains are covered by the per-brain stage-2 banner (ze_sunset_notice_v2_shownin each brain's own DB config) + doctor gates. -
src/commands/repair-jsonb.ts—gbrain repair-jsonb [--dry-run] [--json]: rewritesjsonb_typeof='string'rows in place across 8 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter, subagent_messages.content_blocks, subagent_tool_executions.input, subagent_tool_executions.output). The subagent targets arejsonPayloadOnly: those columns can legitimately hold jsonb string scalars (persistToolExec binds a tool's pre-serialized string payload as-is), so their damage predicate additionally requires container-shaped content (^\s*[\[{]) thatpg_input_is_valid(..., 'jsonb')(PG16+ floor, same as the IS JSON predicateupdateSourceConfigrelies on) actually parses — a plain-text value that merely starts with[or{is never flagged or corrupted. Targets whose table doesn't exist on the brain are skipped viato_regclass(brains without the subagent tables), and one target's failure is recorded on stderr while the run continues — earlier repairs are already committed and the v0_12_2 migration orchestrator JSON-parses stdout.runDoctor'sjsonb_integritycheck counts damage with the same predicate over the same target list. Repairs double-encoded rows on Postgres; PGLite no-ops. Idempotent. Pinned bytest/repair-jsonb.test.ts+test/doctor.test.ts. -
src/commands/orphans.ts—gbrain orphansand MCPfind_orphansshare islanded (no live inbound or outbound links) and legacy inbound-only modes. Read policy scopes candidates and excludes private candidates, endpoints and independent origin pages before counting. Cross-source public links still establish reachability by the documented orphan definition. Reporting exclusions (shouldExcludeFromOrphanReporting) stay centralized; the scoped live-page denominator excludes the same private/reporting rows.--sourceremains explicit; no new remote privacy parameter. -
src/commands/salience.ts—gbrain salience [--days N] [--limit N] [--kind PREFIX] [--json]: pages ranked by emotional + activity salience over a recency window. Mirrors orphans.ts shape (pure data fn + JSON formatter + human formatter). Callsengine.getRecentSalience(opts). Score formula:(emotional_weight × 5) + ln(1 + active_take_count) + 1/(1 + days_since_update). -
src/commands/anomalies.ts—gbrain anomalies [--since YYYY-MM-DD] [--lookback-days N] [--sigma N] [--json]: cohort-level activity outliers. Callsengine.findAnomalies(opts). Two cohort kinds: tag, type. -
src/commands/whoknows.ts—gbrain whoknows <topic>and MCPfind_experts: expertise and relationship routing.findExperts()threads source, page-privacy and holder policy through hybrid retrieval and enrichment, then requires an authorized effective-date read after optional salience work for every caller, including trusted local callers. Candidates absent from that exact(source_id, slug)map are omitted; required admission failures propagate. Optional salience failures retain the existing neutral factor.rankCandidates()usesscore = log(1 + raw_match) × max(0.1, exp(-days/180)) × (0.5 + 0.5 × salience), with salience normalized before ranking and alphabetical slug tie-breaking. Hybrid salience/recency boosts are disabled here to avoid applying them twice. Expert types are filtered in SQL and resolved from the pack byops/insights.ts; a failed pack load supplies an empty type list. CLI supports--explain,--limit,--jsonand thin-client routing. Math is pinned bytest/whoknows.test.ts; exact final admission, mutation races and SQL-failure behavior run on both engines intest/e2e/read-enrichment-privacy.test.ts. -
src/commands/eval-whoknows.ts—gbrain eval whoknows <fixture.jsonl> [--json] [--skip-replay]: two-layer eval gate. Layer 1 quality (hand-labeled fixture, top-3 hit rate ≥ 0.8). Layer 2 regression (eval_candidatesreplay set-Jaccard@3 ≥ 0.4). Sparseness fallback: < 20 replay-eligible rows → Layer 2 auto-skips with stderr warning. Stable JSON envelope withschema_version: 1; exit 0/1/2 for pass/fail/usage.WhoknowsFncallable abstraction makes the gates impl-agnostic;runEvalWhoknows(engine: BrainEngine | null, args)picks the impl at entry — thin-client mode (isThinClient(cfg)) routes per-query throughcallRemoteTool(cfg, 'find_experts', {topic, limit}), local mode callsfindExperts(engine, ...)directly. cli.ts adds a thin-client bypass beforeconnectEngine(dispatch shape undersrc/commands/eval.ts); the regression gate auto-skips in thin-client mode (no DB access toeval_candidates). Public exportsjaccardAtK,topKHit,readFixture,WhoknowsFn, threshold constants pinned bytest/eval-whoknows.test.ts(25 cases incl. null-engine signature contract). -
test/fixtures/whoknows-eval.jsonl— 10-row synthetic placeholder demonstrating the eval-fixture schema ({query, expected_top_3_slugs, notes?}JSONL). End users replace with their own real queries; placeholder uses obviously-example slugs (wiki/people/example-alice). Drivestest/e2e/whoknows.test.ts(seeds a matching synthetic brain, asserts the >=80% gate) and thewhoknows_healthdoctor check. -
src/core/skillopt/+src/commands/skillopt.ts+skills/skill-optimizer/— self-evolving skill optimization grounded in the SkillOpt paper (arXiv 2605.23904).gbrain skillopt <skill>treatsSKILL.mdas trainable parameters of a frozen agent: validation-gated (median-of-3 + epsilon=0.05), budget-capped (preflight estimator), per-skill DB-locked (tryAcquireDbLock('skillopt:<name>', 60min)), atomic-versioned (history-intent-first 5-step commit), body-only mutations (frontmatter forbidden). Rollouts usegateway.toolLoopdirectly with no-op persistence callbacks (zerosubagent_messagespollution) + a read-only tool allowlist derived fromBRAIN_TOOL_ALLOWLISTminusput_page/submit_job/file_upload. Two reflect calls per step; rejected-edit buffer LRU-bounded to 100; bundled-skill gate; bootstrap workflow (sentinel +--bootstrap-reviewed); D_sel floor (>=5 with--splitoverride); audit JSONL viaaudit-writer.ts. Added toALL_PHASESafterpatterns(default OFF; opt-in viagbrain config set cycle.skillopt.enabled true); cycle phase wrapper atsrc/core/skillopt/cycle-phase.tswalks stale skills with per-skill ($0.50) + brain-wide ($2.00) caps. Added toPROTECTED_JOB_NAMES. Surface: dream-cycle phase wrapper;--allbatch mode (src/core/skillopt/batch.ts:runBatchAll);--target-modelsfleet (runFleetparallel per-model receipts underskillopt/fleet/<slug>/); MCP oprun_skillopt(admin scope + per-skillskillopt.allowed_skillsallowlist, NOT localOnly, validatesskill_namekebab-only + confines caller-supplied benchmark/held-out paths to skillsDir for remote callers); Minionskillopthandler +--backgroundwithallowProtectedSubmit: true; write-flavored optimization viasrc/core/skillopt/write-capture.ts:buildWriteCaptureRegistry(virtualput_page/submit_job/file_uploadcaptured in-memory;--write-captureflag); held-out real-user test set viasrc/core/skillopt/held-out.ts(capture infra at~/.gbrain/skillopt-captures/<skill>/<run>.jsonl,--held-out <path>flag,runHeldOutGatecandidate >= baseline). Hermetic via DI seams (opts.chatFnfor optimizer + judge;opts.toolLoopFnfor rollouts; nomock.module).--bootstrap-from-skill→runBootstrapFromSkillinsrc/core/skillopt/bootstrap-benchmark.ts: readsSKILL.mddirectly (norouting-eval.jsonl), makes ONE LLM call emitting a full starter benchmark (tasks + rule judges) as JSONL, parsed line-by-line with skip-bad-line salvage and a min-2-valid-checks-per-task drop; provider/transport errors PROPAGATE (not collapsed tobootstrap_empty).--bootstrap-tasks N(default 15, capped 50);maxTokensscalesmin(8000, max(4000, N*220)). The stderr REVIEW line prints the literalgbrain skillopt <name> --bootstrap-reviewed --split 1:1:1— load-bearing because the default4:1:5split makes a 15-task starter'sD_sel = floor(15/10) = 1, below the>=5floor, so a 15-task benchmark needs--split 1:1:1. Both bootstrap generators shareassertBenchmarkAbsent+readSkillBodyOrThrow;--bootstrap-from-skillis mutually exclusive with--bootstrap-from-routing/--benchmark/--all/--target-models/--resume. Generated rule judges are explicitly WEAK DRAFTS to be strengthened during the review gate. The held-out gate is wired:--held-out <path>is parsed and threaded through every caller (CLI main +--backgroundheld_out_path+ batch/fleetheldOutPath+ therun_skilloptheld_out_pathparam), running at CHECKPOINT ACCEPTANCE so no-mutate/fleet paths can't promote a held-out-failing candidate.assertBundledMutationHeldOutin bundled-skill-gate.ts: bundled +--allow-mutate-bundledrequires a NON-EMPTY held-out (MIN_HELD_OUT_SIZE = D_SEL_MIN_SIZE= 5, derived so they can't desync) or hard-refuses (exit 2), for ALL callers (they funnel throughrunSkillOpt); held-out must be task_id-DISJOINT from the benchmark (overlap rejected — can't catch overfitting).receipt.baseline_sel_scorepopulated + a real final-test eval (test_score+baseline_test_score) scoring best + baseline onsplit.test; sharedscoreSkillOnTasksprimitive (validate-gate.ts) backs baseline/final-test/held-out scoring.--no-mutatewrites proposed.md viawriteProposedin version-store.ts.maxRuntimeMinENFORCED (wall-clock deadline between steps →skillopt_runtime_exceeded→ outcome aborted). Three eval-internal ablation opts onSkillOptOpts(NOT on CLI):reflectMode('both'/'failure-only'),disableValidationGate(greedy-accept),optimizerMode('reflect'/'one-shot-rewrite'), recorded inRunReceipt+ auditrun_startfor replayability;ROLLOUT_SUCCESS_THRESHOLD = 0.5named constant for the partition; one-shot fence-strip is anchored (^```...```$) so an embedded code sample isn't truncated. Claude Haiku 4.5's dateless canonical idclaude-haiku-4-5is insrc/core/anthropic-pricing.ts(aBudgetTracker-capped run on Haiku would otherwise throwno_pricingon the FIRSTchat()of every rollout);runValidationGate(validate-gate.ts) scans settled results forisMustAbortError(error)(fromworker-pool.ts;BUDGET_EXHAUSTEDis inMUST_ABORT_ERROR_TAGS) and re-throws so the caller aborts loudly instead of recording a hollowselScore:0— ordinary non-abort rollout errors still fail-open toscore:0(judge-hiccup posture preserved). Pinned by 152 tests across 18 files (foundation + adversarial + v2 surface + E2E PGLite serial),test/skillopt/bootstrap-from-skill.test.ts(20 cases),test/skillopt/rollout.test.ts,test/skillopt/validate-gate-abort.test.ts(3 cases), held-out ENFORCE + one-shot-rewrite unit cases, and e2e (held-out block/allow, bundled no-mutate, runtime deadline, receipt honesty, held-out disjointness, no-DB-pollution). Drives the Track B SkillOpt benchmark suite in the siblinggbrain-evalsrepo. -
src/core/brainstorm/{domain-bank,orchestrator,judges}.ts+src/commands/{brainstorm,lsd,eval-brainstorm}.ts+src/core/last-retrieved.ts— bisociation-grounded idea generation pair:gbrain brainstorm <question>(defensible, cite-heavy, 4 close × 6 far, judge threshold 4.0/5, save by default) andgbrain lsd <question>(Lateral Synaptic Drift — inverted judge rejecting ideas with resistance >4.5 "too obvious", stale-page bias viapages.last_retrieved_at, 2 close × 12 far, axiomatic inversions required, ephemeral by default). The "domain bank" is prefix-stratified sampling from the user's own brain (SELECT DISTINCT substring(slug from '^[^/]+/[^/]+')cached 1h-TTL inconfigper source) tiebroken byJOIN page_linksconnection_count, with corpus-sampling fallback when fewer prefixes than M exist. Distance normalized to [0,1] via1 - clamp(cosine_distance, 0, 2) / 2.judges.tsexportsrunJudge(config, ideas)+ two configs (BRAINSTORM_JUDGE_CONFIGweighted originality/resistance/thesis_density/concrete_grounding/cognitive_load 0.25/0.20/0.20/0.20/0.15 vsLSD_JUDGE_CONFIGcognitive_load 0.50 + inversion rule). Calibration cold-start fallback: whencalibration_profiles.active_bias_tagsis empty, judge runs without anti-bias context AND stderr-warns. Op-layer write-back insrc/core/operations.tssearch/query/get_pagehandlers firesbumpLastRetrievedAt(engine, pageIds)(fire-and-forget, 5-min throttled via SQL clause, default-on withsearch.track_retrievalconfig escape hatch); internal callers (sync, migrations, dream cycle) bypass the op layer so the LSD stale signal stays clean. The fire-and-forget IIFE is tracked in a module-scopedSet<Promise<unknown>>;awaitPendingLastRetrievedWrites(timeoutMs?: number): Promise<{outcome, pending}>resolves once all tracked promises settle, bounded by a 5sPromise.racetimeout that stderr-warns the pending count.src/cli.tsawaits the drain unconditionally for every op in the op-dispatch finally block BEFOREengine.disconnect(), then a fallbackprocess.exit(0)fires ONLY whenoutcome === 'timeout'ANDshouldForceExitAfterMain(argv)(excludesserveso daemons stay alive) — so the IIFE cannot race disconnect and leave PGLite's WASM keeping Bun's event loop alive.pages.last_retrieved_at TIMESTAMPTZ NULLhas a full (NOT partial) B-tree index covering both NULL and range branches; full forward-reference bootstrap probe on both engines. Frontmattermode: lsdmakes the dream-cycle synthesize phase skip LSD output viaisLsdOutput()insrc/core/cycle/transcript-discovery.tsshort-circuitingisDreamOutput().gbrain eval brainstorm <fixture.jsonl>is a three-axis conjunctive gate (distance + usefulness + grounding — distance alone is gameable).gbrain doctorhas abrainstorm_healthcheck (migration applied,search.track_retrievalsetting, calibration cold-start status).judges.tscomputes the judge token budget viacomputeJudgeMaxTokens(ideaCount, modelId)(named constantsTOKEN_BUDGET_PER_IDEA,TOKEN_BUDGET_ENVELOPE,LEGACY_MIN_MAX_TOKENS,MAX_OUTPUT_TOKENS_CEIL;ANTHROPIC_OUTPUT_CAPSmap: Opus 4.7 32K, Sonnet 4.6 / Haiku 4.5 64K, legacy Claude 3.5 8K) so a large multi-call judge doesn't truncate mid-JSON; with nomodelOverridethe cap routes through the gateway's actual configured chat model viagetChatModel().--savefor both commands persists through the canonical ingestion path:persistSavedIdea(engine, {slug, content, provenanceVia})callsimportFromContent({noEmbed:true, sourcePath})(chunked + tagged + content_hash so search finds it, no embedding cost at save) THEN renders the saved row to disk via the sharedwritePageThroughhelper (file rendered FROM the row so the two sinks can't diverge andgbrain syncdoesn't churn it).formatSaveOutcome(outcome, ctx)returns an honest per-branch message (both-sinks, DB-only when nosync.repo_path/repo-not-a-dir, DB-saved-but-file-errored, total-failure → loudsave FAILED … NOT persistedon stderr + nonzero exit) —--savenever prints "Saved" when the DB write failed.buildIdeaSlug(question, label, nonce?)adds a random nonce suffix (injectable for tests) so two same-day runs sharing the first 60 slug chars don't clobber.--jsoncallers stay DB-only.buildBrainstormFrontmatterObject(result)in orchestrator.ts returns the object form forserializeMarkdown. Pinned bytest/last-retrieved.test.ts,test/e2e/pglite-cli-exit.serial.test.ts(IRON-RULE: realbun src/cli.tssubprocess against a hermetic PGLite tempdir asserts search/get/query exit 0 in <15s + daemon-survival),test/fix-wave-structural.test.ts(asserts the drainawaitis textually BEFOREengine.disconnect),test/brainstorm/{distance,lsd-mode-skip,eval-brainstorm,judges-maxtokens,save}.test.ts. Open Collider source:github.com/CL-ML/open-collider. -
src/core/write-through.ts— shared atomic disk write-through for the canonical ingestion path.resolvePageWriteTarget(engine, slug, sourceId)is the exported single source of truth for the ONE file a (source, slug) pair lives in on disk: a source with its ownlocal_pathwrites at that tree's ROOT (never nested under.sources/); a source without one nests under the host repo (sync.repo_path, default at root, non-default under.sources/<id>/) unless that path is another source's own working tree (leak guard →source_repo_belongs_to_other_source); the page's recordedsource_path(or a containedfile://source_uri) is preferred over a slug-derived name so writes land in the file of record instead of minting a twin; andisWriteTargetContainedrejects hostile rows escaping the tree (path_escapes_source_root). The ok-result also carriessourcePathToBind— the target expressed in the file scanner'ssource_pathconvention (GIT-ROOT-relative when the scan root sits inside a git repo, so a subdirectory-scopedlocal_pathbinds the same form delete-reconcile keys on; scan-root-relative otherwise). The resolver is shared bywritePageThrough, the facts fence writer (src/core/facts/fence-write.ts), and fence forget (src/core/facts/forget.ts) — the fence appends to the page's file, so all writers MUST compute the identical path or the fence lands in a file sync never reads back and the next extract_facts reconcile deletes the fence-owned DB rows.writePageThrough(engine, slug, {sourceId?, frontmatterOverrides?, logger?})resolves the target through it, re-reads the just-written DB row (getPage), renders it viaserializePageToMarkdown, and writes the.mdso the brain has a committable artifact that round-trips throughgbrain sync. Rendering FROM the row means file and row cannot diverge. ATOMIC: writes to a unique temp sibling (<file>.tmp.<pid>.<rand>) +renameSync, cleaning up temp on any failure, so a crash or concurrentgbrain sync/autopilot walking the live git tree never reads a half-written.md(matches the.tmp + renameconvention in import-checkpoint.ts / op-checkpoint.ts). After a successful rename, a row whosesource_pathis still NULL is immediately bound tosourcePathToBind(pages born via put/capture/reverse-write otherwise staysource_path=NULLforever because mtime-watermark incremental sync never rescans an untouched file); NULL-guarded so a scanner-recorded path is never rewritten, best-effort so a failed bind never fails the write. Never throws — returnsWriteThroughResult { written, path?, skipped?: 'disabled_by_config' | 'no_repo_configured' | 'repo_not_found' | 'source_repo_belongs_to_other_source' | 'page_not_found_after_write' | 'path_escapes_source_root' | 'case_insensitive_collision', error? }so the caller decides messaging + exit codes.isWriteThroughDisabled(shared with the fence lane, ~30s per-engine cache) honors thesync.write_throughopt-out. Trust gating (subagent sandbox, dry-run) stays at the CALLER. On a durability-hardened repo (isDurabilityHardened— the gbrain post-commit hook is installed, i.e. the user rangbrain sources harden), a successful write is best-effort COMMITTED viacommitWriteThroughFile(path-limitedgit commit -- <file>, never sweeps unrelated edits; the hook then background-pushes) so write-through content reaches git instead of accumulating uncommitted forever; result carriescommitted?: boolean. Unhardened repos keep write-only behavior. Consumers:put_pageop andgbrain brainstorm/lsd --saveviapersistSavedIdea. Pinned bytest/write-through.test.ts+test/write-through-commit.serial.test.ts. Theput_pageop is that deciding caller for direct writes: a skip outside the deliberate DB-only set (no_repo_configured/disabled_by_config/subagent_sandbox/dry_run) throwsstorage_error, deleting a just-created row first so "created" is never answered for a page with no file backing. -
src/core/model-id.ts—splitProviderModelId(input: string | null | undefined): {provider: string | null, model: string}shared parser for the pricing side. Splits on:first, then/. Defensive contract: null/undefined/empty/whitespace returns{provider: null, model: ''}. Five sites consume it (src/core/anthropic-pricing.ts:estimateMaxCostUsd,src/core/budget/budget-tracker.ts:lookupPricing,src/core/eval-contradictions/cost-tracker.ts:pricingFor,src/core/minions/batch-projection.tsat two call sites,src/core/model-config.ts:isAnthropicProvider) so the pricing + classification surface has no parallel re-implementations ofprovider:modelsplitting — slash-form ids (anthropic/claude-sonnet-4-6) classify correctly instead of falling through to "unknown model". Distinct from the gateway-sideparseModelIdinsrc/core/ai/model-resolver.ts, which throws on bare names because routing needs an explicit provider; this one returns{provider: null, model: 'bare'}because pricing lookups happen against bare model ids. Pinned bytest/model-id.test.ts. -
src/commands/recall.ts— thegbrain recallCLI (facts-first read; local engine or thin-client). Entity→text fallback: a bare positional is entity-first, but when the entity arm matches ZERO facts (and no--grep/--supersessions/--session-idnarrows the read) the query retries as the SQL-level fact-text grep with a one-line stderr note — so keyless/casual usage (gbrain recall coffee) finds the fact either way. One gating predicate + one note formatter (entityTextFallbackApplies/noteEntityTextFallback) shared by the local path and the thin-client mirror so the two can't drift; explicit--grepcallers keep exact semantics (their filter already ran, no fallback surprise).--sincecomposes with the positional entity and with--session-idthroughengine.listFactsSince({entitySlug, sessionId})(one query, cutoff before LIMIT, event time unless--since-last-run), mirroring therecallop's composition insrc/core/ops/facts.ts; pinned bytest/facts-recall-since-composition.test.ts. -
src/commands/transcripts.ts— the transcripts command family, all local-only (ctx.remote=falseby construction).recent: raw.txtcorpus reads vialistRecentTranscripts(same library as the gatedget_recent_transcriptsop).ingest <path-or-glob>: the cross-harness session importer — resolves ONE source id (6-tier chain), threads activePack once, streams progress (phasetranscripts.ingest), and callsrunTranscriptsIngest; embedding is OFF by default (embed backfill is the catch-up lane; the embed flag opts in); the max-bytes flag (validated size string, e.g.4gb) overrides the per-format file/store byte caps for oversized stores while omission preserves each adapter's native default; the since-last watermark is an op-checkpoint (optranscripts-ingest, fingerprint = source + pathspec + format + adapter version + any explicit byte cap viaingestCheckpointFingerprintInput— a checkpoint written under one cap is never silently reused under another, so a capped run's skipped tail can't read as scanned; the pathspec binds the user-stated paths resolved + sorted, and the no-arg/all lane substitutes hostname + sorted harness roots so DB-backed checkpoints shared across a brain's machines never let one machine inherit another's watermark and skip sessions it never scanned) advanced ONLY after a clean, untruncated, non-dry scan; no-arg = confined discovery table, the all flag imports the discovered set.status: found-vs-imported gap table (disk scan vs ONE conversation-pages frontmatter query —executeRaw, both engines — insrc/core/transcripts/discover.ts) — the correctness surface that catches late-arriving sessions no watermark can. The facts flag hands EVERY touched slug (including hash-skipped) tosrc/core/transcripts/ingest-facts.ts. Registered in CLI_ONLY + CLI_ONLY_SELF_HELP + SELF_HELP_WITHOUT_ENGINE (engine-free help).expandTilde(raw)(bare~, leading~/or~\\→ home dir;~userforms and a mid-string~stay literal) is the one tilde expansionexpandPathsapplies to every path spec. -
src/core/transcripts/(directory) — the transcript-adapter seam.types.ts: theTranscriptAdaptercontract —parse(path): AsyncGenerator<ParsedSession, FileDiagnostics>(one FILE may hold many sessions; the generator RETURN value carries bytes/skipped-lines/zero-session diagnostics so an empty file explains itself), format-specific byte caps (JSONL 50MB hard cap — the 10MB default inclaude-code-jsonl.tsbelongs to the hook-lane tail reader, not imports; monolithic export JSON rejects-not-truncates at 200MB), and the ONEbuildTranscriptSlughelper (per-provider dirs matching the conversation-archive layout; id suffixes are sha256 prefixes of the session id — hash12 in slugs, hash16 for the dedup identity — never a cleaned prefix of the source id, which would let same-prefix session ids collide).detect.ts: adapter registry + head-sample detection (explicit format wins; symlinks lstat-rejected) + injectableharnessRoots(the confined discovery surface). Adapters, each with a DATEDSPEC_TARGET+ scrubbed fixture +bytes>0 && sessions==0drift alarm:claude-code.ts(thin wrapper over the shippedclaude-code-jsonl.ts, which also exposes the full-fileparseClaudeSessionFilewith real per-message timestamps — hook-laneparseTranscriptoutput is pinned byte-identical; ownsisClaudeCodeSubagentFile—<session>/subagents/agent-*.jsonllogs are all-sidechain, zero-turn files carrying the PARENT session id, sodiscover.tsand the CLI'sexpandPathsboth skip them and they never read as gap-table backlog or drift; strict parent-dir + basename shape so a project slug containing "subagents" is never hidden),codex.ts(turn selection is STRUCTURAL: user turns fromevent_msguser_message, assistant fromresponse_itemoutput_text; role user/developer response_items are injected preambles and never leak),openclaw.ts(session header + message lines;.checkpoint.*.jsonlsnapshots rejected),hermes.ts(COPY-THEN-READ of state.db + wal/shm sidecars — readonly WAL opens need -shm write access and lock against a live writer; schema from the installed hermes-agent source; PROVISIONAL),grok.ts(Grok Build session dirs — structural turn selection, sidecar-aware discovery; see its own entry),chatgpt-export.ts(themapping-tree current_node walk; branches dropped by design, orphaned parents terminate quietly, latest-leaf fallback; extracted conversations.json only) +claude-export.ts(flat chat_messages); both export adapters load through the sharedexport-json.ts(monolithic JSON over the cap REJECTS with a split hint; a zip or wrong-shape file gets the unzip-first hint) and give id-less conversations content-derived fallback ids so two id-less exports cannot dedup-skip or overwrite each other.render.ts: session → part pages — imessage-slack line format with the regex IMPORTED fromconversation-parser/builtins.ts(round-trip pinned), REAL timestamps (missing ones carry forward; zero-timestamp sessions refused — provenance is never fabricated), anchor-shaped BODY lines backslash-escaped (hostile message content cannot forge speakers), quoted facts/takes fence-marker tokens backslash-escaped at the same per-message seam (gbrain\:facts:begin— a session that read another page cannot plant a live or unbalanced fence on the transcript page; every fence consumer is a substring matcher, so the escape lands inside the token), fail-closed redaction (secret-scan +harvest-private-patterns.txtuser patterns, slack-channel default excluded because it eats issue refs; imperatives COUNTED into hash-coveredtranscript_importfrontmatter, never content_flag), ~300KB message-boundary splitting with 2-message overlap (under the embed-skip threshold), part 1 keeps the base slug,frontmatter.idunique per part.ingest.ts: the engine-facing core — SESSION atomicity (failed sessions skip; integrity failures abort the run), stale-part reconciliation (deletespart > ofleftovers), session metadata banked in the base page's raw_data as the REDACTED copy and healed by content-compare even on hash-skipped re-runs (a run that died beforeputRawData, or a private pattern added after first import, repairs on the next pass),cleanScan/maxSessionTsfor the watermark.ingest-facts.ts: ONErunExtractConversationFactsCoreinvocation (batchslugsselector) inside ONEwithBudgetTracker,isFactsExtractionEnabledpre-checked. Pinned bytest/transcript-adapters.test.ts,test/transcript-render.test.ts,test/e2e/transcripts-ingest-pglite.test.ts,test/e2e/transcripts-writeback-fidelity.test.ts(raw files through the adapters into the gold-extractor facts path). -
src/core/connectors/(directory) — the LIVE chat-history connector layer (front-end to the transcripts pipeline).types.ts: theChatHistoryProvidercontract (mirror ofTranscriptAdapter— leaf module per provider;spoolFormatnames the adapter that parses its output; datedHostSpecTarget).credentials.ts: file-plane store at~/.gbrain/connectors/<provider>.json(0600, dir 0700, atomic tmp+rename), env-above-fileresolveCredentialreturning{cred, source}; credentials NEVER touch the DB /sources.config/ config planes / op payloads.classify.ts: the pure §2A/§2B response classifier — distinguishes a Cloudflare-fingerprint 403 (forbidden_fingerprint, dead-end) from an auth-expired 403 (auth_required, re-auth) and flags a parses-but-no-itemsbody asdrift.client.ts:ConnectorClientmodeled onGitHubClient(plain fetch against a hardcoded ORIGIN + off-origin guard so the cookie/bearer never leaks, browser UA, refresh-once-on-401, Retry-After backoff;fetchImpl/sleep/nowinjectable — NOTfetchWithSSRFGuard, which breaks SNI on Cloudflare hosts).providers/chatgpt.ts+providers/claude.ts(registry inregistry.ts; Perplexity deliberately absent — no adapter): offset/updated pagination with a per-pass 500-page cap and astopBeforewatermark break, epoch/ISO normalization, ChatGPT'sis_archivedsecond pass, org discovery + content-block text assembly for Claude, and arefreshAccessToken(ChatGPT re-mints from the cookie via/api/auth/session; Claude's cookie is terminal).spool.ts: writes fetched conversations in native-export shape (0600, batched, pruned in the orchestratorfinally).sync.ts:runConnectorSync— resolve credential → probe → read the config-scalar watermarkconnectors.<p>.watermark_iso(NOT op_checkpoint, whose 7-day GC would wipe it → full re-fetch) → list towatermark − windowDays→ fetch → spool →runTranscriptsIngest→ advance the watermark ONLY on a fully clean run →logIngestreceipt → stamplast_sync_at→ engine-branched embed kickoff (maybeKickoffEmbed: PostgressubmitEmbedBackfill, PGLiterunEmbedCoreinline — NEVERsubmitEmbedBackfill, which refusesno_worker_surface).config-keys.ts: theconnectors.key builders + the pureisConnectorSyncStale(last, now, floor)dispatch gate.oauth-pkce.ts: dependency-free S256 PKCE loopback (Bun.serve, timing-safe state) — best-effort--try-oauthonly. Surfaces: theconnectors_status/connector_syncops (src/core/ops/connectors.ts, bothlocalOnly), thegbrain connectorscommand family (src/commands/connectors/peeled dir), theconnector-syncminion handler (src/core/minions/handlers/connector-sync.ts, single-flight lock), the autopilotmaybeDispatchConnectorSyncsgate, and theconnectorsdoctor check. Pinned bytest/connectors-*.test.ts+test/e2e/connectors-sync-pglite.test.ts(full pipeline against aBun.servefixture backend, keyless) +test/e2e/connector-sync-handler-pglite.test.ts+test/e2e/doctor-connectors-pglite.test.ts. -
src/commands/integrity.ts—gbrain integrity check|auto|review|extract: bare-tweet detection, dead-link detection, three-bucket repair (auto-repair / review-queue / skip).scanIntegrity()is the shared library function called fromgbrain doctor(sampled at limit=500) andcmdCheck(full scan). Batch-load fast path on Postgres uses a single SQL query (avoids the PgBouncer per-row round-trip timeout), gated byengine.kind === 'postgres'at the call site so PGLite never enters batch; fallbackcatchlogs atGBRAIN_DEBUG=1. Batch projection isSELECT ... ORDER BY source_id, slug(NOTSELECT DISTINCT ON (slug), which would collapse same-slug-different-source pages into one scan) so multi-source brains scan each(source, slug)row independently. Sequential and auto-repair loops uselistAllPageRefs()to enumerate(slug, source_id)pairs and threadsourceIdtogetPage; batch + sequential paths report the same page count on multi-source brains. -
src/core/timeline-dedup-repair.ts— schema-drift self-heal foridx_timeline_dedup. A master-merge migration renumbering can leave a brain's version counter stamped past the index change while the index keeps an old shape — and everyaddTimelineEntrybatch then fails itsON CONFLICTinference, silently breaking timeline writes brain-wide. The version counter can't detect this, so the repair is keyed off the actual index SHAPE. Canonical shape is(page_id, date, md5(summary), source)— md5-keyed (migration v138) because a raw long/incompressible summary would overflow the btree v4 row cap (~2704 bytes) and abort every timeline insert for that page; both engines' insert sites inferON CONFLICTon the md5 tuple, soEXPECTED_COLUMNSMUST carry the md5 form (a raw-shape expectation would make the repair revert v138 on every migrate pass).checkTimelineDedupIndex(engine)returns{tablePresent, indexPresent, columns, needsRepair}(read-only; powers thetimeline_dedup_indexdoctor check; the indexdef column parser is paren-depth-aware somd5(summary)— or any future expression column — parses as ONE column instead of flagging a correct index as drifted forever) andrepairTimelineDedupIndex(engine)dedupes-then-rebuilds to the canonical shape (raw-summary grouping ⟺ md5 grouping modulo negligible collisions).runMigrationsinvokes the repair on every pass (including the no-pending early-return path); idempotent no-op when the index already matches.gbrain apply-migrations --force-schematriggers it on demand. Pinned bytest/timeline-dedup-repair.test.ts. -
src/core/progress.ts— Shared bulk-action progress reporter. Writes to stderr. Modes:auto(TTY\r-rewriting; non-TTY plain lines),human,json(JSONL),quiet. Rate-gated byminIntervalMsandminItems.startHeartbeat(reporter, note)for single long queries.child()composes phase paths. Singleton SIGINT/SIGTERM coordinator emitsabortevents for every live phase. EPIPE defense on both sync throws and stream'error'events. Zero dependencies.emitHumanLineis prefix-aware — inside awithSourcePrefix(id, ...)scope fromsrc/core/console-prefix.tsit prepends[id](and TTY-rewrite mode\r\x1b[2Kcarries the prefix inside the clear-to-EOL escape);emitJsonis intentionally NOT prefixed so NDJSON consumers don't choke on a[id] {...}shape. -
src/core/console-prefix.ts—AsyncLocalStorage<string>-backed per-source line-prefix helper. ExportswithSourcePrefix(id, fn)(runsfnwithidas active prefix; nested wraps replace then restore),getSourcePrefix()(read-only accessor; test seam),slog(...)/serr(...)(prefix-awareconsole.log/console.error), andwithHumanLogsToStderr(fn)(runsfnwith everyslogline, prefixed or not, routed to stderr;runSyncwraps its body in it under--jsonso stdout carries only JSON lines — the envelope plus any JSON status lines — whileserrand directconsole.log(JSON.stringify(..))sites are unaffected; scoped via its ownAsyncLocalStorage, never a process-wide console rebind). Embedded-newline-safe: a multi-line string under prefix[foo]emits[foo] line1\n[foo] line2. Outside a wrap,slog/serrfall through to bareconsole.log/console.errorso single-source callers see identical output (back-compat invariant). Usesrc.id(slug-validated bysources add) NOTsrc.name(free-form) to defeat log-injection through newline/control-character names. Coverage:src/commands/sync.tsperformSync + callees,src/commands/embed.tsrunEmbedCore + helpers,src/core/progress.tsemitHumanLine,src/commands/import.tsrunImport's human-only lines (info()+ the end-of-runImport completesummary — so the first-sync full import undersync --jsonlands on stderr too). -
src/core/cli-options.ts— Global CLI flag parser.parseGlobalFlags(argv)returns{cliOpts, rest}with--quiet/--progress-json/--progress-interval=<ms>/--brain <id>stripped.--brainis the brain-axis (which database) selector: exact-match only (--brain-*per-command flags pass through), value validated against the mount-id regex at parse time, missing/malformed value THROWS — never a silent host fallback.connectEngineinsrc/cli.tsfeeds it (plus the ambientGBRAIN_BRAIN_ID/.gbrain-mount/ mount-path tiers) throughresolveBrainId→BrainRegistry.getBrain, which throwsUnknownBrainErrorfor an unregistered id; mounts get no auto-migrations and keep the host-config AI gateway.getCliOptions()/setCliOptions()expose a module-level singleton so commands reach resolved flags without parameter threading.cliOptsToProgressOptions()maps to reporter options.childGlobalFlags()returns the flag suffix to append toexecSync('gbrain ...')calls in migration orchestrators (propagates--brain=<id>so children stay on the parent's brain).OperationContext.cliOptsextends shared-op dispatch for MCP callers.CliOptionscarriesexplain: boolean.parseGlobalFlagsrecognizes--explainanywhere in argv (stripped before command dispatch).src/cli.tsformatResultforsearch+querycases routes toformatResultsExplainfromsrc/core/search/explain-formatter.tswhenCliOptions.explainis set; falls through to the existing JSON / human formatters otherwise.maybeBackground(opName, fingerprintArgs, runDirect)helper. Same semantics in TTY and cron (no--no-tty-detectflag, no surprise behavior change between contexts): when--backgroundis passed, submits the op as a Minion job viaop_checkpointsfor resumability and returns thejob_id.--background --followexecsgbrain jobs follow <id>so the user sees the same stderr stream they'd get from a direct call. PGLite degrades to inline execution with a clear stderr note ("PGLite worker pool not yet supported; running inline"). Returns a tagged union the caller dispatches on. -
src/cli.tsstrict flag validation +src/core/cli-flag-registry.generated.ts+scripts/generate-flag-registry.ts— pre-dispatch, pre-engine unknown-flag rejection for every command: a flag no handler consults fails loud (unknown flag --x for 'gbrain <cmd>', exit 1;--jsoninvocations also get a structured{status:'error', reason:'invalid_flag'}on stdout) instead of being silently ignored while the un-asked-for real operation runs.validateCommandFlags(command, subArgs)runs after the--helpshort-circuit and before any dispatch or engine connect, in two lanes mirroring dispatch order (CLI_ONLY first —think/salience/anomaliesare both ops AND CLI_ONLY members whose handlers parse flags the op contract doesn't declare): CLI_ONLY commands validate against the generatedCLI_FLAG_REGISTRY(per-command legal sets derived from each command's source — case block + imported modules + one level of relative imports +EXTRA_FLAGS; deliberately over-inclusive, help-text mentions count; regenerate viabun run build:flag-registry); op commands validate viafindUnknownOpFlag, which mirrorsparseOpArgs's traversal (non-boolean flags consume their value token;--key=valueinline form recognized) plus the CLI-local flags consumed outside the op contract (json,explain,help,source,dry-run); the CLI_ONLY token scan is the exportedfindUnknownFlag(args, legal). InparseOpArgs,json/dry_runare CLI-local booleans that never consume a value token, so a trailing--dry-runis a real rehearsal switch feedingmakeContext'sctx.dryRun. Uppercase flag spellings are treated as unknown (every handler is case-sensitive-lowercase, so they'd be silently ignored downstream — the exact class the validator kills). Exempt by contract:call(arbitrary--paraminterface),config(arbitrary set values),jobs submit(handler-defined payload params); everything after a literal--is passthrough and never validated. A command missing from the registry fails OPEN at runtime (never bricks a command);test/cli-flag-validation.test.tspins registry freshness, per-command drift, and consumption evidence — a safety flag (--dry-run,--yes,--force) may only be advertised if the command's source actually reads it. The generator segmentshandleCliOnlyinto per-command text blocks withsegmentDispatchBlocks: bothcase 'X':labels and everyif (command === 'X' …)head — plain, compound (&& args[0] === 'sub': the no-DBeval <sub>bypasses such aseval longmemeval, the<cmd> --helppre-engine branches,agent register) and multi-line — are markers, and ownership follows thecommand === 'X'head, never the condition's tail, so a compound block's flags land on its own command's row (theevalrow is a union across eval subcommands, the registry's shape for every multi-subcommand command); a barecommand === 'X'inside a non-ifexpression is deliberately not a marker. Onlyimport('./commands/*.ts')inside a block counts as a command module (core helpers a block reaches for directly are not scanned — a flag the block consumes through one is already a literal in the block's own text), andisValueOnlyImportskips a destructured import whose bindings are all SCREAMING_CASE constants (a borrowed message string, not a handler). Pinned bytest/generate-flag-registry.test.ts(acceptance AND rejection:--frobnicateis still refused) andtest/eval-longmemeval-cli-smoke.test.ts(the documentedgbrain eval longmemeval … --retrieval-only --by-type --no-trajectory --keyword-onlyinvocation exits 0 as a subprocess). -
src/core/source-id.ts— single canonical source_id validation, dependency-free by design (imported by both engines, cycle, source-resolver, sources-ops).SOURCE_ID_RE(strict: 1-32 lowercase alnum, interior hyphens only, no edge hyphens);isValidSourceId(boolean — for tiers that silently fall back: dotfile, brain_default) vsassertValidSourceId(throws — for tiers that must reject loudly: explicit--source,GBRAIN_SOURCE,cycleLockIdFor);ALL_SOURCES = '__all__'sentinel (deliberately NOT a valid id so it can never collide with a real source or leak into lock ids/path joins;sourceScopeOptstranslates it to an unscoped read for trusted local callers and keeps it unsatisfiable for remote callers, fail-closed).normalizeSourceInput/normalizeFederatedReadInputnormalize the/admin/api/register-clientHTTP body, mirroring the CLI's--source/--federated-readflags: omittedsource→'default'; omittedfederatedRead→undefinedsoregisterClientManualapplies its own[sourceId]default; present-but-invalid values throw so the route returns a structured 400 (invalid_source) instead of failing at INSERT time. Pinned bytest/register-client-source-normalize.test.ts. -
src/core/entity-identity.ts+src/core/ops/entity-identity.ts— cross-source entity identity (cross-source federation). THE IDENTITY KEY IS(source_id, slug): the same real-world entity routinely exists as a different page per mounted source, and nothing else in gbrain asserts "these pages are the same entity". Theentity_identitiestable (migration v137) records that assertion explicitly — member pages grouped under an opaqueentity_idhandle (validateEntityId: slug-flavored, lowercase, max 128 chars), one optional canonical member per group, a page belongs to at most one identity (UNIQUE (source_id, page_id); re-linking MOVES it). Posture is MANUAL-ONLY by design: rows come exclusively from theentity_identity_linkop — no auto-matching or name-similarity heuristics, because a wrong identity merge silently corrupts retrieval. Three ops:entity_identity_link/entity_identity_unlink(write, localOnly — merges change what retrieval unions across sources, too sharp for a remote surface) +entity_identity_list(read; restricts MEMBER visibility to a federated grant or remote scalar scope, while a trusted local caller sees the full cross-source group — cross-source IS the feature). Retrieval union is OFF by default behind theentity_identity.unionconfig key; when on, get_links/get_backlinks merge edges from identity co-members (dedup'd), never widening past the caller's source grant. Same SQL text on both engines (parity by construction); pinned by the entity-identity test + the DATABASE_URL-gated engine-parity suite. Topology context:docs/architecture/brains-and-sources.md. -
src/core/db-lock.ts— generictryAcquireDbLock(engine, lockId, ttlMinutes)over thegbrain_cycle_lockstable. Parameterized lock id so scopes nest cleanly:gbrain-cyclefor the broad cycle (held bycycle.ts) andgbrain-sync(SYNC_LOCK_ID) forperformSync's narrower writer window. UPSERT-with-TTL semantics survive PgBouncer transaction pooling (unlike session-scopedpg_try_advisory_lock); crashed holders auto-release once their TTL expires. Every handle is FENCED to its exact acquisition:DbLockHandle.acquiredAtcaptures the row'sacquired_atas epoch-seconds text (extract(epoch from acquired_at)::text— GUC-independent, unliketimestamptz::textwhich varies with per-session TimeZone/DateStyle across pools) andrefresh()/release()/the cleanup hook all match(id, holder_pid, fence), so a recycled PID or a superseded holder can never refresh or delete a successor's row.refresh()returns a boolean:true= still owned;false= the fenced UPDATE matched 0 rows (stolen or force-cleared — certain loss, the caller must stop relying on mutual exclusion); transient DB errors still THROW (not evidence of a steal; the TTL is the backstop). ExportsLockStolenError(thrown/used as an AbortSignal reason by consumers likecycle.ts's refresher and the supervisor). It also does automatic same-host dead-pid takeover: when the upsert finds a held, NOT-TTL-expired lock whose holder is on this host and provably dead, it reclaims via a guardedDELETE WHERE id=\$1 AND holder_pid=\$2+ one normal-upsert retry returning the standard handle (refresh/release intact). The liveness check is the exportedclassifyHolderLiveness(pid, host, ageMs, opts?)/isHolderDeadLocally(...)(injectableprocess.killseam;HOLDER_TAKEOVER_GRACE_MS = 60_000PID-reuse guard; EPERM classified asaliveso a live process you don't own is never stolen). TTL-expired locks stay the upsert's job; cross-host stays TTL-only.runBreakLock(src/commands/sync.ts) consumes the same predicate. Background reaper:reapDeadHolderLocks(engine)is the periodic sweep the contention path lacked — it deletes locks whose holder isisHolderDeadLocally, scoped to thegbrain-sync:*/gbrain-cycle/gbrain-cycle:*namespaces ONLY (election/supervisor/reindex locks keep TTL-only behavior, untouched), viadeleteLockRowExact(engine, id, pid, acquiredAt)— a snapshot-matched delete (date_trunc('milliseconds', acquired_at) = \$3, so the ms a JS Date keeps survives) that's TOCTOU-safe against a reused PID taking the lock between SELECT and DELETE.cycle.tsruns it at cycle start (before the sync phase);gbrain doctor --fixruns it for no-autopilot brains.selectLockRows(engine, opts?)+ a shared row→LockSnapshotmapper are the single canonical reader backinginspectLock+listStaleLocks+ the reaper.isLockHolderLive(snap, ttlMinutes)is the observability liveness predicate — freshness-keyed (ttl_expiredplus the heartbeat steal-grace), neverprocess.kill, sogbrain jobs supervisor status/gbrain doctorcan report a live supervisor via its queue lock without a PID-reuse false-positive. Pinned bytest/db-lock-auto-takeover.test.ts+test/db-lock-reap.test.ts+test/db-lock-fencing.test.ts.DbLockHandle.refresh(opts?)accepts{signal}(Postgres forwards to executeRawDirect; PGLite ignores);withRefreshingLock's heartbeat aborts the per-tick signal on timeout and guards re-entrancy (15s min cadence vs 30s timeout could stack ticks). -
src/core/sync-concurrency.ts— single source of truth for the parallel-sync policy. ExportsautoConcurrency(engine, fileCount, override?)(PGLite always serial; explicit override clamped to >=1; auto path returnsDEFAULT_PARALLEL_WORKERS=4whenfileCount > AUTO_CONCURRENCY_FILE_THRESHOLD=100),shouldRunParallel(workers, fileCount, explicit)(explicit--workersbypasses the >50-file floor), andparseWorkers(s)(rejects'0','-3','foo','1.5', trailing chars). Used byperformSync,performFullSync,runImport, and the Minionsynchandler so the sites can't drift.DEFAULT_PARALLEL_SOURCES = 4is a SEPARATE constant for the per-source fan-out undergbrain sync --all— kept distinct fromDEFAULT_PARALLEL_WORKERSbecause total live Postgres connections per wave ≈ = 32 at both defaults (each per-file worker opens its ownPostgresEnginewithpoolSize = min(2, resolvePoolSize(2)));sync.tswarns when .resolveWorkersWithClamp(engine, override, commandName, fileCount)wrapsautoConcurrencywith a per-command stderr clamp warning on PGLite (per-(command, requested) dedup via module-scoped warned-once set with_resetWorkersClampWarningsForTest()seam) and is the canonical surface for every bulk-command--workers Nflag (extract-conversation-facts, extract, edges-backfill, reindex-multimodal, reindex, reindex-code); embed.ts deliberately bypasses it and keepsGBRAIN_EMBED_CONCURRENCY || 20.resolveMaxConnections()(readsGBRAIN_MAX_CONNECTIONS, undefined when unset) +clampWorkersForConnectionBudget(workers, perWorkerPool, maxConnections, parentPool)back the opt-in single-sync connection-footprint clamp so a big sync stays under a low pooler cap (parent_pool + workers×perWorkerPool ≤ budget);gbrain doctor'spool_budgetcheck (computePoolBudgetCheck/checkPoolBudgetinsrc/commands/doctor.ts) warns when the budget leaves no room for a worker, pointing atGBRAIN_POOL_SIZE=2. Pinned bytest/pglite-workers-clamp.test.ts. -
src/core/worker-pool.ts— Canonical sliding-pool + bounded-semaphore primitive (used by thesrc/commands/embed.tssliding-pool sites andsrc/commands/eval-cross-modal.ts'srunWithLimitsemaphore). Two exports:runSlidingPool<T>({items, workers, onItem, signal?, onError?, failureLabel?, onProgress?})+runWithLimit<TIn, TOut>({items, limit, fn, signal?}). Atomicity invariant:const idx = nextIdx++is one synchronous JS statement (noawaitbetween read and write — guaranteed by the single-threaded event loop), documented in the module header AND enforced byscripts/check-worker-pool-atomicity.sh(wired intobun run verify), which rejects importingworker_threadsin any consuming file and insertingawaitbetween thenextIdxread and write.MUST_ABORT_ERROR_TAGSset is seeded withBUDGET_EXHAUSTEDfromsrc/core/budget/budget-tracker.ts; tagged errors (matched viaerr.tag === 'BUDGET_EXHAUSTED'to avoid cross-module import) bypassonErrorand hard-abort the pool viaAbortController.abort()to in-flightonItem, then the pool JOINS every worker (each finishes its current item) before rethrowing — the budget cap is a structural ceiling under concurrency and no worker is left running detached past the caller's catch.failures[]shape is{idx, label, error}records (NOT full items; callers supplyfailureLabel(item) => string) for bounded memory under huge brains. Pinned bytest/worker-pool.test.ts+test/scripts/check-worker-pool-atomicity.test.ts. Drives every--workers Nbulk command. -
src/core/embedding-dim-check.ts— facts.embedding dim drift surface.readFactsEmbeddingDim(engine): Promise<FactsColumnDimResult>covers bothvector(N)andhalfvec(N)shapes (migration v40 falls back tovectoron pgvector < 0.7); regex ordering is halfvec-before-vector (substring "vec" appears in "halfvec"; naive/vector/iwould shadow).buildFactsAlterRecipe(dims, configured, type)emits the paste-readyDROP INDEX IF EXISTS idx_facts_embedding_hnsw; ALTER TABLE facts ALTER COLUMN embedding TYPE halfvec(N) USING embedding::halfvec(N); CREATE INDEX idx_facts_embedding_hnsw ON facts USING hnsw (embedding halfvec_cosine_ops) WHERE ...flow (NOT bare REINDEX, which doesn't rewrite the index after a column-type change).assertFactsEmbeddingDimMatchesConfig(engine)is the preflight — throwsFactsEmbeddingDimMismatchError(taggedtag: 'FACTS_EMBEDDING_DIM_MISMATCH'for parity with the worker-pool MUST_ABORT semantics) when configured dim ≠ column width; cached per-engine viaWeakMap; PGLite engines silently skip. Doctor checkfacts_embedding_width_consistency(registered afterembedding_width_consistency) reuses the same helpers with an identical ALTER recipe.decorateEmbeddingDimError(err, slug): maps pgvector's bare "expected N dimensions, not M" write rejection to a namedOperationError('embedding_plane_split', …)carrying the consequence (page NOT written, transaction rolled back) + recovery command; applied atimportFromContent's transaction boundary sogbrain putfails loudly instead of echoing an unexplained provider string; every other error passes through untouched. Pinned bytest/embedding-dim-check-facts.test.ts+test/put-page-embedding-plane-split.test.ts. -
src/core/sort-newest-first.ts— single source of truth for the descending-lex sort thatgbrain importandgbrain syncboth apply. Mutates in place (Array.prototype.sort semantics), returns the same array reference for fluent chaining. Empty/single-element inputs short-circuit. Future ordering changes flip one line here instead of touching two CLI commands. Pinned bytest/sort-newest-first.test.ts(descending order, mixed prefixes, empty, single-element, in-place-mutation contract). -
src/core/cycle.ts— brain maintenance cycle primitive (23 phases;ALL_PHASESis the ordered source of truth).runCycle(engine: BrainEngine | null, opts: CycleOpts): Promise<CycleReport>composes phases in semantic order along the core spine lint → backlinks → sync → synthesize → extract → patterns → recompute_emotional_weight → embed → orphans, with the extraction/graph/consolidation phases (extract_facts, extract_atoms, resolve_symbol_edges, …) slotted between per the ordering comments onALL_PHASES.synthesizeruns after sync (cross-references see a fresh brain) and before extract (auto-link materializes its writes);patternsruns after extract so it reads a fresh graph (subagent put_page setsctx.remote=trueand skips auto-link/timeline by default, so extract is the canonical materialization);recompute_emotional_weightsees the union ofsyncPagesAffected+synthesizeWrittenSlugsincrementally, or all pages when neither anchor is set (full backfill viagbrain dream --phase recompute_emotional_weight).CycleReport.schema_version: "1"is stable;totalsis additive (pages_emotional_weight_recomputed,transcripts_processed,synth_pages_written,patterns_written). Three callers:gbrain dreamCLI,gbrain autopilotdaemon inline path, the Minionsautopilot-cyclehandler. Coordination viagbrain_cycle_locksDB table +~/.gbrain/cycle.lockfile lock with PID-liveness for PGLite; the two handles compose into ONE lock whoserefresh()treats the fenced DB row as the authoritative multi-writer identity and the file half as best-effort freshness — the file lock is refreshed ONLY while the fenced DB refresh reports ownership, so a losing holder can never clobber the successor's file lock on the very tick it detects the steal. A dedicated serialized refresher —startCycleLockRefresher(lock, controller, lockId)(exported; unref'dsetIntervalatmax(15s, TTL/6), env-only overrideGBRAIN_CYCLE_LOCK_REFRESH_MS, in-flight guard so a slow refresh never overlaps the next tick) — heartbeats the lock through long phases (synthesis/patterns/consolidation waits routinely outlive the TTL with no other heartbeat). A fenced miss aborts the controller withLockStolenError; that steal signal combines with the worker's external signal via the exportedanyAbortSignal(signals)(duck-type-tolerant — stubs withoutaddEventListenerare observed by poll; returns{signal, dispose}and dispose detaches the caller-signal listener + clears the poll timer so daemons don't leak), and the run stops at the next phase boundary with a structured partial report carryingreason: 'lock_stolen'(for the 5 long phases — synthesize / extract_atoms / patterns / synthesize_concepts / consolidate — the steal races the phase promise and stops the WAIT immediately; their in-flight work runs to its own bounded timeout because their opts can't carry a signal yet). Transient refresh errors log and retry next tick (the TTL stays the backstop).yieldBetweenPhasesruns between phases;yieldDuringPhaseis in-phase keepalive. Engine nullable; lock-skip on read-only phase selections.CycleOpts.signal?: AbortSignalpropagates the worker's abort signal withcheckAborted()between every phase.CycleOpts.deadlineAtMs(the enclosing minion job's ABSOLUTE wall-clock deadline, threaded fromMinionJobContext.deadlineAtMsby the autopilot-cycle handler; null for directgbrain dreamcallers) flows into patterns, propose_takes, AND synthesize so time-spending phases derive their deadlines from the REAL remaining job budget instead of duplicated literals that collide with the job's own kill timer. Pinned bytest/cycle-lock-steal.serial.test.ts(mid-run steal → partial report, no further phases, successor row intact) +test/cycle-any-abort-signal.test.ts+test/db-lock-fencing.test.ts.runPhaseSyncreturnspagesAffectedviaSyncPhaseResult(threaded torunPhaseExtractas the 4th arg) and takeswillRunExtractPhase: booleansettingnoExtract: phases.includes('extract')sogbrain dream --phase syncdoesn't silently lose extraction. The extract phase callsrunExtractCoreandextractStaleFromDBwithquiet: true(andjsonMode: false), so the helpers' human summaries never land on stdout ahead of thedream --jsonCycleReport while batch-loss diagnostics stay human-readable on stderr (quietis its own knob onExtractOpts— usingjsonMode: trueas a stand-in flipped the stderr channel to JSON events in a plaingbrain dream);totals.pages_extractedcounts pages (pages_processedfrom the targeted pass +stale_pages_drained), not links created.resolveSourceForDir(engine, brainDir)threadssourceIdtoperformSync()so sync reads the per-sourcesources.last_commitanchor (not the drift-prone globalconfig.sync.last_commit).CycleOpts.brainDirisstring | null; when null (checkout-less postgres/Supabase brain) the 6 filesystem phases (lint/backlinks/sync/synthesize/extract/patterns) skip withdetails.reason: 'no_brain_dir'and the DB-only phases run;resolveSourceForDiris null-tolerant.cycleSourceId = opts.sourceId ?? resolveSourceForDir(engine, brainDir)is the canonical per-source scope forextract_facts/extract_atoms/calibration — and forsynthesize(threaded asSynthesizePhaseOpts.sourceIdso synthesized pages land in the cycle's resolved source, not'default') — sogbrain dream --source repo-areconciles repo-a's facts even with no checkout (instead of scoping to'default'while stamping repo-a fresh).deriveStatuscountsedges_resolved/edges_ambiguousas work so an edges-only cycle reportsoknotclean, and scores ONLY attempted phases — the implicit source-cycle exclusion skip-records are bookkeeping and never dilute failure aggregation; thejobs.tsautopilot-cycle+ phase-wrapper handlers passnull(not'.') when no repo is configured. The cycle is SPLIT for autopilot fan-out along thePHASE_SCOPEtaxonomy insrc/core/cycle/phase-scope.ts(see its entry): cycle.ts derivesSOURCE_PHASES/MIXED_PHASES/GLOBAL_PHASES/MAINTENANCE_PHASES(mixed ∪ global, original cycle order) plusSOURCE_FRESHNESS_PHASES(deterministic, non-LLM: lint/backlinks/sync/extract/extract_facts/recompute_emotional_weight — defined in phase-scope.ts) andSOURCE_BACKGROUND_PHASES(LLM-backed/unbounded source work = SOURCE minus FRESHNESS).resolveCyclePhases(requested, sourceId)at the shared runCycle boundary: default/no-source →requested ?? ALL_PHASES(the canonicaldefaultcycle remains full); a named non-default source with NO explicit phases →SOURCE_FRESHNESS_PHASESonly (the freshness keeper's implicitdream --source Xpath must stamp freshness without first draining LLM-backed maintenance); explicit phase lists are honored VERBATIM —dream --source X --phase synthesize(and--input <file>, which implies synthesize) is deliberate operator intent, and--phase orphans --source Xstill narrows orphans viaforceGlobalOrphans. The N-way duplication guard lives at the QUEUE boundary instead: theautopilot-cyclehandler intersects queued per-source payloads withSOURCE_FRESHNESS_PHASES(queued payloads may carry mixed+background phases; an all-rejected or empty list is an explicit no-op skip with reasonall_phases_rejected_by_normalization, never an implicit run, and rejected phases surface on the job result asphases_rejected_by_normalization). On the implicit path, excluded phases surface asskippedwithdetails.reason: 'excluded_from_implicit_source_cycle'+phase_scope. Per-sourceautopilot-cyclejobs enqueuephases: SOURCE_FRESHNESS_PHASESand stamplast_source_cycle_at; the singleautopilot-global-maintenancejob runsMAINTENANCE_PHASES(nosourceId) and stamps the brain-levelautopilot.last_global_atconfig key (LAST_GLOBAL_AT_KEY). SOURCE_BACKGROUND phases have NO automatic lane on multi-source brains (the legacy full-cycle job fires only when the brain has no sources rows) — run them explicitly (gbrain dream --source X --phase extract_atoms) until the background lane lands (see TODOS). The freshness stamp gate isopts.sourceId && phases.length > 0 && engine && !dryRun && !aborted && status ∈ {ok, clean, partial};last_full_cycle_atis still written alongsidelast_source_cycle_aton a per-source success for doctor/legacy readers (not a gate for the brain-wide phases). Pinned bytest/dream-postgres.serial.test.ts+test/jobs-autopilot-cycle-braindir.serial.test.ts+test/autopilot-global-maintenance.test.ts+test/cycle-enabled-phase-completeness.test.ts.runPhaseLint+runPhaseBacklinkscarry theexportkeyword so behavioral tests can drive them directly (internal helpers exposed for test-only consumption; downstream code should NOT depend on them). Pinned bytest/cycle-legacy-phases.test.ts(11 cases across both phases: clean run → status='ok', partial fix → status='warn' withdryRunin details, dry-run path doesn't write, throw-from-lib → status='fail' with the wrapper's try/catch envelope populated). withsrc/core/cycle/extract-atoms.ts+src/core/cycle/synthesize-concepts.ts+src/commands/extract.ts+src/commands/doctor.ts+src/core/op-checkpoint.ts: six daily-driver ops behaviors. (1) Batch idempotency:atomsExistingForHashes(engine, sourceId, hashes[])(exported fromsrc/core/cycle/extract-atoms.ts) answers in one batched SQL roundtrip (never a per-hash loop) returning already-extractedcontent_hash16values; fail-open (SQL error → empty set, extraction proceeds); powered by migration v104pages_atom_source_hash_idx(partial expression index onfrontmatter->>'source_hash'for atom rows wheredeleted_at IS NULL; PostgresCREATE INDEX CONCURRENTLYwith invalid-remnant pre-drop, PGLite plain). (2) Cycle lock TTL + heartbeat:LOCK_TTL_MINUTES = 5;buildYieldDuringPhase(lock, outer)(exported, withLockHandle) callslock.refresh()+ any external hook on every fire, throttled to 30s viamaybeYield, firing both in the main loop AND immediately after everyawait chat(...);synthesize_conceptsuses the same throttled hook. A crashed cycle releases its lock within one short TTL, while the dedicatedstartCycleLockRefresherinterval (see the coordination sentence above) keeps a healthy long-running cycle alive even across a single multi-minuteawait chat(...)— the timer fires during awaits, so no single slow call can silently expire the lock. (3) Progress wiring:progress?: ProgressReporteropt onExtractAtomsOptsandSynthesizeConceptsOpts; cycle.ts passes its phase-level reporter down (NOT a child reporter, which would collide oncycle.extract_atoms.extract_atoms.work); phases only calltick()/heartbeat(), cycle.ts ownsstart()/finish(). (4)by-mentionresume:mentionsFingerprint({source, type, since, gazetteerHash})insrc/core/op-checkpoint.ts— the gazetteer hash is load-bearing (adding entity pages mid-pause shifts the hash → new fingerprint → fresh scan against the new gazetteer, never silent skip); the gazetteer itself is entity-page titles PLUS live-verifiedpage_aliasesentries joined to entity-typed pages (ambiguous aliases and alias-vs-title collisions within a source are skipped), so body mentions of a documented alias link too;gbrain extract links --by-mentionresumes viaop_checkpointswithflushAndCheckpointordering (links flush to DB FIRST, page keys commit to checkpoint SECOND, persist THIRD, so a crash mid-batch leaves the page un-checkpointed and resume re-scans it); persist every 1000 items OR 30s; clean exit clears the checkpoint;--dry-runskips both load and write. (5)sync_consolidationdoctor check (multi-source brains see a paste-readygbrain sync --all --parallel 4 --workers 4 --skip-failed; single-source "not applicable"; SQL errors returnwarnvia the check's own try/catch). (6) Test-isolation:test/cycle-last-full-cycle-at.test.ts+test/schema-cli.test.tsuse per-testGBRAIN_HOME=tempdir. Pinned bytest/cycle/extract-atoms-batch.test.ts,test/cycle/cycle-lock-ttl.test.ts(pinsLOCK_TTL_MINUTES === 5),test/op-checkpoint-mentions-fingerprint.test.ts,test/cycle/extract-atoms-progress.test.ts,test/cycle/synthesize-concepts-progress.test.ts,test/cycle/yield-during-phase-refresh.test.ts,test/cycle/yield-during-phase-throttle.test.ts,test/extract-by-mention-resume.test.ts,test/doctor-sync-consolidation.test.ts. Companionsync --allrecipe block inskills/cron-scheduler/SKILL.md.synthesize_conceptswrites concept pages throughimportFromContent(the same parse→chunk→embed pipeline put_page uses, with put_page'sisAvailable('embedding')→noEmbedgate) soconcepts/pages carrycontent_chunks+ embeddings and are reachable by retrieval (wheresource-boost.tsweights them 1.3×).purgephase (soft-delete TTLs) also GCs staleop_checkpointsrows older than 7 days. Non-fatal on pre-v67 brains (DROP-target-table check before DELETE). The cycle threads its abort signal into the embed phase (runPhaseEmbed(engine, dryRun, signal)) so a timed-out cycle's long embed phase honors cancellation and releasesgbrain_cycle_locksright away instead of after a full backlog run. -
src/core/cycle/phase-scope.ts— the phase-scope taxonomy:PHASE_SCOPE: Record<CyclePhase, 'source'|'mixed'|'global'>maps each of the 23 phases (source: safe to parallelize per source; global: must serialize across the brain; mixed: brain-wide read + page write, so it stays in the default/global-maintenance lane until decomposed), plusSOURCE_FRESHNESS_PHASES— the deterministic, non-LLM subset (lint, backlinks, sync, extract, extract_facts, recompute_emotional_weight) that alone defines source freshness, so LLM-backed enrichment can never hold a freshness stamp hostage. cycle.ts re-exports both and derives the scheduling lists from them (see the cycle.ts entry). Consumed byrunCycle'sresolveCyclePhasesboundary, the autopilot fanout's per-source enqueue (src/commands/autopilot-fanout.ts), the jobs global-maintenance handler, and doctor's routing-federation phase-scope surface (src/commands/doctor/checks/routing-federation.ts). Pinned bytest/autopilot-global-maintenance.test.ts(SOURCE ∪ MIXED ∪ GLOBAL == ALL_PHASES with no overlap; FRESHNESS ∪ BACKGROUND == SOURCE; resolveCyclePhases boundary semantics). -
src/core/cycle/synthesize.ts— Synthesize phase: conversation-transcript-to-brain pipeline, a two-stage cascade where cheap scored triage gates frontier synthesis. Readsdream.synthesize.session_corpus_dir, runsrunTriagePass(exported; bounded pooldream.triage.concurrencydefault 4, wall-clock cache-MISS budgetdream.triage.max_msdefault 5 min — cache hits are free and deferred files reportdeferred: true, never cached, so the next pass continues) whose judgejudgeSignificanceemits{score 0-1 ordinal salience, content_type, segments ≤8 verbatim quotes, entities ≤12, reasons}(non-overlapping bands LOW 0-0.29 / MEDIUM 0.30-0.69 / HIGH 0.70-1.0; head 50%/middle 20%/tail 30% sample withindream.triage.max_charsdefault 24K viasafeSplitIndex; out-of-[0,1] scores areunparseable, never clamped) cached indream_verdictswith the judgingmodel+TRIAGE_VERSION— cache validity requires BOTH to match (switchingmodels.dream.triagere-judges;max_chars/max_tokensdeliberately excluded from validity —dream retriage --forcere-judges under new sampling knobs). Degenerate verdicts (truncated/refusal/unparseable) are never cached. THE gate ispassesTriageGate(triage-rescue.ts):score >= dream.triage.threshold(default 0.5) OR the verified-segment rescue, applied at report construction insiderunTriagePassat READ time — retuning the threshold or rescue knobs re-gates with zero re-judging; the storedworth_processingboolean derives from the fixedDEFAULT_TRIAGE_THRESHOLDconstant (back-compat only, never the live dial). Passing files fan out one subagent per chunk withmax_turnsfromdream.synthesize.max_turns(default 16) and a bounded advisorybuildTriageMapBlock(exported; score/type/entities + chunk-filtered segments, '' for legacy/degraded verdicts so the prompt is unchanged for those) spliced intobuildSynthesisPrompt, withallowed_slug_prefixes(sourced fromskills/_brain-filing-rules.jsondream_synthesize_paths.globs; whendream.synthesize.output_rootis set,loadAllowedSlugPrefixes(outputRoot)remaps thewiki/-rooted globs to the configured namespace, and the same root drives the prompt slug templates; default 'wiki', validated against the slug grammar via the exportedloadOutputRoot). The phase is source-scoped: cycle.ts threadscycleSourceIdasopts.sourceId→ each child'sSubagentHandlerData.source_id→ the subagent tool registry'sOperationContext.sourceId, so put_page writes, collected refs, the summary page, and reverse-writes all target the cycle's resolved source ('default' when unscoped; reverse-writes for the cycle's own source land atbrainDir/<slug>.md, foreign sources underbrainDir/.sources/<id>/). Orchestrator collects slugs fromsubagent_tool_executions(NOTpages.updated_at) and reverse-renders DB → markdown viaserializeMarkdown. Cooldown viadream.synthesize.last_completion_ts, written ONLY on success. Idempotency keysdream:synth-v2:<enc source>:filename:<enc basename>:<hash16>[:c<i>of<n>](byte-stable, pinned bytest/e2e/dream-synthesize-chunking.test.ts; grammar parsed by exportedparseSynthV2Key). Fan-out self-heals idempotency-coalesced rows strandedwaitingin a FOREIGN deaddream-inline-*queue (cancel releases the key slot → re-add into the live run's queue) instead of burning the 35-min wait on a row no worker will ever claim. Opt-in per-source daily submission capdream.synthesize.max_submissions_per_source_per_day(default 0 = off; skips whole files — never partial chunk sets; bypassed for explicit--input/--date/--from/--totargets; count-query failure fails OPEN with a stderr warn).--dry-runruns triage, skips synthesis; deferral-aware headlines append "(N not yet triaged — time budget...)" so a time-boxed cold pass never reads as mass rejection.details.triage(threshold/judged/cache_hits/unreliable/degraded/deferred/below_threshold + rescue_band/rescue_checked/rescue_fired + tokens_in/tokens_out/cost_usd) +details.synthesis(jobs/avg_turns/max_turns_config + quote_verify/spend/children_zero_pages) carry the phase telemetry. Subagent never gets fs-write access.renderPageToMarkdown(exported) stampsdream_generated: true+dream_cycle_dateinto every reverse-write's frontmatter;writeSummaryPagedoes the same on the summary index — this marker is the explicit identity surfaceisDreamOutputchecks intranscript-discovery.ts.stampDreamProvenanceadditionally persists the same marker into thepages.frontmatterJSONB row (merge viaexecuteRawJsonb, raw object bound to$N::jsonb) for every child-written page BEFORE reverse-rendering, so generated pages are DB-queryable and a later put_page write-through (which re-renders from the DB row) can't erase the stamp.judgeSignificance,JudgeClient,runTriagePass,buildTriageMapBlock,parseSynthV2Key,loadSynthConfig,TRIAGE_VERSION, andDEFAULT_TRIAGE_THRESHOLDare exported; the triage model resolves via an explicit pre-read ofmodels.dream.triage(preferred; through exportedresolveAlias) falling back to the standardresolveModelchain (models.dream.synthesize_verdict→ deprecateddream.synthesize.verdict_model→ tierutility).splitTranscriptByBudget(content, contentHash, maxChars)splits oversized transcripts at paragraph boundaries (## Topic:→---→\nladder) using a deterministic offset seeded from the first 32 bits ofcontentHashso retries chunk identically; per-chunk char budget =MODEL_CONTEXT_TOKENS[resolvedModel] × 0.9 × 3.5 chars/token(non-Anthropic ids fall back to a 180K-token safe default + once-per-process stderr warn); operator overridesdream.synthesize.max_prompt_tokens(floor 100K, wins) anddream.synthesize.max_chunks_per_transcript(default 24); per-chunk subagent job/wait timeouts aredream.synthesize.subagent_timeout_ms/dream.synthesize.subagent_wait_timeout_ms(defaults 30/35 min). Legacydream:synth:keys are never produced —loadSuccessfulSynthesisKeys(engine, sourceId, keyPrefix)reads the completed rows of BOTH key families once per phase, so existing brains skip withalready_synthesized_legacy_single_chunk/_chunkedinstead of re-spending the synthesis model, and a transcript whose synth-v2 children already completed skips withalready_synthesized_v2_single_chunk/_chunked(findSynthV2Completion; a cancelled row never counts) BEFORE its link manifest is built — otherwise queue.add's idempotency fast path coalesces onto the completed children and they re-enter writtenRefs, so their old pages get quote-repaired, restamped, reverse-written and re-embedded every night.collectChildPutPageSlugsraw-fetches every (job_id, slug) pair (notSELECT DISTINCT) and rewrites bare-hash6 slugs to<hash6>-c<idx>for chunked children (orchestrator-side, zero Sonnet trust). Cap-hit skips write nothing new todream_verdicts(the pass's cached triage verdict remains — a free cache hit on retry) and record no synthesis job, so raising the cap on next run re-attempts cleanly. Bounds INITIAL prompt size only; tool-loop turn-N accumulation is caught by terminal-error classification insubagent.ts. Verdict routing is gateway-routed:makeJudgeClient(verdictModel)(exported) mirrorstryBuildGatewayClientinsrc/core/think/index.ts— a construction-time provider/key probe returnsnullon a clear miss (unknown provider id viaresolveRecipeAIConfigError, or Anthropic provider with no key viahasAnthropicKey()). The verdict loop wrapsjudgeSignificancein try/catch forAIConfigErrorso mid-run provider failures surface as per-transcriptworth=false, reasons=['gateway error: ...']instead of crashing the phase. Canonical config keymodels.dream.synthesize_verdict(perPER_TASK_KEYSinsrc/core/model-config.ts);JudgeClientsignature preserved verbatim for test-seam stability; CI guardscripts/check-gateway-routed-no-direct-anthropic.shprevents reintroducingnew Anthropic()here or inthink/index.ts. At the queue.add boundary a conditionalanthropic:prefix is applied ONLY when the resolved model has no colon AND starts withclaude-(becauseresolveModelreturns bare ids fromTIER_DEFAULTS/DEFAULT_ALIASESand the subagent validator requiresprovider:modelform) — avoids changing the shared constants which would ripple across everyresolveModelcaller. Pinned bytest/cycle/synthesize-gateway-adapter.test.ts,test/e2e/dream-synthesize-pglite.test.ts(gateway-adapter mid-run AIConfigError catch),test/cycle/regression-pr-wave-r1-r2-r4.test.ts. Drain-loop lock renewal runs through exportedrunDrainRenewalTick(per-call AbortSignal + timeout + re-entrancy guard) — a hung renewLock cannot stack one checked-out slot per interval firing. Pinned bytest/cycle-drain-renewal.test.ts. Execution mode:dream.synthesize.mode(defaultoneshot) threads to every child asdata.modealongsideoneshot_slug_suffix(the structural suffix contract) andrequire_writes: true;details.synthesisadds mode/oneshot_jobs/fallback_jobs/agentic_jobs/fallback_reasons plus the drain telemetry (inline_concurrency_config/effective, drain_ms, queue-wait + runtime p50/p95, dead_jobs, degraded). Pre-retrieval LINK CANDIDATES manifest (link-manifest.ts,dream.synthesize.link_manifestdefault on) is built once per transcript from the cached triage entities/segment notes and spliced intobuildSynthesisPrompttogether with an ALLOWED WRITE PATHS block rendered from the trusted allow-list (the oneshot path never sees a tool schema, so the fence lives in the prompt; rule 2 points at the candidates first). Phase outcome gate: ALL children dead/cancelled → phase fails with SYNTH_ALL_CHILDREN_DEAD (fan-out details preserved on the failure); ANY non-completed child → the cooldown stamp is skipped so released idempotency keys retry next run; 'timeout' (parent stopped waiting) degrades but never triggers the all-dead error. Phase-end deferred-embed closure: whenever the phase wrote pages (regardless of mode — an agentic revert still sweeps debt left by earlier oneshot runs), a 120s-boundedembedStalePages(embed-stale.ts) embeds the NULL-embedding chunks of exactly the pages this phase wrote — never a source-wide sweep; the pre-existing stale backlog stays with the budget-tracked embed-backfill machinery — closing the gap on invocation shapes that never reach the global embed phase (--phase synthesize, autopilot per-source NON_GLOBAL_PHASES). Best-effort (never fails a phase that wrote its pages) and stamps the current embedding signature on fully-re-embedded pages so they stay inside the model-drift invalidation contract. The inline drain lives insrc/core/cycle/inline-drain.ts(re-exported here for patterns.ts +__testing). Budget clamping: clamped to the remaining parent-job budget whenopts.deadlineAtMsis threaded (via patterns.ts'sclampSubagentBudgetstemplate): the clamp re-runs against the live clock PER SUBMIT (the phase is a fan-out with claim-time-anchored child kill switches, so one phase-start clamp only bounds the first child); under the minimum child budget the phase skips honestly (insufficient_cycle_budget) or defers the remaining transcripts (details.budget_deferred_transcripts), the inline drain stops CLAIMING when the budget can't fit another child and unclaimed children are cancelled + deferred (they'd otherwise strand forever in the run's private queue), each completion wait is additionally bounded by the remaining parent budget (floored at 1s so already-terminal children still resolve), and a deferring run neither writes the 12h cooldown timestamp nor counts deferred transcripts as processed — deferral genuinely retries next cycle. THE gate ispassesTriageGate(triage-rescue.ts) applied at report construction insiderunTriagePass: threshold pass OR the verified-segment rescue for band scores[dream.triage.rescue_floor (0.30), threshold)with content_type indream.triage.rescue_content_typesand ≥dream.triage.rescue_min_segments(default 2; 0 = off) of the judge's segments verifying as normalized transcript substrings (≥40 chars, deduped) — reports carryrescued/verified_segments,details.triageadds rescue_band/rescue_checked/rescue_fired + judge tokens_in/tokens_out/cost_usd (null when unpriced), and dream-retriage's reconcile/audit read the SAME predicate. TRIAGE_VERSION is 2 (peak-not-average rubric; segment selection prefers concrete facts/decisions). After slug collection and BEFORE the provenance stamp/reverse-write/embed sweep, the quote verify/repair pass (synthesize-verify.ts,dream.synthesize.quote_verifydefault on) runs on newly-created pages; the phase is wrapped inwithChatPhase('phase:synthesize')(children keep their own job tag — minion_jobs stays the child-spend authority) anddetails.synthesisaddsquote_verify,children_zero_pages(completed children with zero put_page writes — the rule-D disposition), andspend(cost_basis: 'in+out+cache_read'; children summed from minion_jobs tokens_input/output/cache_read priced at the configured synth model, triage from the pass usage; total_usd null unless both price)buildSynthesisPromptis mode-aware (mode: 'agentic' | 'oneshot', passed fromconfig.modeat the sole submit site): agentic children keep the search-tool and final-summary guidance; tool-less oneshot children receive neither, so the prompt never contradictsONESHOT_SYSTEM's JSON-only rule (the oneshot prompt is stored asdata.promptand reused verbatim by the in-job agentic fallback, whose tools stay in the schema). Pinned bytest/cycle-dream-output-root.test.ts. -
src/core/cycle/cycle-date.ts— the dream-cycle calendar-date policy:resolveCycleDateresolves explicit--date>cycle.timezoneconfig > host IANA timezone > UTC, so a run after local midnight buckets into the day the user actually lived instead of rewriting yesterday's UTC-day summary.isValidTimeZonegates configured values (config.ts also validates atgbrain config settime); an invalid configured timezone falls back loudly instead of killing the cycle.utcDateis retained for source-date fallbacks and legacy pages. -
src/core/cycle/extract-atoms.ts— the extract_atoms lens phase: mines eligible pages intoatoms/<date>/<stem>-<hash>pages. Eligibility isCOALESCE(frontmatter->>'atoms_scan_hash','') <> substring(content_hash from 1 for 16); the completion marker (ATOMS_SCAN_HASH_KEYin utils.ts) is EXCLUDED fromcontentHash()'s input so stamping it can't re-arm the very page it marks (and import-file.ts strips it from untrustedremote===truefrontmatter so a remote writer can't suppress mining). Atom identity: page-derived atoms fold the source-page slug into the slug hash (8 chars, NUL-separated) so two same-date pages emitting the same title get DISTINCT slugs; the source CONTENT hash is deliberately NOT folded (a reworded source still upserts rather than duping); transcript atoms keep the legacy title-only 6-char hash.resolvePageAtomSlugadopts a legacy-slug atom with a compatible binding in place on re-extraction (upgrade idempotency — no migration, no duplicate), whileassertAtomImportBindingfail-closed refuses to reuse a slug bound to a DIFFERENT source locator. Both shareisCompatibleAtomBinding(frontmatter, sourcePageSlug)— the ONE definition of compatible: bound to THIS source page, or carrying nosource_slug/source_pathat all (pre-binding-era adoption, not a clobber); asource_path-bound legacy transcript atom or a differentsource_slugis a different origin, and a non-atom page squatting on the slug is refused.source_quoteis verified at extraction time against the exacttruncateUtf8prompt prefix the model saw (locateQuoteadvances by full code points): located quotes persist original characters + [start,end) offsets +source_quote_verified; unlocatable quotes are dropped with the atom kept. Edges write before the completion flip; per-item drain failures stay typed (bounded, sanitized, reconcilable). Transcript items carry the same failure-count/tombstone machinery as pages, in theextract_atoms_transcript_statetable (migrationextract_atoms_transcript_state_table; keyed(source_id, file_path, content_hash)with the 16-char hash prefix, so an edit is a new row and re-eligibilizes):recordItemFailureCountcounts a transcript's malformed output towardMAX_DETERMINISTIC_FAILURES, a zero-yield transcript tombstones immediately (the pageatoms_scan_hashsemantics), andtombstonedTranscriptsForHashesis the batch read-side gate mirroringatomsExistingForHashes(fail-open on error). Reported astombstoned_transcripts(paths), separate from the page-slugtombstoned_for_failures. -
src/core/cycle/extract-atoms-cost-gate.ts— pure cost-gate decision for the extract_atomsBudgetTracker.resolveExtractAtomsCostGate(extractModel, embedModel, overrides?, {explicitBudget?})returns{enforceCap, unpricedModel?, unpricedKind?, zeroPricedEmbedModel?, pricingOverrides?}: the cap is enforceable only when EVERY model billed under the phase's tracker is priceable — the extraction chat model AND the embedding model the atom import (importFromContent, inside the samewithBudgetTrackerscope) calls — with operatorpricing.overridesconsulted first, matchingBudgetTracker.reserve().resolveEmbedModelForCostGate()mirrors the write site'sisAvailable('embedding')gate (null when embedding is unavailable, so the import runsnoEmbedand nothing embeds). Only a DEFAULT cap may be dropped that way: when the operator SETcycle.extract_atoms.budget_usd(explicitBudget), an unpriced EMBED route keeps the cap and is priced at $0 via the returnedpricingOverrides(the caller's map plus the $0 row); an unpriced CHAT model drops the cap either way. The phase passes the decision intomaxCostUsdand hands the gate's overrides (or its own) to the tracker; it warns once either way (unpriced: kind + model + thepricing.overridesremedy, running uncapped rather than latchingbudget_exhaustedon the first item; zero-priced: the embed model + the rate remedy). Pinned bytest/extract-atoms-embed-cost-gate.test.ts(pure cases + a PGLite round-trip with a $0 chat model and an unpriced embed route). -
src/core/cycle/phases/consolidate.ts— the consolidate phase: clusters facts per page and promotes the best claim into a take. Take identity for the upsert lookup is(page_id, claim)restricted to rows this phase authors (kind='fact' AND holder='self') —since_dateis NOT identity (it derives from MIN(valid_from) of the cluster and moves whenever extract_facts re-inserts, which degraded the lookup into duplicate INSERTs); deliberately NOT filtered onactive(a superseded take still owns its claim — skipping it would resurrect a retired claim via the INSERT path);ORDER BY idkeeps the pick deterministic across pre-existing duplicate rows. A resolved take is immutable — its id is reused for consolidation but the row is left untouched. -
src/core/cycle/inline-drain.ts— the dream cycle's private-queue drain.runSubagentsInline(engine, queue, queueName, yieldDuringPhase?, handler?, lockMs?, concurrency?)runs N independentdrainLoops (dream.synthesize.inline_concurrency, clamp [1,8], PGLite forced serial at the callsite) sharing the queue via the same SKIP-LOCKED claim fencing workers use — everything job-scoped (lockToken, abort, timeout timer, keepalive, renewTimer, outcome recording) stays loop-local. Pool hygiene: global housekeeping sweeps are leader-only (loop 0) whilepromoteDelayedruns in every loop; idle claim-poll backs off 1s→5s; a NON-retryable loop-level failure aborts siblings after their current child (a failing child is a job outcome, never a drain crash). Exit only when the queue holds no active/waiting/delayed child (a lease-full bounce's delayed backoff must not strand a child). Outcome routing mirrors the worker:UnrecoverableError→ dead immediately;RateLeaseUnavailableError→releaseLeaseFullJobrequeue without burning an attempt; timeout terminal. Handler invocation is wrapped inwithChatPhase('job:<name>')(worker.ts parity) so each drained child's gateway spend is attributed to the CHILD — a bareawait handler(context)inherits the caller's AsyncLocalStorage phase, and a cycle phase that wraps its own work (dream synthesize wrapsphase:synthesize) would absorb every child's spend into the phase tag.runDrainRenewalTick(per-call AbortSignal + timeout + re-entrancy guard) and the null-safe nearest-rankpercentile()telemetry helper are exported. Pinned bytest/cycle-synthesize-inline-concurrency.test.ts(loop semantics) +test/e2e/dream-synthesize-concurrency-postgres.test.ts(real-Postgres exact-once + lease-ceiling). -
src/core/cycle/link-manifest.ts— pre-retrieval LINK CANDIDATES manifest.buildManifestContext(engine, sourceId?)snapshots the source's slugs + basename index once per phase;buildLinkManifest(engine, ctx, verdict, basename, {outputRoot, sourceId})resolves wikilink candidates ZERO-EMBED from the triage verdict's cached entities/segment notes (basename-index exact/slugified matches first, boundedsearchKeywordFTS second), excludes dream-output prefixes (self-consumption guard), renders- [[slug]] — <deterministic first-two-sentence one-liner>under hard caps (20 pages / 2400 chars) with buildTriageMapBlock's injection posture (whitespace-collapse + length caps + a 'data, not instructions' header). Best-effort everywhere — any failure degrades to the manifest-less prompt. Pinned bytest/cycle-link-manifest.test.ts. -
src/core/cycle/synthesize-verify.ts— Mechanical quote verify/repair on newly-created dream pages (zero LLM). Exports the shared grounding primitivenormalizeForGrounding(whitespace/curly-quote/dash/case folding WITH an offset map back to the original string, so every replacement is a verbatim transcript slice) +normForGrounding(plain form, also used bybuildTriageMapBlockand triage-rescue).verifyAndRepairDreamPages(engine, writtenRefs, transcriptsByPath)scopes to pages whose slug carries the transcript's content-hash suffix (page↔transcript binding; modified pre-existing people/pattern pages are skipped + countedskipped_preexisting— they may quote other sources), extracts paired-quote spans from the page BODY (code fences/inline code/wikilinks/link targets masked; paragraph-paired; odd-mark paragraphs skipped + countedunbalanced; min span 15 chars, ≤200 spans/page), and runs the repair ladder per span: exact substring → keep; normalized match → replace with the verbatim original slice; near match (word-trigram anchors, ≥0.8 token overlap, ambiguity within 0.05 falls through) → replace; else STRIP the quote marks keeping the text — NEVER fabricates, never deletes content. Warn-only counter: numeric/date claims ($amounts, percents, ISO/month dates, 4+ digit numbers, deduped, fences skipped) absent from the normalized transcript →numeric_claim_warns. Write-back ONLY when a span changed, through the canonicalimportFromContentpipeline (page+tags+chunks+links in one transaction, content_hash recomputed,noEmbed— the phase-end sweep backfills; provenance nulls preserve the first-write record); bareengine.putPageis insufficient (pages row only — the embed sweep would embed stale chunk_text). Fail-open per page (read-back miss / write throw → count + continue; abort still unwinds). Kill switchdream.synthesize.quote_verify(default on). Telemetry shape =QuoteVerifyStatsindetails.synthesis.quote_verify. Pinned bytest/cycle-synthesize-verify.test.ts+ the write-path mini-eval harnesstest/cycle-write-path-mini-eval.test.ts. -
src/core/cycle/triage-rescue.ts— THE dream triage gate (passesTriageGate) + the verified-segment rescue (applyTriageRescue). Rescue fires only for scores in[floor, threshold)(floor inclusive; at/above threshold is the plain gate's job) with content_type in the buried-signal allowlist (default mixed/reflection/idea/strategy/people — never routine/technical) and ≥ minSegments of the judge's own segments verifying as normalized transcript substrings (normForGrounding, ≥40 chars, deduped by normalized quote — repetition is not more evidence). $0 — no LLM calls; fabricated segments cannot fire it; works on cached verdicts (gate-time only). Fail-closed on every malformed shape (null score, missing/short/non-string segments), never throws.minSegments: 0is the kill switch (gate degenerates to the plain threshold). ONE-GATE RULE:runTriagePass(report construction —worth/rescued/telemetry/dry-run), the synthesize fan-out, anddream retriage(reconcile-queue cancels +--audit-rejectssampling) all read this predicate — a second hand-rolledscore >= thresholdcheck is how an operator sweep cancels exactly the jobs the rescue admitted. Config:dream.triage.rescue_floor/rescue_min_segments/rescue_content_types. Pinned bytest/cycle-triage-rescue.test.ts+ the rescue suites intest/cycle-synthesize-triage.test.ts/test/dream-retriage.test.ts. -
scripts/check-gateway-routed-no-direct-anthropic.sh— CI guard that fails the build ifsrc/core/cycle/synthesize.tsorsrc/core/think/index.tsreintroduces a runtimenew Anthropic()constructor call or a value-shapedimport Anthropic from '@anthropic-ai/sdk'import. Type-only imports (import type Anthropic from '@anthropic-ai/sdk') stay allowed for adapter types; comment lines (//or*prefixes) are excluded so JSDoc doesn't false-fire. Mirrorsscripts/check-jsonb-pattern.sh. Wired intobun run verify. ExtendGUARDED_FILESwhen migrating another file off direct SDK construction. -
src/core/cycle/patterns.ts— Patterns phase: cross-session theme detection over reflections withindream.patterns.lookback_days(default 30). Names a pattern only when ≥dream.patterns.min_evidence(default 3) reflections support it. Reflection excerpts use the shared UTF-16-safe truncator; a raw.slice(0, 600)can split an emoji pair and make Postgres reject the subagent job's JSONB payload. Single Sonnet subagent; same allow-list path as synthesize (importsloadAllowedSlugPrefixes+loadOutputRootfrom synthesize.ts — the reflections lookup, prompt slug templates, and allow-list all honordream.synthesize.output_root, default 'wiki'). Subagent job/wait timeouts are config keysdream.patterns.subagent_timeout_ms/dream.patterns.subagent_wait_timeout_ms(defaults 30/35 min, mirroring thedream.synthesize.*pair). The phase status reflects the child outcome: non-completedoutcome with zero writes →fail(error codePATTERNS_CHILD_<OUTCOME>); non-completedwith partial writes →warn. Runs AFTERextractso the graph is fresh. The fan-out setsrequire_writes: trueso an all-writes-failed child dead-letters instead of reporting completed. Budget clamping: clamped to the remaining parent-job budget viaclampSubagentBudgetswhen the cycle threadsdeadlineAtMs— the clamp template synthesize.ts reuses;MIN_PATTERNS_SUBAGENT_BUDGET_MSgates an honest skip, andCYCLE_DEADLINE_RESERVE_MSis re-exported here from its base-phase.ts home -
src/core/cycle/extract-facts.ts— extract_facts cycle phase. Fence is canonical: per-page wipe (deleteFactsForPage) + reinsert fromparseFactsFence+extractFactsFromFenceText+engine.insertFacts. The per-page wipe passesexcludeSourcePrefixes: ['cli:']so conversation facts (written byextract-conversation-facts, on pages with NO## Factsfence to recreate them from) survive the reconcile instead of being deleted-with-nothing-to-reinsert. The destructive phase never inherits a failed sync's full-brain walk:slugs: [](a real incremental no-op) is distinguished fromslugs: undefined(full-walk intent) by presence, not length.runPhaseExtractFacts(cycle.ts) surfaces awarn(net_fact_deletion) when the reconcile deletes at leastNET_DELETION_WARN_FLOOR(50) more facts than it reinserts — the exact signature of a conversation-facts wipe. Empty-fence guard refuses when legacy rows (row_num IS NULL AND entity_slug IS NOT NULL) pend backfill (status: warn, hint:gbrain apply-migrations --yes). A phantom-redirect pre-pass runs AFTER the legacy-row guard, BEFORE the main reconcile loop: whenopts.brainDiris set,runPhantomRedirectPass(engine, brainDir, sourceId, dryRun)walks unprefixed-slug pages capped byGBRAIN_PHANTOM_REDIRECT_LIMIT(default 50). The pass returnstouched_canonicals— canonical slugs whose disk fence merged with phantom rows;runExtractFactsUNIONs them into the main reconcile slug set so canonical's DB facts derive from the merged fence in the same cycle (handles phantom-had-only-on-disk-fence).ExtractFactsResultcarries six phantom fields:phantomsScanned,phantomsRedirected,phantomsAmbiguous,phantomsSkippedDrift,phantomsLockBusy,phantomsMorePending. Three bubble toCycleReport.totals(phantoms_redirected,phantoms_ambiguous,phantoms_skipped_drift). -
src/core/facts-fence.ts— the## Factsfence primitives: parse (parseFactsFence), render (renderFactsTable), and strip (stripFactsFence({keepVisibility})— the remote-read privacy boundaryget_page/fetch_page/the chunker apply), plus the remote write-back merge, plus the ONE fence-placement rule:replaceOrInsertFactsFence(body, fenceBlock)replaces an existing fence in place or inserts a fresh## Factssection ABOVE the timeline sentinel (timelineSentinelOffset, #4756 — below it splitBody() files the fence into page.timeline where extract_facts refuses to reconcile it), EOF only when the page has no sentinel; every writer that materializes a fence (upsertFactRow, the phantom-redirect canonical append, the importer's hidden-row merge) routes through it, so no writer can re-grow its own EOF append.restoreHiddenFactRows(incoming, existing)is the row-level, visibility-aware mergeimport-file.tsapplies per fence-bearing column (compiled_truth AND timeline) when a remote caller writes back a page it read with hidden rows stripped — without it, a remote get→edit→put round-trip would silently drop every non-worldfact row. Rules: only non-worldrows of the existing fence are restoration candidates (a caller's deletion of a VISIBLE row stays honored — world rows are never restored); a hidden row whose rowNum is absent from the incoming fence is restored at its stable rowNum (cross-page#F<N>refs survive); a rowNum collision (a caller-authored addition landing on a hidden number the caller never saw) keeps the hidden row's number and renumbers the CALLER's row onto fresh appended numbers (upsertFactRow's append-only contract); same rowNum + same claim keeps the incoming version (idempotent full-content write-throughs); either side parsing with warnings returns null — re-rendering a fence that didn't fully parse would drop the caller's unparsed rows — andfactsGapWarningsurfaces exactly that residual loss. Pure and side-effect-free. Pinned bytest/facts-fence.test.ts+ the remote write-back describes intest/privacy-strip-and-forget.test.ts(row-level merge, world-only deletion honored, timeline-embedded fence round-trip). -
src/core/fence-shared.ts— shared pipe-table primitives for the## Takes(takes-fence.ts) and## Facts(facts-fence.ts) fences:parseRowCells,isSeparatorRow,stripStrikethrough,parseStringCell,escapeFenceCell.parseRowCellsis escape-aware:\|stays inside its cell and decodes back to a literal|(exact inverse ofescapeFenceCell), while any other backslash passes through verbatim so existing fence text (e.g. Windows paths) is byte-stable across a render → parse cycle. This matters because the fence is canonical and reconcile is wipe-and-reinsert — a pipe inside a cell shifting the column layout would corrupt every row behind it on the next reconcile. Pinned by the escape cases intest/facts-fence.test.ts+ the full render → parse → reconcile round-trip intest/e2e/facts-fence-reconcile-postgres.test.ts. -
src/core/entities/resolve.ts— Free-form entity name → canonical slug resolution.resolveEntitySlug(engine, source_id, raw): exact slug → alias-exact (an unambiguouspage_aliaseshit viaresolveAliases, verified against LIVE pages sincepage_aliaseshas no FK — a stale alias row can never point at a deleted page; fail-open on pre-v110 brains missing the table;ResolutionSourcereportsalias_exact) → unambiguous bare-name prefix expansion acrosspeople/<token>-%+companies/<token>-%→ high-specificity fuzzy match for multi-token input (pg_trgm @ 0.7 threshold) → deterministicslugifyholding fallback. Bare-name collisions never use popularity as confidence; shared-token company names below the threshold remain unresolved. Two helpers for the phantom-redirect pass:resolvePhantomCanonical(engine, sourceId, phantomSlug)SKIPS the exact-slug step (a phantom slug'alice'would exact-match itself and no-op the redirect); returns the canonical only when non-null AND contains/.findPrefixCandidates(engine, sourceId, token)is a standalone SQL query returning ALL candidates acrossPREFIX_EXPANSION_DIRS(hardcoded['people', 'companies']) viaslug LIKE ANY($N::text[])over patternsdir/token+dir/token-%, cap of 10 ordered byconnection_count DESC, slug ASC. Pinned bytest/entity-resolve.test.ts(explicit, unique, ambiguous-person, and shared-token-company cases) plustest/phantom-redirect.test.ts(resolvePhantomCanonical 3 cases + findPrefixCandidates 6 cases incl. multi-dir ambiguity and thepeople/aliceberg-doesn't-match-alicefalse-positive guard). -
src/core/cycle/phantom-redirect.ts— Phantom-redirect orchestrator. ExportsrunPhantomRedirectPass(engine, brainDir, sourceId, dryRun): Promise<PhantomPassResult>(per-cycle wrapper acquiring thegbrain-syncwriter lock once for the whole pass, 30s bounded retry, walks up toGBRAIN_PHANTOM_REDIRECT_LIMITunprefixed phantoms) +tryRedirectPhantom(engine, page, sourceId, brainDir, dryRun): Promise<RedirectResult>+stripFenceAndFrontmatterAndLeadingH1(pure body-shape gate helper — strips facts fence incl. preceding## Factsheading and the leading H1; zero residue = phantom). Handler order: body-shape gate →resolvePhantomCanonical(bypasses exact-self-match) →findPrefixCandidatesambiguity check →fenceDbDriftbi-directional check → dry-run early exit → materialize canonical viaserializeMarkdownif DB-only → append phantom fence rows to canonical's disk fence with(claim, valid_from)dedup-guard + row_num continuation →engine.refreshPageBodywith SHA-256 content_hash recomputed via the import-file shape →engine.migrateFactsToCanonical(lossless) →engine.rewriteLinks(DB FK rewrite; wiki-link text rewrite is a documented follow-up) →engine.softDeletePage+engine.deleteFactsForPage(phantom)+fs.unlinkSync(phantomPath).RedirectResult.canonicalpopulated on'redirected'(incl. dry-run preview) so the caller buildstouched_canonicals. Idempotent on re-run: phantom soft-deleted → predicate fails (deleted_at IS NULL); migrate UPDATE matches no rows; dedup-guard prevents double-append. -
src/core/facts/phantom-audit.ts— JSONL audit at${resolveAuditDir()}/phantoms-YYYY-Www.jsonl. Pattern copy ofsrc/core/audit-slug-fallback.ts(ISO-week rotation, honorsGBRAIN_AUDIT_DIR). ExportslogPhantomEvent(record)+readRecentPhantomEvents(days)+computePhantomAuditFilename(now?). Records every outcome:redirected | ambiguous | drift | no_canonical | not_phantom_has_residue | pass_skipped_lock_busy. Best-effort writes — stderr warn on failure, never throws. Separate file fromstub-guard-audit.ts(distinct consumer + lifecycle: stub-guard logs PREVENTIVE blocks; phantom-audit logs CLEANUP decisions, to be read by a futurephantoms_pendingdoctor check). -
src/core/cycle/emotional-weight.ts— Pure functioncomputeEmotionalWeight({tags, takes}, {highEmotionTags?, userHolder?}). Deterministic 0..1 score: tag-emotion boost (max 0.5, case-insensitive match againstHIGH_EMOTION_TAGSseed list), take density (0.1/take, capped at 0.3), take avg weight (0..0.1), user-holder ratio (0..0.1 over active takes; default holder'garry'). Total clamped to [0..1]. Anglocentric / personal-life-biased seed list intentional; override via configemotional_weight.high_tags(JSON array).userHolderoverridable viaemotional_weight.user_holder. -
src/core/cycle/anomaly.ts— Pure stats helpers forfind_anomalies.meanStddevreturns sample stddev (n-1 denominator) and (0,0) for empty input.computeAnomaliesFromBuckets(baseline, today, sigma, limit)takes densified daily-count buckets + today's counts per cohort, returnsAnomalyResult[]. Zero-stddev fallback: cohort fires whencount > mean + 1, withsigma_observed = count - meanas a finite sort proxy (no NaN). Brand-new cohorts (no baseline) havemean=0, stddev=0so the fallback fires at count >= 2. Sorted bysigma_observeddesc, toplimit(default 20).page_slugscapped at 50 per cohort. -
src/core/cycle/recompute-emotional-weight.ts— Cycle phase orchestrator. Two SQL round-trips:engine.batchLoadEmotionalInputs(slugs?)→computeEmotionalWeight(per-row pure function) →engine.setEmotionalWeightBatch(rows). Reads configemotional_weight.high_tags(JSON array, falls back to default seed list on parse error) andemotional_weight.user_holder. EmptyaffectedSlugsshort-circuits with zero-work success. dry-run reports the would-write count without touching the DB. Engine throw bubbles intostatus: 'fail'with codeRECOMPUTE_EMOTIONAL_WEIGHT_FAILso the cycle continues. -
src/core/transcripts.ts—listRecentTranscripts(engine, opts)library reused by both thegbrain transcripts recentCLI and theget_recent_transcriptsMCP op. Readsdream.synthesize.session_corpus_dir+dream.synthesize.meeting_transcripts_dirconfig (same asdiscoverTranscripts); walks.txtfiles withindays; applies theisDreamOutputguard fromtranscript-discovery.ts(skips dream-generated files); returns{path, date, mtime, length, summary}[]sorted newest-first. Summary mode (default true) = first non-empty line + ~250 trailing chars; full mode caps at 100KB/file. Missing/non-existent corpus dirs return[], not error. Trust gate lives in the op handler, not here: the op throwspermission_deniedforctx.remote === true; this is a trusted library function used by both the gated op and the local CLI. -
src/core/operations-descriptions.ts— Constants module for tool descriptions. Pinned viatest/operations-descriptions.test.ts. HousesGET_RECENT_SALIENCE_DESCRIPTION,FIND_ANOMALIES_DESCRIPTION,GET_RECENT_TRANSCRIPTS_DESCRIPTIONplusLIST_PAGES_DESCRIPTION,QUERY_DESCRIPTION,SEARCH_DESCRIPTION. Stable surface for the Tier-2 LLM routing eval — keeping them here keeps the test from binding to whatever is inoperations.tsat test-run time. -
src/core/cycle/transcript-discovery.ts— Pure filesystem walk for synthesize.discoverTranscripts(opts)filters.txtfiles by date range, min_chars, and word-boundary regexexcludePatterns(medicalmatches "medical advice" but NOT "comedical"; power users may pass full regex).readSingleTranscript(path)is thegbrain dream --input <file>ad-hoc path. Self-consumption guard:DREAM_OUTPUT_MARKER_RE(anchored at frontmatter open---\n, optional BOM + CRLF tolerance, scans first 2000 chars fordream_generated: truewith case-insensitive value and word boundary ontrue) drivesisDreamOutput(content, bypass=false). Both functions skip matching files and emit a[dream] skipped <basename>: dream_generated markerstderr log (no silent skips). AnexcludePatternshit is never logged per file (the matching files are sensitive by definition); instead both functions emit ONE[dream] excluded N transcript(s) matching exclude_patterns (<label>: n, ...)stderr line at the end of discovery, naming only the count and the configured pattern labels that fired, never a path, basename, or content.DEFAULT_EXCLUDE_PATTERNSis unchanged (medical,therapy).bypassGuard?: booleanonDiscoverOptsandreadSingleTranscript's opts disables the guard for the explicit--unsafe-bypass-dream-guardescape hatch only — never auto-applied for--input. -
src/commands/dream.ts—gbrain dreamCLI; thin alias overrunCycle. Flags:--dry-run,--json,--phase <name>,--pull,--dir <path>,--input <file>(ad-hoc transcript, implies--phase synthesize),--date YYYY-MM-DD,--from <d> --to <d>(backfill range),--unsafe-bypass-dream-guard(plumbed throughrunCycle.synthBypassDreamGuard→SynthesizePhaseOpts.bypassDreamGuard→discoverTranscripts({bypassGuard})/readSingleTranscript({bypassGuard}); loud stderr warning at synthesize-phase entry; never auto-applied for--input). Conflict detection:--input+--dateexits 2. ISO date validation.--dry-runruns the scored triage pass but skips synthesis (NOT zero LLM calls). Exit 1 on status=failed.resolveBrainDirreturnsstring | null(order:--dir→ resolved--source'slocal_path→ globalsync.repo_path→ null); a checkout-less postgres/Supabase brain runs DB-only phases (incl.resolve_symbol_edges) and skips the 6 filesystem phases withdetails.reason: 'no_brain_dir';runDreamowns the only hard error (no checkout AND no engine). When--sourceresolves but has no on-disk checkout, returns null (DB-only) rather than borrowing another source's globalsync.repo_path(would mix scopes). Pinned bytest/dream-postgres.serial.test.ts.--drain [--window <seconds>]for--phase extract_atoms:runDrain()bypasses the pack-gate and runs the single-hold bounded drain fromsrc/core/cycle/extract-atoms-drain.tsunder the samecycleLockIdFor(sourceId)the routine cycle uses (concurrent autopilot tick defers withcycle_already_running), reporting{extracted, skipped, remaining}. ExitsEXIT_DRAIN_INCOMPLETE=3whileremaining > 0; a null backlog count (count query FAILED) is also exit 3, never a drained success;LockUnavailableError→cycle_already_runningskip (also exit 3). Theextract_atoms_backlogdoctor check (computeExtractAtomsBacklogCheck) surfaces the silent pack-gated backlog with the exact--draincommand; pack-gated cycle skips carry a greppablepack_gated:truemarker.dream retriagedispatches onargs[0] === 'retriage'BEFOREparseArgs(its flag set never collides with cycle flags;dream retriage --helpprints subcommand help engine-free per the same IRON RULE). -
src/core/cycle/transcript-discovery.ts— Pure filesystem walk for synthesize.discoverTranscripts(opts)filters.txtfiles by date range, min_chars, and word-boundary regexexcludePatterns(medicalmatches "medical advice" but NOT "comedical"; power users may pass full regex).readSingleTranscript(path)is thegbrain dream --input <file>ad-hoc path. Self-consumption guard:DREAM_OUTPUT_MARKER_RE(anchored at frontmatter open---\n, optional BOM + CRLF tolerance, scans first 2000 chars fordream_generated: truewith case-insensitive value and word boundary ontrue) drivesisDreamOutput(content, bypass=false). Both functions skip matching files and emit a[dream] skipped <basename>: dream_generated markerstderr log (no silent skips).bypassGuard?: booleanonDiscoverOptsandreadSingleTranscript's opts disables the guard for the explicit--unsafe-bypass-dream-guardescape hatch only — never auto-applied for--input. -
src/commands/dream.ts—gbrain dreamCLI; thin alias overrunCycle. Flags:--dry-run,--json,--phase <name>,--pull,--dir <path>,--input <file>(ad-hoc transcript, implies--phase synthesize),--date YYYY-MM-DD,--from <d> --to <d>(backfill range),--unsafe-bypass-dream-guard(plumbed throughrunCycle.synthBypassDreamGuard→SynthesizePhaseOpts.bypassDreamGuard→discoverTranscripts({bypassGuard})/readSingleTranscript({bypassGuard}); loud stderr warning at synthesize-phase entry; never auto-applied for--input). Conflict detection:--input+--dateexits 2. ISO date validation.--dry-runruns the scored triage pass but skips synthesis (NOT zero LLM calls). Exit 1 on status=failed. Source scope enters the shared 6-tier resolver whenever--source <id>is given ORGBRAIN_SOURCEis set (a null explicit falls through to tier 2: validated +assertSourceExists, so an invalid/unknown env value is a clean exit 1; the__all__sentinel is excluded and falls through to the unscoped routing, which already spans every source); an env scope naming the brain's default-like source keeps the full implicit cycle (#4700 semantics), any other keeps the--sourcefreshness boundary.resolveBrainDirreturnsstring | null(order:--dir→ resolved source'slocal_path→ globalsync.repo_path→ null); a checkout-less postgres/Supabase brain runs DB-only phases (incl.resolve_symbol_edges) and skips the 6 filesystem phases withdetails.reason: 'no_brain_dir';runDreamowns the only hard error (no checkout AND no engine). When--sourceresolves but has no on-disk checkout, returns null (DB-only) rather than borrowing another source's globalsync.repo_path(would mix scopes). Pinned bytest/dream-postgres.serial.test.ts.--drain [--window <seconds>]for--phase extract_atoms:runDrain()bypasses the pack-gate and runs the single-hold bounded drain fromsrc/core/cycle/extract-atoms-drain.tsunder the samecycleLockIdFor(sourceId)the routine cycle uses (concurrent autopilot tick defers withcycle_already_running), reporting{extracted, skipped, remaining}. ExitsEXIT_DRAIN_INCOMPLETE=3whileremaining > 0; a null backlog count (count query FAILED) is also exit 3, never a drained success;LockUnavailableError→cycle_already_runningskip (also exit 3). Theextract_atoms_backlogdoctor check (computeExtractAtomsBacklogCheck) surfaces the silent pack-gated backlog with the exact--draincommand; pack-gated cycle skips carry a greppablepack_gated:truemarker.dream retriagedispatches onargs[0] === 'retriage'BEFOREparseArgs(its flag set never collides with cycle flags;dream retriage --helpprints subcommand help engine-free per the same IRON RULE). -
src/commands/dream-retriage.ts—gbrain dream retriage: re-scores the corpus via the sharedrunTriagePass(withmaxMs: 0— operator sweeps run to completion;--limitslices the discovered list caller-side) and reconciles the queued private-queue backlog: synth-v2 rows verdict-gated, plus any row stranded in a provably-deaddream-inline-*queue regardless of key family (patterns children, legacy grammars) — doctor's orphaned_private_queue check points here, so the repair selects everything the check can flag. Liveness uses ownership correlation (mirrors the doctor check): a livegbrain_cycle_locksrow suppresses conversion only for queues born at/after itsacquired_at; an older crashed cycle's queue stays repairable while a new cycle runs (unknown birth/acquisition stays fail-safe possibly-live). Spend-gated: upfront estimate viacanonicalLookupon the resolved triage model, confirmation aboveSPEND_CONFIRM_USD($5, indream-retriage-constants.ts; unpriced models gate onUNPRICED_CONFIRM_FILES=500) unless--yes(--jsonnon-interactive requires--yesabove the gate);--max-usdsoft-stops via the pass'sshouldStopseam (estimate-based).--reconcile-queue(opt-in — cancels queued work): selects waiting/delayed/pauseddream:synth-v2:%jobs across ALL queues, parses keys withparseSynthV2Key, then per row, reading THE shared gate (passesTriageGatefromtriage-rescue.ts— threshold pass OR the verified-segment rescue; a hand-rolledscore >= thresholdhere would cancel exactly the jobs the rescue admitted): matched below the gate → cancel; matched above the gate butwaitingin a staledream-inline-*queue → cancel asconverted_for_resubmit(the backlog conversion — cancelled rows release their idempotency slot so the next cycle re-adds into a live drain); matched above the gate in a live queue → keep; matched-but-unscored → keep (never cancel on missing data); unmatched → keep unless--cancel-unmatched; key-source vsdata.source_iddisagreement → skip +source_mismatch; status re-checked immediately before each cancel (rows turnedactiveare skipped; residual race matches cancelJob's best-effort contract). Legacydream:synth:keys are excluded at the SQL LIKE filter — never candidates.--sourcescopes cancels (other_sourcecounted);--threshold/--since/--force/--dry-run(zero judge calls, zero cancels — cached scores only, uncached files reportneeds_triage).--audit-rejects <n>re-judges N stride-sampled files that the gate actually REJECTED (below threshold AND not rescued — auditing a rescued file as a rejection would misreport the disagreement rate) with the SYNTHESIS model and reports that rate — the operator calibration loop. Reports carryrescued/verified_segmentsso a rescued file reads as accepted everywhere the command renders. Exit 0 success (even when nothing cancelled), 1 no engine/corpus, 2 usage or declined spend gate. Pinned bytest/dream-retriage.test.ts. -
src/commands/friction.ts+src/core/friction.ts—gbrain friction {log,render,list,summary,diff}reporter. Append-only JSONL under$GBRAIN_HOME/.gbrain/friction/<run-id>.jsonl. Schema is a flat extension ofStructuredAgentError; every claw-test run opens with aphase-marker/startmeta record carryingagent+scenario+harness_schema(agent-name resolution depends on it). Render groups by severity → phase, defaults to--redactfor md output (strips$HOME/$CWDto placeholders so reports paste safely in PRs).diff --base <run-or-agent> --compare <run-or-agent>is the cross-agent instrument: exact run-id wins, else agent name resolves to that agent's latest run; identity is(kind, phase, normalized 80-char message prefix — digit runs collapsed so durations/counts don't split identities)overkind ∈ {friction, delight}as MULTISETS (per-severity counts + totals are the compared attributes:count_changed= volume,severity_changed= distribution shape via exact integer proportion test, so a delight→friction flip or a 2×error+1×nit → 1×error+2×nit redistribution always surfaces; markers/interrupted feed the compatibility banner, which warns on scenario/version mismatch); output labels sections "unique to" — an instrument, never a blame-attributor. Run-id resolves from --run-id>$GBRAIN_FRICTION_RUN_ID>standalone.jsonl. Skills the claw-test exercises carry a_friction-protocol.mdcallout so agents know when to log friction. -
src/commands/claw-test.ts+src/core/claw-test/—gbrain claw-test [--scenario <name>] [--live --agent <name>]. End-to-end "fresh user" friction harness. Two modes: scripted (CI gate, agent-free) and live (real agent subprocess, $1–2 in tokens). SetsGBRAIN_HOME=<tempdir>for hermeticity and captures gbrain's--progress-jsonevents from each child's stderr to verify expected phases ran (import.files,extract.links_fs,doctor.db_checks). Scripted phases: setup → install_brain (gbrain init --pglite) → import (--no-embed) → query → extract → verify (gbrain doctor --json; top-level status ishealthy|warnings|unhealthy) → render. Live mode STAGES the scenario before the agent turn (fresh-install: brain pages + AGENTS.md stub + init; upgrade: seed-first viaseed-pglite.ts, NO init — the migration is the scenario under test), prepends a per-rungbrainPATH shim so the BRIEF's baregbrainruns this checkout, handsBRIEF.mdto the agent runner, then verifies a scenario-declared success ORACLE (oracle: {query, min_results, files_exist}in scenario.json; upgrade uses a non-mutating schema-version probe viareadPgliteSchemaVersion— doctor would auto-migrate and pass a do-nothing agent). Child-side friction merges into the parent's friction file before tempdir cleanup. Four runners ship (src/core/claw-test/runners/{openclaw,hermes,grok,opencode}.ts; shareddetectBinary/filterAllowlistEnvlive inagent-runner.ts): openclaw invokesopenclaw agent --local --agent <name> --message <brief>; hermes invokeshermes -z <brief>($HERMES_BIN>which hermes;HERMES_HOMEpassthrough is the env-allowlist delta; sharedBASE_ENV_ALLOWLIST+validateBinPathEnvlive inagent-runner.ts); grok (xAI Grok Build, observed shapes indocs/mcp/GROK-CLI-PIN.md) invokesgrok -p <brief> --output-format plain($GROK_BIN>which grok; deltaGROK_HOME+XAI_API_KEY; writes a version preamble into the transcript and warns loudly when the operator's~/.claude.jsonregisters gbrain — grok reads vendor MCP configs for trusted folders); opencode (SST, observed shapes indocs/mcp/OPENCODE-CLI-PIN.md) invokesopencode run <brief> --format default($OPENCODE_BIN>which opencode; delta = the EXPLICIT multi-provider keys XAI/Google/Gemini/OpenRouter — BASE carries only Anthropic+OpenAI — plus XDG dirs, OPENCODE_CONFIG(_DIR), and OPENCODE_DISABLE_AUTOUPDATE; OPENCODE_CONFIG_CONTENT deliberately absent, it is a config-shadow channel; no--auto— MCP tools fire without it; bare-semver version preamble is the SST-vs-claimant discriminator; warns loudly when the user-global opencode config carries mcp.gbrain). Live-lane posture: the OPERATOR's configured agent + hermetic brain; the fully hermetic lanes aretest/e2e/install-real-hermes.serial.test.ts,test/e2e/install-real-grok.serial.test.ts, andtest/e2e/install-real-opencode.serial.test.ts(split-gated a step past grok: opencode's anonymous free tier drives MCP tool calls keyless, so even the nonce SMOKE — with a STRUCTURALgbrain_*tool_use assert viaparseOpencodeJsonl— runs in the keyless tier; the paid anthropic leg self-validates its pinned model id against the authedopencode modelslist before any spend) (grok door is split-gated: keyless compat tier needs only the binary —mcp doctoris grok's honest discriminator, proving the seven-verb surface keyless; paid SMOKE additionally needsXAI_API_KEYand asserts a per-run nonce fact, never the committed one). Transcript capture (transcript-capture.ts) usesfs.createWriteStreamwith'drain'-event backpressure (so a 256KB burst cannot stall the child). Env knobs (harness escape hatches, all optional):GBRAIN_BIN_OVERRIDE(child gbrain binary; validated absolute/no-dotdot/no-metacharacter because it's interpolated into the PATH shim — under the bun runtime the harness otherwise synthesizes a launcher so children never exec bun itself),GBRAIN_CLAW_PHASE_TIMEOUT_MS(per-phase child wall clock, default 5 min),GBRAIN_CLAW_AGENT_TIMEOUT_MS(live agent turn wall clock, default 10 min). -
skills/_friction-protocol.md— shared cross-cutting convention skill (like_brain-filing-rules.md). Tells agents when to callgbrain friction logand how to choose a severity. Routes to friction CLI from any skill the claw-test exercises. -
scripts/check-progress-to-stdout.sh— CI guard against regressing to\r-on-stdout progress. Wired intobun run testviascripts/check-progress-to-stdout.sh && bun testin package.json. -
docs/progress-events.md— Canonical JSON event schema reference. Additive only. -
src/core/markdown.ts— Frontmatter parsing + body splitter.coerceFrontmatterString(v)coerces a non-stringtitle/slug/typeto a deterministic string at parse time so a YAML-typed value never reaches.toLowerCase()and throws (title: 2024-06-01parses as aDate,title: 1458as a number, and a throw would block the sync bookmark from advancing); aDatebecomes its UTC ISO date (2024-06-01, machine-independent and matching the on-disk token, unlikeString(date)),null/undefinedbecome'', everything else usesString().splitBodyrequires an explicit timeline sentinel (<!-- timeline -->,--- timeline ---, or---immediately before## Timeline/## History). Plain---in body text is a markdown horizontal rule, not a separator.inferTypeauto-types/wiki/analysis/→ analysis,/wiki/guides/→ guide,/wiki/hardware/→ hardware,/wiki/architecture/→ architecture,/writing/→ writing (plus existing people/companies/deals/etc heuristics).resolveSourceLocalFilePathmaps Git-root-relativepages.source_pathvalues into a source whoselocal_pathscopes a repo subdirectory; it finds the Git root with filesystem checks only, strips the exact root-relative scope, rejects unsafe/non-markdown paths, and when given a slug it tolerates historical basename-relativesource_pathrows by checking the slug directory only after the direct path is absent. It leaves write containment to the caller. -
scripts/check-jsonb-pattern.sh— CI grep guard. Fails the build if any source contains (a) the${JSON.stringify(x)}::jsonbinterpolation pattern (postgres.js v3 double-encodes it), or (b)max_stalled INTEGER NOT NULL DEFAULT 1in any schema source file (must be DEFAULT 5 to preserve SIGKILL-rescue). It also invokesscripts/check-jsonb-params.mjsand propagates its exit code. Wired intobun test. -
scripts/check-getpage-scoped-write.mjs— CI scanner for the unscoped-check/scoped-write source-isolation bug class: flags any non-test src file containing BOTH agetPage(call with no second argument (or theX ? {sourceId} : undefinedany-source-when-unset ternary) AND a write-path call (putPage(/importFromContent(/importFromFile(). Fix pattern:getPage(slug, { sourceId: x ?? 'default' })(mirror the write's schema default); opt-out markergbrain-allow-unscoped-getpage: <reason>for documented read-only first-match sites (span, preceding lines, or trailing same-line comment). Grandfathered allowlist is EMPTY. Comment/string-aware balanced-paren span walker (same skeleton as check-jsonb-params.mjs); argv-overridable roots; wired ascheck:getpage-scopein verify CHECKS + guards-manifest +test/fixtures/guards/check-getpage-scoped-write.mjs/{bad,good}/+test/check-getpage-scoped-write.test.ts. -
scripts/check-jsonb-params.mjs— AST-lite CI guard for the POSITIONAL jsonb double-encode form the template grep above misses: anexecuteRaw/executeRawDirect/.unsafe()call whose balanced arg span bindsJSON.stringify(x)into a bare$N::jsonbcast. Walks each call's balanced span respecting strings/templates/comments, handles generic-typed calls (executeRaw<T>(), and allows the sanctioned forms ($N::text::jsonb,$N::text[],executeRawJsonb,sql.json, an inlinejsonb-guard-okcomment). PGLite's nativedb.queryis deliberately not scanned (it parses text→jsonb, so the bug can't occur there). Heuristic by design (whole-span correlation; can't see aJSON.stringifyassigned to a variable before the call) — the real backstop is the DATABASE_URL-gated e2e parity tests. Scan roots overridable via argv for its self-test (test/check-jsonb-params.test.ts). -
scripts/check-source-id-projection.sh— CI grep guard for the multi-source bug class. Grepssrc/core/postgres-engine.ts+src/core/pglite-engine.tsforSELECT.*FROM pagesprojections matching therowToPagefeeder shape (id + slug + type + title) and fails ifsource_idis missing.Page.source_idis required at the type level; a projection dropping the column producesPagerows withsource_id: undefinedwhile TypeScript's: stringlies about it. Wired intobun run verify. -
scripts/guards-manifest.tsv+scripts/guard-self-test.sh— THE single registry ofscripts/check-*CI guards (52 guards) and its self-test harness. Every guard is classifiedscanner(greps/parses repo sources — must eventually carry fixtures),buildfresh, orrepostate(exempt-with-reason, not fixture-tested).guard-self-test.sh(bun run check:guard-self-test, wired intobun run verify) runs eachselftest=yesscanner against known-bad (must exit non-zero) and known-good (must pass) fixture trees undertest/fixtures/guards/<guard>/{bad,good}/via theGBRAIN_GUARD_ROOTenv seam, and fails the build when a newscripts/check-*script is missing from the manifest — so a guard whose pattern rots into a permanently-green no-op fails CI instead of masquerading as coverage. The manifest registers and classifies guards but does not itself schedule them —run-verify-parallel.sh'sCHECKSarray remains the execution list, and a registered guard is not automatically wired into verify. New guard = new manifest row (+ fixtures if scanner) + aCHECKSentry if it should gate pushes. -
scripts/merge-lcov.ts+scripts/coverage-diff-gate.ts+scripts/coverage-baseline-gate.ts+scripts/update-coverage-baseline.ts+scripts/render-coverage-summary.ts+scripts/coverage-gate-exemptions.txt+scripts/coverage-baseline.json— the coverage measurement + gating cluster; the operating guide is docs/TESTING.md "Coverage lanes and gates".merge-lcov.tswalks artifact dirs forlcov.info+lane-manifest.json, sums DA hits per file:line, normalizes paths repo-relative, and emits a merged lcov + summary JSON (src-only totals/per-dir/per-file, thelineHitsextension the diff gate consumes, and never-loaded src files as count + sorted list — deliberately never a percentage, since physical lines ≠ executable lines);--manifest-expectpins the lane set, and a missing/incomplete lane or ashardlane withlcovCount != 1(the xargs-batching tripwire) marks the summarydegraded: true— still exit 0 (degraded is data, and both gates go report-only on it).coverage-diff-gate.tsgates added/changed gate-scoped lines (non-test, non-generatedsrc/**.ts) at ≥80% covered plus zero changed-but-never-loaded files; report-only unlessCOVERAGE_GATE_ENFORCE=1; a[coverage-exempt: reason]commit trailer passes with a loud warning;coverage-gate-exemptions.txtrows (exact path or trailing-/prefix; SHRINK-ONLY — additions need a graduation review) are excluded from the gate but still reported ([e2e-exempt]/[subprocess-undercount]); exit contract: 0 = pass or report-only, 1 = fail while enforcing, 2 = infrastructure error (never conflated with a coverage verdict).coverage-baseline-gate.tsreads the baseline viagit show origin/master:scripts/coverage-baseline.json(never the working tree, so a PR can't weaken its own bar) and compares corpus-matched sections only (--corpus prCorpus|fullCorpus), failing on >0.5pp global or >1.0pp per-dir drops;provisional: truein the baseline (the current state — both corpus sections unseeded) keeps it report-only regardless of enforcement;update-coverage-baseline.tswrites the working-tree baseline (per-file detail limited to the committedwatchlist) and--promoteflipsprovisional: false.render-coverage-summary.tsrenders the summary JSON as markdown on stdout for$GITHUB_STEP_SUMMARY, including the behavioral-vs-structural counts fromscripts/structural-suites.tsv. Wiring: 14 PR-corpus lanes in test.yml (10 matrix shards + serial + the three dedicated slow jobs) uploadcoverage-*artifacts and the advisorycoverage-reportjob merges + renders + runs both gates report-only (deliberately absent fromtest-status/cache-writeuntil graduation); schedule-onlycoverage-full-{unit,serial,slow,e2e}+coverage-full-reportin e2e.yml produce the self-contained nightly fullCorpus number (full e2e glob included) and thecoverage-full-mergedtrend artifact. Collection isCOVERAGE_DIR-opt-in intest-shard.sh/run-serial-tests.sh/run-e2e.sh— unique coverage dir per bun process (a reused dir overwriteslcov.info), lane manifest written only on a green run,run-e2e.shrequires an ABSOLUTECOVERAGE_DIRand honorsE2E_FILE_TIMEOUT_SECS(both deliberately non-GBRAIN_-prefixed to survive the hermetic env scrub). Bun/JSC emits line records only (function coverage is informational) and no subprocess coverage, sosrc/cli.tsundercounts. Pinned bytest/scripts/merge-lcov.test.ts,test/scripts/coverage-diff-gate.test.ts,test/scripts/render-coverage-summary.test.ts. -
scripts/check-module-size.sh+scripts/module-size-limits.tsv— the module-size ratchet (bun run check:module-size, wired intobun run verify). The TSV commits a per-filewc -lceiling (path max_lines policy note); four rules, all violations reported before a single exit 1: a file above its ceiling fails (raise a ceiling only as a conscious TSV edit); a ceiling more than 50 lines above the measured size fails (stale slack after a shrink — lower it so the ratchet holds); a TSV row whose path no longer exists fails (remove the row); an unlistedsrc/**/*.ts(excluding*.generated.ts/*.test.ts) above the 1500-line new-file cap fails (split it or add a row). Policyregion-exempt(onlysrc/core/migrate.ts) counts lines OUTSIDE the append-onlyexport const MIGRATIONS = […];region, so the migrations array grows freely while the surrounding runner logic stays ratcheted. Self-test seams:GBRAIN_GUARD_ROOT,GBRAIN_MODULE_SIZE_SLACK,GBRAIN_MODULE_SIZE_NEWFILE_CAP. -
scripts/classify-tests.ts+scripts/structural-suites.tsv— suite-level behavioral-vs-structural test classification (the intent axis described in docs/TESTING.md "File taxonomy"). Content-based detectors — repo-anchoredreadFileSync/Bun.filereaders, exec-scan grep windows oversrc|scripts|docs, and thedoctorSource()/doctorFileSource()helpers — mark a suite STRUCTURAL when its assertions read repo source/doc text rather than executing product code; tmpdir-anchored reads don't count, and files with detectors but no attributable suite land in anunknownbucket emitted as comment rows (surfaced, never silently dropped). Modes: bare = rewrite the TSV;--check= byte-for-byte regenerate-and-diff freshness (wired asbun run check:structural-manifestinbun run verifyviascripts/check-structural-manifest.sh);--summary= counts only. Fix misclassifications in the detector list, never by hand-editing the TSV.render-coverage-summary.tsconsumes the TSV for the behavioral-vs-structural line in the CI coverage report. -
scripts/build-pglite-snapshot.ts—bun run build:pglite-snapshot: bakes a post-initSchema()PGLite data dir intotest/fixtures/pglite-snapshot.tar+ a version file (schema hash line, thendims=/model=lines recording the embedding shape it was baked with). Idempotent (hash short-circuit ~40ms when fresh; rebuilds stale) and concurrency-safe (atomicmkdirlock attest/fixtures/.pglite-snapshot.lockwith staleness-verified takeover — a live lock is never stolen; tar written first, version file last, so a crash can't leave a fresh-looking torn fixture; waiter bounded byGBRAIN_SNAPSHOT_LOCK_TIMEOUT_MS, default 120000; an exhausted waiter facing a still-live lock proceeds unlocked as a last resort — the loader's hash/shape gate validates the version file, not the tar bytes). Called through the sharedensure_pglite_snapshothelper inscripts/lib/test-env.sh(also home ofdetect_cpus+detect_available_mem_mb; sourced byrun-unit-parallel.sh,test-shard.sh,run-slow-tests.sh,run-serial-tests.sh,run-verify-parallel.sh, andrun-e2e.sh— default-on, opt outGBRAIN_NO_SNAPSHOT=1, no-op when a parent already exported the path, non-fatal on build failure with a one-line "active" echo so a silent cold-init fallback stays visible) and directly byscripts/ci-local.sh; every caller exportsGBRAIN_PGLITE_SNAPSHOT. The loader side istryLoadSnapshot+computeSnapshotSchemaHash(exported fromsrc/core/pglite-engine.ts): the coverage-immune hash reads raw file bytes for the schema/migration entry modules and imported helpers, including grant SQL/policy/repair dependencies and withdrawal triggers (keep its dependency list and CI cache keys aligned when adding a helper); any hash or embedding-shape mismatch warns once and falls through to normal cold init — the snapshot is an optimization, never authoritative. The loader memoizes per process: the schema hash computes once (source bytes are static for the process lifetime) and the version file + ~42MB tar are read once per (path, process) instead of once per engine construction (a full suite constructs 600+ engines); a terminally-unusable path (missing/stale/torn) memoizes as null and is never retried, the tar blob loads lazily only after the FIRST caller passes the shape gate, and the dims/model shape gate itself is deliberately NOT memoized (tests reconfigure the gateway mid-process — a mismatched engine must still fall back to cold init). Accepted limitation: a snapshot rewritten mid-process is not observed; the only writer runs before test fan-out. Test seams__snapshotMemoStatsForTests/__resetSnapshotMemoForTests. Pinned bytest/snapshot-shape-guard.test.ts. -
docker-compose.ci.yml+scripts/ci-local.sh— Local CI gate.bun run ci:localspins up fourpgvector/pgvector:pg16services (postgres-1..4) +oven/bun:1with named volumes (gbrain-ci-pg-data-{1..4},gbrain-ci-node-modules,gbrain-ci-bun-cache), runs gitleaks on host, smoke-testsscripts/run-e2e.shargv handling, runs guards + typecheck, then the Tier 1 default: 4-shard parallel unit + E2E (xargs -P4, one Postgres per shard; unit phase keepsDATABASE_URLunset).--no-shardfalls back to the legacy unsharded sequential flow (debug aid);--diffruns the diff-aware selector unsharded. Also runs apgbouncerservice (edoburu/pgbouncer,POOL_MODE: transaction,AUTH_TYPE: plain— pg16 stores SCRAM verifiers, so the userlist must hold the plaintext password;IGNORE_STARTUP_PARAMETERSwhitelists gbrain'sstatement_timeout/idle_in_transaction_session_timeoutstartup params the way the Supabase pooler does) fronting postgres-1 on host portGBRAIN_CI_PGBOUNCER_PORT(default 6543); every E2E invocation exportsGBRAIN_PGBOUNCER_URL(pooled; dedicatedgbrain_pgbouncerdatabase so it never races thegbrain_testTRUNCATE fixtures) +GBRAIN_PGBOUNCER_DIRECT_URL, consumed bytest/e2e/pgbouncer-teardown.test.ts— which reproduces the transaction-mode teardown failure in the local gate.--no-pullskips upstream pulls;--cleannukes named volumes. Postgres host port defaults to 5434; override withGBRAIN_CI_PG_PORT=NNNN. Stronger gate than PR CI's 2-file Tier 1 set. -
scripts/select-e2e.ts+scripts/e2e-test-map.ts— Diff-aware E2E test selector. Reads three git sources (committedorigin/master...HEAD, working-treeHEAD, andgit ls-files --others --exclude-standardfor untracked NOT-gitignored files), classifies as EMPTY / DOC_ONLY / SRC. Fail-closed: EMPTY → all files; DOC_ONLY (every path matches the README/CLAUDE/AGENTS/CHANGELOG/TODOS allowlist) → empty stdout; SRC → escape-hatch paths (schema, package.json, skills/) trigger all, else the hand-tunedE2E_TEST_MAPglob narrows, and an unmapped src/ change still emits ALL files (never silently nothing). Pure-function exportsselectTests,classify,matchGlob.bun run ci:select-e2eprints the current selection on stdout.test/select-e2e.test.tscovers all 4 branches plus 3 guards (skills/, untracked files, unmapped src/) — 24 cases. -
scripts/run-e2e.sh— Sequential E2E runner. Accepts an optional argv-driven file list (used byci:local:diff) and a--dry-run-listflag that prints the resolved file list and exits (used byci-local.sh's startup smoke-test). Falls back totest/e2e/*.test.tsplustest/phantom-redirect-engine-parity.test.tswhen invoked with no args (the phantom-redirect Postgres arm is only reachable through a DATABASE_URL-bearing lane; the unit wrappers strip the URL, so this lane must carry it). This wrapper is the database-URL opt-in boundary: it exportsGBRAIN_TEST_ALLOW_DATABASE_URL=1so the bunfig preload guard (test/helpers/database-url-guard-preload.ts) lets the run start, unsetsGBRAIN_DATABASE_URL(the e2e suite runs onDATABASE_URLonly — an ambientGBRAIN_DATABASE_URLwould pass the opt-in yet reach CLI-subprocess paths with no name floor), and its GBRAIN_* env scrub preservesGBRAIN_E2E_ALLOW_DBso the name-floor escape hatch the guard's own error message names stays usable. It also preservesGBRAIN_CI_DISABLE_TEST_ENV_FILE=1, keeping CI runs from loading checkout-local.env.testingcredentials after the shell-to-Bun handoff. It also exportsGBRAIN_TEST_KEEP_PROVIDER_KEYS=1so the unit-lane provider-key strip preload (test/helpers/provider-keys-preload.ts) leaves the real keys that live embed/parity e2e tests skip-gate on. Each file runs under a gtimeout/timeout wedge backstop (default signal: SIGTERM — the bun test child installs no JS-level handler for it, so kernel-default termination applies) — 180s default, with a per-file override for known-slow files (skills.test.tsgets 420s: the real ingest-skill run replays every migration, so its floor grows as master adds migrations); the cap is a wedge backstop, not a per-test budget, and barebun(no outer cap) is the fallback when neither timeout binary is installed. It activates the PGLite schema snapshot like the other runners: sourcesscripts/lib/test-env.sh+ensure_pglite_snapshotafter the--dry-run-listearly exit (list mode stays instant; non-fatal on build failure), re-exportsGBRAIN_PGLITE_SNAPSHOTas an ABSOLUTE path (e2e tests spawn CLI subprocesses with varying cwd — a relative path silently misses the tar there), and its env keep-list preserves the var; e2e files that assert the path TO post-initSchema()state carry per-filedelete process.env.GBRAIN_PGLITE_SNAPSHOTopt-outs with one-line reasons. -
scripts/llms-config.ts+scripts/build-llms.ts— Generator forllms.txt(llmstxt.org-spec web index) +llms-full.txt(inlined single-fetch bundle). Curated config drives both. Runbun run build:llmsafter adding a new doc.LLMS_REPO_BASEenv lets forks regenerate with their own URL base.FULL_SIZE_BUDGET(600KB) caps the inline bundle; generator WARNs if exceeded. Committed output has no runtime consumer; committed for GitHub browsing and fork-safe fetching. -
AGENTS.md— Local-clone entry point for non-Claude agents (Codex, Cursor, OpenClaw, Aider). MirrorsCLAUDE.mdintent via relative links. Claude Code keeps usingCLAUDE.md. -
docs/UPGRADING_DOWNSTREAM_AGENTS.md— Patches for downstream agent skill forks to apply when upgrading. Each release appends a new section; includes diffs for brain-ops, meeting-ingestion, signal-detector, enrich. -
src/core/schema-embedded.generated.ts— AUTO-GENERATED from schema.sql (runbun run build:schema; the.generated.tssuffix exempts it from the module-size ratchet's new-file cap, per the guard's generated-file carve-out) -
src/schema.sql— Full Postgres + pgvector DDL (source of truth, generates schema-embedded.generated.ts) -
src/core/search/expansion.ts— Multi-query expansion via Haiku. ExportssanitizeQueryForPrompt+sanitizeExpansionOutput(prompt-injection defense-in-depth). Sanitized query is only used for the LLM channel; the original query still drives search.expandQueryreturns the original first plus 2-3 variants;hybridSearchre-enforcesqueries[0]= the caller's query and dedupes repeats, then fuses each variant's vector list as a rolevariantarm throughsrc/core/search/fusion-lists.ts(see that entry), where the variant arms sharesearch.expansion_variant_budgetas total RRF weight —null(every bundle's default) is legacy equal weight, the configuration under which the LongMemEval receipt shows expansion halving strictrecall_all@5(93.19% plain hybrid vs 54.89% with expansion, paired +3 / -183;docs/architecture/RETRIEVAL.md"Multi-query expansion"). The harness recordsexpansion_variantsper row and--expansion-replayserves them back so a budget sweep changes only the budget. -
recipes/— Integration recipe files (YAML frontmatter + markdown setup instructions) -
docs/guides/— Individual SKILLPACK guides (broken out from monolith) -
docs/integrations/— "Getting Data In" guides and integration docs -
docs/architecture/infra-layer.md— Shared infrastructure documentation -
docs/ethos/THIN_HARNESS_FAT_SKILLS.md— Architecture philosophy essay -
docs/ethos/MARKDOWN_SKILLS_AS_RECIPES.md— "Homebrew for Personal AI" essay -
docs/guides/repo-architecture.md— Two-repo pattern (agent vs brain) -
docs/guides/sub-agent-routing.md— Model routing table for sub-agents -
docs/guides/skill-development.md— 5-step skill development cycle + MECE -
docs/guides/idea-capture.md— Originality distribution, depth test, cross-linking -
docs/guides/quiet-hours.md— Notification hold + timezone-aware delivery -
docs/guides/diligence-ingestion.md— Data room to brain pages pipeline -
docs/designs/HOMEBREW_FOR_PERSONAL_AI.md— 10-star vision for integration system -
docs/mcp/— Per-client setup guides (Claude Desktop, Code, Cowork, Perplexity) -
BrainBench retrieval benchmark (P@5/R@5 corpus + harness): lives in the separate gbrain-evals repo. Not installed alongside gbrain. Distinct from the in-repo cross-harness memory conformance suite (
gbrain eval brainbench—src/eval/brainbench/, corpus atevals/brainbench/, methodology indocs/eval/BRAINBENCH.md). -
skills/_brain-filing-rules.md— Cross-cutting brain filing rules (referenced by all brain-writing skills) -
skills/RESOLVER.md— Skill routing table (based on the agent-fork AGENTS.md pattern) withskills/manifest.json: schema-author wired into the dispatcher with the full functional-area trigger list (compressed routing pattern per the dispatcher convention). -
skills/conventions/— Cross-cutting rules (quality, brain-first, model-routing, test-before-bulk, cross-modal) -
skills/_output-rules.md— Output quality standards (deterministic links, no slop, exact phrasing) -
skills/signal-detector/SKILL.md— Always-on idea+entity capture on every message -
skills/brain-ops/SKILL.md— Brain-first lookup, read-enrich-write loop, source attribution -
skills/idea-ingest/SKILL.md— Links/articles/tweets with author people page mandatory -
skills/media-ingest/SKILL.md— Video/audio/PDF/book with entity extraction -
skills/meeting-ingestion/SKILL.md— Transcripts with attendee enrichment chaining -
skills/citation-fixer/SKILL.md— Citation format auditing and fixing -
skills/repo-architecture/SKILL.md— Filing rules by primary subject -
skills/skill-creator/SKILL.md— Create conforming skills with MECE check -
skills/daily-task-manager/SKILL.md— Task lifecycle with priority levels -
skills/daily-task-prep/SKILL.md— Morning prep with calendar context -
skills/cross-modal-review/SKILL.md— Quality gate via second model -
skills/cron-scheduler/SKILL.md— Schedule staggering, quiet hours, idempotency -
skills/reports/SKILL.md— Timestamped reports with keyword routing -
skills/testing/SKILL.md— Skill validation framework -
skills/soul-audit/SKILL.md— 6-phase interview for SOUL.md, USER.md, ACCESS_POLICY.md, HEARTBEAT.md -
skills/webhook-transforms/SKILL.md— External events to brain signals -
skills/data-research/SKILL.md— Structured data research: email-to-tracker pipeline with parameterized YAML recipes -
skills/minion-orchestrator/SKILL.md— Unified background-work skill. Two lanes: shell jobs viagbrain jobs submit shell --params '{"cmd":"..."}'(operator/CLI only; MCP throwspermission_deniedfor protected names) and LLM subagents viagbrain agent run(user-facing entrypoint). Shared Preconditions block, parent-child DAGs with depth/cap/timeouts,child_doneinbox for fan-in, PGLite--followinline path for dev. Triggers narrowed to"gbrain jobs submit"+"submit a gbrain job"sostats/prune/retryquestions fall through togbrain --help. -
templates/— SOUL.md, USER.md, ACCESS_POLICY.md, HEARTBEAT.md templates -
skills/migrations/— Version migration files with feature_pitch YAML frontmatter -
src/commands/publish.ts— Deterministic brain page publisher (code+skill pair, zero LLM calls) -
src/commands/backlinks.ts— Back-link checker and fixer (enforces Iron Law). Repair-inserted references land as UNDATED rows (- Referenced in [Title](path)) in a dedicated## Referenced bysection placed ABOVE the timeline region — retroactively stamping the repair date onto a timeline would forge events that never happened on that date; only live capture writes dated timeline entries.insertBacklinkEntry(content, bodyStart, entry)never touches bytes beforebodyStart, locates the timeline boundary viafindTimelineSplitIndex(existing timeline sentinels take precedence over bare## Timeline/## Historyheadings), appends to an existing## Referenced bysection or creates one above the timeline, and is CRLF-tolerant;insertTimelineEntryis an alias for existing import sites.buildBacklinkEntry(sourceTitle, sourcePath)takes no date; dir-shaped sources get an extension-less brain-slug link so the next check pass credits the fresh row instead of re-flagging (idempotency). -
src/commands/lint.ts— Page quality linter (catches LLM artifacts, placeholder dates).runLintCore(opts)walks the tree ONCE;--fixapplies fixes during that same scan and reports per-page results through theonPageIssues(relPath, issues, fixedCount)callback (issues = what remains after this run's fix attempt for the page;fixedCount= fixes actually applied), so the summary's auto-fixed count is the true count — a second scan run after fixing would see already-fixed files and report zero.onScanStartfires once with the collected page count for progress wiring. Pinned bytest/lint-fix-single-pass.test.ts. On a durability-hardened brain (isDurabilityHardenedon the target dir, or the page's directory for a single-file target) every non-dry-run--fixrepair is committed path-limited viacommitWriteThroughFileright after thewriteFileSync— the same contract as put_page write-through, so the cycle, thelint/lint-fixminion handlers and the CLI never leave hardened repairs as uncommitted drift for the sync phase to flag (pinned bytest/cycle-lint-durability.test.ts). Lint ruleshuge-page(flags pages exceedingcontent_sanity.bytes_warn) andscraper-junk(flags pages matching any junk pattern). Both reuseassessContent()fromsrc/core/content-sanity.tsso lint, doctor, and ingest share one assessor.lint.tslifts DB config when~/.gbrain/is reachable; falls back to file/env on CI. Pinned bytest/lint-content-sanity.test.ts. withsrc/commands/sources.ts:gbrain linthas amarkup-heavyrule (flags pages whose prose-vs-markup ratio exceedscontent_sanity.max_markup_ratio, reusingassessContentSanityso lint/gate/scan share one assessor); pinned bytest/lint-content-sanity.test.ts.gbrain sources audit <id>is disposition-aware: its dry-run disk scan reports would-quarantine / would-reject / would-flag counts driven by the effectivecontent_sanity.junk_disposition+ markup config, so an operator previews the gate's verdict before sync. Thecontent-sanity-auditJSONL (src/core/audit/content-sanity-audit.ts) records the quarantine/flag dispositions. -
src/commands/report.ts— Structured report saver (audit trail for maintenance/enrichment) -
src/core/destructive-guard.ts— three-layer protection against accidental data loss.assessDestructiveImpact(engine, sourceId)counts pages/chunks/embeddings/files/facts for a source (a fact-only source — a revoked agent's workspace, since facts are the primary agent write lane — is data at stake, not empty; pre-facts brains degrade to 0), plusoauthClientCount— OAuth clients whosesource_idreferences it.checkDestructiveConfirmation(impact, opts)is the fail-closed gate (--confirm-destructiverequired when data is present;--yesalone is rejected). FK-RESTRICT lifecycle:clientsReferencingSource(engine, sourceId)lists ALL physical OAuth-client rows referencing a source viaoauth_clients.source_id— the FK is PHYSICAL (ON DELETE RESTRICT ignoresdeleted_at), so soft-deleted (revoked-but-retained) rows BLOCK a hard delete too and come back taggeddeleted; pre-migration brains withoutdeleted_atfall back to untagged referents (42703-retry idiom) and brains without the table have none by construction.formatClientReferentsBlockrenders the shared refusalsources remove/sources purgeprint (naming each client —[revoked, retained]for soft-deleted rows — + the revoke command) so the raw Postgres FK violation never reaches the operator.softDeleteSource/restoreSource/listArchivedSources/purgeExpiredSourcesdrive the source-level archive lifecycle viasources.archived BOOLEAN,archived_at TIMESTAMPTZ,archive_expires_at TIMESTAMPTZ;purgeExpiredSourcesSKIPS client-referenced sources via a physicalNOT EXISTS(soft-deleted clients count) so recurring maintenance keeps sweeping the rest instead of aborting. Page-level analog:BrainEngine.softDeletePage/restorePage/purgeDeletedPagespluspages.deleted_at TIMESTAMPTZand a partial purge index. The MCPdelete_pageop rewires tosoftDeletePage; opsrestore_page(scope: write) andpurge_deleted_pages(scope: admin,localOnly: true) round out the surface.get_page/delete_page/restore_page(src/core/ops/pages.ts) accept an explicit per-callsource_idthat is honored or rejected loudly, never silently dropped:get_pageresolves it through the caller's read grant ('__all__'spans every source for trusted local callers, the granted sources for remote callers), whiledelete_page/restore_pagereject'__all__', target exactly one source, and — for every caller other than trusted local CLI (ctx.remote === false) — accept only the caller's write authority (ctx.auth.sourceId, falling back toctx.sourceIdfor legacy tokens and unauthenticated transports; the federated read grantallowedSourcesconfers no delete/restore access), echoing the targetedsource_idin the response. Pinned bytest/pages-source-scoping-4329.test.ts. Search visibility (buildVisibilityClauseinsrc/core/search/sql-ranking.ts) hides soft-deleted pages and archived sources fromsearchKeyword/searchKeywordChunks/searchVectorin both engines. The autopilot cycle'spurgephase callspurgeExpiredSources+engine.purgeDeletedPages(72)so the 72h TTL is real. -
src/commands/pages.ts—gbrain purge-deleted [--older-than HOURS|Nd] [--dry-run] [--json]operator escape hatch. Mirror ofgbrain sources purgefor the page-level lifecycle. Hard-deletes pages whosedeleted_atis older than the cutoff; cascades to content_chunks/page_links/chunk_relations. -
src/core/op-checkpoint.ts— DB-backed checkpoint primitive for long-running ops. Migration v67 introducesop_checkpoints (op TEXT, fingerprint TEXT, completed_keys JSONB, updated_at TIMESTAMPTZ, PK(op, fingerprint)). Per-op fingerprint helpers (embedFingerprint,extractFingerprint,reindexFingerprint,integrityFingerprint,purgeFingerprint) computesha8(canonical-JSON(relevant-params))so re-running with the same params resumes fromcompleted_keysand re-running with different params (e.g.--limit 100vs--limit 200) starts fresh. Cross-worker safe on Postgres (DB row, no file-lock race); PGLite degrades gracefully. The 7-day TTL GC runs in the cycle'spurgephase. All writes (recordCompleted,clearOpCheckpoint) route throughengine.executeRawDirect+withRetry(BULK_RETRY_OPTS)so they survive Supavisor pool exhaustion, andrecordCompletedreturnsboolean(banked vs failed-after-retries) — the 9 non-sync consumers keep its REPLACE-into-completed_keyssemantics. Resumable sync uses the additiveappendCompleted(key, deltaKeys)/appendCompletedOnce(the latter no-retry for the SIGTERM path) which INSERT a delta into theop_checkpoint_pathschild table (migration v115:(op, fingerprint, path)PK, FK toop_checkpointsON DELETE CASCADE) via a single writable-CTEunnest(\$3::text[])write — O(delta), never an O(N²) full-set rewrite.loadOpCheckpointreturns theUNION ALLof legacycompleted_keys+ child-table paths (deduped in JS), so an in-flight upgrade loses nothing. The legacy arm is gated onjsonb_typeof(completed_keys) = 'array'so a non-array (scalar) parent row can't makejsonb_array_elements_textthrow "cannot extract elements from a scalar" and take down the whole union (which would discard the valid child rows and lose all banked progress for the key); a third union arm flags the corruption so the loader logs it once and keeps the child rows. Migration v119 adds theop_checkpoints_completed_keys_arrayCHECK (jsonb_typeof(completed_keys) = 'array') — a DB-enforced, always-on guard that makes the scalar-corruption class structurally impossible going forward; the migration repairs any pre-existing scalar to'[]'underLOCK TABLE ... IN SHARE ROW EXCLUSIVE MODEandsrc/core/schema-embedded.generated.ts+src/core/pglite-schema.tsship the same CHECK on fresh installs (a loader hit implies schema drift, a disabled constraint, or an out-of-band writer).recordCompletedbinds its array through$3::text::jsonb(NOT a bare$3::jsonb) so postgres.js.unsafe()doesn't double-encodeJSON.stringify(sorted)into the scalar string that CHECK rejects (PGLite parses it silently, so only real Postgres surfaces the bug). A DATABASE_URL-gatedtest/e2e/op-checkpoint-jsonb-parity.test.ts(its own CI job) asserts the array shape on real Postgres.syncFingerprint({sourceId, lastCommit})keys the sync rows. Pinned bytest/op-checkpoint.test.ts(incl. delta-append, union read, cascade clear, durable-write boolean, and the scalar-parent guard).import-checkpoint.tsstays a separate file-backed checkpoint — both systems coexist without conflict (unifying them would mean async-propagating the four sync call sites insrc/commands/import.ts; deferred). -
src/core/brain-score-recommendations.ts— pure data layer consumed by bothgbrain doctor --remediation-plan/--remediateandgbrain features.computeRecommendations(checks, opts)returnsRemediation[]with stableid, content-hashidempotency_key,severity,est_seconds,est_usd_cost,depends_on(references stable ids, not check names — so plan order is reproducible).classifyChecks(report)triages every doctor check three-state intoremediable | human_only | blocked(human_onlycovers RLS warnings and other human-judgment gates;blockedcovers dependency chains where a parent check failed).maxReachableScore(checks)computes the ceiling for empty/under-configured brains (no entity pages → graph_coverage caps at 70; no embedding key → embedding_coverage caps at 60). Cost estimates pull fromanthropic-pricing.ts(synthesize/patterns/consolidate) andembedding-pricing.ts(embed jobs). Pinned bytest/brain-score-recommendations.test.ts(~27 cases incl. determinism, content-hash idempotency, DB-backed checkpoint provenance, three-state triage). -
src/core/abort-check.ts— one canonical place for cooperative-abort checks across gbrain's long loops.isAborted(signal?)→ boolean (for loops thatbreakand return partial progress).throwIfAborted(signal?, label?)throws anAbortError(name === 'AbortError') at phase boundaries, preferring the signal'sreason('wall-clock'/'lock-lost'/'shutdown') so the unwind self-describes.anySignal(internal, external?)composes two signals into one that fires when EITHER does (platformAbortSignal.anywith a manual-relay fallback), returning the internal unchanged when there's no external so non-aborting callers pay nothing. Threading these checks throughrunPhaseEmbed → runEmbedCore → embedAll(Stale)/embedPagelets the embed phase bail and releasegbrain_cycle_locksimmediately (an embed phase that ignored its abort signal would hold the lock and make later autopilot cycles skip withcycle_already_running). Coverage spans every long cycle-reachable phase:extract(incrementalextractForSlugs+ the full-walkextractLinksFromDir/extractTimelineFromDir, all viarunSlidingPool's signal),extract_facts(per-page loop + the per-pageembedsignal +runPhantomRedirectPass's 30s lock-retry),consolidate's bucket loop, andlint(which is synchronous, so itawaits a periodic yield to let the signal land).runCycleadds a terminal abort check before stampinglast_full_cycle_atso a cancelled cycle never reports a completed full run, plus a per-phaseduration_mswarning that names any phase overrunning the worker's 30s force-evict deadline. Pinned bytest/abort-check.test.ts+test/cycle-abort.test.ts. -
openclaw.plugin.json— ClawHub bundle plugin manifest -
.codex-plugin/plugin.json+.codex-plugin/mcp.json+.agents/plugins/marketplace.json— the Codex plugin lane: manifest (skills → the committedplugin/tree; mcpServers → the NON-root mcp.json), the MCP declaration (serve --surface starter --source-guard, code-derivedenv_varspassthrough), and the codex-native marketplace. Version lockstep with package.json + the claude/openclaw manifests pinned bytest/codex-plugin-manifest.test.ts. -
.claude-plugin/plugin.json+.claude-plugin/marketplace.json— the Claude Code plugin lane: inline MCP declaration (command/args/cwd via${CLAUDE_PLUGIN_ROOT}; noenvblock — Claude passes the parent env through); the Claude marketplace carries the full plugin PLUS the persona variant entries (gbrain-coding/gbrain-daily→plugin-variants/), while the codex marketplace intentionally stays single-entry until codex's multi-entry handling gets its observation run — variant names are pinned toskills/plugin-lanes.json#personasbytest/codex-plugin-manifest.test.ts. -
.agents/gbrain-launcher— shared plugin MCP launcher (sh, Unix-only) for the Codex, Claude Code, and OpenClaw (openclaw.plugin.jsonmcpServers.gbrain) lanes: GBRAIN_BIN → ~/.bun/bin/gbrain → PATH resolution, one stderr resolution line, GBRAIN_SURFACE substitute-or-append override forserveargv, actionable exit-127 recovery copy, no auto-install by design. Behavioral branches all pinned in the manifest test. -
skills/plugin-lanes.json— curation record for the plugin lanes: lane set = (openclaw bundle ∖ base_exclusions) ∪ additions, a reason per entry;starter_gapsis the generated snapshot of per-skill beyond-starter MCP ops (refresh via--write-gaps). The openclaw lane's own curation is untouched — plugin users ARE the brain host (the downstream-vs-host inversion). -
scripts/generate-plugin-tree.ts+scripts/check-plugin-tree.sh+plugin/— generator, byte-diff drift gate (in the skills commit gate), and the committed curated skill tree both plugin lanes ship (67 skills + shared conventions/_*.md deps + a generated README carrying the CLI-primary starter note). -
test/e2e/codex-plugin-install-real.serial.test.ts+test/e2e/claude-plugin-install-real.serial.test.ts— the plugin doors: clean-tree staging (git archive HEAD), real marketplace add/plugin add, snapshot + exec-bit + non-root-mcp.json pins, tools/list starter-surface oracle, cold-home fast-fail, --source-guard block/allow, coexistence + removal probes, auth-gated SMOKE turns. CI: theplugin-doorsjob in heavy-tests.yml (pinned binary provisioning + expected-pass-count refuse-green). -
src/commands/capture.ts+src/commands/serve-http.ts+src/core/{operations,import-file,types,utils,facts/absorb-log,brainstorm/{orchestrator,error-classify},scope,postgres-engine,pglite-engine}.ts— ingestion hardening. Capture frontmatter merge viamergeCaptureFrontmatter(uses shared data-onlydata-frontmatter, preserving metadata beyondparseMarkdown);/ingestnull-guard + outer try/catch envelope with!res.headersSentguard; dedup via separate normalize-for-hash (normalizeForHashstrips BOM/CRLF/whitespace/NFKC) + body-after-frontmatter-strip on the DB hash (excludescaptured_at+ingested_atso capture-cli timestamp variations don't invalidate the chunk cache); friendlypages_source_id_fkrewrite viamaybeRewriteSourceFkErroron BOTH local + thin-clientcallRemoteToolcatch blocks;facts:absorb'No database connection' suppression via typedinstanceof GBrainError && e.problemcheck + first-occurrence stack-trace info log (module-scoped_hasLoggedDisconnectedFactsAbsorbflag, test seam_resetFactsAbsorbDisconnectedFlagForTests); CLI help discoverability (captureinCLI_ONLY_SELF_HELP+ pre-engine-bind--helpshort-circuit inhandleCliOnly+ aBRAINsection inprintHelp); binary-file guard viadetectBinaryNullByte(buf)first-8KB NUL scan on--file(Buffer-read, no encoding) and--stdin(readStdinBufferaccumulator); provenance write-through — put_page accepts 3 optional params (source_kind, source_uri, ingested_via;ingested_atserver-stamped) + trust gate (whenctx.remote !== falseIGNORE client params, server stampsmcp:put_page, fail-closed) + COALESCE-preserve UPDATE semantics (omitting params on a later put_page preserves prior values; first-write-wins);/admin/api/register-clientscopes normalization vianormalizeScopesInput(raw: unknown)insrc/core/scope.ts(accepts string/string[]/missing; rejects['read write']space-in-element shape, non-string elements, empty array, unknown scopes; deduped + sorted); brainstorm timeout surfacing via an orchestrator-level try/catch atrunBrainstormentry (single-point wrap covers every internal SQL site, classifies SQLSTATE 57014 via postgres.js.code/.sqlState/ message fallback intoStructuredAgentErrorcodebrainstorm_timeoutwith a hint covering all 3 PG cancel sub-causes); read-path surfaces all 4 provenance columns viagetPageprojection +rowToPage3-state optional read +Pageinterface; canonical source resolver routes capture throughresolveSourceWithTier(engine, parsed.source, cwd); thin-client--sourcerejection (server-side OAuth client registration owns source scope); thesource_kindtaxonomy is closed (capture-cli | put_page | mcp:put_page | webhook | file-watcher | inbox-folder | cron-scheduler),--sourcemaps to source_id only. Tests:test/capture-build-content.test.ts,test/capture-runcapture.test.ts,test/put-page-provenance.test.ts,test/scope-normalize.test.ts,test/cli-help-discoverability.test.ts,test/brainstorm-timeout.test.ts; extendedtest/facts-absorb-log.test.ts,test/import-file.test.ts,test/e2e/engine-parity.test.ts,test/e2e/serve-http-ingest-webhook.test.ts. Report atdocs/v0.38-smoke-test-report.md. Follow-ups in TODOS.md: SQL-shape rewrite oflistPrefixSampledPagesfor PgBouncer, magic-byte allowlist for binary detection,--source-kindoverride flag, ingest_capture handler migration, provenance-history table, facts:absorb root-cause trace.
BrainBench — in a sibling repo
The retrieval-quality BrainBench — the public benchmark for personal-knowledge
agent stacks (P@5/R@5/MRR/nDCG corpus + harness) — lives in
github.com/garrytan/gbrain-evals. It
depends on gbrain as a consumer; gbrain never pulls in the ~5MB eval corpus or
the pdf-parse dev dep at install time. The name "BrainBench" primarily
refers to the in-repo cross-harness memory conformance suite
(gbrain eval brainbench — see the src/eval/brainbench/ and
evals/brainbench/ entries above and docs/eval/BRAINBENCH.md); this section
covers the separate retrieval benchmark.
gbrain's public API surface (the exports map in package.json) is what
gbrain-evals consumes: gbrain/engine, gbrain/types, gbrain/operations,
gbrain/pglite-engine, gbrain/link-extraction, gbrain/import-file,
gbrain/transcription, gbrain/embedding, gbrain/config, gbrain/markdown,
gbrain/backoff, gbrain/search/hybrid, gbrain/search/expansion,
gbrain/extract. Removing any of these is a breaking change for the
gbrain-evals consumer.
Hindsight calibration (key files cluster)
Calibration teaches gbrain how the user tends to be wrong and uses that
knowledge at every advice surface: a six-migration schema (v67-v72),
three cycle phases, eight expansions, one admin tab.
Convention skill at skills/conventions/calibration.md has the agent-
facing rules.
Migration v80: the takes_resolution_consistency CHECK accepts
quality='unresolvable' AND outcome=NULL as the 4th valid resolution state. The
column-level CHECK on resolved_quality (takes_resolved_quality_values) enumerates
all 4 states. Take.resolved_quality, TakeResolution.quality, and
takes-fence.ts:TakeQuality are 4-state. TakesScorecard carries unresolvable_count
unresolvable_rate;resolvedstays 3-state (correct+incorrect+partial) so scorecards stay comparable over time.finalizeScorecard:unresolvable_rate = unresolvable_count / (resolved + unresolvable_count), NULL when both 0. Spec doc atdocs/architecture/calibration-quality-gate-spec.md. Pinned by R1-R5 intest/takes-resolution.test.tsandtest/migrate.test.ts's v80 structural + PGLite round-trip suite (CHECK admits unresolvable+NULL, still rejects partial+true and unresolvable+true|false, pre-v80 NULL/NULL rows survive).
src/core/cycle/base-phase.ts— abstractBaseCyclePhaseclass. EnforcessourceScopeOpts(ctx)threading at the type level; rules out the source-isolation leak class structurally for every new phase. Inherits source-scope, budget meter, error envelope, progress reporter. propose_takes / grade_takes / calibration_profile all extend it. The ONE home ofCYCLE_DEADLINE_RESERVE_MS(60s carved out of the enclosing job's remaining wall-clock — wait-poll + worker force-evict grace + cleanup headroom; patterns.ts re-exports it) and ofBasePhaseOpts.deadlineAtMs(the enclosing minion job's absolute deadline, threaded via runCycle; null/unset for directgbrain dreamcallers — phases then fall back to their derived defaults).src/core/cycle/propose-takes.ts— LLM scans markdown prose, proposes gradeable claims to thetake_proposalsqueue. Candidate discovery excludesextract_receiptpages before the LLM call so the phase never re-ingests its own operational receipts; nullable legacy page types remain eligible. Idempotency cache on(source_id, page_slug, content_hash, prompt_version)composite unique index. Fence-dedup: existing canonical takes passed to the extractor as context. Ships a stub prompt; tuned prompt arrives via the synthetic corpus build. Per-page extractor failures log a warning and continue, but a whole-run condition (perclassifyGlobalLlmErrorinsrc/core/ai/errors.ts) breaks the page loop with a single combined warning line, setsaborted_global_error, and records a halt in the rollup — auth/billing on the first hit, bare rate_limit only afterRATE_LIMIT_HALT_STREAK(3) consecutive hits (a successful call resets the streak). Status:failwhen the halt happened with ZERO successful extractor calls (the whole LLM lane is down), otherwise any warnings fold intowarn+(N warning(s))summary suffix, so swallowed failures can't read as a cleanok.llm_calls_succeeded/llm_calls_failed/haltedland in details. The phase wall-clock deadline is DERIVED, never a literal (a literal equal to the autopilot-cycle handler anchor, with non-co-started clocks, would make the cleandeadline_hitpartial-completion path structurally unreachable): explicitopts.deadlineMswins (test seam), elseresolveProposeTakesDeadlineMs(deadlineAtMs, now)=PHASE_DEADLINE_FRACTION_OF_JOB(0.8, headroom for grade_takes + calibration_profile which run after it with no deadline of their own) × (remaining job budget −CYCLE_DEADLINE_RESERVE_MS), clamped to the fallback (0.8 × theautopilot-cycleentry inHANDLER_DEFAULT_TIMEOUT_MS— a missing anchor throws at module LOAD, failing the whole cycle visibly). A fractioned value underMIN_PROPOSE_TAKES_BUDGET_MS(2 min) resolves to null and the phase returns an honestskippedwithreason: 'insufficient_cycle_budget'(after the cheap provider probe, before any rollup/DB write — records neither a halt nor a completed round; next cycle retries with a fresh budget). Pinned bytest/propose-takes.test.ts+test/propose-takes-per-claim.test.ts+test/cycle-phase-deadline-drift.test.ts. The kind vocabulary isTAKE_KIND_VALUESimported fromsrc/core/takes-fence.ts(the fence enum — no hand-copied set); an unknown kind maps through the legacy-kind table, elsetake.src/core/cycle/grade-takes.ts— walks unresolved takes older than 6 months, retrieves evidence, asks judge model, caches verdict. Auto-resolve DISABLED by default. Conservative thresholds: >=0.95 single OR >=0.85 ensemble 3/3 unanimous.aggregateEnsemblereuses the cross-modal substrate; fires on the borderline 0.6-0.95 band. Writes totake_grade_cache. Same global-error posture as propose-takes: a judge failure that classifies as a whole-run condition breaks the take loop withaborted_global_error(auth/billing first hit; rate_limit after 3 consecutive takes;failstatus when zero judge calls succeeded, elsewarn+ warning count;judge_calls_succeeded/judge_calls_failed/haltedin details). Rejected ensemble judges are classified the same way —Promise.allSettlednever flattens a revoked key or exhausted spend limit into a silent null verdict. Per-take auto-apply failures stay per-take.src/core/cycle/calibration-profile.ts— aggregates resolved takes into 2-4 narrative pattern statements + active bias tags. Voice-gated viagateVoice(). Cold-brain skip when <5 resolved. Writes tocalibration_profileswith audit columns (voice_gate_passed,voice_gate_attempts,grade_completion).src/core/calibration/voice-gate.ts— singlegateVoice()function, mode parameter (pattern_statement|nudge|forecast_blurb|dashboard_caption|morning_pulse). 2 regens then template fallback fromsrc/core/calibration/templates.ts. Haiku judge with mode-specific rubrics; all rubrics structurally forbid clinical/preachy voice.src/core/calibration/cross-brain.ts— 4-rule contract for cross-brain calibration reads. Local-first → mount-fallback (only withcanReadMountsForCtx(ctx)true) → cross-brain attribution viasource_brain_id+from_mount→ subagent prohibition closes the OAuth-token-to-cross-brain-leak surface. All 4 rules pinned intest/cross-brain-calibration.test.ts.src/core/calibration/nudge.ts— real-time pattern surfacing.evaluateAndFireNudge(opts): threshold check (conviction > 0.7, holder match, slug-derived domain hint matches active bias tag) → cooldown probe (14d via take_nudge_log) → fire + log. STDERR-only output; multi-channel deferred.src/core/calibration/take-forecast.ts— Brier-trend at write time. Pure math over existingTakesScorecard; no LLM. Returnspredicted_brier,bucket_n,overall_brier. Insufficient-data branch atMIN_BUCKET_N = 5.batchForecastmemoizes per (holder, domain) tuple.src/core/calibration/gstack-coupling.ts— outcome-driven learnings coupling.writeIncorrectResolution(opts)shells out to thegstack-learnings-logbinary. Config gatecycle.grade_takes.write_gstack_learnings(default false for external users). Namespace prefixgbrain:calibration:v0.36.1.0:so--undo-wavecan scrub.src/core/calibration/svg-renderer.ts— server-rendered SVG for the admin SPA Calibration tab. Pure functions: data → SVG string. Inlines design tokens; XSS-safe viaescapeXml(). Four renderers:renderBrierTrend,renderDomainBars,renderAbandonedThreadsCard,renderPatternStatementsCard. SPA renders via<TrustedSVG>wrapper behindrequireAdmin.src/core/calibration/undo-wave.ts—undoWavereverses calibration's mutations: unsetstakes.resolved_*for wave-applied resolutions (cross-checks resolved_by so manual writes persist), deletes calibration_profiles, purges nudge logs, marks grade-cache rows applied=false.--dry-runshows counts without writing. Idempotent on wave_version match.src/core/calibration/think-ab.ts— A/B harness.runAbTrialcalls thinkRunner twice (baseline + with-calibration), records preference tothink_ab_results.buildAbReportaggregates over a 30-day window; flagscalibration_net_negativewhen n>=20 + win rate < 45% on decisive trials.src/core/calibration/recall-footer.ts— formatter for the morning-pulse calibration block. Cold-brain branch when <5 resolved. Opt-in via the wiring layer.src/core/eval-contradictions/calibration-join.ts— cross-reference.tagFindingWithCalibration(finding, profile)returns bias-tag context for contradictions matching active patterns. Returns null when profile missing (output unchanged when no profile exists).src/core/think/prompt.ts— anti-bias prompt shaping.withCalibrationoption onbuildThinkSystemPromptadds anti-bias rules.buildCalibrationBlock()emits the<calibration>XML.buildThinkUserMessagehas TWO shapes: default (question first), and with-calibration (retrieval → calibration → question) when opt-in. Wired intorunThinkviaopts.withCalibration+opts.calibrationHolder.src/commands/calibration.ts— CLI:gbrain calibration(read + print),--regenerate,--undo-wave <ver>,ab-report. MCP opget_calibration_profile(scope: read) backs the same data path. Source-scoped viasourceScopeOpts(ctx).src/core/owner-holder.ts— single source of truth for "the brain owner" holder string.DEFAULT_OWNER_HOLDER = 'self'(matches the consolidate facts→takes writer +docs/takes-vs-facts.md);resolveOwnerHolder({override, configValue})returns override >emotional_weight.user_holderconfig >'self'. Consumed by the calibration_profile cycle phase,gbrain calibrationCLI, theget_calibration_profileop,think's calibration block,emotional-weight'sDEFAULT_USER_HOLDER, and doctor'scalibration_freshness. Pure; unit-tested intest/owner-holder.test.ts. Does NOT unify owner-identity fragmentation (self/brain/people-<owner>) — tracked separately.src/commands/takes.ts— the fullgbrain takessubcommand dispatcher (list/search/embed/add/update/supersede/resolve/propose/scorecard/calibration/extract/revisit).takes list --limit N --offset Nvalidates integers at the CLI; the engine clamps limit (default 100, cap 500) and floors offset at 0.takes revisit <slug>opens $EDITOR on the source page with a<!-- gbrain:revisit -->cursor marker.src/core/take-proposals.ts— owns thetake_proposalsqueue's row contract.normalizeTakeProposalRowcoerces Postgres driver BigInt/string values (id,weight,promoted_row_num) to numbers at the query boundary so both engines return the same numeric row shape.admin/src/pages/Calibration.tsx— Calibration tab. Single-column layout.<TrustedSVG>wrapper handlesdangerouslySetInnerHTMLfor the server-rendered SVG.admin/src/index.css—--text-muted: #777(WCAG AA contrast bump to ~5.5 on the #0a0a0f bg).test/fixtures/calibration/extract-takes-corpus/— synthetic prompt-tuning corpus. Ships 5 representative pages; full 50-page + 10-page holdout generated bygbrain calibration build-corpus. All anonymized per CLAUDE.md placeholder list.scripts/check-synthetic-corpus-privacy.sh— CI guard inbun run verify. Greps for explicit dollar amounts + verifies non-essay fixtures reference at least one placeholder name.test/regressions/v0.36.1.0-iron-rule.test.ts— R1-R5 IRON-RULE inventory; pins all 5 rules in one place.DESIGN.md— repo-root design system. Formalizes the de facto admin tokens. Calibration target for future/plan-design-reviewand/design-review.
Schema packs: mutation surface (key files cluster)
The schema-pack mutation surface: six foundation modules + a mutate skeleton + stats/sync data plane + CLI verbs + MCP ops + a first-class agent skill.
Key files:
src/core/atomic-write.ts— the shared atomic file writer for brain-repo markdown writers: unique tmp sibling (.tmp.<pid>.<rand>) → write loop until every byte lands (writeSync may legally short-write under disk pressure; a silent short write could atomically install truncated content) → fsync → close → optionalverify(onDiskBytes)callback (throw = abort, tmp removed, target untouched) → mode-preserving atomic rename → best-effort parent-directory fsync (rename durability). Consumed bysrc/commands/backlinks.ts(which verifies withparseMarkdown({validate:true})before the rename). Rename prevents torn writes, NOT lost updates — read-modify-write callers pair it withwithPageLock(backlinks does). Unifying the per-module copies (skillopt/apply-edits, write-through, lint) is a filed TODO.src/core/schema-pack/pack-lock.ts— AtomicO_CREAT|O_EXCLper-pack lock. DELIBERATELY NOT theexistsSync + writeFileSyncTOCTOU shape fromsrc/core/page-lock.ts. Default 60s TTL, refresh every 10s whilewithPackLock(fn)runs,--forcesemantics = "steal stale lock" NOT "skip locking." Lock path per-pack so two packs never block each other.src/core/schema-pack/write-vocabulary.ts— write-time pack-vocabulary enforcement shared by the write surfaces (capture op,add_linkop,gbrain captureCLI).loadActivePackForWriteVocabulary(ctx)pairs the engine's DB-planeschema_packkey with FILE-ONLY config (theloadActivePackForLocalEngineposture) while threadingremote(fail-closed: anything not strictly false is remote) +sourceId; NEVER throws — null means "no resolvable pack", which callers MUST treat as "no vocabulary to enforce" (the write proceeds exactly as before).packDeclaresPageType/packDeclaresLinkType+ message/suggestion formatters reject an EXPLICIT undeclared name with the pack's declared vocabulary in the error so agents self-correct without agbrain schema explainround-trip. The DEFAULTnotepath is never checked here — callers validate EXPLICIT names only, so baregbrain captureworks under a pack that doesn't declarenote.loadActivePackForWriteVocabularyalso swallows a rejectinggetConfig;previewNamesbounds the vocabulary preview in the error (the first 12 names +(N total)). Pinned bytest/write-vocabulary.test.ts.src/core/schema-pack/mutate-audit.ts— ISO-week JSONL at~/.gbrain/audit/schema-mutations-YYYY-Www.jsonl. Privacy-redacted: type names → sha8, prefixes → first slug segment only, matchescandidate-audit.tsprivacy posture. Logs BOTH success AND failure events so theschema_pack_writabilitydoctor check has signal.summarizeMutations()is the cross-surface parity primitive.src/core/schema-pack/registry.ts—resolvePackwalks theextendschain (depth cap viaEXTENDS_DEPTH_WARN/EXTENDS_DEPTH_HARD_CAP), RETAINS each ancestor manifest, materializesborrow_from, and composes all of it intoresolved.manifestthroughmergeInheritedManifest. Every downstream consumer readsresolved.manifest, so doing the merge here is what makes inheritance visible without per-consumer wiring.borrow_fromis selective (only the namedtypes/link_types, and only from the target's OWN declarations), non-transitive, and fail-closed — a missing target throwsUnknownPackErrorvialoadByName, matching the extends path; an omitted category borrows none of it. The alias graph + closure hash are computed on the MERGED manifest, so a cross-pack alias cycle surfaces asAliasCycleErrorat resolve.manifest_sha8/packIdentitystay the CHILD's own bytes — a parent edit does not move the child's identity, so the invalidation path is what keeps a child honest.invalidatePackCache(name?)walks the extends-chain reverse-graph (editing a parent pack must not leave children stale).tryCachedPack(name)TTL-gated fast path: insideSTAT_TTL_MS(default 1000ms, envGBRAIN_PACK_STAT_TTL_MS) returns cached without statting; outside the window it stats every TRACKED file — the extends chain PLUS every borrowed pack — and cascade-invalidates on mtime change (cross-process detection), so editing a borrowed pack invalidates its borrowers. Pinned bytest/schema-pack-registry.test.ts+test/schema-pack-merge.test.ts.src/core/schema-pack/merge.ts— the pure child-wins composition helper behindresolvePack.mergeInheritedManifest(ancestorsBaseFirst, child, borrowed)returns the fully-composed manifest; precedence is child → borrowed → nearest parent … → base. SIX ingest/query-shaping fields inherit:page_types,link_types,frontmatter_links,enrichable_types,filing_rules,takes_kinds.phases+calibration_domainsare DELIBERATELY child-only — they gate real cycle execution (cycle.tspackDeclaresPhase), so inheriting them would silently run phases a pack never declared;mapping_rules,migration_from,extends,borrow_from, and the identity fields are child-only too (all ride the...childspread).mergePageTypescarries the ordering contractinferTypeFromPackdepends on (first-path_prefix-match-wins, array order): the BASE (root,extends: null) pack is the ordered foundation/tail; an override of a base type keeps the base POSITION (Map.setupdates the value, keeps insertion order) so base's curated priority survives; a genuinely-new type from ANY non-base layer — child, borrowed, or a middle pack — is PREPENDED nearest-first, so a more-derived prefix wins regardless of chain depth.mergeByKeykeeps the first occurrence per key walking highest-precedence-first (the order-insensitive keyed fields);frontmatter_linkskeys onpage_type\x00link_type— a NUL, not a space, because both are unconstrained strings and a space-join would collide{"a b","c"}with{"a","b c"}.mergeUnionbackstakes_kinds: UNION not replace, because the Zod default makes an omitted field indistinguishable from an explicit one — so a child can ADD kinds but CANNOT narrow below base ∪ parent. Pure + deterministic: no disk, no engine. Pinned bytest/schema-pack-merge.test.ts.src/core/schema-pack/best-effort.ts—loadActivePackBestEffort(ctx)returnsResolvedPack | null. Single source of truth for the pack-aware wiring sites.nullmeans EMPTY FILTER (NOT hardcoded defaults, which would silently violate the pack).src/core/schema-pack/type-usage.ts— stored-type classifier behind the alias-footgun visibility surfaces:classifyStoredType(type, pack)→ canonical | alias_of (with the canonical type +path_prefixes[0]filing directory) | undeclared, over a STRUCTURAL pack shape so import-file's thinactivePackand the full manifest both satisfy it.sanitizeTypeForDisplaystrips control chars + caps length (type strings come from frontmatter and get echoed into terminals);renderTypeWarningSummaryrenders the once-per-type-per-run lines. Consumers:importFromContent(advisoryImportResult.type_warningat the typeExplicit site — the type is still stored literally, zero filing change), sync/import summary aggregation (+SyncResult.type_warningsso worker-driven syncs surface counts in job results), thestored_type_is_alias/stored_type_undeclareddata-plane lint rules, all gated by configschema.type_warnings(default on; lint rules always active). Pinned bytest/schema-type-usage.test.ts.src/core/schema-pack/lint-rules.ts— 14 pure rule functions.withMutation's pre-write validation gate composes the 10 file-plane rules; the 4 DB-aware rules (extractable_empty_corpus,mutation_count_anomaly,stored_type_is_alias,stored_type_undeclared) need an engine (CLI--with-db; the stored-type pair acceptsLintOpts.sourceIdscoping, not yet threaded from the CLI). Single source of truth consumed by CLI lint + MCPschema_lint+ the pre-write validation gate. File-plane rulelink_regex_catastrophic_backtrack— advisory ReDoS pre-screen flagging the classic nested-quantifier shapes ((a+)+,(a*)*,(a+)*,(\w+)+) in a link_type'sinference.regexviaNESTED_QUANTIFIER_RE. WARNING not error: a hard reject would disable the whole pack on upgrade (pages fall back to legacy typing). The runtime input-length cap inredos-guard.tsis the actual safety net; this rule tells the pack author to fix the pattern.src/core/schema-pack/redos-guard.ts+src/core/schema-pack/link-inference.ts— ReDoS hardening for pack inference regexes.redos-guard.tsprovidesMAX_REGEX_INPUT_CHARS(default 64_000, envGBRAIN_MAX_REGEX_INPUT_CHARS) — a hard input-length cap, the real runtime safety net (catastrophic backtracking needs a long input; a link-extractioncontextis normally a sentence or short paragraph). Over the cap,runRegexBoundedthrows the taggedRegexInputTooLargeErrorand the regex is skipped (degrade-to-mentions) without entering thenode:vm.link-inference.ts:inferLinkTypeFromPackno-budget branch (test contexts) routes throughrunRegexBoundedso the input-length cap + per-regex vm timeout (PER_REGEX_TIMEOUT_MS = 50) apply on every path. Defensive hardening + diagnostics; the deterministic ~3100-file sync-wedge root cause remains open. Pinned bytest/redos-hardening.test.ts+test/schema-pack-lint-rules.test.ts.src/core/schema-pack/query-cache-invalidator.ts—invalidateQueryCache(engine, sourceId?)DELETEs query_cache rows so cached search results bound to old page types don't survive a schema mutation.src/core/schema-pack/mutate.ts— 8-stepwithMutationskeleton (bundled-guard → lock → read → mutator → validate → atomic write → audit → invalidate) backs the 11 single-mutation primitives:addTypeToPack,removeTypeFromPack(with reference check),updateTypeOnPack,addAliasToType,removeAliasFromType,addPrefixToType,removePrefixFromType,addLinkTypeToPack,removeLinkTypeFromPack,setExtractableOnType,setExpertRoutingOnType. Each primitive's business-rule validation + transform is factored into abuild*Mutator(...)pure(manifest) => manifestfunction shared withapplyMutationsAtomic(theschema_apply_mutationsbatch entry point) so single-call and batched mutations can never validate differently.applyMutationsAtomiclocks + reads the pack file ONCE, applies + lint-validates every mutation in the batch against an in-memory manifest, and callswritePackManifestat MOST ONCE — only after the whole batch checks out — so a batch that fails partway leaves the pack file byte-identical to its pre-batch state. Atomic single write via.tmp + fsync + rename— the pack file on disk is NEVER partial, for either a single mutation or a batch. Inline minimal JSON→YAML emitter so YAML packs stay YAML (does NOT preserve comments — pin pack.json if you care about layout).src/core/schema-pack/stats.ts—runStatsCore(engine, opts)returns per-source + aggregate page counts + coverage % +dead_prefixes(declared prefixes with zero matching pages — agent drilldown signal). Multi-source aware (sourceIds[]federated,sourceIdsingle, or whole-brain). PGLite + Postgres parity viaexecuteRaw. Empty brain → coverage:1.0 (vacuous truth).src/core/schema-pack/sync.ts—runSyncCore(engine, opts)chunked UPDATE in 1000-row batches per declared prefix. Concurrent writers never block on a single row >100ms. Write-side scoping viactx.sourceIddirectly (NOTsourceScopeOpts, which inherits OAuth read federation). Idempotent on--applyre-run.src/commands/schema.ts— 14 CLI verbs in the dispatch table:add-type,remove-type,update-type,add-alias,remove-alias,add-prefix,remove-prefix,add-link-type,remove-link-type,set-extractable,set-expert-routing,stats,sync,reload.withConnectedEngineroutesloadConfig()through the canonicaltoEngineConfig()helper and passes the complete result (database_urlanddatabase_path) to factory construction and connect, so PGLite schema commands open the configured brain. Lifecycle-grouped help text (Inspection / Activation / Authoring / Discovery+repair). Pinned bytest/schema-cli-database-path.serial.test.ts.skills/schema-author/SKILL.md— Agent dispatcher for "evolve the schema pack." Triggers: 15+ phrasings incl. "add a page type", "my brain has untyped pages", "propose new types from my corpus", "backfill page types". Explicit Non-goals callout tobrain-taxonomist(files one page) andeiirp(schema-check during iteration) so agents pick the right surface. 7-phase workflow: brain → assess → propose → apply → sync → verify → commit. Lists every gbrain schema CLI verb + every MCP op the skill uses.brain_first: exemptfrontmatter. Required conformance sections: Contract, Anti-Patterns, Output Format.skills/conventions/schema-evolution.md— Canonical convention: "when to add a type vs alias vs prefix." Decision tree: <20 pages → don't pack-codify; 20-100 → alias or narrow prefix on existing type; 100+ → first-class type. Don'ts section + "when to remove a type" + "when to commit the pack" all answered in one place.
Pack-aware wiring is partial. Three follow-ups are filed in TODOS.md: enrichment-service.ts
union widening ('person' | 'company' → string), facts/eligibility.ts
pack-aware ELIGIBLE_TYPES wiring, and 3 doctor checks (schema_pack_coverage,
schema_pack_writability, schema_pack_mutation_audit).
src/core/vector-index.ts+src/commands/doctor.ts:embedding_column_registry— shared pgvector HNSW eligibility policy.hnswIndexExpected(columnType, dims)derives the answer from the canonicalvector/halfvecdimension caps already used by migration index generation. Doctor reports an HNSW-less active embedding column as a healthy exact-scan configuration when its declared width exceeds the applicable pgvector cap, and only emits the index repair recipe when an index is actually supported. Pinned at both cap boundaries bytest/vector-index-lifecycle.test.ts.
Agent bootstrap cluster (the paste-in desktop-agent install)
Normative docs: docs/designs/AGENT_BOOTSTRAP_DESIGN.md (scope) +
docs/designs/AGENT_BOOTSTRAP_PLAN.md (implementation, review-finding IDs inlined).
User-facing contract: docs/guides/bootstrap.md. Runbook the paste block fetches:
BOOTSTRAP_FOR_AGENTS.md (root; carries a version stamp CI pins to VERSION).
src/commands/bootstrap.ts— thegbrain bootstrap {status,interview,render,repo,hooks,verify,uninstall,attach,cloud-setup-script}dispatcher (plus the machine-levelharnesssubcommand — see thecore/bootstrap/harness.tsentry below). Engine-free everywhere exceptverify(which opens/closes its own engine — safe because verify runs with no live serve, before host registration).cloud-setup-scriptis a pure printer (printstemplates/bootstrap/cloud-setup-script.shfor the cloud environment's setup step) and is dispatched entirely BEFORE workspace resolution, so it works from any cwd, including HOME or cwd still matches) ISprocess.env.HOME(falling back toos.homedir()only when HOME is unset) — an unqualified run from a freshly-SSH'd shell must not silently stage a home-directory-scalegit add -Aover~/.ssh/and friends; the refusal is still appended to<home>/bootstrap/install.jsonl(keyed by the rejected candidate path) before returning.status,uninstall, andharnessare exempt from this guard (HOME_WORKSPACE_GUARD_EXEMPT):statusonly reads and prints a report,uninstallremoves exactlyreceipt.created_paths(each containment-checked) — never agit add/commit/push, never a workspace-wide scan — andharnessoperates only onhomeand never even receives awsargument (runHarness, machine-level wiring), so the valueresolveWorkspacereturns for it is used only forLogCtx.wsbookkeeping. Exemptingstatus/uninstallkeeps the recovery path reachable for the guard's own victims: someone who bootstrapped into$HOMEneedsstatusto see what's there anduninstallto remove it, both run with--workspacepointing AT $HOME; the refusal message for the still-guarded subcommands names that recovery path. Known gap: a process already launched with HOME pointed away from the real account home is not caught (Bun'sos.homedir()mirrors process-start HOME rather than doing an independent uid lookup, so it collapses to the same wrong value). Mutating subcommands run under the workspace bootstrap lock; render is gated on interviewcomplete && confirmedand hard-refuses when the workspace origin is a PUBLIC remote (identity files must never land in a public repo — the same template-door gatestatusenforces; unverifiable visibility warns and proceeds, treating the origin as public); the provider key routes to the 0600 config sink and never touches interview state; every MUTATING subcommand appends a line to<home>/bootstrap/install.jsonl(the read-onlystatusandcloud-setup-scriptdo not log).hookson Claude Code writes the committed carrier in a cloud sandbox (writeCommittedClaudeHooks) and the gitignored local file otherwise;uninstalltears down the durability wiring and removes this workspace's per-root push/debounce state, and its opencode teardown sweeps BOTH merged global filenames (under the opencode config-dir bootstrap lock) plus the projectopencode.json, expectation-keyed on this workspace's source id (skipOtherSource) — a gbrain entry from a different workspace is skipped with a note, never silently deleted. Consent answers resolve FAIL-CLOSED:consentAnswertreats a hand-edited/unusable interview value (non-string, empty) as declined — loudly, with a re-record note — never falling through to a permissive bank default;hookson Codex prints a corrective note when a persistedprojectMCP_SCOPE answer is found (raw state read, not the resolver —codex mcp addhas no scope flag, registrations are always user-global) with safe clear instructions.GBRAIN_BOOTSTRAP_ABORT_AFTERis the deterministic kill-mid-phase test seam.src/core/bootstrap/format.ts—agent.jsonmanifest (format_version 1, provisional;initializedsentinel distinguishes a template clone from a bootstrapped workspace) + the machine-local install receipt (<home>/bootstrap/receipt.json) that proves THIS machine ran bootstrap; uninstall is keyed to the receipt, never the repo manifest. Atomic writes;readManifestnever throws (typed states incl. conflict markers).src/core/bootstrap/assets.ts— every template + the question bank embedded via Bunwith { type: 'file' }imports (the chunkers/code.ts pattern) so the compiled binary renders with no repo checkout;DERIVED_TOKENS(GITHUB_REPO_URL, CORPUS_RETENTION_DAYS) is the non-bank half of the template token set the CI bijection guard checks.templates/bootstrap/— the ten{{TOKEN}}identity templates (AGENTS/CLAUDE/SOUL/USER/MEMORY/HEARTBEAT/ACCESS_POLICY/GITHUB/memory-README/gitignore),questions.json(12 asked / 6 required; consent keys;persist:falsesink keys), andtemplate-repo/— the VENDORED deterministic render the release job diffs against before publishing the public template repo. Generic placeholder content only (privacy iron rule; CI-asserted).src/core/bootstrap/interview.ts— interview state at<ws>/state/interview.json(committed; multi-device re-render source). Read-back confirm hash:--confirmmust present the hash of the exact answer set shown to the human, and ANY later answer change clears the confirmation — the single-batch self-confirm attack is structurally impossible. Set-time enforcement: length caps, reject-lists, allowed-lists, control-char strip,{{escaping. Conflict-markered files return agent-readable errors, not stack traces.src/core/bootstrap/render.ts— token substitution with interview values treated as data (line-leading#/<!--/fence escaping), hard-fail on unresolved tokens, never-clobber + timestamped backups on--force, blank-line collapse, byte floors scaled to answered count.--minimalis the deterministic placeholder mode the template-repo generator uses (byte-identical across runs; leaves required tokens as literal fill-me markers; writesinitialized:false).--onlynever writes agent.json.src/core/bootstrap/lock.ts— the bootstrap-run mutex (atomic mkdir + pid liveness + age guard + ownership token; steal requires dead pid AND stale age) and the family's shared typedBootstrapError(GH_MISSING/GH_AUTH carry exit 2 = human action needed).src/core/bootstrap/repo.ts/attach.ts/uninstall.ts— private-repo lifecycle.createPrivateRepo: gh gates, slugified name probe,gh repo create --private --source --push, privacy verified viagh api .private(rate-limit/5xx is VERIFY_UNAVAILABLE, distinct from not-private) before any push, idempotency keyed off the remote URL. A pre-existing origin is adopted (disposition 'adopted') when the authed gh user owns it, there's no recordedrepo_url, and it is SAFE — empty or already carrying our history (assertAdoptableOrigin; a foreign-content repo is refusedORIGIN_NOT_EMPTY, never a silent no-op); this is the create-repo-first path. Org-owned origins and anything else are refused and pointed at attach. Repo-local git identity is set in both create and adopt paths before commit;repo_urlis recorded only after a successful push.attachWorkspace(machine two): requires aninitializedmanifest, writes this machine's receipt, returns structured wiring steps.uninstallWorkspace: receipt-keyed, refuses under a live serve (read-only lock probe — never opens the engine), removes exactly receipt-recorded paths + marker-keyed host entries, keeps the brain unless--delete-brainAND bootstrap created it; never wholesale-deletes the gbrain home. All gh/git through an injectable ExecRunner seam.src/core/bootstrap/hooks.ts+host-specs.ts— host wiring.host-specs.tsis the ONE module owning host-format assumptions (dated spec targets with verifiedAt + doc references: claude-code hooks/settings shapes incl. the 10,000-char hook-output cap and the five hook events; codex mcp-add argv, the streamable-HTTPurl+http_headers = { Authorization = "Bearer <t>" }config shape (loads on codex-cli 0.147.x and 0.149.x; inlinebearer_tokenis rejected at config load by >=0.149),codexConfigPath()honoring CODEX_HOME,claudeUserSettingsPath()honoring CLAUDE_CONFIG_DIR/HOME-else-homedir(), the directory playing the role of~/.claude— NOT forclaudeUserMcpConfigPath, whose default lives at the HOME level as~/.claude.json), which also feedsclaudeUserSkillsDir()/claudeProjectSkillsDir()(native SKILL.md discovery, the harness-bridge install targets; user scope attested, project scope provisional-from-docs) — and the opencode shapes:opencodeConfigDir/opencodeGlobalConfigPath(XDG-only resolution; the.jsoncname preferred for parity withopencode mcp add; OPENCODE_CONFIG/_CONFIG_DIR/_CONFIG_CONTENT deliberately NOT honored — observed INERT in opencode 1.18.18, honoring them would write registrations into a file opencode never reads),opencodeProjectConfigPath,opencodeGlobalSiblingPath(the OTHER member of the global filename pair — writers reconcilemcp.<name>across it because opencode merges both), andOPENCODE_HAS_HOOKS=false— 'gbrain does not wire opencode's plugin/event system yet', not 'opencode has no hooks'). The settings writers are path+marker parameterized:writeClaudeHooksAt/removeClaudeHooksAttake an explicit settings file and a marker VALUE (workspace installs stampbootstrap-v1in.claude/settings.local.json; harness installs stampbootstrap-harness-v1in user-scope settings or a --project dir — the two coexist and each removal strips only its own;refuseOnForeignGbrainMarkerblocks same-file double-wiring), fail-closed on broken JSON (a settings file that doesn't parse THROWS with the path — a writer must never relocate or overwrite a config it can't read), mode-preserving realpath-resolved atomic writes (dotfile symlinks survive), and fixed-or-timestamped backups.addPermissionsAllowEntry/removePermissionsAllowEntrymanage the harness lane'smcp__<name>headless pre-approval with set semantics and NO marker (ownership rides the harness receipt; foreign entries always survive); the add path fails CLOSED when an existingpermissionskey orpermissions.allowcarries a shape it doesn't understand (host security policy is never rewritten on a guess), while removal leaves shapes it can't read untouched and reports nothing-to-remove.registerClaudeMcp/registerCodexMcpbuild argv only (Claude Code takes--scope, project default; Codex has no scope flag —codex mcp addis always user-global;-e/--env GBRAIN_SOURCEso MCP writes land in the workspace source, andserve --surface fullpinned so a pre-existingmcp_surface: verbsconfig row can't silently narrow the bootstrap op surface). The opencode workspace lane execs NOTHING — registration is the directopencode-json.tswrite with an INVERTED scope default (user-global; opencode spawns project-config servers with no trust prompt, soMCP_SCOPE=projectis an explicit opt-in that writes the committed-candidateopencode.jsonwith a PATH-resolved command and prints the sharing warning), verification is config parse-back + a best-effortopencode mcp list --pureprobe — the probe spawns via a held Bun.spawn handle from a fresh EMPTY temp-dir cwd (never the invoking cwd: opencode merges a project opencode.json from cwd and spawns its servers with no trust prompt), is SKIPPED entirely for project scope (printed note; parse-back is authoritative) and on plugin-bearing configs (mcp listis a code-execution surface), and on timeout actually kills the child (SIGTERM → SIGKILL, bounded pipe drain, code 124 into the could-not-confirm branch;probeSpawnis the injectable seam). User-scope writes also reconcile the SIBLING global filename first (ours → removed with a note; foreign → refuse naming both files) so a merge-shadow registration can never survive.detectHarnessprobesOPENCODE/OPENCODE_PID(set in opencode's bash-tool children, observed 1.18.18).src/core/bootstrap/codex-hooks.ts— the codexhooks.jsonwriter (SessionEnd capture lane), built entirely on the datedCODEX_HOOKS_SPEC_TARGET(verified 2026-08-25 against pinned codex-cli 0.147.0, live + tag-source evidence; re-run the observation gate on codex version bumps). The three load-bearing facts: hooks.json is TOP-LEVEL deny-unknown-fields (ownership rides the legaldescriptionslot + a command-substring token, never a_gbrainkey); user-layer hooks are TRUST-GATED and fail SILENTLY (the writer lands BOTH the hooks.json entry AND its[hooks.state."<path>:session_end:<g>:<h>"].trusted_hashconfig.toml entry — sha256 of the recursively-key-sorted compact canonical JSON of the normalized handler identity — inside a gbrain-managed marker block, backing config.toml up to.hooks.bakso it never clobbers the MCP writer's.bakrollback anchor); SessionEnd handlers are hard-killed at 3s (the command captures stdin to a mktemp file and detaches a nohup grandchild running the realgbrain hook session-end --harness codex— live-verified end-to-end). Deliberately NOGBRAIN_SOURCEin the command: hooks.json is user-global, so a baked source would stamp every codex session on the machine; session-end resolves everything from the payload. a fresh install APPENDS our group last while a re-run REPLACES it IN PLACE (a single well-formed install never shifts a foreign group's trust index; dropping DUPLICATE gbrain groups can shift foreign groups sitting after them — the writer prints a note naming the re-trust); the whole command is wrapped in a top-levelsh -c '…' <bin>so a non-POSIX $SHELL (fish/csh) never parses the POSIX script; the foreign-trust-entry guard is a tolerant [hooks.state…] header match (not byte-exact); a golden known-answer hash vector pins the canonical recipe; removal also deletes the gbrain description when byte-identical to ours and NOTES the inherent index shift for foreign groups that sat after ours; unparseable hooks.json is never touched, on write or removal; the residual stale-trust-index case (user reorders their own groups) degrades to silent non-execution, which doctor'scodex_hooks_never_firedrung names. Pinned bytest/codex-hooks-writer.test.ts.writeCommittedClaudeHooksis the second, COMMITTED hook carrier: it writes marker-keyed entries into the workspace's checked-in.claude/settings.jsonusingbuildPortableClaudeHookCommand(PATH-resolvedgbrain, fail-open when the binary is absent) so teammates cloning the repo inherit the hooks;committedHookEvents(ws)feeds the local writer'scarriedEventsso the two carriers never double-wire an event, andremoveClaudeHooksstrips both.src/core/bootstrap/codex-toml.ts— the ONE direct codex-config writer (codex mcp addcannot express an inline credential, and framework-spawned codex inherits no shell profile for the env-var lane). One[mcp_servers.<name>]table between full-line markers carryingurl+http_headers = { Authorization = "Bearer <token>" }(NOT inlinebearer_token, which codex-cli >=0.149 rejects at config load;parseCodexBlockBearerin harness.ts keeps a legacybearer_tokenread fallback so existing tokens still recover); everything outside survives byte-for-byte. Foreign-server detection PARSES the config (Bun.TOML.parse, no dependency) with our block stripped — a header-only regex would false-negative inline-table/dotted/quoted spellings into a codex-bricking duplicate table; rewrites re-anchor at EOF; renders are parse-validated with an ours-keys-exactly assert before rename; damaged markers refuse; 0600 tmp/target/.bak (the .bak carries the previous token on re-runs); CRLF preserved, missing trailing newline repaired. Also exportsrenderCodexHttpServerBlock({name, url, bearerToken})— a MARKER-FREE, parse-validated[mcp_servers.<name>]TOML render for paste-into-config surfaces (gbrain agent register's codex block); rendering only, never writes a file. Pinned bytest/codex-toml.test.ts.src/core/bootstrap/serve-health.ts— serve/healthprobe + scopes version-skew floor, re-exported byharness.ts.probeServeHealth(mcpUrl, fetchFn, timeoutMs=3000)GETs<base>/healthand returns{ok, version?, engine?, detail?}, never throws;fetchFnis an explicit argument (no ambient fetch, no engine, no config).isServeOlderThanScopes(v)compares against the PINNEDSCOPES_MIN_SERVE_VERSIONconstant — a comparison against the moving CLI VERSION would false-flag every scope-aware serve on the next release. An older serve verifies scoped tokens as FULL ACCESS, so callers print the floor line unconditionally.src/core/bootstrap/opencode-json.ts— the ONE direct opencode-config writer (workspace stdio lane, harness remote lane, connect--install). ALL edits ride jsonc-parsermodify/applyEdits(comments/formatting/EOLs survive byte-for-byte outside the edited range — opencode's ownmcp addpreserves comments, and JSONC is its effective grammar for BOTH.jsonand.jsoncfilenames, which MERGE when both exist). Ownership is a 4-state STRUCTURAL FINGERPRINT (opencodeEntryKind→ ours-same-source | ours-other-source | foreign | absent; GBRAIN_SOURCE EQUALITY for local entries, receipt-url match or the{env:GBRAIN_REMOTE_TOKEN}interpolation for remote), never a marker key (a future strict-schema flip must not brick the host). Distinct read-failure classes (ENOENT create / empty-as-{} / unreadable refuse); foreign refusal on write AND remove; post-render validation (our entry round-trips + every other key survives) keeps the original on failure; 0600 for inline-bearer targets; backups are UNIQUE per operation (<config>.bak-<hex>, returned in the result withwrittenText, the exact rendered bytes) so overlapping runs can never clobber each other's snapshot, and a backup is chmod'd 0600 whenever the COPIED content carries an inline bearer (write AND remove paths); the ours-other-source refusal is caller-appropriate (url +--forcewording on the remote/expect-url path, GBRAIN_SOURCE wording on the local path);removeOpencodeMcpEntrytakes a caller expectation + optionalskipOtherSource— the uninstall lane passes THIS workspace's source id so an ours-other-source match becomes a calm skip-with-note (another workspace's registration is never deleted; foreign still refuses);reconcileOpencodeSiblingGlobalclears a same-name gbrain entry from the OTHER global filename before a global write (foreign → refuse naming both files) — opencode merges both, so a leftover sibling entry is a shadow registration; the bun-run ownership lane is ANCHORED (an exactgbrain/gbrain-*path segment in some arg — a gbrainy-fork cli path is NOT ours, fail-closed);parseOpencodeEntryBearerrecovers the harness--statustoken url-matched only;opencodeRemoteEntryExistsis the stdio-lane ownership arbiter (codexBlockOwnsName analog). Callers holdacquireBootstrapLock(config-dir → opencode-dir ordering). Pinned bytest/opencode-json.test.ts(incl. the fingerprint truth-table suite).src/core/bootstrap/atomic-write.ts— the ONE atomic config-file writer for bootstrap host surfaces: symlink-target-resolving, mode-inheriting (freshModefor new files,forceModefor secret-bearing targets), random-suffix tmp + rename, tmp unlinked best-effort when write/chmod/rename throws (ENOSPC/EACCES never strand.tmp-litter), and the existsSync→realpathSync race falls back to fresh-path resolution instead of throwing raw ENOENT. hooks.ts (JSON), codex-toml.ts (TOML+EOL), and opencode-json.ts (JSONC) all swap through it; serialization and EOL policy stay caller-side.src/core/bootstrap/harness.ts—gbrain bootstrap harness: machine-level wiring of framework-spawned Claude Code/Codex/opencode sessions to a RUNNINGgbrain serve --http, noagent.json. The opencode target mirrors the codex posture: forced-wire on explicit--harness opencode(the JSONC writer needs no CLI), one managedmcp.<name>remote entry with the inline bearer header (0600) viaopencode-json.ts, rotation across a url change recognized through the PRIOR receipt's url, failed-smoke rollback restoring the run's UNIQUE backup or removing a fresh entry — guarded by a content compare against the exact text this run wrote, so a NEWER registration that landed after the lock released is never clobbered (rollback skips with a note; the fresh mint is revoked either way), with the backup unlinked once consumed or once the smoke verifies —--removeclassifying against the receipt url (not-ours skips with a note), and--statusbearer recovery viaparseOpencodeEntryBearer(url-matched). Consent block in the honesty register (reach as fact, transcript capture its own numbered item +--no-capture, off-ramps in the same breath; non-TTY requires--yes); /health probe with a loopback guard on--url(remote brains aregbrain connect's charter;--tokenmakes it a pure registrar); mint-first rotation (previous token revoked BY ID only after every target confirms + the bearer smoke passes — clients are never dead mid-swap; revoke-by-name never happens); the smoke is canary-gated: a random same-format bearer must FAIL auth before the real token is sent, so a loopback impostor is caught whichever way it answers, and ANY failed smoke rolls back symmetrically — fresh registrations removed, replaced ones restored (an unrestorable replacement fails the target honestly), the freshly-added pre-approval stripped, and the fresh mint revoked immediately so nothing live stays pointed at an unverified endpoint; thepermissions.allowpre-approval only lands after the MCP registration itself confirms; stale prior-target cleanup runs AFTER the smoke passes, so a run that failed to establish its replacement never unwires working prior wiring; write-ahead harness receipt (targets persist aspendingat mint time and flip per-target, so a crash leaves consumable state); registration ownership checks (--forceto replace a foreign-url server;--removeskips what it no longer owns); user-XOR-project hook scopes;GBRAIN_HOOK_LANE=harnesson hook commands sogbrain hookyields to a workspace bootstrap install in the cwd (Claude Code merges settings scopes — same event must not fire twice); the hookGBRAIN_SOURCE, the receiptsource_id, and the token grant bind to a validated--source(scalar write floor), else to the sourceHarnessDeps.resolveHookSourceresolves through the SAME chain the serve's resolve-IPC binding runs (resolveSourceWithTier: env → dotfile → local_path →sources.default→ sole populated non-default → seed default) with that source's federated read set as the grant, so the hooks'turn_contextclaim is neversource_mismatch; a resolution landing ondefault/__all__is the federated floor. Lookup failures are NOT papered over withdefault: a typo'd--sourceor stale env/dotfile throwsSOURCE_UNRESOLVEDbefore any mint or receipt; an explicit--sourcewith an unopenable engine binds unverified with a warning; a live PGLite serve (engine cannot open) refuses with theLIVE_SERVEescape hatches when no token was supplied and, on the--tokenlane without--source, wires the hooks UNPINNED with a warning (an unpinned hook resolves through the live serve's own binding; the receipt records the federated floordefaultplussource_pinned: false); any other lookup failure throwsSOURCE_UNRESOLVEDnaming--source; Postgres degradation + serve-version-skew honesty lines (isServeOlderThanScopespinned toSCOPES_MIN_SERVE_VERSION, the first scope-aware release, so later CLI bumps never re-trigger the warning);--statusprobes the live truth with host-config token recovery (redacted; the Claude Code lane recovers a bearer ONLY from a registration whose URL matches the receipt — never another install's credential; the codex fallback reads OUR managed block at the receipt-recorded path, itsurlkey not yet compared — TODOS.md; the opencode fallback IS url-compared insideparseOpencodeEntryBearer) and honestverify: unavailabledegrades, with a cron-honest exit contract: 0 only when serve + token + every target verify and the rotation has converged (honest degrades count), 1 on an unreachable serve, a failed token verify, failed/pending targets, unconverged rotations, or a half-removed receipt (zero targets, minted token awaiting deferred revoke — also a doctor FAIL); no receipt prints honestly and exits 0 in plain mode, 2 under--json;--jsonon apply emits ONLY the final JSON document on stdout (prose → stderr);--removeis engine-free-first and defers the token revoke with exact instructions under a live PGLite serve. Locks on the gbrain HOME, plus the host config dir around codex/opencode writes, removals, rollbacks, and stale cleanup (stale prior-target cleanup nests the codex/opencode config-dir lock inside the held claude config-dir lock — same claude-first ordering as apply/remove, same-dir skip — so its read-modify-write can't interleave with a concurrent config-dir-locked writer);bootstrap uninstallholds the HOME lock across the whole teardown and runs harness removal FIRST (revoke needs the DB alive;--delete-brainwould destroy harness.json) and treats NO_RECEIPT/HOME_GUARD/RECEIPT_MISMATCH as "no workspace install" once harness wiring is cleared. Pinned bytest/bootstrap-harness.serial.test.ts+test/e2e/bootstrap-harness-lifecycle.serial.test.ts.src/core/token-mint.ts— programmatic legacy-token mint/revoke for the harness lane:mintLegacyToken(scopes → the TEXT[] column; requiredtakesHoldersper-token allow-list, harness default['world']; optionalpermissions.source_idfederation array mirroring the stdio lane'slocalFederatedSourceIdsgrant, element 0 = write floor;RETURNING id) andrevokeLegacyTokenById(never touches same-name siblings). ExportsTOKEN_ID_RE, the canonical token-id shape shared with theauth revoke --idCLI gate. CanonicalhashToken/generateTokenfromsrc/core/utils.ts. Pinned bytest/token-mint.test.ts.src/commands/hook.ts— engine-freegbrain hook {session-start,user-prompt,stop,session-end,compact}(zero engine modules in the import graph; a hook must NEVER contend for the PGLite writer lock). WhenGBRAIN_HOOK_LANE=harness(set on harness-mode hook commands), each event PARSES the cwd's workspace settings carriers —.claude/settings.local.jsonAND the committed.claude/settings.json— and yields silently only when a livebootstrap-v1hook entry wires THAT event: the workspace install wins over user-scope harness wiring so the merged settings scopes can't fire the same event twice, unwired events still run, and a repo committing marker-lookalike strings in unrelated fields can't disable the machine-wide capture lane (fail-open — a read/parse hiccup means the event runs normally). user-prompt: stdin hook JSON → transcript-path confinement → last-4-turns window + cross-turn dedupe (the transcript'shook_additional_contextattachments — the blocks WE previously injected — ridepriorContextText, deduplicated and capped atPRIOR_CONTEXT_MAX_BYTES(32KB, so the advisory payload can never blow the IPC message cap; one oversized block is skipped without evicting smaller ones), so a page is volunteered once per session, not once per mention; structured extraction only, never raw-turn substring matching) → IPC turn_context (with a feedback-loopchannel,--harness <claude-code|codex>, default claude-code) →hookSpecificOutput.additionalContextunder an 800ms self-deadline; every path fails open (exit 0, empty stdout) with a typed reason in the heartbeat. Listed in cli.ts'sSTARTUP_HOOK_SKIP_COMMANDS(per-prompt invocations must never spawn a detached check-update child; membership is pinned by a source grep — the runtime path no-ops under NODE_ENV=test). session-start: file-plane digest (allowlisted MEMORY.md sections, push staleness, prior failures) + crashed-session recovery push gated on an initialized manifest. session-end: confined full-transcript parse → redacted corpus write (session-id filename dedup, retention prune; the corpus scan runs highEntropy whenever the relay is on — the relay child derives its egress task line from the corpus text — and stays vendor-prefix-only otherwise) → parser-drift detection (bytes>0 && turns==0is loud) → best-effort workspace push. session-start recovery + session-end pushes run in a DETACHED child so the hook returns immediately (a synchronous inline push would block harness startup on a dirty tree); the corpus write is atomic and clears the stale ingested/in-progress sidecars so a resumed session re-ingests its appended transcript. Heartbeat JSONL is counters/reasons only by construction;readHeartbeatTailfeeds doctor.GBRAIN_HOOKS=0kills all events. Session-end capture dispatches per harness throughcaptureSpecFor(io.harness)(src/core/transcripts/capture-spec.ts): claude-code and codex each pin their OWN confinement root + parser; unknown/undefined/opencode resolve to the claude spec (golden-pinned). The codex lane confines tocodexSessionsDir()+codexArchivedSessionsDir()(codex moves rollouts into the flat archived store), parses rollouts viaparseCodexHookTranscript, and falls back to bounded id-matched discovery across both stores when the SessionEnd payload carriestranscript_path: nullor a path that no longer exists (missing_path;unreadableis reserved for EACCES/IO faults) — the fallback and prior-run relay failures are DEFERRED heartbeat reasons, applied after current-session reasons so they never mask one (first-degrade-wins). ONE gate covers the whole optional Memorable seam —memorableGateAllowed(cfg)in hook-heartbeat.ts: the config flag (integrations.memorable.enabled === true, file plane), theGBRAIN_MEMORABLEkill switch (any common negative spelling, trimmed; env can only ever disable), AND the gbrain-authored consent stamp (memorable-consent.json— see hook-heartbeat.ts). Off is the default: no tool-call collection, no receipt, no spawn. On, session-end runs the sharedredactedToolCallsJson(span-filtered to the corpus window, highEntropy ALWAYS on — those args are the one artifact that leaves the machine) andrecordAndRelayReceipt(receipt dedup by post-redaction content hash → CLI-side consent evidence →resolveMemorableBin→ detached fire-and-forgetmemorable recordspawn; both capture lanes trim memorable-relay.jsonl). Fail-closed egress guards: a scanner-import failure skips the receipt+relay entirely (memorable_relay_skipped_unscanned— parity with the openclaw lane), and a newest-mtime discovery GUESS (no session-id match) writes the local corpus but never relays (memorable_relay_skipped_newest_guess— the newest rollout can be a different, still-running session). A payload without session_id adopts the transcript's own id post-parse (never a shared unknown.txt corpus), and the gate answer is hoisted BEFORE the parse and collection is opt-in, so the default gate-off population never collects tool calls (collectToolCalls: memorableAllowed). Operator-facing doc:docs/memorable-agents.md(install isnpm i -g memorable-cli; closed source, no source install, no embedding model to configure).src/core/transcripts/claude-code-jsonl.ts— the Claude Code transcript parser as a dated spec-target (tool_use/tool_result/thinking/image/sidechain/summary/compact-boundary shapes; placeholders for non-text content); also extractsinjectedContextBlocks— thehook_additional_contextattachment lines a gbrain hook previously injected (verified live against claude CLI 2.1.224; marker-filtered, so a foreign hook's blocks are excluded and another tool's output can't suppress volunteering — a same-user mislabeling guard, not an authenticity check), the user-prompt hook's cross-turn dedupe input;confineTranscriptPath(contained under~/.claude/projects,.jsonl, lstat-rejects symlinks, byte cap; under WSL, a Windows drive-literaltranscript_path— Claude Code on the Windows host invoking hooks viawsl.exe— is translated throughsrc/core/wsl-paths.tsand then held to the SAME confinement, rooted Windows-side, so translation never widens what a hook may read). Fixtures:test/fixtures/conversation-formats/claude-code.jsonl(synthetic, privacy-guarded) +test/fixtures/hook-transcript.jsonl(real captured hook round-trip).parseTranscriptadditionally returnstoolCalls, but collection is OPT-IN (collectToolCalls: true; the bare parse — every per-prompt lane — collects nothing, so tool INPUTS are never retained for users who never opted in): the tool name + input args for everytool_useblock, oldest → newest, each joined to itstool_resultoutcome bytool_use_idand then stripped of that id so no transcript-internal identifier reaches a consumer; every STRING in a collected input is bounded toTOOL_CALL_VALUE_MAX_CHARS(32k) with an explicit…[N chars omitted]marker (capToolCallInput, shared with the codex lane — one receipt-size ceiling for every harness). It is a parallel extraction, never a change toentryToTurn— that function's placeholder-only[tool: name]rendering is load-bearing for the token-budget-constrained ambient-recall path; existing callers that ignore the field see no behavior change.src/core/transcripts/grok.ts— the Grok Build (grok CLI) transcript adapter: one session directory (~/.grok/sessions/<url-encoded-cwd>/<uuid>/chat_history.jsonl) = one session;GROK_HOMErelocates the root. Turn selection is STRUCTURAL (mapGrokLine, exported so the datedSPEC_TARGETmapping is pinned): user turns aretype:'user'text blocks WITHOUTsynthetic_reason(injected system_reminder/task_completed rows aretyped, i.e. intentionally text-free, as are tool_result-only user arrays); assistant turns need non-empty stringcontent(tool-only rows —content''/null WITHtool_calls— aretyped); system/reasoning/tool_result/backend_tool_call rows aretyped; unknown row typesskip. A recognised HUMAN-turn row whose content this parser cannot decode (usercontentmissing / non-string / non-array / an array holding non-block entries; assistant non-stringcontentwith notool_calls) ismalformedand counted as SKIPPED, never typed, so a file made only of such rows is neverexpectedEmpty— the ingest drift signal fires (bytesRead > 0, sessions 0,cleanScan=false, watermark frozen) andzeroSessionsReasonnames the malformed row count, instead of an upstream schema change silently advancing the watermark past whole conversations.chat_history.jsonlcarries NO per-message timestamps: session times come from the siblingsummary.json(created_at/last_active_at; a malformed summary is ignored, an unparseable date is never fabricated) and, because grok writessummary.jsonat session END, an in-progress session or partial rsync falls back to the log file's own mtime — a real filesystem time — withraw.timestamp_sourcestamped'summary.json'vs'file_mtime'so consumers can tell them apart (without the fallback the render refusal would freeze the--since-lastwatermark for the whole grok root). Every message carries the session bounds (last message =last_active_at, the rest = start). Session id: summary → UUID dir name →stableIdFromPath(a sha256 prefix of the log path, never the barechat_historybasename, which would collapse every such session onto one page). Sidecars (updates.jsonl, events.jsonl, rewind_points.jsonl, summary.json, prompt_history.jsonl) are not sessions — discover.ts's grok-sidecar filter is scoped to PROVEN grok trees and the head sniff rejects claude-family keys (grok detection runs after claude-code; an oversized system head still detects through the truncated-line sniff); a percent-encoded cwd segment that fails to decode yieldscwdundefined with the session id intact.src/core/transcripts/capture-spec.ts+codex-hook-lane.ts— the per-harness session-end capture seam.capture-spec.ts: the satisfies-anchoredCAPTURE_SPECSrecord ({confine, parse, discover?}) for the STDIN-DRIVEN lanes (claude-code, codex);captureSpecFor()resolves unknown/undefined/opencode to the claude spec (golden-pinned bytest/codex-hook-lane.test.ts); openclaw is deliberately NOT a member (its lane is in-process, trusted-plane — see context-engine.ts).codex-hook-lane.ts:confineCodexTranscriptPath(the same S3#8 ladder as the claude root, pinned at BOTHcodexSessionsDir()ANDcodexArchivedSessionsDir()— codex MOVES a rollout into the flatarchived_sessionsstore on archive, so fencing only the live store refused a still-valid path; both CODEX_HOME-resolved since the spawner IS codex; an explicitly pinned test-seamrootstays single-root; ENOENT/ENOTDIR on the path ismissing_path— the rung hook.ts gates its discovery fallback on — whileunreadableis reserved for EACCES/IO faults, so the codex session-end heartbeat reason for a nonexistenttranscript_pathafter a failed discovery istranscript_missing_path, nottranscript_unreadable; no WSL branch v1),parseCodexHookTranscript(ParsedTranscript shape over the adapter's exportedmapCodexLine; same OPT-INcollectToolCallscontract asparseTranscript, and the samecapToolCallInputbound on observed args; head+tail over budget so session_meta identity survives; tool calls from the OBSERVED args keys —custom_tool_call.inputfixture-verified,function_call.argumentssource-verified at rust-v0.147.0; tolerant JSON-string parse; NOresultjoin — 0.147.0 persists no success flag on*_outputrows;compactedrows become boundary positions), anddiscoverNewestCodexRollout(bounded newest-first walk over the dated live store, then a flat pass over the archived store under the SAME 4096 dirent cap — skipped once the cap trips; id-in-filename match, and an id-matched archived hit keepstranscript_discovered(never relabelled_newest, which hook.ts reads as a guess and bars from the relay); symlink-reject — the fallback fortranscript_path: nullor a moved rollout, and the seam for SIGKILL'd sessions, which never fire SessionEnd). Engine-free by construction; never imports discover.ts. Pinned bytest/codex-hook-lane.test.ts+test/codex-hook-lane-archived.test.ts+ the runHook-driven matrix intest/memorable-relay.serial.test.ts.src/core/wsl-paths.ts— WSL Windows-drive path translation shared by the doctor asset checks (src/commands/doctor-asset-paths.ts) and hook-transcript confinement (confineTranscriptPathabove). ExportsWINDOWS_DRIVE_PATH_RE,translateWindowsPath(p, mountRoot)(C:\Users\x\file→<mountRoot>/c/Users/x/file; null for non-drive input so callers keep the original untouched),parseWslAutomountRoot(conf)(the/etc/wsl.conf[automount] rootvalue, default/mnt), anddetectWslMountRoot()(null off-WSL). Mechanical detection + translation only — POLICY stays with callers (skip vs stat vs confine): the hook lane confines the translated path to the known Claude config tree, doctor decides skip-vs-stat. Pinned by the WSL cases intest/claude-code-jsonl.test.ts.src/core/context/turn-context.ts— server-side per-turn assembly: reflex pointers + volunteered pages (≤3) + hot facts (alwaysvisibility=['world']— the IPC path never widens what MCP would return) under a "data, not instructions" envelope, trimmed to ≤8KB (the harness caps hook output at 10,000 chars). The ambientcontext_packanddeltamodes are world-only across every arm by default; only an explicit trusted-localinclude_privatewidens entity cards, changed pages, threads, and facts together. The result exposespointersAND post-trimvolunteered— exactly what the rendered text carries — so the IPC delivery point can log the feedback loop without ever counting a trimmed-out page. Reuses the hot-memory cache keyed by typed sessionId. Engine-agnostic.src/core/context/resolve-ipc.ts(IPC v2) — discriminated-union requests (absentkind= legacy resolve;turn_contextcarriesprotocol: 2+ a shared secret from a 0600 file in the data dir, plus an additivechannelfor feedback-loop attribution — wire channel claims are validated to the harness channels at the logging site, anything else logs as the default hook channel), handler map, named response types, per-kind timeouts/size caps, socket + parent dir permissions set before exposure, server-side source binding (cross-source requests rejected), protocol echo (a response without it = stale serve → loud degradation). Bind policy:startResolveIpcServerconnect-probes the path first — a live owner makes it return null (that serve stays the IPC provider; a transient serve never unlinks a live socket), only a dead/absent owner's entry is unlinked (unconditionally, since Bun on win32 cannot stat the AF_UNIX socket file it leaves behind) before listen. The connection handler processes exactly ONE request per connection (trailing bytes mid-await never double-process a line or double-log a delivery); the client clamps a too-big request below the message cap by dropping the advisorypriorContextTextBEFORE any conversation turn. Delivery seams:onDelivered(resolve kind) andonTurnContextDelivered(turn_context kind) both fire ONLY after the response write succeeds — a block abandoned before the serve responded is never counted (serve's callback logs the delivered block's volunteered pages + pointers tocontext_volunteer_eventsunder the request channel); write-accept still isn't proof of injection (the client can trim/drop after receipt), which is why thevolunteer_channelsdoctor check reconciles counts against the hook heartbeat. v1 clients and servers interoperate untouched. Serve-delegated sync rides the same socket as three additive secret-gated kinds (sync_start/sync_status/sync_abort, wire shapes + fail-closed option validation insrc/core/context/sync-ipc.ts): start+poll, one line per connection, O(1) handlers with no server budget race; a pre-delegation serve answersunknown_kind:*without the protocol echo and the client degrades to the typed stale-serve refusal.src/core/context/turn-context.ts— server-side per-turn assembly: reflex pointers + volunteered pages (≤3) + hot facts (alwaysvisibility=['world']— the IPC path never widens what MCP would return) under a "data, not instructions" envelope, trimmed to ≤8KB (the harness caps hook output at 10,000 chars).deltapages carryupdated_atfromPage.updated_at_iso(the column's microseconds, projected bylistPages), sonext_cursor.sinceand the sessionlast_wake_at(read back viato_charinsession-state.ts) resume from the exact row — a millisecond-rounded cursor re-delivers same-millisecond pages on the next wake; thedeltaop keeps an already-canonical 6-digit ISOsinceverbatim instead of re-rounding it through a JS Date. The ambientcontext_packanddeltamodes are world-only across every arm by default; only an explicit trusted-localinclude_privatewidens entity cards, changed pages, threads, and facts together. The result exposespointersAND post-trimvolunteered— exactly what the rendered text carries — so the IPC delivery point can log the feedback loop without ever counting a trimmed-out page. Reuses the hot-memory cache keyed by typed sessionId. Engine-agnostic.src/core/context/resolve-ipc.ts(IPC v2) — discriminated-union requests (absentkind= legacy resolve;turn_contextcarriesprotocol: 2+ a shared secret from a 0600 file in the data dir, plus an additivechannelfor feedback-loop attribution — wire channel claims are validated to the harness channels at the logging site, anything else logs as the default hook channel), handler map, named response types, per-kind timeouts/size caps, socket + parent dir permissions set before exposure, server-side source binding (cross-source requests rejected), protocol echo (a response without it = stale serve → loud degradation). The connection handler processes exactly ONE request per connection (trailing bytes mid-await never double-process a line or double-log a delivery); the client clamps a too-big request below the message cap by dropping the advisorypriorContextTextBEFORE any conversation turn. Delivery seams:onDelivered(resolve kind) andonTurnContextDelivered(turn_context kind) both fire ONLY after the response write succeeds — a block abandoned before the serve responded is never counted (serve's callback logs the delivered block's volunteered pages + pointers tocontext_volunteer_eventsunder the request channel); write-accept still isn't proof of injection (the client can trim/drop after receipt), which is why thevolunteer_channelsdoctor check reconciles counts against the hook heartbeat. v1 clients and servers interoperate untouched. Serve-delegated sync rides the same socket as three additive secret-gated kinds (sync_start/sync_status/sync_abort, wire shapes + fail-closed option validation insrc/core/context/sync-ipc.ts): start+poll, one line per connection, O(1) handlers with no server budget race; a pre-delegation serve answersunknown_kind:*without the protocol echo and the client degrades to the typed stale-serve refusal.src/core/facts/visibility.ts—resolveDefaultVisibility(engine)/resolveVisibilityParam: the ONE resolver behind all four facts-visibility default sites (facts.default_visibilityconfig key; explicit caller value always wins; invalid values fail closed to private). Bootstrap sets the workspace brain's default toworldso the principal's own sessions can recall their facts — a documented, security-relevant knob. NOTE the deliberate asymmetry with the ambient-writeback TEMPLATE resolver (writeback-config.ts below): this READ/extract default fail-closes UNSET to private; write GUIDANCE treats unset as world (therememberverb's own default) because telling agents to write private-by-default on a default install makes their facts invisible to their own later remote sessions.- Ambient memory writeback cluster (opt-in,
memory.auto_writeback = off|salient|all, default off; guidedocs/guides/ambient-writeback.md):src/core/facts/ttl-parse.ts— dependency-free TTL grammar leaf (parseTtlShorthandnon-throwing typed results;validateTtlConfig— duration-shorthand-only, positive, ≤365d for the transient-TTL config).ops/facts.ts'sparseTtlParamwraps it with the byte-identical frozen verbError copy; the engine-free hook child shares the grammar without touching the gateway-reaching ops graph.src/core/facts/writeback-config.ts— config resolution, fail-closed OFF.gbrain config set memory.*DUAL-WRITES (file mirror first for the engine-free readers, then the AUTHORITATIVE DB plane;src/commands/config.tsMEMORY_DUAL_PLANE_KEYS branch, which also stamps the resolvedmemory.visibility_posturefile mirror for the engine-free harness renderer — and afacts.default_visibilityset/unset RE-STAMPS that mirror so installed blocks can always converge; a DB-write/delete failure on these keys exits NON-ZERO naming the unchanged runtime value, andconfig getreports them DB-first with a drift warn). MOUNT RULE: when the selected brain is not the host (--brain/GBRAIN_BRAIN_ID/.gbrain-mount), every dual-plane lane (set, unset,unset --pattern, the posture re-stamp) writes the DB row ONLY and says so — the machine-local mirror gates the HOST's Stop hook, and enabling a team mount must never opt the host's conversations into banking; an unresolvable brain selection also skips the mirror (fail toward not mutating host state). The engine path resolves mode/ttl from the DB ONLY — the machine-global file mirror never enables a per-brain gap (a mounted/selected brain can't inherit another brain's opt-in); a provided fileCfg serves drift detection (plane_drift: DB row ABSENT + file enabled; explicit DB 'off' is intent, never drift) and the LKG override (a file-mirror explicit 'off' beats a cached ENABLED bundle on read failure).resolveWritebackConfigcaches a per-engine LAST-KNOWN-GOOD bundle ATOMICALLY (mode+ttl+visibility together; no LKG + read failure = OFF+read_error).ambientOptsFrom(wb, {remember, extractFacts})gates the section on the caller's ACTUAL callable set — noremember⇒ no section (bound-client fences, clamped surfaces).visibilityPostureFromRawowns the rule: unset → world, only explicit non-world → private.src/core/facts/writeback-instructions.ts— the ONE ~15-line instruction-section builder feeding BOTH the MCPinstructionscomposition and the bootstrap-managed harness blocks (surfaces structurally cannot drift). Mode-conditional candidate policy, literal resolved TTL, surface-honest extract_facts mention, visibility posture with the "world = agents on this brain, not the public internet / never widen" rule.src/core/facts/writeback-gate.ts— the engine-free zero-LLM Stop-hook salience gate: frozen skip-reason vocabulary (empty/too_short[CJK-aware 20→10 floor]/ack_or_greeting/slash_command/question_only/quoted_or_tool_output/bulk_paste>8KB), first hit wins; ok returns the NFC/collapsed normalized text + sha256-hash24 — the turn's idempotency key (same turn ⇒ same.wb-filename ⇒ embedding-free dedup that works keyless).src/core/facts/writeback-audience.ts— personal-vs-shared classifier for the consent nudge. Declaration (brain.audience, stamped by company-brainify's Phase-5 handoff, the bootstrap interview's SURFACE_MULTIUSER answer — applied the momentinterview --setrecords it, BEFORE the agent runs init — or the operator) BEATS the conservative heuristic (≥3 distinct non-automation MCP clients active 30d viamcp-usage.ts,likely_automationexcluded);reasons[]always names the evidence. The machine-global file-mirror declaration speaks for the HOST brain only — a mounted/selected brain falls through to its own DB row or heuristic. Fail directions: config read failure →unknown(nudge stays silent), usage-read failure with a reachable DB →personal(missing multi-client infra is personal-shaped). Never enables anything.src/core/onboard/writeback-nudge.ts— the fire-once consent ASK (init epilogue +runPostUpgrade): double gate (sentinelmemory.auto_writeback_notice_shown+ setting unset), personal-audience only, suppressed on mounts/thin-clients/env bypass,[AGENT]-relayed disclosure + ask (non-TTY IS the relay path — deliberately unlike runInitNudge), sentinel stamped AFTER printing (a decline is permanent; a shared classification does NOT burn the sentinel), whole body try/caught (init/upgrade must survive a crashing nudge). The recurring reminder issrc/core/advisor/collect-writeback-consent.ts(info,ask_user, NO dispatch_id — consent is never--apply-able; local-only; fires only AFTER the sentinel so the advisor is never the first ask).src/core/bootstrap/instructions-block.ts— the managed harness block (<!-- gbrain:ambient-writeback:begin/end -->): spliceCompiledBlock discipline verbatim (idempotent replace-interior, THROW on damaged markers, marker-equal body lines neutralized), header names mode + serve endpoint (multi-brain last-wins stays visible), remove/strip helpers never delete a user file. Harness lane (src/core/bootstrap/harness.ts):kind:'instructions'targets (additive under receipt version 1 — old binaries won't unwire them, documented) for user-scope CLAUDE.md +$CODEX_HOME/AGENTS.md(paths + the AGENTS.override.md exclusivity probe are dated PROVISIONAL spec entries inhost-specs.ts); enabled ⇒ install, disabled ⇒ CONVERGE (strip blocks + receipt targets + advisory line; a strip FAILURE is recorded as a failedinstructionsreceipt target so --status/--remove/the next converge keep tracking the orphan); registrar mode (non-loopback --url) NEVER installs blocks — the local mirror speaks for the local brain, and the remote brain's own MCP instructions carry the contract when its operator opts in; a block installs only when the SAME host's MCP target confirmed (both lanes — a foreign-registration refusal or failed write must not leave a block directing saves at a server this run never registered), and a failed final smoke STRIPS the blocks this run installed (targets marked failed) so no block outlives its rolled-back registration;--remove/--statuswired.- Stop-hook backstop lane:
hookStopinsrc/commands/hook.ts(own 2s deadline inside Stop's 10s cap, fail-open exit 0, file-plane gate, 128KB-tail transcript parse with ONE 2MB-cap retry when no user turn is found — turn OFFSET from EOF ≠ turn size; a big tool_result would otherwise silently lose the turn — then gate → secret-scannedbankWritebackTurn[corpus-segments.ts:<session>.wb-<hash24>[.src-<sourceId>].txt— the source segment lets the SWEEP fallback file the turn into the session's own source; scanner-unavailable = fail-closed skip] →requestContextPack(bankOnly, flushCorpusFile, trigger:'writeback-bank', sourceId from GBRAIN_SOURCE); every outcome a typedwriteback-bankheartbeat reason — by-design skips [gate reasons, no_user_turn, flush_skip_*] rideoutcome:'ok', 'degraded' is INFRA faults only) →src/mcp/context-pack-handler.tstags.wb-basenameslane:'writeback'→checkpoint-harvest.ts's writeback lane (AUTHORITATIVE serve-sidememory.auto_writebackre-check with{gate:true}+ loadConfig drift input — off ⇒ terminal.ingested {skipped:'writeback_off'};plane_drift/invalid mode/read_error/non-transport extraction skip [extract_skipped_*: refusal/content_filter/malformed_output class] ⇒ NO sidecar, file survives for the sweep; salient ⇒notabilityFilter:'medium-and-up';source:'hook:writeback'; NO manifest publish; per-session prompt-harvest capWRITEBACK_SESSION_CAP=30in a true-LRU counter map [delete-before-set on increment], budget burned only on real enqueues, overflow = ACK-only skip the sweep batch-extracts; heartbeats underevent:'writeback'with PERSISTED inserted/duplicate/superseded). The sweep's corpus pass recognizes.wb-files, applies the same gate semantics, routes to the FILENAME-banked source (isValidSourceId-checked, pass source as fallback), and recordsskipped_reasonin its terminal sidecar (src/core/sweep.ts) — leftover banked turns never bypass the OFF gate and never silently zero out. wb-file state machine: ack-only skips leave the file for the sweep;writeback_offis the one terminal skip sidecar. - Read-time TTL validity (there is no sweeper): ACTIVE fact reads in BOTH engines (
listFactsByEntity/Since/BySession, bothfindCandidateDuplicatesbranches,countUnconsolidatedFacts,getFactsHealthactive buckets) carryAND (valid_until IS NULL OR valid_until > now())—valid_untilis temporal validity, not retention; history paths (listSupersessions,findTrajectory,--asof) deliberately still see lapsed rows; a re-stated expired fact re-inserts fresh. Pinned bytest/facts/ttl-validity.test.ts+ engine-parity additions. - Diagnostics:
src/commands/doctor/checks/memory-writeback.ts(memory_writeback, BRAIN category) — quiet-ok when off, but the OFF branch still probes receipt + canonical instruction paths and WARNS on a lingering block (off-but-still-instructing) with the converge command; plane-compare warns when the file mirror disagrees with the DB row (the Stop hook acting on the wrong truth) naming the re-sync; a read_error reports the state as UNKNOWN, never "off (default)"; resolved bundle + BOTH visibility postures + audience/reasons; instruction blocks receipt-vs-live-probe-vs-drift compare (drift fix names theconfig set+bootstrap harnesscombo — the config set re-stamps the posture mirror so the re-render converges even after an out-of-band visibility flip) + AGENTS.override.md detection — and a receipt target in statefailedis a STANDING warn regardless of the live probe (the one physical-survival path — a smoke-rollback strip that itself threw — leaves a current-looking block directing sessions at a rolled-back endpoint, and only the receipt state knows) naming thebootstrap harness --yes/--removeconverge; validity-lapsed count; 7d counters (verbs usage sidecar's additiveremember_status[stamped insrc/mcp/dispatch.ts, ALL-MCP-callers semantics labeled honestly] +writeback/writeback-bankheartbeat events;turns_bankedcounts flush_skip_* — banked, enqueue declined) with the local/lossy disclosure. Null-engine runs report the file mirror honestly. - Tests:
test/writeback-config.test.ts,test/mcp-instructions-writeback.test.ts,test/facts/writeback-gate.test.ts,test/hook-writeback-stop.serial.test.ts, writeback-lane cases intest/checkpoint-harvest.serial.test.ts,test/bootstrap-instructions-block.test.ts+ harness serial additions,test/writeback-nudge.serial.test.ts,test/doctor-memory-writeback.serial.test.ts, the hermetic 5-steptest/ambient-writeback-lifecycle.serial.test.ts, and the NON-GATING real-codex door (test/e2e/bootstrap-real-codex.serial.test.ts, ambient-writeback describe).
src/core/sweep.ts+src/commands/sweep.ts— the serve-resident maintenance sweep (the lock owner closes the persistence loop): facts-fence reconciliation (zero-LLM, reuses the cycle extractor with a slug subset), deterministic link/timeline extraction over recent workspace pages (the same cores asgbrain extract— remote put_page deliberately skips these, the sweep is where the graph compounds), and spend-gated corpus ingest (skipped keyless; sidecar-marked exactly-once). Bounded, fail-soft, never throws; armed at serve startup (3s, best-effort) and on 10-min idle ticks through the injectable timer seam, everything unref'd;GBRAIN_SWEEP=0kills it.gbrain sweep --onceis the trusted CLI seambootstrap verifyuses (CLI-only, never over MCP). The link pass threads the samelink_resolution.cross_sourceopt-in and configuredsources.defaultthe CLI extract lanes pass intoresolveCandidateSources, so an edge into another source is created (flag on) or counted ascross_source_linkinskipped(flag off) — never silently dropped, and the reconcile never deletes a cross-source edgeextract links --source dbcreated.src/core/context/sync-ipc.ts+src/core/serve-sync-runner.ts+src/commands/sync-delegate.ts— serve-delegated sync.sync-ipc.tsis the LEAF wire module: theDELEGATED_SYNC_OPTION_FIELDStable is the single source of truth for the validator, the CLI wire-builder, and the serve-side SyncOpts builder;validateDelegatedSyncOptionsrejects unknown keys fail-closed (repoPath/skipLock/lockId/concurrency are unreachable from the socket), requirestimeoutSeconds(0 = the explicit--no-hard-deadlineunbounded encoding; everything else clamps to 24h), andtoWireSyncResulttruncatespagesAffectedto 50 + a true total under the 256KB message cap.serve-sync-runner.tsis the serve-side module singleton: one job at a time (correctness still rests on thegbrain-sync:<source>row lock inside performSync), clientToken attach (a lost-ack retry finds its own running OR retained-terminal job instead of duplicate-running), per-job deadline timer,[serve-sync]lifecycle stderr lines, an idempotent sharedshutdownDelegatedSync()both serve shutdown paths await BEFOREengine.disconnect()(the disconnect-mode drain is allowAbort:false after_dbis nulled, so settle writes need the live engine; a registered drainer remains as backstop), andmaybeDrainDeferredEmbeds(delegated jobs always run noEmbed — the cost gate lives in runSync — and the serve drains stale embeds afterwards viarunEmbedCore, keyless-safe, cleared only when a drain finds nothing left; wirenoEmbedrecords that the USER declined embeds and suppresses the drain).sync-delegate.tsis the CLI half: read-only holder probe (probeLivePgliteHolder), DEFAULT-DENY argv gate (any unclassified token refuses by name — a silently dropped--excludewould perform the wrong sync), engine-free source tiers (resolveSourceIdEngineFree), 1s poll loop tolerant of event-loop-blocked serves with a PID-probe backstop,unknown_job→ serve-restarted resume hint, Ctrl-C →sync_abort(second Ctrl-C exits 130), and runSync'spull_failedverdict mirroring. Mounts and non-PGLite configs never delegate; opt-outs--no-delegate/GBRAIN_SYNC_NO_DELEGATE=1(client) andGBRAIN_SERVE_SYNC_IPC=0(serve). Pinned bytest/context/sync-ipc-validation.test.ts,test/sync-delegate-ladder.test.ts,test/serve-sync-runner.serial.test.ts,test/context/resolve-ipc-sync-kinds.test.ts, andtest/e2e/sync-delegation-under-serve.serial.test.ts(gating tier1 step).src/core/capability.ts— config-plane keyless/keyed detection + the honest capability report (per-provider keyless banner: OpenAI = semantic search + auto-extraction, Voyage = semantic search, Anthropic = auto-extraction) rendered by verify and the runbook. The extraction probe resolves through the SAME sharedresolveEffectiveChatModelthe gateway's reconfigure fallback uses (GBRAIN_MODEL> servable file pin > key-aware tier default), so a stale unservable pin degrades identically in the report and at runtime. Key/env fold comes frommergedProviderEnv(src/core/ai/provider-env.ts). Accepted limitation (documented in the module): DB-plane overrides (models.default,facts.extraction_model) are invisible to this engine-less probe; the runtime gate infacts/extract.tsand the engine-aware pre-enqueue gate infacts/backstop.tsare the backstops.src/core/secret-scan.ts— pattern scanner for USER workspaces (own minimal allowlist +<ws>/.gbrain-scan-allowper-finding overrides — deliberately NOT the repo's.gitleaks.toml, which is a public-repo CI fixture policy); redacted previews only;redactFindingsis the corpus-write mode.src/core/backup/status-file.ts— ENGINE-FREE core of the monthly backup-coverage check: the cached verdict (~/.gbrain/backup-status.json, schemagbrain-backup-status-v1; fail-open load, atomic tmp+rename write,invalidateBackupStatus()fired by the fix paths —bootstrap reporeceipt write,sources harden,workspacePushfinish on ok when the cache carried a warn or failing/unpushed totals (a routine healthy push leaves an ok cache alone, preserving the monthly throttle)), the notice renderer (backupNoticeText(s, 'human'|'aggregate')— human ≤300 chars names assets, aggregate is counts-only for remote surfaces; the remote-privacy pin), and the bounded nag gate shared by every render channel (backupNagGate(channel, s)onbackup-nag-state.json, OWN schemagbrain-backup-nag-v1— skillpackloadNagStatewould drop the extra fields on round-trip, so only the pure policy fnsdecideNagAction/recordNagDisplayare reused). Budget = AND of: 24h cross-channel dampener (last_shown_at), per-channel ceiling 3 per pseudo-version (YYYY-MM(checked_at)+ no-remote fingerprint — month rollover or a changed verdict re-surfaces; a same-month recompute with an unchanged verdict stays quiet), and a global cap of 3 recorded impressions per month (global_shown_count/global_month).backupNagReadOnlyConsultis the OpenClaw context-engine's never-writes variant;backupSpawnDue/recordBackupSpawndebounce the session-end detached spawn (no sidecar file);maybeEmitBackupNagis the cli.ts startup rail body (skip set incl. serve/call/jobs;BACKUP_LOCAL_ONLY <n>machine marker + human line). Off switches:GBRAIN_BACKUP_CHECK=0, configbackup.check_enabled; interval envGBRAIN_BACKUP_CHECK_DAYS> configbackup.check_interval_days> 30; values <1 or non-numeric fall back to the 30-day default (DAYS=0never means "always stale";config setrejects <1 outright). Tests:test/backup-status-file.serial.test.ts.src/core/backup/coverage.ts— engine-side compute for the backup check.computeBackupCoverage(engine, {localGitProbes})assesses: source repos (deduped bydiscoverGitRoot, capped 500 roots with a logged skip count;originRemoteState— a positive tri-state, deliberately NOT sync-git'shasOriginRemote, which collapses probe failures into a false "no remote" — plushasRemoteTrackingRef(origin configured but never pushed is alsono_remote),aheadCount,isWorkingTreeDirty— local read-only git subcommands only, no network), the bootstrap workspace (receiptrepo_url+ push statuses, file plane only), db_only tiering (info row with thegbrain export --dirfix), harness skill dirs (info — installed copies), and the DB-only-brain worst case (pages>0, nothing git-backed → warn on PGLite / info with different copy on postgres — the engine.kind branch).no_remotealone flipsoverall: 'warn'.getBackupStatusis the single choke point (fresh cache → file read; probed results persist, probe-less results NEVER persist; a failed compute returns the prior cache — never clobbers).maybeRefreshBackupStatusInProcess(engine)is the serve-side single-flight refresher — called frommcp/dispatch.tsgated onopts.transport === 'stdio'(the locality axis; 'http'/UNSET ⇒ fail-closed, pinned by test), recomputes on stale-or-absent OR warn+>24h, 1h attempt floor. Trust boundary: same asextraction-sync.ts— git probes only in trusted-local contexts. Consumers:commands/backup.ts(gbrain backup status|check, PGLite-lock fallback to cache),advisor/collect-backup-coverage.ts,doctor/checks/backup-coverage.ts(backup_coverage, localOnly probes / remote cache-only),commands/sync.tspost-sync stale-only refresh, hook session-end detached spawn. Docs:docs/operations/backup-check.md. Tests:test/backup-coverage.serial.test.ts,test/advisor-backup-coverage.serial.test.ts,test/doctor-backup-coverage.serial.test.ts,test/hook-backup-notice.serial.test.ts,test/mcp-backup-nag.serial.test.ts.src/core/workspace-push.ts—gbrain sources push: deny-glob backstop (tracked*.pglite/.env*refused regardless of .gitignore state) → stage FIRST → secret-scan the STAGED index blobs viagit cat-file(closes the scan-then-stage TOCTOU — scanned bytes == committed bytes) → commit FIRST → divergence-safe pull → push, under one cross-platform lock (mkdir-atomic; flock is not a dependency — macOS). The pre-push secret gate FAILS CLOSED: an unreadable, oversized (>PUSH_MAX_SCAN_BYTES), or otherwise unscannable staged blob returnsblocked_unscannable(nothing committed) instead of sailing through — only a confirmed staged deletion is skipped, and binary/NUL-sniffed blobs are scanned anyway. Statuses map to exit codes insrc/commands/sources.ts:pushed/skipped_in_flight→ 0;blocked_secrets/blocked_tracked_deny/blocked_unscannable/refused_visibility→ 5;pull_conflict/push_failed/other → 1. Refuses public AND unverifiable remotes (never fail-open); pushes even on clean trees; writes<home>/bootstrap/push-status.json. Parent-repo-aware (a source may be a subdirectory of the workspace repo).src/core/gbrain-home.ts— the single GBRAIN_HOME resolution choke point (delegates to config's parent-dir semantics; 0700 on create) — durability, push, hooks, and bootstrap all route through it so home semantics cannot drift.src/core/bootstrap/verify.ts+status.ts— verify is the definition of done: fail-soft check suite over the REAL write path (put_page op → write-through file underbrain/→ in-process sweep → graph floor via link tables → recall), the keyless magic-moment check (## Factsfence → zero-LLM reconciliation → world-visibility read-back), source_id collision resolution (as the one bootstrap subcommand holding an engine: a manifestsource_idalready registered to a DIFFERENT checkout → derives a stableworkspace-<8char-path-hash>, persists it to agent.json, names the re-register steps — every consumer readsmanifest.source_id), theembedding_planecheck (keyed installs live-embed ONE probe string and compare the RETURNED width to the actualcontent_chunks.embeddingwidth — a plane split is a named FAIL with the recovery command, keyless passes, a dead probe warns; runs BEFORE the roundtrip so a keyless-passing roundtrip can never certify a brain whose keyed writes all fail), token sweep, byte floors, secret scan, deny globs, repo privacy, hooks smoke (in-process IPC), capability report, first-run tour; snapshots kept last-5 under<home>/bootstrap/. status owns the ordered PHASES list (the runbook defers to it), artifact-first detection, install.jsonl, the runbook version-stamp skew check, and the support blob doctor/agents relay verbatim.src/core/bootstrap/template-repo.ts+scripts/generate-template-repo.ts— deterministic public-template generation (render--minimal+ placeholder manifest + stamped README); published only by the release workflow after diffing against the vendored tree.scripts/check-bootstrap-tag.sh/scripts/check-bootstrap-templates.sh— CI guards: sanctioned distribution ref only (latest-stable; the release job advances it after assets publish) + runbook stamp == VERSION; template↔question-bank token bijection + placeholder-only assertion + offline generator↔vendored byte-diff + runbook-phase↔status.ts consistency. Both skip gracefully when their subjects are absent.src/core/cycle/synthesize-concepts.tsconcept-quality addendum — eligible groups are processed deterministically by tier, descending atom count, then concept slug so the fixed LLM budget reaches the strongest evidence first regardless of database row order. Every page and phase receipt distinguishesllm, intendeddeterministic_tier,budget_fallback, anderror_fallbacksynthesis modes. The narrative call's output cap is sized from the resolvedmodels.dream.synthesizemodel viaresolveSynthMaxOutputTokens: 500 for non-thinking models, the gateway'sTHINKING_MODEL_MAX_OUTPUT_TOKENS(no phase-private number — DeepSeek v4 truncates at 8192-class caps) when the gateway's sharedisThinkingModelmatches (name-matched Claude 5 or recipe-declaredthinking_by_default; unknown providers count as non-thinking), so a tier-reasoningthinking model no longer spends the whole budget on reasoning and persists a template stub or truncated chain-of-thought as the concept narrative. Pinned bytest/cycle/synthesize-concepts-token-cap.test.ts. Right after each concept page write the phase banks concept<->member-atom provenance edges throughengine.addLinksBatch(link_source: 'concept-provenance';synthesized_fromconcept->atom,synthesizesatom->concept; audit sitecycle.synthesize_concepts.provenance), both endpoints scoped to the cycle's source, so the concept pages are graph-reachable (backlinks, relational recall, doctorgraph_signals_coverage/orphans) even though the prompt forbids enumerating atoms in the body. The dedicatedlink_sourcekeeps reconcile passes from pruning them;ON CONFLICT DO NOTHINGmakes re-runs the backfill; a failed or zero-row edge write lands inlink_warnings[](phasewarn) — NOT infailures[], which means "LLM-failed → template fallback" downstream and drives the rollup'shalt_delta/round_completed_delta— and never aborts the page write. Atom discovery (thetype = 'atom'scan) is scoped to the cycle source (opts.sourceId ?? 'default') so same-slug atoms from other sources are never grouped into this source's concepts. Pinned by the#4589describe intest/cycle/extract-atoms-synthesize-concepts.test.tsandtest/cycle/synthesize-concepts-source-scope.test.ts.
Google connector + open-loop engine (key files cluster)
User-facing contracts: docs/guides/google-connect.md (setup + the typed error
catalog) and docs/guides/open-loops.md (detection + close semantics).
Hosted-relay server design: docs/designs/HOSTED_OAUTH_RELAY.md (the CLI-side
seams are frozen in this repo; the server is gbrain.io's build). Agent-facing
operation: skills/google-loops/SKILL.md.
-
src/core/creds/vault.ts— the generic credential vault: one home for every outbound credential gbrain holds (Google OAuth today; provider-agnostic by design, providers register undersrc/core/creds/providers/). Two backends behind one frozen interface:FileVaultBackend(~/.gbrain/credentials.json, 0600, atomic writes — the CLI/self-host default) and the DB-backedEngineVaultBackendshape hosted gbrain.io implements. Custody rules: secrets live ONLY in the vault (never the config plane;sources.configstores a credential-id pointer, mirroring how github sources store an env NAME);list()returns redacted metadata only. No CLI imports — prompts live insrc/commands/. -
src/core/creds/errors.ts— the typed credential error catalog (CredentialError): every connect/refresh failure a user can hit maps to one code with four user-facing fields ({code, problem, cause, fix, doc_url}), rendered two ways — a conversational fix-first one-liner (stderr /[SHOW USER]blocks) and structured JSON (--jsonenvelopes). Single source of truth for the troubleshooting table indocs/guides/google-connect.md— update both together. No CLI imports; hosted reuses it. -
src/core/creds/providers/google.ts— Google OAuth2 client, BYO + relay-minted; hand-rolled fetch (no googleapis dependency,fetchImpl-injectable). PKCE S256 withaccess_type=offline&prompt=consentso a refresh token is always minted;client_refon the vault entry routes refresh (byo→ direct against the Google token endpoint with the user's own client;hosted-relay→ through the relay).invalid_grantis sub-classified into the catalog: clock skew (local clock vs Google'sDateresponse header), the 7-day Testing-mode expiry (age heuristic on last_refresh_ok_at/connected_at), and plain revocation.GOOGLE_SERVICE_SCOPESis read-only by construction — the connector never writes to Google. -
src/core/creds/redirect.ts— how the authorization code gets back to us:RedirectStrategy = 'loopback' | 'paste' | 'hosted-callback'. Loopback is an ephemeral 127.0.0.1 listener; paste mode needs NO listener (the fixedPASTE_REDIRECT_URIfails to load, the user pastes the full address-bar URL back) and is auto-selected bysniffHeadless(SSH/WSL/container/no-display). Google's device-code flow is NOT an option — Gmail/Calendar/Contacts scopes are excluded from it; don't re-litigate. -
src/core/creds/relay-client.ts— the typed CLIENT half of the gbrain.io zero-retention consent relay (createSession→ user clicks the consent URL → poll a one-timeclaim; the relay deletes tokens on first successful claim or at session TTL). Inert unlessGBRAIN_OAUTH_RELAY_URLis set (unset = the BYO flow, which always works).test/creds-relay-client.test.tsis the server conformance spec — a server that round-trips those fixtures is compatible. -
src/core/creds/export.ts— versioned passphrase-encrypted credential bundles (scrypt N=, r=8, p=1 → AES-256-GCM) for machine moves and hosted-upgrade transfer. A bundle carries selected vault entries PLUS the provider client records they depend on (Google refresh tokens are bound to the client that minted them — moving one without the other produces dead tokens). Format frozen here; hosted's import endpoint conforms to it. -
src/core/google/types.ts— pure data shapes for the google source kind: normalized Gmail/Calendar/People payloads,GoogleSourceConfig(account pointer, services, historyDays, managed dir), and theGoogleSourceStatecursor file persisted at<managed dir>/.google-source.json(gmail history id, downward-moving backfill floor, per-service syncTokens). No I/O. ExportsDEFAULT_CALENDAR_ID = 'primary'(the Calendar API's own alias; every'primary'literal in google-clients/google-source/sources/sources-ops resolves through it) andGoogleSourceConfig.calendarId(one calendar per source).GoogleSourceState.calendar_idrecords which calendarcalendar_sync_tokenwas minted for — a token is only valid against its own calendar; absent on legacy state, which is therefore primary's. -
src/core/google/access.ts— the pluggable Google-access seam:GoogleAccessProvider(getAccessToken/forceRefresh) withCommandAccessProvider(--access command: any CLI that prints a token — gog/gcloud/a gateway's mint command; parsed as a bare token line or JSON{token|access_token, expiry|expires_in}, cached until expiry with a 60s margin, 30s exec timeout, failures =access_command_failedcarrying the stderr tail) andEnvAccessProvider(--access env: a live token read from a NAMED env var each call, refreshed outside gbrain; missing =access_env_missing). The vault flow'sGoogleTokenProvidersatisfies the same interface. The token command executes only in the locally-running sync (google config keys are unreachable over MCP; same trust class as recipe health-check argv). -
src/core/google/google-clients.ts— hand-rolled Gmail/Calendar/People REST clients (house style, no googleapis dependency): auth via anyGoogleAccessProvider(vault-backedGoogleTokenProviderby default), 401 → forceRefresh + single retry, Retry-After honored (delta-seconds AND http-date), 403 accessNotConfigured →api_not_enabledcarrying the exact enable deep link (project number extracted from the client id), uniform pageToken pagination with a safety cap,fetchImplinjectable for tests.extractCalendarMethod(part)walks the MIME tree for atext/calendar/application/icspart and reads the iCalendarmethod=parameter from the PART's ownContent-Typeheader (headers[]on nested parts withformat=full— Gmail'sMessagePart.mimeTypeis the BARE media type, so a mimeType-only parse would read '' for every real invite and silently downgrade the structural signal to the subject fallback); a mimeType that still carries params is the fallback parse; a bare.icsfilename with a non-calendar MIME type claims nothing.CalendarClient.listEvents({calendarId})sweeps one calendar (defaultDEFAULT_CALENDAR_ID). -
src/core/google/google-render.ts— pure render functions ({relPath, markdown}out; no I/O, no engine): thread pages underemails/YYYY/MM/(type email), events undercalendar/YYYY/MM/(type meeting), contacts underpeople/(type person). Gmail deep links are code-generated via the typedemailCitationscaffold, never LLM-composed. The noise/signature rulesrecipes/email-to-brain.mdspecifies as prose for agent-authored collectors are implemented here — keep the recipe and this module in sync.isCalendarSystemMail({calendarMethod, subject})gates BOTH loop detectors: the PRIMARY signal is acalendarMethodin theCALENDAR_METHODSallowlist (RFC 5546 — REQUEST, REPLY, CANCEL, PUBLISH, COUNTER, DECLINECOUNTER, REFRESH, ADD; case-insensitive); an empty or unrecognised method is NOT a stamp and falls to the subject fallback, which is anchored to the start of the subject, refuses any Re:/Fwd: prefix, and matches only Calendar's OWN localised headers (Invitation:,Updated invitation:,Accepted:,Declined:,Tentative:,Canceled event:+ localised equivalents) — a generic word likeNotification:is a human/vendor subject and never matches.renderThreadPagestampsparticipants:(every address on the thread) ANDsenders:(sorted unique message AUTHORS only — the listloops mute sendergates on, so muting one person cannot silence a whole group thread). Pinned bytest/google-render.test.ts. -
src/core/google/google-source.ts— the google source kind sweep (mirrors github-source.ts: API-backed, materializes markdown under the source's managed dir, flows through the standard import pipeline — chunks, embeds, aliases, links). Sweep order contacts → calendar → gmail: alias rows must exist before the loop detector resolves counterparties. Per-service independent cursors: contacts/calendar syncToken commits only after that service's fully-successful sweep (410 GONE drops the token and re-runs windowed); gmail delta rides history.list with an expired-history windowed fallback; the INITIAL backfill drains newest→oldest with a batch-committed floor cursor so a killed 50k-message backfill resumes at the floor instead of restarting, and the historyId anchor is captured BEFORE the backfill so the delta lane takes over with zero gap (overlap re-renders are idempotent). Emits thesync.google_materializeprogress phase (docs/progress-events.md). Access resolves per source config: the vault (default), a token-printing command (g_access: command— gog/gcloud/gateway; identity entry synthesized, scope preflight trusts the configured services, send-as aliases fetched live best-effort), or a named env var (g_access: env). Secrets never land insources.config(the command/env NAME is config; tokens are not). Honesty invariants:last_sync_atis gated on the GMAIL sweep's success (it feedsgbrain waiting's trust-critical staleness gate, which protects loop freshness — a gmail sweep with real thread failures must not advance it); contacts/calendar failures mark the runpartialbut do not block the stamp, and a source without gmail in its services stamps unconditionally. A thread whose fetch fails on consecutive sweeps lands in the poison ledger and is skipped (steady-state runs honor the ledger;--fullretries with a fresh one), so one bad thread can't wedge the sync while a poison-skip alone doesn't count as failure. Contact reconcile resolves by contact resourceName (DB lookup by contact id FIRST): deletion tombstones carry onlyresourceName + deleted(no names — slug derivation yields null) and a renamed contact derives a DIFFERENT slug, so both must land on the existing page by id, never by slug. The calendar sweep binds its sync token to its calendar:sweepCalendarcomparesstate.calendar_id(legacy state without it is treated as primary's) with the configuredcalendarId; on a mismatch it logs[google] calendar changed (<old> → <new>), discards the token and takes the windowed first-sync path (nosyncTokenon the wire), and a freshly banked token stores the calendar id beside it — pairing a re-pointed source with the previous calendar's cursor would import a foreign delta. Pages from the previous calendar are not reconciled on the switch (logged; a filed follow-up).enqueueLoopsExtractionenqueues EVERY eligible thread (newest first) under a waiting-depth budgetLOOPS_EXTRACT_ENQUEUE_CEILING − waitingthat counts only THIS source's waitingloops_extractjobs (data->>'sourceId'), so one account's stalled backlog cannot pin another source's budget at 0; overflow is a logged deferral (a deferred thread re-candidates when it changes) and a failed probe fails open. The budget probe counts PENDING rows —status IN ('waiting','delayed','active')— so a retry backlog parked indelayedduring a flapping provider cannot stack past the ceiling. With no chat provider available (isAvailable('chat')false) the enqueue is skipped entirely and one stderr line per sweep names the reason and the count; pages still import and the threads re-candidate on their next touch or a--fullsweep. -
src/core/google/loop-detect.ts— the zero-LLM thread-state machine: last substantive message inbound + user in To: + unanswered ≥24h →unanswered_inbound; last outbound + contains a question + unanswered ≥72h →unanswered_outbound; a reply closes the loop (closed_by: reply_detected). Precision IS the product: noise senders, list mail (List-Unsubscribe), CC-only delivery, FYI/forwards without a question, self-threads, and muted senders/threads never open loops — pinned by the labeled fixture corpus intest/google-loop-detect.test.ts(every false-positive class gets a fixture before its fix). Pure verdict function + a thin apply step called per touched thread from the sync; the apply step must never fail the sync. -
src/core/google/loops-extract.ts— the LLM half of the open-loop engine: ONE extractor per recent thread page projects each commitment into THREE substrates in the same pass — theopen_loopsrow (dedupcommit:<sha8>), a facts row via writeSingleFact (kind=commitment, fence-first, deduped; its id lands onopen_loops.fact_idso entity cards / recall / context_pack see the commitment through existing read paths with zero new read code), and a typed edge thread-page → person-page (owes_to/awaiting_reply_from) for relational search.extract_factsnever runs separately on google-source email pages. Guardrails: injection-hardened input, ALL-or-nothing parse barrier (a malformed model response writes NOTHING), a structural eligibility gate (loopExtractionEligibility— Gmail labels,List-Unsubscribe, calendar part, who wrote the message; no vendor lists) plus the per-source waiting-depth enqueue budget (LOOPS_EXTRACT_ENQUEUE_CEILING, a spend backstop — seegoogle-source.ts), only the lastLOOPS_EXTRACT_WINDOW_DAYS(30) of mail (the deep backfill is never extracted), kill switchloops.extraction_enabled(default ON for google sources). Suppressions gate the lane before any model/facts/edge write: a muted thread id, or a muted SENDER —fm.from∪fm.senders, every address that AUTHORED a message, so a muted counterparty who wrote earlier in the thread still suppresses — returnsreason: 'suppressed'; recipients/CC never count (muting one person must not hide everyone else's commitments in a group thread, and an outside sender must not be able to dodge extraction by CC'ing a muted address). Pages without asenders:field fall back tofm.fromalone until their next re-render. Eligibility computes the substantive (non-noise, non-calendar) message set FIRST andowner_participatedrequires the owner to have written a SUBSTANTIVE message — a pure-calendar thread the owner RSVP'd to staysno_substantive_messages. When the chat provider is unavailable,runLoopsExtractthrowsLoopsExtractRetryableError(reasonllm_unavailable) rather than returning a skipped result: a completed no-work row would hold the revision-keyed idempotency slot forever, while a retried-then-dead row frees it so the same revision re-enqueues on the next sweep. -
src/core/loops/loops-store.ts— SQL accessors for theopen_loops+loop_suppressionstables overengine.executeRawwith IDENTICAL SQL text on both engines (parity by construction, the sources-ops.ts pattern — no per-engine method twins). JSONB discipline: evidence binds through$N::text::jsonb, never a bare::jsonbcast over JSON.stringify. Loops close by state transition, never delete: reply-driven auto-close flips status todoneand stampsclosed_by, keeping the audit trail. The upsert's DO UPDATE carries a manual-close guard in its WHERE: a closed row (done/dropped/stale) only reopens on GENUINELY newer activity, so a routine sweep re-seeing the same thread never resurrects a hand-closed loop, and the staleness auto-close (closed_by: 'staleness') is likewise guarded against the upsert's reopen.removeSuppression(engine, sourceId, kind, value)is the exact inverse of the mute write — deletes the one(source_id, kind, value)row with the same lower-casing, returns{removed: boolean}(false = was not muted, callers treat it as a no-op success), and never reopens loops: suppressions gate NEW detection only. -
src/core/ops/loops.ts— the op surface:open_loops(read),loops_close(write),loops_mute(write).open_loopsis deliberately NOT localOnly (hosted serves it over HTTP to the authenticated owner); instead it applies fail-closed evidence redaction forctx.remote !== false— counts, counterparty, summary, due date only; verbatim quotes, Gmail deep links, and the injectabletextdigest are trusted-local only. The result carries the google sources' last-successful-sync ages + astaleflag (>24h) so callers can refuse stale-but-confident output on a trust-critical surface. Per-call scope paramssource_id/all_sourcesresolve throughresolveRequestedScope(an MCP client bound to another source can reach the google source's loops; remote callers stay in-grant, out-of-grantsource_idis denied); a scope with NO google source returnsno_google_sources: trueand the digest says the engine has nothing to read instead of a false "You are clean". Scope is fail-closed too: an UNSCOPED remoteopen_loopsread is refused outright (the op must not rely on transports to enforce it — an unscoped remote read would span every source), and the write ops (loops_close,loops_mute,loops_unmute) require a single-source remote scope that matches the caller's grants — trusting a caller-suppliedsource_idfor a remote write would let any remote client plant (or lift) suppression rows cross-source;loops_unmutereverses a suppression viaremoveSuppressionand reports{removed}honestly. -
src/commands/google.ts—gbrain google connect|status|calendars|disconnect(+setupdispatch). Agent-first contract: every subcommand supports--jsonemitting{ ok, status, next_action: { command?, user_message? }, error? }(calendarsaddsaccount+calendars[]— id/summary/primary/accessRole — and carries thesources add --calendar-idtemplate innext_action.command; it resolves a lone connected account, exits 2 when there are none or several without--account, and throwsnot_connectedfor an unknown--account); human copy the harness must relay verbatim is fenced in[SHOW USER] ... [/SHOW USER]blocks; secret intake is--client-json <path|->(preferred) / envGOOGLE_CLIENT_ID+GOOGLE_CLIENT_SECRET/ TTY prompt, with raw argv flags accepted but documented last-resort; connect is an idempotent state machine (detects what exists, performs only the missing step — re-running is the documented fix for most errors). Engine-free exceptstatus's best-effort linked-sources listing (degrades without an engine). Funnel events append to~/.gbrain/integrations/google/heartbeat.jsonl; account addresses are hashed to a short non-reversible tag, never raw. The vault entry'smeta.scopesrecords what Google ACTUALLY granted (the token response'sscope— the consent screen lets users uncheck scopes; the requested set is only the fallback), which is what lets downstream preflights reportscope_missinginstead of opaque per-sweep 403s. -
src/commands/creds.ts—gbrain creds list|remove|export|import: the provider-agnostic vault surface; never prints a secret (list is always redacted). Provider-SPECIFIC connect flows live in their own commands. Export custody: a loud per-credential warning when a byo Google entry's consent screen is not known published-to-Production (its 7-day Testing expiry travels with the tokens). -
src/commands/loops.ts—gbrain waiting [--top N] [--json] [--stale-ok]+gbrain loops list|show|done|drop|mute: all paths dispatch through the trusted-local op layer (handleToolCall, remote:false) so CLI and MCP share one behavior.waitingREFUSES when EVERY google source has gone >24h without a successful sync and prints the exact fix (--stale-okbypasses; per-source sync ages are always reported) — stale-but-confident output is worse than none. Output per counterparty: what's owed, evidence quotes, Gmail deep links, entity-card context, a paste-ready digest. Reads default to the__all__brain span (loops live in google sources, notdefault— a default-scoped read would say "all clean" while people wait);--source <id>narrows explicitly. An unqualifiedmute/unmuteresolves the brain's google source (neverdefault) through the SAME resolution — an unmute can never aim at a different source than the mute it reverses — and refuses with the exact fix when none or multiple exist.unmute sender <email> | thread <id>is exact and forward-only; a repeated unmute is a no-op success (removed: false, exit 0) so scripts can call it unconditionally. -
src/commands/google-setup.ts+google-setup-tail.ts— the one-command orchestrator behindgbrain google setup: connect (skipped when tokens exist) → source registration (skipped when registered) → first bounded sync under a wall-clock budget (the newest-first backfill floor means whatever lands is the NEWEST mail — exactly whatwaitingneeds; the remainder resumes on every later sync, and setup says so honestly) → the firstgbrain waitingdigest in the same session as consent. Split in two so the connect half stays engine-free. Every step idempotent; re-running resumes wherever the last run stopped. -
src/commands/doctor/checks/google-oauth.ts— thegoogle_oauthdoctor check: zero-network vault health (live refresh probes belong togbrain google status; doctor stays fast and offline-safe). fail: a connected account with an expired access token AND no successful refresh in >2 days (refresh is broken — revoked, rotated client, or the Testing-mode expiry already hit); warn: a consent screen not known Production whose last proof of life is ≥5 days old (the proactive day-6 re-auth demand, cheaper than a dead pipeline on day 8); ok: accounts healthy or nothing connected (the connector is optional, not an error). -
src/core/agent-install/+scripts/setup-in-agent.sh— Package-versioned non-root installation into a verified persistent root. Absolute launchers isolate ambient routing and credentials, receipts track ownership, pending enablement, non-secret failure codes, and an exact repair command, and repair retains database contents. Upgrades journal pending schema migrations before switching runtime metadata; retries use schema-only initialization without invoking host orchestration. Busy results remain structured and retryable without deleting live locks. Native instructions/routines remain pending until observed in the actual harness. -
src/core/backup/archive.ts+src/core/backup/snapshot.ts— Private bounded full-PGLite snapshots with file inventories, checksums, credential-reference exclusions and fresh-root restoration. Hold the database lock, detect changing inventories, validate private staging before publication, rebase managed paths, detach external roots, and quarantine nonterminal restored jobs without deleting spending evidence. Never replace the source installation. -
src/core/harness/registry.ts— Canonical adapter identities, supported transports, credential-delivery mechanisms, guide links and dated evidence; legacy connection/register vocabularies derive their facts here.scripts/build-harness-docs.tsemits the user-facing adapter reference. -
src/core/harness/credentials.ts+delivery.ts+install.ts— Private credential handoff and host delivery journal, ownership-preserving native configuration and isolated thin launchers. A committed active client and matching current secret are prerequisites for recovery. Configuration removal and server revocation are separate actions. -
src/core/harness/capabilities.ts+src/mcp/capabilities.ts— Shared effective-grant orientation for OAuth whoami and authenticatedgbrain://capabilities. The resource preserves the exact seven-tool surface; source, operation, scope, fence and server-gate restrictions remain enforced at dispatch. No worker probe or per-tool database query during discovery. -
src/core/harness/verify.ts— Randomized server-side memory verification with separate transport, identity, authority, read, write/readback, delegation and cleanup verdicts. Lost mutation responses are reconciled before retry; server success never certifies native-harness activation or cross-conversation behavior. -
src/commands/mcp.ts+mcp-provision.ts+harness-connect.ts— Host-side grant/preview/CAS and recoverable private delivery, followed by installation inside the intended harness. Running PGLite hosts are administered through their authenticated API, never by opening a second database process. CLI status uses the delivery-aware exit verdict.