Single-Node Subsystems

Stage 1 of the architecture program builds the production single-node server machinery inside mongreldb-core. This page tours the subsystems that have landed and how to observe them. None of them change the on-disk database format: existing databases open unchanged.

Resource groups and the memory governor (S1E)

Every unit of admitted work is classified into one of eight WorkloadClass values (control, replication, oltp, interactive_sql, ai_retrieval, analytics, maintenance, backup) and belongs in a ResourceGroup that bounds its concurrency, queue depth, memory, temporary disk, work units, CPU weight, priority (0-255), and result size. ResourceGroupRegistry::with_defaults() seeds one configured group per class; the control and replication groups are pinned — they cannot be removed and cannot be re-registered without reserved capacity. Groups serialize deterministically (sorted by name) so they can later replicate as cluster settings.

The node-level MemoryGovernor owns nine memory pools (MemoryClass: page cache, decoded cache, query execution, result buffering, AI candidates, compaction, replication, backup, network buffers). A subsystem reserves bytes with try_reserve(bytes, class) and holds the returned Reservation guard; dropping the guard releases the bytes. pressure() is the fraction of the configured maximum in use; as it rises the governor escalates in a fixed order, each level with its own threshold and a hysteresis band so the level does not flap around a boundary:

1. RejectLowPriority — new compaction/backup reservations are rejected.
2. EvictCaches — registered reclaimable caches are driven through evict_reclaimable; the page caches implement Reclaimable, and PageCache::with_governor(governor, class) attaches one.
3. SpillOperators — spill_trigger() reports the level and request_spill_grant turns it into a typed grant that releases memory reservation bytes in exchange for moving operator working memory to disk; the disk side is the spill manager (next section).
4. ThrottleMaintenance — maintenance work yields to foreground work.

At every level the reserved floor holds: replication and network-buffer (control-plane) memory is never fully starved. Observe the governor with stats() — GovernorStats reports per-class usage, pressure, the current escalation level, and granted/rejected reservation counters.

Wired: Database::open constructs exactly one governor per core (OpenOptions::memory_budget_bytes, default DEFAULT_MEMORY_BUDGET_BYTES = 1 GiB — a reservation cap, not a preallocation) and exposes it through db.memory_governor(). Both page caches reserve under it and register as reclaimable, so escalation step 2 drives real cache eviction. Database::open also seeds ResourceGroupRegistry::with_defaults() (db.resource_groups()); the server mirrors a process-level registry on AppState for SHOW RESOURCE GROUPS and hierarchical scheduler admission.

Spill manager (S1E-004)

When the governor enters escalation step 3, spill-eligible operators move working memory to disk through the SpillManager. Database::open constructs one per core, rooted at <db-root>/temp/spill (OpenOptions::temp_disk_budget_bytes, default DEFAULT_TEMP_DISK_BUDGET_BYTES = 4 GiB of live spill files), and exposes it through db.spill_manager(). Query engines open a per-query SpillSession with begin_query(query_id, cap_bytes) or begin_query_in_group(query_id, group) — the per-query cap comes from ResourceGroup::temporary_disk_bytes.

Spill files carry the spec's four properties:

• Query-ID namespaced — every query spills under its own temp/spill/q-<hex>/ subdirectory, so cancellation removes exactly one directory.
• Checksummed — every frame carries a CRC32C (the WAL's Castagnoli CRC); the sealing trailer carries a SHA-256 over all plaintext payloads plus the frame count, verified on read.
• Bounded — the per-query cap and the node-global cap are enforced with the governor's add-then-validate-rollback protocol; overflow is the typed SpillError::BudgetExceeded.
• Encrypted when database encryption is enabled — with a meta DEK present, every frame payload is sealed AES-256-GCM with a fresh random nonce; otherwise frames are plaintext.

Cleanup is total: the SpillHandle RAII guard deletes its file on drop, an unfinished SpillWriter deletes its partial file, a dropped SpillSession removes the whole per-query directory, and SpillManager::open sweeps every stale entry a prior process run left behind — spill files never outlive the process that created them. The governor's request_spill_grant is the memory-side counterpart: an eligible reservation trades bytes for the grant before the operator writes through SpillWriter::append, finishes into a handle, and reads back through SpillReader.

Distributed fragment execution binds the planner's per-fragment spill allowance through FragmentControl::begin_spill onto the core SpillManager (begin_query with max_spill_bytes). Single-node SQL sessions that stage large transactions also use encrypted spill frames for pending ops.

Persistent online jobs (S1F-002/S1F-003)

The JobRegistry tracks long-running schema/data jobs — index builds, column backfills, schema validation, materialized-view rebuilds, key rotation, large imports — through the seven JobState values: Pending, Running, Paused, Cancelling, RollingBack, and the terminal Succeeded and Failed. Legal transitions are enforced by JobState::can_transition; illegal ones fail with JobError::IllegalTransition.

The registry is mirrored to a JOBS file next to CATALOG on every state mutation, written through the same temp-write + fsync + atomic-rename + parent-dir-fsync path the catalog checkpoint uses (an 8-byte magic, a SHA-256 integrity tag over the body — or AES-256-GCM under the encryption feature — and a versioned JSON envelope). Crash recovery runs at JobRegistry::open: Running jobs park as Paused with their last durable checkpoint, and interrupted cancels and rollbacks land in Failed.

Operators observe and steer jobs through submit / get / list / pause / resume / cancel; cancellation_token(job_id) exposes the cooperative cancel signal. run_build_publish(registry, job_id, job) drives a BuildPublishJob through the seven phases in order — record pending definition, pin snapshot, build hidden generation, catch up committed deltas, validate, publish atomically, release old generation — persisting a checkpoint after each, so a resumed drive skips completed phases. Phases must be idempotent and publish must be atomic. Fault-injection hooks (job.<phase>.before / job.<phase>.after) fire at every phase boundary.

Wired: Database::open opens exactly one registry per core (the sibling JOBS file persists across reopen) and exposes it through db.job_registry(); crash recovery runs inside that open. The landed driver is synchronous: callers (admin SQL, ops jobs, tests, engine entry points) advance a job through run_build_publish phase-by-phase with durable checkpoints — no background executor threads are required for correctness. Concrete kinds (IndexBuild, ColumnBackfill, …) are represented in JobKind and driven through that same API.

Versioned catalog commands (S1F-001)

Every logical catalog mutation is expressed as a versioned CatalogCommand wrapped in a CatalogCommandRecord: an explicit encoding version (CATALOG_COMMAND_FORMAT_VERSION = 1; unknown versions fail closed on decode) plus a monotonic catalog_version assigned on apply. The CATALOG file is demoted from sole authority to a checkpoint with no on-disk format change — the bounded retained command history (COMMAND_HISTORY_LIMIT = 256) rides the existing checkpoint and the CatalogSnapshot WAL payload.

Catalog::apply_command validates, applies, bumps catalog_version, and appends the record; application is deterministic, and replaying the same record against the same catalog version is an idempotent no-op. apply_command_and_checkpoint additionally rewrites the checkpoint through the existing atomic write path. Observe state with catalog_version() and commands_since(version) — the latter returns a strict suffix when the bounded history has compacted past version.

required_permission(command) is the authorization map for each command: Ddl for table/column/index, trigger/procedure, and materialized-view commands; Admin for user/role/grant/revoke, security-policy, resource-group, and job-definition commands. Enforcement (landed): Database emitters call require / require_for against the caller's principal on the public mutation paths; catalog-command apply stays deterministic and permission-agnostic so the same record can replay on a replica without re-checking session identity.

Lock manager (S1B-003)

LockManager provides key and predicate locking with deadlock detection: two modes (LockMode::Shared and LockMode::Exclusive — deliberately no Update mode; conversion deadlocks are resolved by the detector instead) over four LockKey families: Row (one physical row), Key (primary-key or unique-constraint bytes), Range (serializable predicate protection), and Barrier (schema_barrier(), sequence_barrier(name)).

Grants are strict FIFO per key: a reader arriving behind a queued writer never barges ahead of it. The wait-for graph is rebuilt on every enqueue and grant; cycles are found deterministically and the victim is chosen deterministically — lowest explicit priority first, then the youngest transaction (largest u64 ID). The victim's acquire fails with LockError::Deadlock, bridged to MongrelError::Deadlock (victim and cycle preserved) so callers see the precise ErrorCategory::Deadlock (taxonomy category 9) with the same retry-the-whole-transaction discipline as a write conflict. Waits honor a deadline (LockError::DeadlineExceeded) and cooperative cancellation (LockError::Cancelled). A transaction holds at most one in-flight acquire, re-acquisition is re-entrant, and release_all(txn_id) runs exactly once when the transaction ends.

Wired: Database::open constructs the manager (db.lock_manager()), and the engine paths acquire through it. Every commit, abort, and rollback funnels into release_txn_locks. DML commits hold the schema barrier Shared; DDL operations take it Exclusive for the duration of the entry point, so schema changes exclude concurrent DML and one another. The commit path claims primary-key and unique-constraint keys Exclusive, takes FK parent-protection locks while checking foreign keys, and serializes auto-increment fills on per-table sequence barriers. A deadlock victim's error surfaces at the SQL boundary as MongrelError::Deadlock. SQL SELECT ... FOR UPDATE acquires exclusive row locks through Database::lock_rows_for_update for the open SQL transaction (released on COMMIT/ROLLBACK). Serializable predicate protection records table-level scans for SSI certification.

Version-retention pins (S1C-004)

A historical version may be reclaimed only when it is older than the oldest pin of every source. The six PinSource values: TransactionSnapshot (oldest active MVCC reader, projected from the SnapshotRegistry), HistoryRetention (the configured rolling window, also projected), BackupPitr, Replication, ReadGeneration (cursors and immutable read generations), and OnlineIndexBuild.

Every mounted table owns a PinRegistry; subsystems that still read old versions register a PinGuard (cheap, deregisters on drop). GC folds the registry into the table's version floor: gc_versions reaps a retiring run only when its retire epoch is at or below the oldest pin of every source, so no reader can lose versions it still needs. Observe the live set with Database::version_pins_report() — one PinsReport per table, one PinInfo per active source (oldest epoch, when the oldest pin was taken, live pin count).

Wired: the per-table registries, the GC floor, and the diagnostics report all operate today. Live call sites include ReadGeneration pins for immutable generations, BackupPitr for backup boundaries, Replication pins during apply_replicated_records, and OnlineIndexBuild during rebuild_indexes_from_runs. Database-level backup run-file pins (backup_pins) continue to protect GC of in-use run files alongside the per-table pin registry.

Storage-mode marker (_meta/storage-mode)

Every database root carries a durable, versioned, checksummed marker at _meta/storage-mode (spec section 5.3, Stage 2 groundwork) declaring which runtime owns the directory: Standalone, ServerOwnedStandalone, or ClusterReplica { cluster_id, node_id, database_id }. The marker is written atomically (temp + rename + fsync) and is never rewritten in place — spec section 5.2 forbids in-place conversion, so the write fails closed when the existing marker disagrees. Databases created before the marker existed have no file: they open as Standalone and the marker is backfilled on first open, purely additively.

Cluster replica open rules. A ClusterReplica directory is owned by the cluster node runtime:

• every normal open path (Database::open, open_with_options with default options, and the encrypted/credentialed variants) rejects it with a typed error naming the cluster, node, and database ids;
• the cluster node runtime opens it with Database::open_cluster_replica, which fails closed unless the marker exists and exactly matches the offered identity (opening the wrong replica would corrupt two clusters);
• a backup validator may open any mode — including ClusterReplica — read-only through OpenOptions::offline_validation, the only way to open a cluster replica outside the cluster runtime.

Every replica open (cluster runtime or offline validation) is read-only: user writes fail with MongrelError::ReadOnlyReplica because mutations arrive through the replicated apply path (Database::apply_replicated_records / apply_replicated_catalog_command, driven by the consensus engine sink). Read-only also means admin-level maintenance such as hot_backup is rejected on a live replica core (mutations arrive only through the replicated apply path); replica backups stage offline from the quiesced root (see Replicated High Availability). The open gate matrix is integration-tested in crates/mongreldb-core/tests/stage2_gate.rs::storage_mode_open_gate_matrix.

Landed residual surfaces

• ResourceGroupRegistry::with_defaults() is constructed inside Database::open (Database::resource_groups) and mirrored on the server AppState for SHOW RESOURCE GROUPS.
• Distributed fragment control binds spill allowance through FragmentControl::begin_spill → SpillManager::begin_query.
• SQL FOR UPDATE lock acquisition and serializable table-level SSI predicate recording.
• Credentialed handle attaches and per-handle read-only enforcement (shared-handle path).
• Catalog mutations route through CatalogCommand with required_permission checks.
• Pluggable embeddings: see Embeddings and retrieval.