Daemon Mode (mongrelDB-server)

September 2, 2026 · View on GitHub

By default, MongrelDB runs embedded inside your application. But sometimes you want a long-lived database process that multiple applications can share - like a traditional database server, but lightweight.

The mongreldb-server daemon solves this. It opens the database once, keeps indexes and caches warm in memory, and serves queries over HTTP.

Installation

# Install from crates.io
cargo install mongreldb-server

# Or install a release binary
VERSION=v0.56.0
ASSET=mongreldb-server-linux-x64 # use mongreldb-server-linux-arm64 on ARM64 Linux
curl -L -o /usr/local/bin/mongreldb-server \
  "https://github.com/visorcraft/MongrelDB/releases/download/${VERSION}/${ASSET}"
chmod +x /usr/local/bin/mongreldb-server

# Or build from source
cd crates/mongreldb-server
cargo build --release

Starting the Daemon

# Basic: start serving a database on the default port (8453)
mongreldb-server ./my_database 8453

# With authentication (Bearer token required on all requests)
mongreldb-server ./my_database 8453 --auth-token my-secret-token

# With connection limit (max 100 concurrent requests)
mongreldb-server ./my_database 8453 --auth-token my-secret-token --max-connections 100

# Open a credential-enforced database and require HTTP Basic authentication.
MONGRELDB_DB_USERNAME=admin \
MONGRELDB_DB_PASSWORD='database-password' \
mongreldb-server ./my_database 8453 --auth-users

The daemon opens the database, builds indexes (if needed), and starts listening for HTTP requests on 127.0.0.1:8453.

History retention

The daemon defaults to 1024 retained commit epochs. Override startup behavior with MONGRELDB_HISTORY_RETENTION_EPOCHS. Authenticated administrators can inspect or change the durable window while the daemon runs:

GET /history/retention
PUT /history/retention
Content-Type: application/json

{"history_retention_epochs": 1024}

Both responses contain history_retention_epochs and earliest_retained_epoch. The routes require ADMIN permission when catalog authentication is enabled. Increasing the window cannot restore history that was already pruned.

Operations: drain and configuration reload

Graceful drain and configuration reload (the operations surface) are admin routes; like /audit and /history/retention, they require ADMIN permission when catalog authentication is enabled.

POST /admin/drain
Content-Type: application/json

{"drain_deadline_ms": 30000}

POST /admin/drain initiates a graceful drain: the server stops admitting new SQL and sessions, cancels in-flight queries (server_shutdown), closes sessions (rolling back staged transactions), then shuts the storage core down with the given deadline — in-flight commit-critical work finishes, durable state is synced, and the file lock is released. The body is optional; drain_deadline_ms defaults to MONGRELDB_DRAIN_DEADLINE_MS, else 30000. A deadline overrun answers 409 and leaves the core draining (new operations stay rejected); a later call with a larger deadline resumes the shutdown. Draining is terminal: restart the process to serve the database again. Every mutating HTTP surface (/sql, /sessions/*, /tables/* writes, /txn, /kit/txn, /kit/create_table, procedure and trigger mutations) answers 503 once a drain starts.

GET /admin/drain

Reports the core lifecycle state (open, draining, closing, closed; also opening during startup and poisoned after an unrecoverable internal error), whether SQL is still admitted, in-flight query and live session counts, and the last drain's outcome record (initiated, completed, deadline_ms, lifecycle, detail).

POST /admin/reload
Content-Type: application/json

{"sql_max_output_rows": 100000}

POST /admin/reload re-reads the mutable subset of server configuration and applies it live; SIGHUP triggers the same reload from the environment. Reloadable fields (environment variable → live effect):

  • MONGRELBL_SLOW_QUERY_MS — slow-query log/metrics threshold.
  • MONGRELDB_SQL_DEFAULT_TIMEOUT_MS — default per-query timeout (clamped to the maximum).
  • MONGRELDB_SQL_MAX_TIMEOUT_MS — maximum per-query timeout.
  • MONGRELDB_SQL_CANCEL_GRACE_MS — cancellation grace on session close and admin drain.
  • MONGRELDB_SQL_MAX_OUTPUT_ROWS / MONGRELDB_SQL_MAX_OUTPUT_BYTES — per-request output ceilings.
  • MONGRELDB_HISTORY_RETENTION_EPOCHS — re-applied to the core (equivalent to PUT /history/retention).

An absent/empty body re-reads everything from the environment; present body fields (the _ms/row/byte names above, plus history_retention_epochs) override the environment field-by-field and must be positive. The response echoes the applied values. Everything else is static node configuration and stays restart-only: listen address/port, data directory, authentication token/mode, connection and session capacity, session idle timeout, SQL/AI/ retained-page admission concurrency, the request-bytes bound, SQL idempotency TTL and capacity (the receipt binding includes the expiry policy), and the pre-cancellation and retained-page bounds. Drain, reload, and authorization failures on these routes are recorded in the security audit log with the calling principal and outcome.

Cluster administration

The /admin/cluster/* endpoints and the cluster / node CLI subcommands expose the cluster bootstrap and membership workflows (Stage 2/3, spec §11.1). The node data directory is the database directory itself: cluster records live under <db-dir>/cluster-meta/. A server whose database directory carries no cluster identity keeps working exactly as before — the status endpoint reports "standalone" and the mutating endpoints answer 409. See Sharded Cluster for the machinery behind these surfaces.

All three endpoints are admin routes: like /admin/drain, they require ADMIN permission when catalog authentication is enabled.

Cluster status

curl http://127.0.0.1:8453/admin/cluster/status

Reports one JSON view: "mode": "cluster" with the persisted identity (cluster id + node id), membership (node descriptors with locality, capacity, lifecycle state, build version, and version advertisement), member_endpoints, the database_group descriptor, a key-free trust summary (ca_cert_pem, node_cert_pem, allowed_node_ids, has_node_key — the node private key is never served), and this binary's version_info; or "mode": "standalone" with a hint when no identity exists. Corrupt or unsupported metadata answers 500 and is audited as admin.cluster.status.fail.

Drain a member

curl -X POST http://127.0.0.1:8453/admin/cluster/node/drain \
  -H "Content-Type: application/json" \
  -d '{"node_id": "<32 hex digits>"}'

Moves a member from Up to Draining in the persisted membership record. The body is optional; node_id defaults to this node's own identity. Replies {"member": <updated descriptor>}. Errors: 400 for an unparsable node_id, 404 for an unknown member, 409 on a standalone node or an illegal state transition (only Up drains). The initiation, outcome, and failure are audited (admin.cluster.drain, .ok, .fail).

Remove a member

curl -X POST http://127.0.0.1:8453/admin/cluster/node/remove \
  -H "Content-Type: application/json" \
  -d '{"node_id": "<32 hex>", "confirm_token": "<64 hex>"}'

Moves a member to Decommissioned (permanent). confirm_token is required and is obtained out of band from the CLI (see below) — it is never served over HTTP, never written to the audit log, and never echoed in a response. node_id defaults to this node's own identity. Replies {"member": <updated descriptor>}. Errors: 400 when confirm_token is absent, 403 for a wrong token, 404 for an unknown member, 409 on a standalone node or an illegal transition (removal is permitted from Up, Draining, or Down). Audited like drain (admin.cluster.remove, .ok, .fail).

Cluster CLI

One-shot subcommands; they operate on the data directory and exit without starting the daemon:

# Create the cluster on this node (id, initial membership, database raft
# group descriptor, trust configuration). Prints cluster_id and node_id.
mongreldb-server cluster init --data-dir <dir> \
  --endpoints 10.0.0.1:8453,10.0.0.2:8453 \
  --locality region=us-central,zone=a

# Provision this node into an existing cluster from an invite.
mongreldb-server cluster join --data-dir <dir> \
  --cluster-id <32 hex> --endpoints 10.0.0.1:8453 \
  --allowed-node-ids <hex,hex,...>

# Print the status report (cluster or standalone view).
mongreldb-server cluster status --data-dir <dir>

# Membership transitions (default target: this node's own identity).
mongreldb-server node drain  --data-dir <dir> [--node-id <hex>]
mongreldb-server node remove --data-dir <dir> [--node-id <hex>] [--confirm-token <hex>]

Flags: --data-dir (required everywhere); --endpoints (comma-separated host:port; init advertises the first as its RPC address unless --rpc-address is given); --locality (key=value tiers); --cluster-id (join); --trust-dir (directory holding ca-cert.pem, node-cert.pem, node-key.pem; default <data-dir>/trust); --allowed-node-ids (comma-separated admitted node ids; default: this node — required on a first join, which mints its node id during join); --node-id and --confirm-token (membership transitions).

Safety rules, all fail-closed: a node is bound to one cluster for the lifetime of its data directory (re-init or re-join is rejected; joining a different cluster requires an explicit wipe); trust material must be readable and PEM-armored; and node remove is a two-step confirmation — run without --confirm-token it prints the token and changes nothing, run with it the removal proceeds. Mutating workflows serialize on a bootstrap.lock file under cluster-meta/; a crash mid-workflow can leave it behind (the file records the pid and time — delete it to unblock). Joined nodes keep only the validated invite until the meta group wires membership through (Stage 2F/3A), so their cluster status shows an empty membership, and node drain/node remove — which edit the bootstrap cluster.json — answer "not initialized" there.

Running as a daemon (--daemon mode)

The --daemon flag forks the server into the background, detaches from the terminal, and writes a PID file:

# Start in background with a PID file
mongreldb-server ./my_database --daemon

# Custom PID file location
mongreldb-server ./my_database --daemon --pidfile /var/run/mongreldb.pid

# With auth + encryption
mongreldb-server ./my_database --daemon --port 8453 --auth-token my-secret --passphrase my-encryption-key

The server handles SIGINT (Ctrl+C) and SIGTERM gracefully - it flushes all tables, writes pending data to disk, removes the PID file, and exits with code 0. SIGHUP reloads the mutable configuration subset live (see "Operations: drain and configuration reload"); it never interrupts serving.

Keeping the daemon running (auto-restart)

For production deployments, use a process supervisor to ensure the daemon restarts automatically if it crashes or the host reboots.

Install the systemd unit file (shipped at crates/mongreldb-server/mongreldb-server.service):

# Copy the binary and unit file
sudo cp mongreldb-server /usr/local/bin/
sudo cp mongreldb-server.service /etc/systemd/system/

# Edit the unit file to match your database path and auth settings
sudo nano /etc/systemd/system/mongreldb-server.service

# Enable and start
sudo systemctl daemon-reload
sudo systemctl enable mongreldb-server
sudo systemctl start mongreldb-server

# Check status
sudo systemctl status mongreldb-server

# View logs
sudo journalctl -u mongreldb-server -f

The unit file template:

[Unit]
Description=MongrelDB Server
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/mongreldb-server /var/lib/mongreldb 8453
Restart=always
RestartSec=3
User=mongreldb
Group=mongreldb
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

With Restart=always, systemd restarts the daemon within 3 seconds if it crashes, and automatically starts it on boot.

Docker

# Pull the multi-arch release image
docker pull ghcr.io/visorcraft/mongreldb-server:v0.56.0

# Run with auto-restart
docker run -d \
  --name mongreldb \
  --restart=always \
  -p 8453:8453 \
  -v ./my_database:/data \
  ghcr.io/visorcraft/mongreldb-server:v0.56.0 \
  /data --port 8453 --auth-token my-secret

The --restart=always policy restarts the container on crash, daemon exit, or host reboot (when Docker itself starts).

supervisord

[program:mongreldb]
command=/usr/local/bin/mongreldb-server /var/lib/mongreldb --port 8453
directory=/var/lib/mongreldb
autostart=true
autorestart=true
startsecs=3
stderr_logfile=/var/log/mongreldb/error.log
stdout_logfile=/var/log/mongreldb/output.log
user=mongreldb

Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongreldb
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongreldb
  template:
    metadata:
      labels:
        app: mongreldb
    spec:
      containers:
      - name: mongreldb
        image: mongreldb-server:latest
        args: ["/data", "--port", "8453"]
        ports:
        - containerPort: 8453
        volumeMounts:
        - name: data
          mountPath: /data
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: mongreldb-data

Kubernetes restarts pods automatically via its health checks and self-healing mechanisms.

Authentication

The daemon supports three auth modes - they can be combined:

  1. Token (--auth-token <token>): every request must carry Authorization: Bearer <token>. A single string compare - fast.
  2. User (--auth-users): every request must carry Authorization: Basic <base64(user:pass)> against a catalog user (Argon2id-verified). The matching Principal is injected into request extensions for permission checks.
  3. Both (--auth-token and --auth-users): token or valid user credentials accepted.

When no flag is set (the default), auth is disabled for local development.

# Token only - fastest for service-to-service traffic.
mongreldb-server ./my_database 8453 --auth-token my-secret-token

# User auth - per-identity credentials, verified against the catalog.
mongreldb-server ./my_database 8453 --auth-users

# Both - token OR valid user accepted.
mongreldb-server ./my_database 8453 --auth-token my-secret-token --auth-users --max-connections 100

# Bearer token
curl -H "Authorization: Bearer my-secret-token" http://127.0.0.1:8453/health
# → "ok"

# Basic auth against a catalog user
curl -u alice:s3cret-pw http://127.0.0.1:8453/health
# → "ok"

# No credentials
curl http://127.0.0.1:8453/health
# → 401 Unauthorized

Manage users on a running daemon through the SQL endpoint (or any of the other surfaces documented in Users, Roles & Permissions):

# Create the first admin user before enabling --auth-users in production.
curl -X POST http://127.0.0.1:8453/sql \
  -H "Authorization: Bearer my-secret-token" \
  -H "Content-Type: application/json" \
  -d '{"sql": "CREATE USER alice WITH PASSWORD '\''s3cret-pw'\''; ALTER USER alice ADMIN"}'

require_auth databases

When a database has require_auth = true (see Credential Enforcement), database-open authentication and HTTP authentication are separate layers.

Set the database-handle credentials as a pair in the process environment:

MONGRELDB_DB_USERNAME=admin \
MONGRELDB_DB_PASSWORD='database-password' \
mongreldb-server ./my_database 8453 --auth-users

The daemon reads both variables once, removes both from its environment before daemonization or worker-thread startup, opens the database, and zeroizes its password buffer immediately after the open returns. There is deliberately no database-password command-line flag because process arguments are commonly visible to other local users and process monitors. Inject the variables from a restricted service-manager secret or environment file. Setting only one, either to an empty value, or using invalid UTF-8 is a startup error.

For an existing require_auth database, the variables authenticate the daemon's database handle. For a database with no catalog yet, they atomically create a credential-enforced database with that user as the first admin. They also compose with --passphrase for encrypted databases.

The HTTP boundary must independently configure --auth-users, --auth-token, or both. The binary refuses to start a require_auth database without either mode, and the library router rejects every route if constructed that way. Token-only HTTP mode is valid when the database handle was opened with the current admin credentials above: bearer requests execute as that exact admin principal. With --auth-users, every request's HTTP Basic credentials are atomically verified and resolved against the current catalog, then that exact principal is checked at the storage layer. Dropping and recreating a username does not let the new identity inherit the old identity's queries, sessions, cursors, or idempotency receipts. An under-privileged principal receives 403 Forbidden.

Connection Pooling

When --max-connections N is set, the daemon caps concurrent in-flight requests via a ConcurrencyLimitLayer. Requests beyond the limit wait in a queue. Default: unlimited (all requests handled immediately).

Request bounds

Every request is bounded (connections, sessions, in-flight requests, request bytes, result bytes, idle time); the daemon is asynchronous and never allocates an OS thread per connection:

  • Connections / in-flight requests: --max-connections N (above).
  • Sessions: --max-sessions N (default 256); at capacity, session creation returns 503.
  • In-flight SQL executions: MONGRELDB_SQL_MAX_CONCURRENT (default: CPU count); excess requests queue, and queue wait counts toward the query deadline.
  • Request bytes: MONGRELDB_MAX_REQUEST_BYTES (default 2 MiB). Requests whose declared Content-Length exceeds the limit are rejected before the body is read with a structured 413 (REQUEST_BODY_TOO_LARGE, category resource exhausted); chunked bodies are hard-capped at the same limit.
  • Result bytes/rows: MONGRELDB_SQL_MAX_OUTPUT_BYTES (default 64 MiB) and MONGRELDB_SQL_MAX_OUTPUT_ROWS (default 1 000 000).
  • Idle time: --session-idle-timeout <s> (default 300) reaps idle sessions, discarding any staged transaction.

Structured errors

Structured error responses carry the stable error taxonomy (spec 9.7) on the error object: category (one of twenty stable names) and category_code (its stable numeric code, never reused) are the programmatic contract; code and message are diagnostic detail. Clients must handle categories, never messages.

API Endpoints

All requests use JSON for parameters. Query results come back as Arrow IPC bytes (a binary format for columnar data).

Health Check

curl http://127.0.0.1:8453/health
# → "ok"

Table Management

# List tables
curl http://127.0.0.1:8453/tables

# Create a table
curl -X POST http://127.0.0.1:8453/tables \
  -H "Content-Type: application/json" \
  -d '{"name": "events", "columns": [...]}'

# Drop a table
curl -X DELETE http://127.0.0.1:8453/tables/events

SQL

curl -X POST http://127.0.0.1:8453/sql \
  -H "Content-Type: application/json" \
  -d '{
    "sql": "SELECT count(*) FROM events WHERE amount > 500",
    "format": "arrow",
    "query_id": "00112233445566778899aabbccddeeff",
    "timeout_ms": 30000,
    "max_output_rows": 10000,
    "max_output_bytes": 8388608
  }'
# Response includes X-MongrelDB-Query-ID.

Clients should generate the 32-hex-character query ID with a cryptographically secure random generator before sending a buffered request. Predictable IDs are not safe cancellation capabilities. Body query_id and timeout_ms values take precedence over X-MongrelDB-Query-ID and X-MongrelDB-Timeout-Ms headers.

max_output_rows and max_output_bytes must be positive. They apply to JSON, buffered Arrow, and Arrow streams and are clamped to the daemon-configured maxima.

Retried SQL writes

Buffered JSON writes accept idempotency_key in the body or Idempotency-Key in the header. If both are supplied, they must match. Keyed reads are rejected. Keyed requests must contain exactly one supported durable write: INSERT, UPDATE, DELETE, TRUNCATE, or the supported table, view, index, trigger, and policy DDL forms. Here, view DDL means durable materialized views; ordinary CREATE VIEW and DROP VIEW are session-scoped and therefore rejected for idempotent daemon writes. Multi-statement SQL, transaction controls, and transient/session commands such as NOTIFY, LISTEN, ATTACH, DETACH, SHOW, EXPLAIN, and PRAGMA are rejected before an intent is persisted.

curl -X POST http://127.0.0.1:8453/sql \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: create-event-42" \
  -d '{"sql":"INSERT INTO events (id) VALUES (42)"}'

The key is owner-bound and also binds the normalized, literal-aware SQL fingerprint, the parameter-list hash (currently the empty list because /sql has no separate bind array), effective output options, and pooled-session identity, plus the server-enforced receipt expiry policy. Reuse with different semantics or a different expiry policy returns IDEMPOTENCY_KEY_REUSE_MISMATCH. Every successful keyed write, including a write that matches no rows, returns a durable receipt instead of result rows. Retrying returns that receipt without parsing or executing the SQL. A receipt is HTTP 200 even when post-commit cancellation or serialization failed; inspect status, outcome, and terminal_error rather than treating HTTP success as clean response completion. For a write that durably committed, the receipt response additively carries commit_receipt — the core commit log's irrevocable receipt for the idempotency record (transaction_id, commit_ts_physical_micros/commit_ts_logical/commit_ts_node_tiebreaker, log_term/log_index, durability) — anchored in the durable TXN_IDEMPOTENCY ledger so the key → original-receipt binding survives restart under the commit log's authority (spec §10.2 S1B-005): an identical retry returns the same receipt, a changed request conflicts. The field is absent for no-op writes (nothing committed) and for receipts recorded before this unification. Receipt and intent files contain HMAC-authenticated hashes and outcome metadata, never the raw key, owner, SQL, parameters, or result rows. Encrypted databases derive the HMAC key from the database KEK. Plain databases create a random, database-local key at _meta/server-idempotency.key; it is not derived from an auth token, username, or password.

Before execution the daemon durably publishes an intent. It durably publishes the receipt after a known successful terminal outcome, including a commit or a no-op. Unix uses a file fsync, atomic rename, and parent-directory fsync; Windows uses a file flush and write-through atomic replacement. These cannot be one atomic filesystem/database operation. If a process or power failure occurs between commit and receipt persistence, the intent remains and the same key returns non-retryable QUERY_OUTCOME_UNKNOWN; the daemon never re-executes that write. Verify the database outcome before operator recovery. Completed receipts expire after MONGRELDB_SQL_IDEMPOTENCY_TTL_SECS; indeterminate intents do not expire into unsafe re-execution. MONGRELDB_SQL_IDEMPOTENCY_MAX_ENTRIES bounds all unique persisted scopes, including receipts, live executions, crash-left intents, and corrupt entries. Each owner is limited to one quarter of that capacity, with a minimum of one slot, so one tenant cannot consume the whole store. The daemon fails closed with a capacity error when either limit is reached. Crash-left intents remain durable outcome-unknown tombstones and continue consuming capacity until an operator verifies the database outcome and performs recovery. They cannot safely auto-expire into key reuse: without an atomic database receipt, expiry would risk repeating a write that committed before the crash.

SQL idempotency format v5 and Kit idempotency format v3 replaced older unkeyed checksums. Existing older entries fail closed as outcome-unknown; they are never accepted as replays. The stores use descriptor-relative, no-follow filesystem operations for directories, entries, and capacity locks. A symlink, non-regular entry, forged JSON document, failed authentication tag, or unavailable integrity key blocks execution. Atomic receipt publication uses a fresh collision-resistant temporary name, so an old fixed .tmp path cannot redirect or replace a receipt.

Filesystem permissions remain the trust boundary. The plain-database integrity key is created as an owner-only file on Unix, but a process running as the database owner that can read that key can authenticate its own replacement entries. Encrypted databases do not persist this server key; they derive it from the in-memory database KEK. Run the daemon under a dedicated OS account and do not grant other processes write access to the database root.

QUERY_OUTCOME_UNKNOWN responses and retained statuses encode committed and all commit/statement counters as JSON null. Clients must preserve that tri-state value. Only explicit false proves no commit.

Retained SQL pagination

JSON reads can opt into a process-local retained snapshot. The request must be exactly one query and must name every output column that may be returned.

curl -X POST http://127.0.0.1:8453/sql \
  -H "Content-Type: application/json" \
  -d '{
    "sql":"SELECT id, created_at, large_payload FROM events ORDER BY id",
    "max_output_rows":100000,
    "max_output_bytes":67108864,
    "pagination":{
      "page_size_rows":100,
      "projection":["id","created_at"],
      "max_page_bytes":262144,
      "max_page_tokens":65536
    }
  }'

curl -X POST http://127.0.0.1:8453/sql/continue \
  -H "Content-Type: application/json" \
  -d '{"cursor":"sp1:..."}'

max_output_rows and max_output_bytes cap the complete retained projected result. Page limits cap each response. Each response reports exact projected JSON bytes and estimated_tokens = ceil(bytes / 4); this is a transport hint, not a model tokenizer. The global retained-memory limit also charges a conservative per-row/per-column allocation overhead, not only JSON text bytes. Cursors are signed, owner-bound, server-instance-bound, and expire with the retained result. Repeating a cursor returns the same page. Writes cannot create cursors. The SQL engine may compute unprojected columns, but only the named projection is serialized, retained, and returned. A daemon restart invalidates all cursors.

The Rust remote client exposes both protocols directly:

use mongreldb_client::{MongrelClient, SqlPageOptions};

let client = MongrelClient::new("http://127.0.0.1:8453")?;
let receipt = client.sql_write_idempotent(
    "INSERT INTO events (id) VALUES (42)",
    "create-event-42",
)?;

let first = client.sql_page(
    "SELECT id, created_at FROM events ORDER BY id",
    SqlPageOptions::new(100, vec!["id".into(), "created_at".into()]),
)?;
if let Some(cursor) = first.next_cursor.as_deref() {
    let second = client.continue_sql_page(cursor)?;
    // consume second.rows
}
# Ok::<(), mongreldb_client::ClientError>(())

AsyncMongrelClient provides async methods with the same names. Client-side validation rejects empty or oversized keys, cursors, projections, and zero limits before network I/O.

# Request cancellation from another connection.
curl -X POST \
  http://127.0.0.1:8453/queries/00112233445566778899aabbccddeeff/cancel

# Inspect safe status metadata. Raw SQL is never returned.
curl http://127.0.0.1:8453/queries/00112233445566778899aabbccddeeff

# Negotiate support instead of guessing from a version string.
curl http://127.0.0.1:8453/capabilities

Cancellation capability version 2 accepts an owner-bound cancellation before the matching SQL request registers. The later request is cancelled before SQL admission or parsing. Send the same X-Session-ID on the cancel request when the SQL belongs to a pooled session. Pre-registration cancellations are process-local, bounded, and short-lived; a daemon restart clears them. Admin metrics expose only their current entry and byte counts, never IDs or SQL. Unknown cancel requests also use a bounded per-owner fixed-window rate limit, including repeated requests for the same query ID. Rate-limit exhaustion returns HTTP 429.

Status includes safe timing trace fields for queueing, planning, execution, and serialization, the cancel-requested and cancel-observed phases, and the commit-fence outcome. It never includes raw SQL or parameters.

Query status and cancellation are owner-or-admin operations. Unknown status lookups and not-owned IDs return 404 with QUERY_NOT_FOUND; a valid unknown cancel request creates the owner/session-bound pre-cancellation above. Cancellation after the durable commit fence returns 409 with CANCEL_TOO_LATE. A client transport timeout or disconnected socket does not by itself prove that a buffered server query stopped. Official clients send a separate cancellation request.

SQL execution limits use these environment variables:

MONGRELDB_SQL_DEFAULT_TIMEOUT_MS
MONGRELDB_SQL_MAX_TIMEOUT_MS
MONGRELDB_SQL_MAX_CONCURRENT
MONGRELDB_SQL_MAX_ACTIVE_QUERIES
MONGRELDB_SQL_FINISHED_QUERY_TTL_SECS
MONGRELDB_SQL_PRE_CANCEL_TTL_MS
MONGRELDB_SQL_PRE_CANCEL_MAX_ENTRIES
MONGRELDB_SQL_PRE_CANCEL_MAX_BYTES
MONGRELDB_SQL_PRE_CANCEL_MAX_PER_OWNER
MONGRELDB_SQL_PRE_CANCEL_RATE_WINDOW_MS
MONGRELDB_SQL_PRE_CANCEL_RATE_PER_OWNER
MONGRELDB_SQL_CANCEL_GRACE_MS
MONGRELDB_SQL_MAX_OUTPUT_BYTES
MONGRELDB_SQL_MAX_OUTPUT_ROWS
MONGRELDB_SQL_IDEMPOTENCY_TTL_SECS
MONGRELDB_SQL_IDEMPOTENCY_MAX_ENTRIES
MONGRELDB_SQL_PAGE_TTL_SECS
MONGRELDB_SQL_PAGE_MAX_ENTRIES
MONGRELDB_SQL_PAGE_MAX_RETAINED_BYTES
MONGRELDB_SQL_PAGE_MAX_PER_OWNER

The query deadline starts before the SQL semaphore: queue wait, planning, execution, response serialization, and stream backpressure all count toward it. Closing a session cancels its queued and active queries. Graceful server shutdown rejects new SQL, cancels queued and running work, lets commit-critical writes finish, and records tasks that exceed cancellation grace.

Prepared statements are bound per session to the SQL text, the declared or first-observed parameter types, the catalog version, and the schema versions they were planned against. POST /sessions/{id}/prepare returns the session-scoped statement_id alongside the statement name and accepts an optional param_types array of canonical type names (NULL, BOOL, INT64, FLOAT64, TEXT, BYTES). Execution with a mismatched parameter list fails with PREPARED_PARAMETER_MISMATCH rather than coercing silently. On an incompatible catalog/schema change the next execute invalidates the statement and replans it from the bound SQL; a stale plan never executes silently. If the statement can no longer be planned (for example its table was dropped), execute fails with 409 SCHEMA_VERSION_MISMATCH (category schema version mismatch, code 16) and the statement must be re-prepared.

Prepared-statement DELETE /sessions/{id}/statements/{name} accepts the same X-MongrelDB-Query-ID and X-MongrelDB-Timeout-Ms controls as /sql and returns X-MongrelDB-Query-ID.

Typed Kit API

The daemon serves a typed Kit API with authoritative constraint enforcement:

Compatibility note: default_value is interpreted as a literal JSON scalar. The legacy behavior of treating default_value: "now" or default_value: "uuid" as dynamic defaults has been removed. Use default_expr: "now" or default_expr: "uuid" for dynamic defaults.

# Get the full schema catalog
curl http://127.0.0.1:8453/kit/schema

# Atomic typed write batch (put/upsert/delete with idempotency keys)
curl -X POST http://127.0.0.1:8453/kit/txn \
  -H "Content-Type: application/json" \
  -d '{"operations": [...], "idempotency_key": "..."}'

# Typed query with conditions
curl -X POST http://127.0.0.1:8453/kit/query \
  -H "Content-Type: application/json" \
  -d '{"table": "events", "conditions": [...], "limit": 1000, "offset": 10000}'

Keyed /kit/txn, trigger DDL, and procedure-call writes use a shared durable idempotency store. Keys must contain 1 to 256 bytes. Before execution the daemon fsyncs an intent bound to the authenticated owner, operation, and exact payload. After execution it fsyncs the exact HTTP status and JSON response. Reusing a key with different input returns IDEMPOTENCY_KEY_REUSE_MISMATCH. If the commit outcome or receipt publication is uncertain, the intent remains and every retry returns QUERY_OUTCOME_UNKNOWN without re-executing the write.

Basic-auth ownership uses the user's immutable catalog ID and creation epoch, not the username. Dropping and recreating the same username creates a different owner that cannot access old sessions, query statuses, continuation cursors, or idempotency receipts. Bearer ownership uses a domain-separated SHA-256 digest; the token itself is never stored. /kit/txn checks current permissions before receipt lookup and binds the current security version plus every target table's table and schema IDs. Permission changes or drop-and-recreate table changes therefore cannot replay an old response.

The store uses MONGRELDB_SQL_IDEMPOTENCY_TTL_SECS (24 hours by default) for completed receipts and MONGRELDB_SQL_IDEMPOTENCY_MAX_ENTRIES (4096 by default) for the global capacity; one owner may use at most one quarter of the global capacity. Indeterminate intents do not expire automatically. Full or unavailable stores reject a keyed write before execution. Legacy unverified Kit v2 and _idem/*.json cache files, and SQL v4 entries, fail closed after upgrade; archive or remove them only after all retry windows from the older daemon have safely expired.

Row-Level Operations

# Put a row
curl -X POST http://127.0.0.1:8453/tables/events/put \
  -H "Content-Type: application/json" \
  -d '{"row": [1, 42, 2, "alice@test.com", 3, 95.5]}'

# Count rows
curl http://127.0.0.1:8453/tables/events/count

# Commit pending writes
curl -X POST http://127.0.0.1:8453/tables/events/commit

The legacy Rust client's put method uses exact tagged JSON for values JSON cannot represent safely: binary bytes and UUID/JSON payloads use lowercase hex, decimals use a canonical unscaled integer string, and intervals use canonical component strings. Embeddings and floats must be finite. Invalid UTF-8/JSON, non-finite numbers, malformed tags, and wrong embedding dimensions are rejected instead of being converted to NULL or lossy text.

Successful legacy transactions and table, procedure, and trigger drops return status: "committed" with matching numeric epoch and canonical epoch_text. A client that loses or cannot validate a write response reports QUERY_OUTCOME_UNKNOWN; it never reports a plain transport/decode error as a known abort.

Procedures and Triggers

# List/create/drop/call stored procedures
curl http://127.0.0.1:8453/procedures
curl -X POST http://127.0.0.1:8453/procedures -d '...'
curl -X POST http://127.0.0.1:8453/procedures/my_proc/call -d '{"args": {...}}'

# List/create/drop triggers
curl http://127.0.0.1:8453/triggers

Compaction

# Compact all tables
curl -X POST http://127.0.0.1:8453/compact

# Compact one table
curl -X POST http://127.0.0.1:8453/tables/events/compact

The daemon also runs a background auto-compactor that sweeps every 30 seconds and merges any table with 8+ sorted runs. Auto-compaction does not reset the WAL. There is no HTTP /checkpoint route. PRAGMA wal_checkpoint over the SQL endpoint flushes tables and runs GC; it does not publish a fresh empty WAL. To shrink recovery, stop the daemon and run Database::checkpoint() / mongreldb-kit checkpoint against the data directory. See Maintenance & Operations.

Change Data Capture (NOTIFY / LISTEN)

The daemon publishes change events to a broadcast channel. Applications can subscribe via the GET /events endpoint, a Server-Sent Events (SSE) stream (text/event-stream). Each event carries an event: type (change or notify), a JSON data: payload (ChangeEvent { channel, table, op, epoch, message }), and a stable <commit_epoch>:<operation_index> id:. Send Last-Event-ID to resume from a previous id; a retention gap returns 409 before the stream starts, or a terminal gap event if the client falls behind:

# Stream change events
curl -N http://127.0.0.1:8453/events
# event: change
# id: 5:0
# data: {"channel":"","table":"events","op":"put","epoch":5,"message":null}
#
# event: notify
# id: 6:0
# data: {"channel":"alerts","table":"","op":"notify","epoch":6,"message":"threshold exceeded"}

SQL NOTIFY channel [, 'payload'] publishes a notification on a named channel; LISTEN channel is accepted (subscribers connect via /events).

Replication (WAL Streaming)

The daemon exposes GET /wal/stream?since=<seq> which streams committed WAL records as newline-delimited JSON ({ seq, txn_id, op }). A follower polls this endpoint and applies records to a local database copy:

use mongreldb_client::ReplicationFollower;

let mut follower = ReplicationFollower::new("http://leader:8453", "/local/copy")?;
let n = follower.sync()?;  // fetch + count new records
println!("applied {n} records, up to seq {}", follower.last_seq());

This enables async leader→follower replication for read scaling and disaster recovery.

Connecting from Rust

use mongreldb_client::MongrelClient;

let client = MongrelClient::builder("http://127.0.0.1:8453")
    .bearer_token("token")
    .build()?;

// SQL
let batches = client.sql("SELECT * FROM events WHERE score > 90")?;

// Row count
let count = client.count("events")?;

// Typed Kit operations
let schema = client.kit_schema()?;

Use .basic_auth(username, password) for HTTP Basic. AsyncMongrelClient provides the same builders and typed AI routes for Tokio applications.

Connecting from Node.js

const { RemoteDatabase } = require('@visorcraft/mongreldb');

const db = new RemoteDatabase('http://127.0.0.1:8453');

const count = db.count('events');
const arrowBytes = db.sql('SELECT * FROM events LIMIT 100');

// Compact tables remotely
db.compact();
db.compactTable('events');

How It Works

The daemon holds the Database open with all indexes in memory. HTTP requests are handled asynchronously (axum + tokio), with an optional concurrency limit. Indexes and caches stay warm between requests.

Security

The daemon listens on 127.0.0.1 by default. For production:

  1. Use --auth-token for service-to-service traffic, or --auth-users for per-identity credentials (see Users, Roles & Permissions).
  2. Use --max-connections to prevent resource exhaustion.
  3. Put a TLS-terminating reverse proxy (nginx, Caddy) in front for HTTPS.

For the recommended single-node + HNSW Dense + auth deployment recipe, see Production: single-node HNSW Dense + auth.