Performance-critical-path rules
May 25, 2026 · View on GitHub
A handful of files in this workspace sit on the request-handling hot path: every GetTs / GetTsBatch RPC, every window extension, every Raft propose/apply touches them. Regressions in these files surface as elevated end-to-end latency or reduced batch throughput, not as test failures, so we enforce a small set of source-level rules on top of the regular clippy/build checks. Files on the critical path carry this marker on the first non-blank line below the canonical copyright header — above any module-level doc, inner attribute, or use:
// #[PerformanceCriticalPath]
This is a comment marker, not a proc macro. Enforcement is by review plus a CI guard (scripts/check-critical-path.sh). If you are editing a marked file, the rules below apply.
Rules
- No synchronous I/O on the hot path. Disk, gRPC, RocksDB, and other control-plane I/O must be behind a bounded async boundary. If a function in a marked file needs I/O, await a previously-started future or spawn the work onto a background task; never call a blocking op inline. This rule is not grep-enforceable — it depends on review.
- No
tracing::info!or higher log levels. Usetracing::debug!or lower. Hot-path logs at info-or-higher volume fill dashboards and force synchronous writes when the subscriber is configured to flush. The guard rejectstracing::info!/warn!/error!, the bareinfo!/warn!/error!shortcut (viause tracing::info;), and the matching_span!macros. - No
println!. Same volume problem as info logs, plus it bypasses thetracingfilter entirely. - No long synchronous compute. Anything measured in milliseconds belongs on a background worker. The hot path is for routing, packing, and enqueue — not compute.
Two related rules are intentionally not enforced by this guard — they live in other layers of the build:
- No panics on recoverable paths. Enforced at the workspace level by the panic policy (
clippy::unwrap_used+clippy::expect_usedaswarn, withcargo clippy ... -- -D warningsmaking the warning fatal). - No
std::sync::Mutexheld across an.await. Planned: enableclippy::await_holding_lock = "deny"workspace-wide as a follow-up. The clippy lint is precise — it knows the difference between a sync mutex held across an await point (a real bug) and one held synchronously (fine) — whereas a grep-based "nostd::sync::Mutex::newin this file" rule false-positives on legitimate non-async uses.
CI guard
scripts/check-critical-path.sh runs in CI as the critical-path job in .github/workflows/ci.yml. It:
- Finds every
*.rsfile in the repo carrying the#[PerformanceCriticalPath]marker. The scan reads the whole file (so a marker that has drifted down is never missed) and matches the marker only as a standalone comment line, so prose that mentions`#[PerformanceCriticalPath]`in passing is not treated as a marker. - Checks each marker is well-placed: it must sit within the first
SCAN_WINDOWlines, derived as the canonical copyright header's line count (read fromscripts/header.txt, so it tracks the header automatically) plus a small allowance for the blank separator and the marker. A marker found below the window is reported as a misplaced-marker violation rather than silently dropped from enforcement. - For each marked file, greps for the banned patterns listed in the script (the
BANNEDarray). - Prints violations with line numbers.
CI runs in strict mode — the workflow exports CRITICAL_PATH_STRICT=1, so any violation fails the build. The script itself defaults to warn-only when CRITICAL_PATH_STRICT is unset, which is convenient for local iteration while preparing a new marker candidate. To mirror CI locally, run CRITICAL_PATH_STRICT=1 ./scripts/check-critical-path.sh.
New files added to the marker list must be compliant before marking — the guard does not accept pre-existing violations on newly-marked files, even in warn-only mode. "Warn-only" is a timing buffer for the initial rollout of a new banned pattern, not a license to ship non-compliant annotations.
To adjust the list of banned patterns or the strict-mode toggle, edit the script and update this doc in the same commit.
Marker placement
Place the marker on the first non-blank line below the canonical copyright header (currently 22 lines) — typically line 24, with one blank line separating header and marker — and above any module-level doc comment (//!), any inner attribute (#![...]), and any use statement. The guard allows the marker within the first SCAN_WINDOW lines (the header's line count plus a small allowance), so it must sit at the top; pushing it below — e.g. under a long //! module doc — is reported as a misplaced-marker violation rather than silently disabling enforcement. The marker is a plain // line comment, not Rust syntax; it does not interfere with the file's //! module doc (which still attaches to the module) or with any #![cfg_attr(...)] inner attribute (such as the panic-policy attribute in each library crate's lib.rs).
The marker is per-file. If you split a marked module into child files, mark every child file that remains on the hot path — the guard does not inherit markers through mod.
After placing the marker, verify CRITICAL_PATH_STRICT=1 ./scripts/check-critical-path.sh is clean on that file and add the path to the list below in the same commit.
Current critical-path files
Files in this list are compliant with the rules above — the guard is green against them today. Grouped by subsystem; within each group, files are ordered roughly by call frequency along the per-request path.
Per-request RPC path
crates/tsoracle-server/src/service.rs—TsoServiceImpl::get_ts, the per-request RPC handler. Every gRPCGetTsenters here and dispatches into the allocator and (when a window extension is needed) the consensus driver.crates/tsoracle-core/src/allocator.rs— the window allocator state machine. Everytry_grantandwould_grantcall goes through here.crates/tsoracle-core/src/clock.rs— theClocktrait andSystemClock.now_msis read on every grant attempt and on every window-extension prepare.crates/tsoracle-core/src/timestamp.rs— the packedTimestamp(u64)type.Timestamp::packconstructs the returned value for every issued grant.
Window-extension / consensus path
crates/tsoracle-driver-file/src/driver.rs—FileDriver. The fsync on window extension is the durability boundary for non-replicated deployments.crates/tsoracle-driver-openraft/src/driver.rs—OpenraftDriver, the bridge implementingConsensusDriver::persist_high_wateron top of anyOpenraftHighWaterHost.crates/tsoracle-driver-openraft/src/log_entry.rs— the single command type replicated through the openraft log; encoded on every propose, decoded on every apply.crates/tsoracle-driver-openraft/src/state_machine.rs—HighWaterStateMachine::apply, which runs on every committed entry.
Client path
crates/tsoracle-client/src/driver.rs— client-side coalescing driver. Every concurrent waiter passes throughdriver_task's select loop.
The two driver crates' lib.rs files are intentionally NOT marked: each has been refactored into a thin module-declaration shell, and the hot-path code now lives in the sibling files listed above. The marker is per-file, so placing it on a shell lib.rs would scan the wrong file — the guard does not inherit through mod.
Files considered but intentionally unmarked
crates/tsoracle-server/src/server.rs— owns the leader-watch supervisor task. The task emits one-shottracing::error!death-rattle messages when the watch loop returns an error or panics, so operators get a visible signal that serving has stopped. Those calls fire at most once per process lifetime and are not on the per-request path, but the grep-based guard cannot distinguish a one-shot supervisory log from per-request logging. The per-request handlers themselves live inservice.rs, which IS marked.