Observability
July 20, 2026 · View on GitHub
This document explains how Sōzu emits metrics, logs, audit events, and trace
context, where the seams are, and what conventions to follow when adding new
instrumentation. For the inventory of currently emitted metrics and access-
log fields, see configure.md. For the H2 mux internals see
h2_mux_internals.md.
Topology
Each Sōzu worker is a single-threaded mio event loop with three concurrent observability surfaces:
┌──────────────────────────────┐
│ worker process │
│ │
accept loop ──┐ │ ┌─ thread-local METRICS ─┐ │ ┌─ statsd UDP ─┐
TLS handshake │ │ │ │──┼──────▶│ network drain│
H2 mux │── incr! ─┼─▶ │ Aggregator │ │ └──────────────┘
H1 editor │ count! │ │ (counters, gauges, │ │
backend pool │ gauge! │ │ HDR-time histograms) │──┼──┐
socket I/O │ gauge_ │ │ │ │ │ ┌─ sozu CLI ─┐
… │ add! │ └─────────────────────────┘ │ └─▶│ local drain│
│ time! │ │ └────────────┘
│
│ log_context! ─────────────► main log stream
│ (structured prefix) (info/warn/error)
│
│ log_access! ─────────────► access log stream
│ (RequestRecord) (ASCII or protobuf)
│
│ SubscribeEvents bus ───────► control-plane events
│ (EventKind) (incl. audit log line — MUX-style, at info level)
└──────────────────────────────┘
Three properties shape every extension:
- Single-threaded per worker — no
Arc<Mutex>inside the loop.METRICSisthread_local!(lib/src/metrics/mod.rs). - Edge-triggered epoll via mio — anything queued from the read path must
signal_pending_writeon the readiness tracker (feedback_epollet_signal_pending_writein agent memory). This affects metric flush timing only when the metrics socket itself stalls — usually transparent to instrumentation authors. &'static strkeys everywhere — counters and gauges accept only&'static strkeys (the type signature ofcount_add/set_gauge). Per-error / per-kind breakdowns must materialise the keys at compile time.
Metric primitives
| Macro | Signature | When to use | Example |
|---|---|---|---|
incr!(key) / incr!(key, cluster_id, backend_id) | &'static str | Increment by 1. The 3-arg form labels by cluster+backend. | incr!("h2.frames.rx.data"); |
count!(key, value) | &'static str, i64 | Increment by N (e.g. byte counts). | count!("bytes_in", n as i64); |
decr!(key) | &'static str | Decrement by 1. Pair with a prior incr!. | decr!("http.active_requests"); |
gauge!(key, value) | &'static str, usize | Absolute snapshot. ⚠️ Last-writer-wins across emit sites — only safe for proxy-level state with a single emitter (e.g. client.connections). | gauge!("client.connections_max", n); |
gauge_add!(key, delta) / gauge_add!(key, delta, cluster, backend) | &'static str, i64 | Lifecycle delta. Aggregates correctly across emit sites. Pair every +1 with a -1 on every close path. | gauge_add!("backend.pool.size", 1); |
time!(key, ms) / time!(key, cluster_id, ms) | &'static str, usize | Latency in ms. Stored as HDR histogram in the local drain. | time!("backend_response_time", cluster, ms); |
Gauge correctness
The most common metric bug is a gauge!() emitted from a per-connection /
per-stream context: the value the dashboard sees is "whatever the last
emitter wrote", not aggregate state. Default to gauge_add!() for any
metric that is incremented from more than one emit site. The H2 connection
gauges (h2.connection.{active_streams,window_bytes,pending_window_updates})
were converted to gauge_add! lifecycle deltas plus impl Drop symmetric
teardown after this exact bug went unnoticed for a while.
Gauge underflow
Past production incidents (a650ad69, d2f01ed4) all came from
gauge_add!(-1) running without a paired +1 having run earlier. Both
MetricValue::update and AggregatedMetric::update saturate the value to
0 and emit a single error! log line on underflow, in both debug and
release builds; neither panics. The metric is still wrong on the next
snapshot, but the process keeps running and the log line names the
offending key. Two patterns prevent this class of bug:
- Co-locate increments and decrements with the state being tracked. If
the gauge measures "active backend connections", emit the
+1at the same site that creates the backend connection and the-1at the same site that closes it. - Move teardown into
impl Dropwhen the close paths are scattered across the codebase (graceful shutdown, force-disconnect, panic-unwind).impl Drop for ConnectionH2is the canonical example: it subtracts whatevergauge_add!(+N)the connection ever emitted, regardless of which close path runs.
Cardinality budgets
StatsD (UDP) is forgiving but Prometheus / influxdb is not. Two rules:
- Static keys only. A key is
&'static str— synthesised at compile time viaconcat!, a literal, orLazyLock-leaked at startup. Per-IP labels MUST hash into a bounded bucket table (seelib/src/server.rs::PER_SOURCE_BUCKETSfor the canonical 256-bucket pattern withLazyLockstatic-string array). - Opt-in granularity.
MetricDetail(proto) /MetricDetailLevel(config) lets operators chooseprocess | frontend | cluster | backend. Mirrors HAProxy'sextra-countersopt-in. New label dimensions should respect this knob; the wiring layer is a follow-up MR.
Local drain reset cadence
The local drain (LocalDrain in lib/src/metrics/local_drain.rs) keeps two
maps: proxy_metrics (proxy-wide) and cluster_metrics (per-cluster +
per-backend). Both are cumulative since worker start. There is no
automatic timer that resets them — the wall-clock if now.minute() == 0 && now.second() == 0 block that used to clear cluster_metrics every UTC
hour was removed because it produced apparent spikes / drops to zero in
operator dashboards, and because gauges had to be preserved across the
clear anyway (long-lived H2 sessions could close after the clear and
underflow the gauge counter). Operators who want to reset issue
sozu metrics clear (proto: MetricsConfiguration::Clear), which wipes
both maps in one shot AND wipes the master process's own main_metrics.
Per-cluster entries are also dropped on RemoveCluster /
RemoveBackend IPC so retired clusters do not leak metric keys; on the
StatsD network_drain the same two events drop the cluster's
cluster_metrics, backend_metrics, and queued MetricLines, so the
wire side goes silent immediately (any unsent statsd interval for the
cluster is discarded — bounded by the one-second network-drain cadence
hard-coded in NetworkDrain::send_metrics).
RemoveCluster also arms a per-drain tombstone (removed_clusters: HashSet<String>) so subsequent emissions for the same cluster id are
dropped on the floor rather than resurrecting the row via
entry().or_default(). This matters in production: the per-proxy
remove_cluster paths in lib/src/http.rs / https.rs / tcp.rs
drop cluster config but do NOT close in-flight sessions, so long-lived
H2 / WebSocket / TCP sessions continue emitting access-log /
response-time / gauge metrics for the removed cluster. Without the
tombstone those emissions would keep growing the cluster row until the
last session closed; with it, the wire and the local drain stay quiet.
The tombstone is cleared on AddCluster for the same id (a cluster can
come back after a remove) and on sozu metrics clear (operator-initiated
full reset).
Implication for dashboards: counters in sozu metrics output are
monotonic; charts should compute rate() / irate() rather than
treating successive snapshots as windowed counts. Histograms accumulate
every sample since worker start, so Percentiles.p_99 is the lifetime
p99, not a windowed one. Per-bucket counters are u64 so saturation is
not a concern under realistic uptime / RPS combinations.
Operator clear caveat: issuing sozu metrics clear during live traffic
resets in-flight gauge accuracy. Sessions that opened before the clear
and decrement gauges on close (e.g. connections_per_backend) land on
the saturating-to-zero path and emit one error! log line per
occurrence, naming the offending key. This is intentional; the gauge
recovers as new sessions arrive. Operators who want to reset counts but
keep live gauges intact should not use sozu metrics clear — wait for
the cumulative counters to roll over in their dashboard math instead.
Log primitives
Structured prefixes via per-protocol log_context! / log_module_context! /
log_context_lite! macros — defined per file:
| Prefix | File | Carries |
|---|---|---|
MUX | protocol/mux/mod.rs | session ULID, peer/local, frontend, backend list |
MUX-H2 | protocol/mux/h2.rs | …plus position, state, total RST counts, draining |
MUX-H1 | protocol/mux/h1.rs | …plus stream id, parked, close_notify |
MUX-ROUTER | protocol/mux/router.rs | renders [session req cluster backend] via HttpContext::log_context() |
MUX-CONN / MUX-CONV / MUX-PARSER / MUX-PKAWA / MUX-STREAM | corresponding files | module-level only (no per-session context) |
KAWA-H1 | protocol/kawa_h1/mod.rs | session, frontend, request/response parsing phase |
RUSTLS | protocol/rustls.rs | SNI/ALPN byte lengths, version, source, frontend |
PIPE | protocol/pipe.rs | addresses, frontend/backend status & readiness |
TCP | tcp.rs | frontend, backend, peer (cached on SessionTcpStream) |
SOCKET | socket.rs | session, peer, local, RTT, state |
Conventions:
- Use the macro defined in the file. Do not call
log::info!/log::error!directly from protocol code — the prefix tag is load-bearing for log-search. - Tier severity by intent:
debug!/trace!for expected idle closes, timeouts, noisy state.warn!/error!for real protocol errors or invariant breaks. (Seefeedback_log_context_before_theorisingfor the reasoning.) - When an
HttpContextis in scope, prefer$http_ctx.log_context()(kawa_h1/editor.rs:587) over hand-rolling aLogContext { ... }struct literal — the helper is the canonical formatter.
Sensitive-value logging boundary
Debug is a production logging projection in Sōzu, not a lossless inspection
format. Generic log sites format command requests, worker responses, retained
tasks and state, router/listener errors, and TLS runtime objects with {:?}.
Every type reachable from those paths must therefore produce bounded metadata
when its fields can contain operator- or customer-controlled material.
The protected material includes certificate and private-key contents, chains,
certificate names and fingerprints, certificate-query domains and results,
cluster identifiers, HTTP routing hosts/paths/methods/tags, redirect and
rewrite values, TCP SNI and ALPN values, custom-answer keys and bodies, and
header names and values. Their log projections may retain only operationally
useful metadata such as socket addresses, enum variants, booleans, presence
flags, collection counts, and aggregate byte lengths. Certificate and key
slots render as [redacted] in addition to their lengths.
The canonical [session request cluster backend] prefix is the explicit
correlation envelope and is not a generic object dump: its cluster/backend
slots remain lossless so operators can join lines for one request. The same
identifiers must still be bounded when they appear inside Debug output,
retained task/state dumps, error text, or ad-hoc message fields. This exception
is limited to the named correlation slots; it does not permit raw authority,
path, method, SNI, ALPN, certificate SAN, or request payload fields in the
verbose Session(...) body.
Apply the boundary at every layer that can reach a log sink:
- Generated protobuf types that carry sensitive fields are listed in
command/build.rsviaprost_build::Config::skip_debug; their bounded implementations live incommand/src/proto/mod.rs. The top-levelRequestand itsRequestTypeenum render only the exhaustive request kind, so string-valued verbs remain bounded both directly and insideWorkerRequest. - Configuration, response, retained-state, gatherer, router, listener, and TLS
wrappers must summarize their own collections and string-keyed maps rather
than relying on an enclosing type to hide them.
TaskContainerrenders only the concrete task kind and timeout presence; it never delegates to a retainedGatheringTask'sDebugimplementation. - Direct
debug!,info!,warn!,error!, and failure-message projections must interpolate the same counts, kinds, addresses, and byte lengths. A safeDebugimplementation does not protect a log statement that formats a raw field directly. TLS preread/handshake and mux routing logs therefore render cluster/authority/SNI/ALPN/SAN counts, match kinds, and aggregate byte lengths rather than individual values. - Error and poisoned-lock paths follow the same rule; exceptional control flow is not permission to disclose the rejected value or the retained object.
This is deliberately a logging-only boundary. Protobuf wire encoding,
Serde/JSON output, retained-state keys, certificate-query response payloads,
and the raw fields stored in StateError, RouterError, and
RetrieveClusterError::SniAuthorityMismatch remain unchanged. These error
types bound only Display/Debug; callers matching their public variants
still receive the complete identifiers, frontend keys, rejection reasons,
hosts, paths, methods, SNI values, and authorities.
RequestHttpFrontend::Display, which is used as an operational/state key, also
remains lossless. Callers that are explicitly authorized to retrieve these
values continue to receive them; generic diagnostics do not.
Regression tests use a distinct long sentinel for every protected field and
assert all three parts of the contract: the sentinel is absent, the expected
count/length metadata is present, and output stays within a fixed size bound.
Coverage must include the leaf Debug implementation, generic/nested command
wrappers, retained state or tasks, direct log statements, and runtime TLS and
error paths as applicable.
Per-flood-detector helper macro pattern
When a violation funnel (H2FloodViolation, RustlsError, H2Error) needs a
&'static str metric key per variant, define a concat!-based macro:
macro_rules! h2_error_metric_key {
($prefix:literal, $error:expr) => {
match $error {
H2Error::NoError => concat!($prefix, ".no_error"),
H2Error::ProtocolError => concat!($prefix, ".protocol_error"),
// …14 arms, exhaustive — adding a new H2Error variant fails the build here
}
};
}
Then wrap with one helper per direction:
fn metric_for_goaway_sent(error: H2Error) -> &'static str {
h2_error_metric_key!("h2.goaway.sent", error)
}
This keeps the breakdown in lock-step with the underlying enum (the build
breaks on a new variant) while preserving &'static str semantics.
Access logs
Schema lives in command/src/logging/access_logs.rs::RequestRecord with the
protobuf wire shape in command/src/command.proto::ProtobufAccessLog. Adding
a field requires:
- Field on
RequestRecord<'a>(Rust struct,lib/src/protocol/...populates). - Field on
ProtobufAccessLog(proto, append a new optional tag — never reuse or reorder existing tags). - Populate at every emit site:
- H1:
lib/src/protocol/kawa_h1/mod.rs::log_request - H2 mux:
lib/src/protocol/mux/stream.rs::generate_access_log - TCP:
lib/src/tcp.rs::log_request - WS / WSS post-upgrade pipe:
lib/src/protocol/pipe.rs::log_request
- H1:
- Update
RequestRecord::duplicate()inaccess_logs.rsso the protobuf path serialises the new field. - Document the field in
configure.md§OpenTelemetry / §TLS handshake metadata.
The TLS metadata fields (tls_version, tls_cipher, tls_sni, tls_alpn)
are sourced from the rustls handshake context in lib/src/https.rs and
plumbed via mux::Context into HttpContext. The pipe path picks them up
via Pipe::set_tls_metadata called from https.rs::upgrade_mux.
Tracing — current state
This is W3C traceparent passthrough only. No span lifecycle, no OTLP
exporter, no SDK dependency. Behind the opentelemetry compile-time
feature flag:
traceparentis parsed inlib/src/protocol/kawa_h1/editor.rs::on_request_headers(the same callback runs on H1 and on H2 frames decoded viapkawa.rs).- A new span ID is generated per Sōzu hop; the value is rewritten on the
outgoing request and stored on
HttpContext.otel. - Access logs surface
trace_id,span_id,parent_span_id. - Access logs include
start_time(proto field 30,Uint128, nanoseconds since epoch) — a wall-clock timestamp captured at the start of the request viaSessionMetrics::mark_request_start(). Consumers reconstructing OTel spans should prefer this field overtime - request_time, which mixesCLOCK_REALTIMEandCLOCK_MONOTONICand produces unreliable start timestamps on short-lived requests.
To go further (real spans, OTLP exporter, B3/Datadog propagation), see the
"Out of scope" section in configure.md. It would land
behind a new feature flag rather than expanding the scope of opentelemetry.
Control-plane audit trail
Every privileged mutation on the unix command socket (AddCluster,
RemoveCertificate, ActivateListener, …) goes through three observability
surfaces:
-
An
Eventof the matchingEventKindis published on theSubscribeEventsbus. The 14 mutation variants are enumerated incommand/src/command.proto::EventKind. -
An
incr!("config.<verb>")counter is bumped (e.g.config.cluster_added). -
A structured audit log line is emitted at
info!level in the MUX-family layout (keywordCommand(...)rather thanSession(...), which names a data-plane session). Every free-form field (target,actor_comm,actor_user,socket,reason) is sanitized at render time — control chars (\x00..=\x1f,\x7f) are replaced with?so attacker-influenced input cannot forge additional audit lines via embedded\t/\n/ ANSI escapes. Rendered form (ANSI colours off):[01HXS4GZ9EYP3F2R7K8M6B4N2C 01HXS4H5K2QR9C7PVWXY8T6ZNA my_app -] AUDIT Command(verb=cluster_added, actor_uid=1000, actor_gid=1000, actor_pid=12345, actor_user=florentin, actor_comm=sozu, client_id=42, socket=/run/sozu/sozu.sock, target=cluster:my_app, result=ok, sozu_version=1.1.1)Field reference
Mandatory fields (always present):
verb— stable static identifier for the audited operation (e.g.cluster_added,state_loaded,listener_updated). Oneconfig.<verb>statsd counter per verb.actor_uid/actor_gid/actor_pid— peer credentials fromSO_PEERCREDon the unix command socket.unknownon read failure / non-Linux builds.actor_user— resolved POSIX account name (getpwuid_r(uid)at accept time).unknownwhen NSS has no match for the UID.actor_comm—/proc/<pid>/commat accept time (up to 15 chars), lets SOC distinguish thesozubinary running with thecommandsub-command from ad-hoc shells that share a UID.client_id— per-accept monotonic counter. Distinct from thesession_ulidbracket slot, which survives as a grep-correlation key across every verb a single sozu CLI invocation emits.socket— path of the command socket the client connected through. Lets multi-instance sozu deployments that share a SIEM sink pick which instance emitted each line.target— free-form verb-specific descriptor (e.g.cluster:my-cluster,file:/var/lib/sozu/state.bin,stop:hard,listener:http:127.0.0.1:8080). ForUpdateHttp/Https/TcpListener, each patched field is rendered asfield=old→new. ForReplaceCertificate, both old and new cert fingerprints are included (certificate:<addr>:old=<fp>:new=<fp>). Sanitized.result—okorerr.sozu_version—CARGO_PKG_VERSIONat build time. Forensic pin for mixed-fleet audit streams.
Optional fields (appear when relevant):
error_code— structured failure bucket:dispatch_error,worker_failure,worker_timeout,peer_cred_unavailable,invalid_input,io_error,other. Present whenresult=err.reason— truncated human-readable failure detail (max 256 chars, sanitized). Pairs witherror_code.elapsed_ms— wall-clock time between request acceptance and audit emission. Set on completion-time lines.fanout— worker fan-out outcome:ok,partial,timeout, orlocal_only. Set on completion-time lines for verbs that scatter to workers.workers—<ok>/<err>/<expected>per-worker counts. Pairs withfanout.request_sha256— truncated (64-bit, 16 hex chars) SHA-256 of the protoRequestwire-encoding. Set for verbs that flow throughworker_request. Useful for dedupe / replay detection.
Dedicated sink:
audit_logs_targetOperators can route audit lines to a dedicated file (distinct from
log_target) via theaudit_logs_targetconfig option. Set to a plain filesystem path (e.g./var/log/sozu/audit.log) to have every audit line also appended there. The file opensO_APPEND | O_CREATwith mode0o640(owner read+write, group read), so granting anauditgroup tail-only access is one filesystem ACL away. ANSI escape sequences are stripped before writing to the dedicated sink so the file stays SIEM-parseable even whenlog_colored = true. Write failures log a warning but never block the mutation.None(default) keeps audit lines routed only throughlog_target.JSON sink:
audit_logs_json_targetFor SIEM pipelines that prefer not to parse the human-readable line,
audit_logs_json_targetwrites a structured JSON object per line to a dedicated file. SameO_APPEND | O_CREAT | 0o640semantics. Schema is stable; every key is always present, missing values are JSONnull:{ "ts": "2026-04-23T13:14:15.123456Z", "boot_generation": 0, "session_ulid": "01HXS4GZ9EYP3F2R7K8M6B4N2C", "request_ulid": "01HXS4H5K2QR9C7PVWXY8T6ZNA", "actor": { "uid": 1000, "gid": 1000, "pid": 12345, "user": "florentin", "comm": "sozu", "role": "user" }, "client_id": 42, "connect_ts": "2026-04-23T13:14:14.500000Z", "socket": "/run/sozu/sozu.sock", "verb": "cluster_added", "target": "cluster:my_app", "result": "ok", "cluster_id": "my_app", "backend_id": null, "sozu_version": "1.1.1", "build_git_sha": "9a1b2c3d4e5f", "extras": { "elapsed_ms": 17, "fanout": {"status": "ok", "workers_ok": 2, "workers_err": 0, "workers_expected": 2}, "request_sha256": "0123456789abcdef" } }Both sinks are independent — set both for an operator-friendly tail stream + a machine-parseable archive.
Retention policy
PCI-DSS 10.7 requires audit trails to be retained ≥ 1 year, with the most recent 3 months immediately available for analysis. ISO 27001 A.8.15 recommends similar but defers to organisational policy.
audit_logs_targetandaudit_logs_json_targetare append-only files under your operator's logrotate / archival pipeline — the recommended shape on Clever Cloud Linux deployments is:/etc/logrotate.d/sozurotates/var/log/sozu/audit*.{log,jsonl}daily, compresses withxzafter 1 day, archives off-host after 90 days.- Off-host archive bucket retains for 400 days to clear the PCI-DSS 1-year window with cushion.
- Restrict on-host read access via
setfacl -m g:audit:r-x /var/log/sozu/; only theauditgroup should be able to tail the live file. - The
Server.boot_generationfield stamped on every line lets log analysers stitch sessions across hot-upgrade re-execs without trusting PIDs.
Sōzu does not rotate or compress audit files itself — that is delegated to the OS-level rotator. Operators who want native rotation should use
logrotatewith thecopytruncatedirective (since sōzu keeps the file handle open) or sendSIGHUPafter rename to trigger a re-open (not yet implemented — TODO).Two lines per worker-fanning verb
Verbs that fan out to every worker (AddCluster, RemoveHttpFrontend, UpdateHttpsListener, AddCertificate, …) emit two audit lines:
- Attempt-time:
result=okmeans "accepted by the main process state". Fires immediately afterstate.dispatchsucceeds. No fanout / elapsed_ms. - Completion-time: emitted when every worker has responded (or the
scatter deadline fires). Carries
fanout=ok|partial|timeout,workers=<ok>/<err>/<expected>,elapsed_ms, and — onresult=err—error_code+reason.
Operators correlate the two via the shared
[session_ulid request_ulid …]bracket.Local-only verbs
Verbs that don't fan out —
SoftStop/HardStoprequest,LoggingLevelChanged,UpgradeMain/UpgradeWorkerinit,SubscribeEvents,SaveState, andLoadStatecompletion — emit a single audit line carryingresultand (when applicable)error_code+reason.LoadStateandSaveStateembedok:<n> errors:<n>counts intarget=file:<path>.Bracket slots follow the
[session_ulid request_ulid cluster_id|- backend_id|-]convention shared withMUX/MUX-ROUTER/RUSTLS/PIPE/TCPlines.
The actor_uid is captured at unix-socket accept time via SO_PEERCRED
(bin/src/command/server.rs, stored on ClientSession.actor_uid). Failed
syscalls or non-Linux builds collapse to actor_uid=unknown rather than
panicking. This satisfies PCI-DSS 10.2 / ISO 27001 A.8.15 / SOC 2 audit-trail
requirements without an external audit shim.
To add a new audited verb:
- Add the
EventKindvariant to the proto (preserve existing tags). - Update
Display for Eventincommand/src/proto/display.rs. - In the
bin/src/command/requests.rshandler:- Push the
Eventto the bus (find an existingEventKind::CLUSTER_ADDEDemit site to copy from). - Emit
incr!("config.<verb>"). - Emit the audit line via the
audit_log_context!macro inbin/src/command/requests.rs— it renders the verb into the MUXSession(verb=..., actor_uid=..., client_id=..., target=..., result=...)block automatically.
- Push the
Extension checklist
Before merging an instrumentation change:
- Metric keys are
&'static str(compile-time literal orLazyLock-leaked). - Per-direction / per-error / per-frame breakdowns use a
concat!-based helper macro so the build breaks on a new enum variant. - Cardinality is bounded — either no labels, or labels from a fixed-size table, or hashed into a bucket of known size.
- Gauges that are emitted from more than one site use
gauge_add!(lifecycle delta) and pair every+Nwith a-Non the close path. Considerimpl Dropfor symmetric teardown. - New protocol modules define their own
log_context!/log_module_context!with a unique prefix tag. - Types and direct log sites reachable from sensitive command, state, routing, or TLS data expose only bounded counts/lengths/kinds/addresses, with sentinel-absence and fixed-output-bound regression tests.
- New access-log fields land on
RequestRecord,ProtobufAccessLog(new tag), all four emit sites, andRequestRecord::duplicate(). -
configure.mdis updated in the same changeset. - Privileged control-plane verbs land an
EventKind+config.<verb>counter + audit log line (MUXSession(...)layout,info!level, routed via theaudit_log_context!macro).
See also
configure.md— full inventory of currently emitted metrics, access-log fields, OpenTelemetry passthrough config.h2_mux_internals.md— H2 mux state machine and flood-detector design.lib/src/protocol/mux/LIFECYCLE.md— stream/slot lifecycle withfile.rs:LINEcitations.CLAUDE.md— agent conventions including log macro discipline, metric macros list, and security-sensitive areas.