Benchmarks

Purpose

The benchmark is a reproducible measurement harness, not a source of universal performance claims. It connects to a real PostgreSQL/PGMQ database and writes a JSON result document with schema pgmq-cpp.benchmark.v1.

Never run it against a shared, staging, private, or production database. It creates, purges, and drops queues named:

• pgmq_cpp_bench_send
• pgmq_cpp_bench_consume
• pgmq_cpp_bench_consume_dlq
• pgmq_cpp_bench_notify
• pgmq_cpp_bench_notify_dlq

A scope guard attempts to drop all five queues in reverse order on both normal completion and C++ exception paths. Cleanup is best-effort so that a cleanup failure cannot hide the benchmark result; abrupt process or host termination can still leave project-owned queues for inspection or manual cleanup.

Build and run

Use a Release build for publishable numbers:

export VCPKG_ROOT=/path/to/vcpkg
cmake -S . -B build/linux-release -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_TOOLCHAIN_FILE="$VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" \
  -DVCPKG_TARGET_TRIPLET=x64-linux \
  -DBUILD_TESTING=OFF \
  -DPGMQ_CPP_BUILD_CLI=OFF \
  -DPGMQ_CPP_BUILD_EXAMPLES=OFF \
  -DPGMQ_CPP_BUILD_BENCHMARKS=ON
cmake --build build/linux-release --target pgmq-cpp-benchmark

./build/linux-release/benchmarks/pgmq-cpp-benchmark \
  --dsn 'postgresql://pgmq:pgmq@127.0.0.1:55432/pgmq' \
  --messages 10000 \
  --output build/benchmark-linux-release.json

With a multi-configuration Windows build:

cmake --build --preset windows-release --target pgmq-cpp-benchmark
.\build\windows-release\benchmarks\Release\pgmq-cpp-benchmark.exe `
  --dsn 'postgresql://pgmq:pgmq@127.0.0.1:55432/pgmq' `
  --messages 10000 `
  --output build\benchmark-windows-release.json

PGMQ_TEST_DSN may replace --dsn. The default message count is 500 and the default output is pgmq-cpp-benchmark.json. The benchmark refuses to run without an explicit database connection.

Measurements

One invocation records:

Section	Method
send_single	Sequential one-message SDK sends
send_batch	SDK batch sends in fixed batches of 100
consume_scaling	Preload the queue, then run regular workers at 1, 2, 4, and 8 threads
handler latency	Runtime handler duration p50/p95/p99/max for each worker count
queue wait	Database enqueue-to-observation duration plus local steady-clock claim-queue-to-handler duration, p50/p95/p99/max
idle_worker	Three seconds with no work, notifications disabled, 250 ms polling
idle CPU	Approximate process CPU time divided by wall time
empty-read rate	Estimated from polling wake-up count
wakeup_latency	One polling-only and one LISTEN/NOTIFY wake-up with a 500 ms fallback

Consume elapsed time ends when the last handler reports completion; the subsequent clean shutdown verifies that terminal actions drain, but the final ack drain is not included in that throughput interval. Handler latency excludes database acknowledgement. These definitions must stay attached to quoted numbers. Queue wait deliberately does not subtract a database timestamp from the application wall clock; the split-domain calculation excludes the transit gap between the database observation and local claim registration.

The wake-up comparison is a single sample per mode, suitable as a smoke signal, not a statistically stable latency distribution. Run repeated independent invocations before drawing conclusions.

Metadata written automatically

The JSON records:

• pgmq-cpp version;
• PostgreSQL version and server_version_num;
• PGMQ version;
• operating-system family;
• compiler identifier/version;
• debug versus release;
• reported hardware concurrency;
• message count, batch size, and visibility timeout.

Metadata that must accompany published results

Also record outside the JSON:

• exact commit of pgmq-cpp;
• CPU model and assigned cores;
• RAM;
• storage type;
• bare metal, VM, WSL, or container limits;
• database host topology;
• Docker/image digest when applicable;
• PostgreSQL settings changed from defaults;
• connection string with credentials removed;
• warm-up policy and number of repetitions;
• background load;
• median and spread across runs.

Without this information, numbers are local observations, not comparable claims.

Recommended protocol

1. Start the pinned project integration database or another isolated PGMQ 1.12.0 instance.
2. Confirm the benchmark reports the intended PostgreSQL/PGMQ versions.
3. Build Release with the same compiler/options used for comparison.
4. Run one warm-up whose output is not reported.
5. Run at least five measured invocations with the same --messages.
6. Retain every raw JSON document.
7. Report median plus min/max or an appropriate confidence interval.
8. Change only one independent variable between comparisons.
9. Stop and remove only project-owned database resources.

For worker-scaling analysis, inspect the 1/2/4/8 series rather than quoting only its maximum. Database capacity, client pool size, host cores, and acknowledgement round trips can all become the bottleneck.

Interpretation cautions

• A debug build is useful for correctness, not representative throughput.
• Loopback database results do not predict remote network performance.
• Empty-handler throughput measures the queue/runtime path, not application work.
• The operating-system process CPU timer gives aggregate process CPU time; the resulting percentage is only an approximation and can exceed one core's capacity when several threads run.
• Queue wait combines a database-clock segment and a local steady_clock segment. It avoids host/database wall-clock skew but omits the response transit gap between those segments.
• Notification latency is affected by producer commit, PostgreSQL scheduling, listener state, and the polling fallback.
• PGMQ notification throttle changes event volume, not queue correctness.
• Unlogged queues may be faster but have different crash durability and must not be compared as if equivalent to durable queues.

Results status

Final-source local Windows run — 2026-07-26

One Release run completed successfully on the local Windows development host after the benchmark cleanup fix. It used MSVC 19.41, PostgreSQL 18.4, PGMQ 1.12.0, 500 messages, batches of 100, and a 30-second visibility timeout. The raw result is retained at local-windows-pg18-pgmq-1.12.0.json.

The environment record is:

Property	Observed value
Machine	ASUS TUF Gaming A16 FA607PV
CPU	AMD Ryzen 9 7940HX, 16 cores / 32 logical processors
Physical RAM	33,517,625,344 bytes (about 31.2 GiB)
Container runtime	Docker Desktop 4.36.0 / Engine 27.3.1, WSL2 Linux backend
Database image	ghcr.io/pgmq/pg18-pgmq@sha256:9410102ea5fc16428be5b7203fdfef372827039f57ae05874e8c2e6f45e15336
Container limits	No per-container CPU or memory limit recorded
Database storage	Local Docker volume; exact physical-media mapping not proven

The run observed:

Measurement	Observed value
Sequential single send	286.309 messages/s
Batch send, batch size 100	22,879.630 messages/s
Idle process CPU, three-second sample	0.0%
Estimated empty reads	3.986 queries/s
Polling wake-ups	11 in 3.011 seconds
Polling-only wake-up, one sample	409.2320 ms
LISTEN/NOTIFY wake-up, one sample	10.1499 ms

Worker scaling and empty-handler latency were:

Workers	Throughput (messages/s)	Handler p50 (ms)	p95 (ms)	p99 (ms)	max (ms)
1	173.384	0.0042	0.0075	0.0091	0.0138
2	381.355	0.0050	0.0102	0.0128	0.0248
4	706.972	0.0047	0.0104	0.0159	0.0420
8	1,364.847	0.0046	0.0111	0.0158	0.0502

The JSON also retains the split-domain queue-wait distributions for all worker counts. Its rate fields were checked against the stored message counts and elapsed seconds. After the run, an exact query of pgmq.meta found zero rows matching pgmq_cpp_bench%, confirming that the three working queues and two explicit dead-letter queues were removed.

This is one short local run without a discarded warm-up or repetitions. It is not a cross-platform result, statistically stable benchmark, production capacity claim, or comparative marketing result. The inspection did not capture transient CPU frequency, global Docker Desktop limits, background load, changed PostgreSQL settings, or exact physical storage mapping.

The exact execution command is recorded in verification.md. Future results must retain their raw JSON and the environment metadata required above; numbers must never be inferred from the benchmark source.