Reliability semantics

Terms

• claim: a PGMQ read that increments read_ct and advances vt.
• visibility lease: the interval until vt; other consumers should not receive that message during the interval.
• acknowledgement: deleting or archiving the source message.
• attempt: the read_ct value returned by the claim.
• claim token: (queue_name, msg_id, read_ct) in regular mode.
• side effect: any durable action performed by a handler.

Regular mode: at-least-once

Regular mode guarantees no successful acknowledgement before a handler returns a success disposition and the runtime still proves ownership. It cannot guarantee that a handler runs only once.

The critical crash window is:

claim -> handler external effect succeeds -> process dies -> ack never commits

After vt, PGMQ makes the message visible again. Another attempt repeats the handler. This is expected at-least-once behavior, not data loss.

Applications should put a stable business key in the message, for example payment_id, image_id + transformation_version, or an event UUID. The handler should make its destination operation conditional on that key:

• an inbox/deduplication table with a unique constraint;
• an idempotency key supported by the remote API;
• an upsert whose final state is deterministic;
• compare-and-set on the target business version.

An in-memory “seen” set is not sufficient because it disappears on restart and does not coordinate multiple processes.

Ownership and the read_ct guard

PGMQ's public completion and visibility functions identify a message by msg_id, not by a per-delivery receipt handle. msg_id alone is therefore unsafe for a long-running worker:

1. Worker A reads message 10 with read_ct=1.
2. A stalls until its vt expires.
3. Worker B reads message 10; PGMQ returns read_ct=2.
4. A resumes. Deleting by msg_id=10 alone would delete B's live claim.

pgmq-cpp treats read_ct as the claim generation. Every regular-mode renew, ack, retry, or dead-letter action requires:

same msg_id AND same read_ct AND vt is still in the future

The check and mutation occur under the same PostgreSQL transaction/row lock. When it fails, A is stale: the runtime emits lease_lost, disables its acknowledgement, and requests its stop token.

This guard protects queue ownership. It cannot undo a side effect that A already performed, and a C++ handler can ignore its stop token. Handlers must therefore remain idempotent and cooperatively cancellable.

Lease renewal

One scheduler batches due renewals by queue. A successful conditional update extends each owned message to now + visibility_timeout.

The read projection returns both the PGMQ vt as Message::visible_at and PostgreSQL clock_timestamp() as Message::database_observed_at. The runtime subtracts those two database-clock values, clamps the remaining duration to the configured lease, and maps it onto steady_clock. Lease scheduling therefore does not compare the database timestamp with the application host's wall clock and is not shifted by later wall-clock adjustments. PostgreSQL still makes the authoritative ownership decision with vt > clock_timestamp(). The conversion does not subtract response transit after the database observation, so every renewal and terminal operation still uses the database ownership guard and fails closed if the lease has expired.

• Renewal stops as soon as the claim is finished.
• A transient database failure emits lease_renewal_failed.
• Before the known deadline, the scheduler retries with a short bounded delay.
• At or beyond the deadline, failure becomes lease_lost.
• A renewal that affects no row immediately becomes lease_lost.
• Process death stops renewal; the message eventually becomes visible.

Choose lease_renewal_interval comfortably below visibility_timeout, leaving room for database stalls and scheduling delays. Configuration requires it to be positive and strictly shorter than the visibility timeout.

Concurrent consumers

PGMQ reads use PostgreSQL row locking and SKIP LOCKED. Multiple threads and processes can claim work without receiving the same message while its claim remains valid. The ownership guard prevents a stale claimant from later acknowledging a newer generation.

This is not a claim that two handler invocations can never overlap. If an old handler outlives its lease and ignores cancellation, a new valid claimant may run concurrently. Idempotency remains required.

Handler results and attempts

Disposition	Regular-mode effect
success/delete	Guard ownership, delete, commit
success/archive	Guard ownership, archive, commit
retry	Guard ownership, move vt by explicit or computed backoff, commit
dead-letter	Guard ownership, send DLQ envelope and delete source in one transaction
abandon	Stop renewal/ack and wait for current vt to expire

The default retry policy is five maximum attempts, 500 ms initial delay, factor 2, five-minute cap, and ±20% jitter. read_ct supplies the attempt number. If a handler asks to retry when the attempt has reached max_attempts, the runtime dead-letters instead.

Retry backoff is a duration, not an application-wall-clock deadline. Regular mode and transactional mode both bind validated bigint milliseconds and move visibility relative to PostgreSQL clock_timestamp(). Application and database wall-clock offset therefore does not change either retry delay. Caller-supplied absolute enqueue/visibility times remain absolute instants and are a separate API contract.

Both modes accept retry durations only in the inclusive range 0..9,007,199,254,740 ms (about 285 years). PostgreSQL interval multiplication resolves through float8; this bound keeps the internal microsecond value at or below 2^53, so every accepted integer millisecond is preserved exactly. HandlerResult::retry, queue registration for the configured RetryPolicy, and both runtime reschedule paths validate the same range. Negative or larger values produce pgmq::Error with ErrorCategory::invalid_input and operation worker_retry before any visibility SQL. A retry factory error thrown inside a handler follows the normal non-retryable handler-error path; the runtime emits handler_failed and applies its dead-letter classification. The runtime applies the same normalization to a manually constructed invalid HandlerResult, so bypassing the factory does not cause repeated handler re-entry.

A pgmq::Error thrown by a handler is retried only when its category is classified retryable. Other std::exception failures default to retry. Non-retryable pgmq::Error failures default to dead-letter. Explicit handler results remain preferable when the business classification is known.

Dead-letter atomicity

The runtime writes an ordinary PGMQ message to the configured DLQ and deletes the source in one PostgreSQL transaction. If either operation fails, the transaction rolls back and the source is not silently lost.

The envelope contains:

• schema pgmq-cpp.dead-letter.v1;
• original queue and msg_id;
• original read_ct;
• original body and headers;
• bounded error summary;
• dead-letter timestamp.

After a DLQ item has been selected and claimed, replay sends the preserved body/headers back through PGMQ and deletes that DLQ item in one transaction. Batch replay commits all selected send/delete pairs or none. The earlier selection read is outside that transaction and still increments read_ct; the CLI releases visibility best-effort on dry-run or failure. Replay is a new delivery and must not bypass idempotency.

Transactional mode

Transactional mode guarantees PostgreSQL atomicity:

claim + handler SQL + delete/archive + commit

All operations use one connection and transaction. A throw, cancellation, abandon, disconnect before commit, or process death before commit rolls back the business SQL and acknowledgement together. Success becomes durable only at commit.

This assumes the trusted handler follows the WorkerTransaction contract: execute() must not contain transaction-control SQL or SQL that acknowledges or changes the current task. The narrowed type removes direct convenience methods but cannot parse and sandbox arbitrary SQL.

For retry/dead-letter, a savepoint removes handler business changes while retaining the claim in the outer transaction; the retry schedule or atomic DLQ move is then committed. Transactional retry delay is calculated against the database clock, including when a test clock deliberately places the application wall clock far in the future.

The guarantee does not include external effects. A transaction handler that calls another system reintroduces the regular-mode ambiguity and should use an outbox or idempotency protocol.

There is also an unavoidable client-observation ambiguity if the network fails during COMMIT: PostgreSQL commits atomically, but the client might not learn which outcome occurred. Reconcile using durable business/queue state rather than blindly replaying non-idempotent work.

pop is different

Client::pop reads and deletes in the PGMQ function. If the application crashes after pop returns but before processing completes, the message is gone. It is appropriate only when at-most-once loss is acceptable or the returned data is immediately transferred into another atomic boundary.

The worker runtime does not use pop.

FIFO qualifications

FIFO functions control database selection order. Runtime concurrency still matters:

• grouped may return several messages from one group in a batch;
• grouped_round_robin may return multiple group layers;
• parallel handlers can therefore complete such a batch out of order;
• grouped_heads returns at most one current head per group and is the appropriate strategy for parallel strict per-group progression.

FIFO ordering does not prevent duplicate attempts after failure, and messages without x-pgmq-group share one upstream default group.

Notifications

Notifications improve idle pickup latency; they do not change delivery semantics.

• PGMQ sends an empty notification after insert, subject to throttle.
• PostgreSQL delivers only after commit.
• identical notifications may be coalesced;
• disconnected listeners miss notifications;
• notification setup and reconnect have races.

The worker always executes a read after waking and always keeps a positive polling fallback. Queue rows—not notification events—are the source of truth.

Shutdown

Shutdown stops claims before requesting active handlers to stop. Pending local claims are not acknowledged. A clean drain joins the runtime threads.

After grace-period expiry:

• the result reports timeout and remaining in-flight count;
• acknowledgements are disabled for those claims;
• renewal stops, so the messages eventually return;
• handlers are not forcibly killed.

The host must keep handler dependencies alive until handlers return or terminate the process according to its own policy. A handler that completed an external effect near shutdown can still be retried, so shutdown does not remove the need for idempotency.

Storage qualifications

Durable queues use normal PostgreSQL tables. Unlogged queues trade durability for reduced write overhead and may be truncated after a PostgreSQL crash; do not use them for work that must survive database failure. Partitioned queues depend on correct pg_partman installation and retention configuration.