Failure model

This document states the behavior the implementation is designed to produce. Observed test evidence belongs in verification.md; expected behavior is not a substitute for a run.

Assumptions

• PostgreSQL 14–18 and PGMQ 1.12.0 operate according to their documented transaction and visibility semantics.
• Queue tables are durable unless the caller explicitly chooses QueueStorage::unlogged.
• PostgreSQL evaluates whether a stored PGMQ visibility timestamp has expired. Caller-supplied absolute delivery/visibility TimePoint values must identify the wall-clock instant the caller intends.
• Transactional retry sends a millisecond duration and PostgreSQL derives the deadline from clock_timestamp() inside the transaction; it does not depend on the application wall clock.
• Both worker modes accept retry durations only from 0 through 9,007,199,254,740 milliseconds. This exact-interval bound is validated before visibility SQL.
• Regular-worker lease scheduling instead uses visible_at - database_observed_at from one database query and maps that duration to steady_clock; it does not subtract the host wall clock.
• Handler code may fail, block, ignore cancellation, or perform non-transactional side effects.
• Transactional handlers are trusted to follow the WorkerTransaction contract and not use arbitrary execute() SQL to end the transaction or acknowledge/change the current task.
• Processes, database connections, and notification sessions can disappear at any instruction boundary.
• There is no distributed transaction with an external service.

Failure matrix

Failure	Detection	Queue/business outcome	Duplicate risk	Required response
Handler returns retry	explicit result	business effect is caller-defined; source is rescheduled	Yes	Make prior effects idempotent
Explicit retry delay or configured policy is negative or exceeds 9,007,199,254,740 ms	HandlerResult::retry, queue registration, and runtime guards	invalid_input / worker_retry occurs before visibility SQL; an in-handler or aggregate-bypass result emits handler_failed and is dead-lettered once	Prior external effects may repeat	Correct the duration/policy; do not use an absolute timestamp as a duration
Handler throws retryable pgmq::Error	exception category	regular mode schedules retry; transactional mode rolls back handler savepoint then schedules	Yes	Inspect event/error context
Handler throws non-retryable pgmq::Error	exception category	atomic DLQ move	Low for queue move; prior external effect may repeat	Inspect DLQ
Other handler exception	catch-all	retry, then DLQ at attempt limit	Yes	Classify explicitly when possible
Regular lease renewal fails before deadline	renewal event	short bounded retry while original deadline remains	Possible if outage outlasts lease	Handler should observe stop request if lease becomes lost
Lease expires or conditional renewal affects no row	lease_lost	no ack/retry/DLQ by stale worker; message may have a newer owner	Old and new handlers may overlap	Idempotency; stop cooperatively
DB disconnect before a new SDK operation	libpq health check	reconnect before sending operation	Normal operation semantics	Monitor reconnects
DB disconnect during one autocommit statement	libpq error	server outcome can be unknown; SDK does not replay blindly	Producer write may have committed	Reconcile/idempotently retry
DB disconnect inside caller transaction	error; reconnect disabled	connection loss causes server rollback unless commit already completed	Commit outcome can be unknown	Reconcile durable state
Connection pool exhausted beyond pool_acquire_timeout	timeout-category pgmq::Error	operation was not sent	None from that operation	Increase capacity, shorten leases, or reduce contention
Runtime thread creation fails during WorkerRuntime::start()	thread constructor exception	already launched runtime threads are stopped and joined; a later start() rebuilds all per-run state	Work claimed during a partially completed multi-queue start can become visible again after its lease	Log the failure, relieve the process resource limit, then retry start()
Invalid QueryResult row/column lookup	std::out_of_range, or std::invalid_argument for a null C-string name	query has already completed; result access fails without another database operation	None	Correct the index/name or check result shape
Notification lost/coalesced/throttled	usually not directly knowable	polling later finds row	None from notification alone	Keep polling enabled
Listener disconnect	listener reconnect counter/event	LISTEN and configuration restored; race covered by poll	None from notification alone	Monitor reconnect rate
Notification setup unsupported/denied	setup error in direct SDK; worker falls back	worker polls	Higher latency/query load	Fix version/privileges if notifications desired
Process crash in regular handler	no local cleanup	lease stops renewing; source becomes visible after vt	Yes	Restart worker; use idempotency
Process crash in transactional handler before commit	PostgreSQL session rollback	claim, handler SQL, and ack all roll back	Handler may rerun, but database partial state does not persist	Restart worker
Shutdown grace timeout	ShutdownResult and event	no remaining ack; renewal stops	Yes after lease expiry	Application controls process lifetime
Metrics/event sink failure	callback exception is swallowed	task processing continues	None	Make sink independently reliable
PostgreSQL crash with durable queue	PostgreSQL recovery	committed rows follow PostgreSQL durability	Depends on commit acknowledgement ambiguity	Follow PostgreSQL recovery procedures
PostgreSQL crash with unlogged queue	PostgreSQL semantics	unlogged queue can be truncated	Data loss	Do not use for durable work
Partition maintenance error	PostgreSQL/pg_partman error	affected operation fails	Depends on caller retry	Repair pg_partman/retention configuration

Deterministic process-crash boundaries

Crash tests use a separate project-owned worker process and deterministic sync points. The expected state transitions are:

Kill point	Regular mode	Transactional mode
Before claim	Message remains visible/unclaimed	Message remains visible/unclaimed
After claim, before handler	Invisible until lease expiry, then redelivered	Open transaction rolls back on connection death; message is available
During handler	Redelivered after lease expiry	Handler SQL and claim roll back
External side effect complete, before ack	Side effect may be duplicated after redelivery	External effect may be duplicated; it was never covered by PostgreSQL atomicity
Before transaction commit	Not applicable to regular external work as one atomic unit	Business rows and ack both absent after rollback
After ack/commit has completed	Source is deleted/archived; no redelivery	Business rows and ack both present

“After commit” must mean the test process crossed a synchronization point after PostgreSQL returned successful commit—not merely that COMMIT was sent.

The verification report must record total unique tasks, final queue/business state, and observed duplicate count. If a crash scenario was not run, its status is not run, not inferred from this table.

Unknown commit outcome

A network failure can occur after PostgreSQL commits but before the client receives the success response. The database remains atomic, but the caller does not know which side of commit occurred.

The library intentionally does not reconnect and replay a transaction. Applications should:

1. choose a stable operation id;
2. store it with a unique constraint in the same transaction;
3. query durable state after reconnect;
4. retry only through an idempotent path.

The same issue exists for a producer send executed as one autocommit statement: if the connection fails mid-response, the message may exist. Put an application event id in its body/headers if duplicate sends matter.

Regular completion failures

Regular terminal actions run in their own transaction after the handler:

• ownership is locked and checked;
• the mutation is made;
• commit determines success.

If ownership has changed, the action fails closed as lease_lost. If the database operation itself fails, the runtime emits handler_failed with completion context and does not claim success. The existing message eventually becomes visible unless the server actually committed and only the response was lost. This conservative behavior favors no silent deletion over optimistic acknowledgement.

Dead-letter failure

The DLQ send and source delete share one transaction. Expected invariants:

• commit: destination exists and source does not;
• rollback/failure: source remains and destination does not;
• stale ownership: neither move nor delete is attempted.

The destination queue is created at runtime start. Failure to create it causes startup to fail instead of deferring a predictable dead-letter error.

Notification failure

Notifications are intentionally absent from correctness invariants:

• the trigger may throttle a burst;
• PostgreSQL may coalesce repeated same-channel empty-payload notifications;
• a producer transaction can roll back, in which case no committed message exists;
• a listener may be disconnected;
• LISTEN restoration has a race.

The only durable fact is the queue row. Polling checks that fact.

Shutdown failure

std::stop_token is cooperative. A handler that blocks forever cannot be safely terminated by the runtime.

On timeout, shutdown() returns without pretending the drain succeeded. It disables acknowledgements for unfinished regular claims and stops renewal. The application must not destroy objects referenced by the handler while it is still executing. Depending on deployment policy, it may log and keep waiting, escalate to process termination, or isolate handler work in a separately killable process.

No timeout path acknowledges unfinished work.

Explicitly outside the model

pgmq-cpp does not solve:

• Byzantine database behavior or silent storage corruption;
• arbitrary code continuing safely after memory corruption;
• exactly-once calls to external systems;
• atomic commits across databases;
• recovery of an unlogged queue after PostgreSQL crash;
• application logic that uses a non-unique or changing idempotency key.