Client Backpressure Tests

July 15, 2026 · View on GitHub

Introduction

The YAML and JSON files in this directory are platform-independent tests meant to exercise a driver's implementation of retryable reads. These tests utilize the Unified Test Format.

Several prose tests, which are not easily expressed in YAML, are also presented in this file. Those tests will need to be manually implemented by each driver.

Drivers should test that retries do not occur immediately when a SystemOverloadedError is encountered. This test MUST be executed against a MongoDB 4.4+ server that has enabled the configureFailPoint command with the errorLabels option.

Let client be a MongoClient
Let collection be a collection

Now, run transactions without backoff:

Configure the random number generator used for jitter to always return 0 -- this effectively disables backoff.

Configure the following failPoint:

    {
        configureFailPoint: 'failCommand',
        mode: 'alwaysOn',
        data: {
            failCommands: ['insert'],
            errorCode: 2,
            errorLabels: ['SystemOverloadedError', 'RetryableError']
        }
    }

Insert the document { a: 1 }. Expect that the command errors. Measure the duration of the command execution.

   const start = performance.now();
   expect(
    await coll.insertOne({ a: 1 }).catch(e => e)
   ).to.be.an.instanceof(MongoServerError);
   const end = performance.now();

Configure the random number generator used for jitter to always return a number as close as possible to 1.
Execute step 3 again.
Compare the time between the two runs.
```
assertTrue(absolute_value(with_backoff_time - (no_backoff_time + 0.6 seconds)) < 0.6 seconds)
```
The sum of 2 backoffs is 0.6 seconds. There is a 0.6-second window to account for potential variance between the two runs.

Test 2: REMOVED

Test 3: Overload Errors are Retried a Maximum of MAX_RETRIES times

Drivers should test that overload errors are retried a maximum of MAX_RETRIES times. This test MUST be executed against a MongoDB 4.4+ server that has enabled the configureFailPoint command with the errorLabels option.

Let client be a MongoClient with command event monitoring enabled.
Let coll be a collection.

Configure the following failpoint:

    {
        configureFailPoint: 'failCommand',
        mode: 'alwaysOn',
        data: {
            failCommands: ['find'],
            errorCode: 462,  // IngressRequestRateLimitExceeded
            errorLabels: ['SystemOverloadedError', 'RetryableError']
        }
    }

Perform a find operation with coll that fails.
Assert that the raised error contains both the RetryableError and SystemOverloadedError error labels.
Assert that the total number of started commands is MAX_RETRIES + 1 (3).

Test 4: Overload Errors are Retried a Maximum of maxAdaptiveRetries times when configured

Drivers should test that overload errors are retried a maximum of maxAdaptiveRetries times, when configured. This test MUST be executed against a MongoDB 4.4+ server that has enabled the configureFailPoint command with the errorLabels option.

Let client be a MongoClient with maxAdaptiveRetries=1 and command event monitoring enabled.
Let coll be a collection.

Configure the following failpoint:

    {
        configureFailPoint: 'failCommand',
        mode: 'alwaysOn',
        data: {
            failCommands: ['find'],
            errorCode: 462,  // IngressRequestRateLimitExceeded
            errorLabels: ['SystemOverloadedError', 'RetryableError']
        }
    }

Perform a find operation with coll that fails.
Assert that the raised error contains both the RetryableError and SystemOverloadedError error labels.
Assert that the total number of started commands is maxAdaptiveRetries + 1 (2).

Test 5: Overload Errors with baseBackoffMS override base backoff

Drivers SHOULD test that overload errors with baseBackoffMS override the default backoff duration. This test MUST be executed against a MongoDB 9.0+ server that has enabled the configureFailPoint command with the errorLabels option.

Let client be a MongoClient.
Let coll be a collection.
Configure the random number generator used for exponential backoff jitter to always return a number as close as possible to 1.

Configure the following failPoint:

    {
        configureFailPoint: 'failCommand',
        mode: 'alwaysOn',
        data: {
            failCommands: ['insert'],
            errorCode: 462, // IngressRequestRateLimitExceeded
            errorLabels: ['SystemOverloadedError', 'RetryableError']
        }
    }

Insert the document { a: 1 }. Expect that the command errors. Measure the duration of the command execution.

   const start = performance.now();
   expect(
    await coll.insertOne({ a: 1 }).catch(e => e)
   ).to.be.an.instanceof(MongoServerError);
   const end = performance.now();

Run the following command to set up baseBackoffMS on overload errors.

client.admin.command("setParameter", 1, externalClientBaseBackoffMS=50)

Execute step 5 again.
Assert that the server attached baseBackoffMS to the error and that the driver parsed it.

Run the following command to disable baseBackoffMS on overload errors.

client.admin.command("setParameter", 1, externalClientBaseBackoffMS=0)

Assert absolute bounds on each run's duration.
```
assertGreaterEqual(exponential_backoff_time, 0.6)
assertGreaterEqual(with_base_backoff_ms_time, 0.3)
assertLess(with_base_backoff_ms_time, 0.6)
```
A run can never be faster than the sum of its backoffs. With jitter pinned to 1, the default backoffs are 0.2 + 0.4 = 0.6s and the baseBackoffMS=50 backoffs are 0.1 + 0.2 = 0.3s.