Execution Reconciliation

September 23, 2026 ยท View on GitHub

Execution reconciliation aligns the venue's actual order and position state with the system's internal event-sourced state. Use this guide to understand startup state recovery and the continuous checks that detect runtime discrepancies.

Unresolved live command outcomes are one source of state divergence. For how Nautilus classifies local failures, definitive results, and unknown outcomes, see Command outcomes.

For the complete node lifecycle, see Live trading. For the available settings and recommended values, see Configure a live trading node.

Reconciliation model

Live execution reconciles local state against venue reports. Backtesting controls both order execution and the resulting state, so it does not need venue reconciliation.

Two scenarios:

  • Cached state exists: report data generates missing events to align the state.
  • No cached state: all orders and positions at the venue are generated from scratch.

:::info[Position reports are market exposure] An explicit position report is authoritative. During startup reconciliation, the engine either aligns to that report within reconciliation tolerances or fails closed.

Authoritative position reports:

  • An explicit open report, including quantity and direction.
  • An explicit flat report.

Not evidence of a flat position:

  • A missing report.
  • A null quantity.
  • A venue that does not publish positions.

The fill window does not decide whether the report is authoritative. Missing reports do not mean flat.

By default, generate_missing_orders is enabled. The engine generates the orders and fills needed to align local state to the report. Disabling generation does not allow an unresolved report through startup.

This guarantee covers reports included by the position-report and instrument filters, with reconciliation enabled. Unresolved reports prevent actors and strategies from starting. :::

:::tip Persist all execution events to the cache database. This reduces reliance on venue history and gives reconciliation the retained order and position state needed to interpret short history windows. :::

Component responsibilities

LiveNode owns the ExecutionManager and schedules recurring reconciliation. The manager tracks activity, retries, and fill identities, interprets cached state, and prepares reconciliation events. ExecutionEngine applies events to orders and positions and handles individual execution reports.

The UML diagram shows ownership and dependencies. A filled diamond denotes ownership; dashed arrows point from a caller to a component it uses. The kernel owns the engine and shared cache; it is omitted here to focus on reconciliation.

classDiagram
    direction LR

    namespace nautilus_live {
        class LiveNode
        class ExecutionManager
    }
    namespace nautilus_execution {
        class ExecutionEngine
    }
    namespace nautilus_common {
        class ExecutionClient {
            <<interface>>
        }
        class Cache
    }

    LiveNode *-- ExecutionManager : owns
    LiveNode ..> ExecutionClient : requests recurring reports
    LiveNode ..> ExecutionEngine : dispatches through kernel
    ExecutionManager ..> ExecutionClient : polls reports for standalone checks
    ExecutionManager ..> ExecutionEngine : applies startup events
    ExecutionManager ..> Cache : reads state and registers external orders
    ExecutionEngine ..> ExecutionClient : routes commands and requests reports
    ExecutionEngine ..> Cache : updates orders and positions

The live client facade shares one adapter instance between the node and engine. Pending report requests can retain client borrows while the event loop handles other work. Instrument updates are deferred until those borrows are released, then flushed on request completion or cancellation.

Within nautilus-live, the source modules divide these responsibilities as follows:

ModuleResponsibility
node/mod.rsNode lifecycle, event loop, and event dispatch.
node/reconciliation.rsRecurring report tasks, deadlines, cancellation, and result handling.
execution/manager.rsReconciliation state, decisions, and individual reconciliation checks.
execution/reconciliation.rsShared types, state-independent decisions, and targeted report requests.

The separate nautilus_execution::reconciliation module supplies report-to-event and arithmetic operations shared with the execution engine.

At startup, the manager publishes raw reports, applies order and fill events, verifies historical fill application, and then evaluates positions against the updated cache. During continuous position checks, the node coordinates authoritative fill queries and dispatch before asking the manager to generate synthetic events. Activity revisions detect local changes during requests or callbacks; applying authoritative fills defers synthetic reconciliation until a fresh position report.

The manager remains available without the node feature. Standalone callers can use its individual polling methods and apply the returned events themselves. Standalone position polling directly returns synthetic discrepancy events; the node adds the authoritative-fill recovery sequence.

Execution-client origins

An execution-client origin is a write-once binding between an order and the client responsible for its execution.

An origin is recorded:

  • From an explicit client on submission, or from the final client selected after routing and venue validation and before transport.
  • When non-synthetic external orders are materialized during startup reconciliation, from the reporting mass-status client.
  • When external orders are materialized from runtime venue reports and the report's account matches exactly one registered client that handles the instrument venue.

An origin may be absent for:

  • Cache data written before resolved origins were persisted.
  • External orders whose runtime report does not identify exactly one registered client by account and instrument venue.
  • Synthetic reconciliation orders.

The built-in cache backends enqueue a resolved origin for persistence before transport. Their writes remain asynchronous, so enqueue order does not guarantee that the origin is durable before the order reaches the client.

At startup, each client's mass status is checked against the cached origins: an order the client reports is expected to be bound to that same client. A missing origin logs an aggregated warning and remains compatible with existing cache data. A conflicting origin logs an aggregated deprecation warning and reconciles for compatibility. A future release rejects the conflict as a startup error. See the origin rows in Startup reconciliation.

This is separate from external order claims, which attribute venue-sourced orders to a strategy. The execution-client origin records which client an order belongs to.

Reconciliation reports

The execution engine consumes four reconciliation report variants from live adapters. Each variant has a different normal role when its matching order is absent from the cache. Explicitly bounded history can instead use order-only fill projection.

VariantPurposeMissing-order action
OrderStatusReportOrder state update.Creates an order and infers any reported fill.
FillReportStandalone fill.Creates a market order, then applies fill metadata.
OrderWithFillsOrder state plus fills.Creates an order, applies fills, and infers residue.
PositionStatusReportVenue position snapshot.Logs the report; positions remain fill-derived.

When to use each variant

Adapters choose the variant that matches the venue event:

  • Use OrderStatusReport for order lifecycle updates when fill details arrive on a separate stream.
  • Use FillReport for a venue-initiated closure that has a fill but no user-level order. Hyperliquid liquidations follow this pattern.
  • Use OrderWithFills when one venue event contains both an order status and its fills. Binance Futures uses this for exchange-generated ADL, liquidation, and settlement orders.

Snapshot freshness and fill corrections

A snapshot must not undo a fill that occurs after the state it describes. Receiving a snapshot after a stream event does not make the snapshot newer: REST requests and stream delivery can overlap during recovery. For example, a request can observe zero filled quantity, a stream can then deliver a fill of five units, and the older response can arrive last. Interpreting that response as a correction would wrongly void the five units.

Snapshot corrections require a distinction between:

  • A snapshot that predates a fill: its lower filled quantity does not establish that the fill was voided.
  • A snapshot that covers the fill and reports a reduction: the reduction can represent a genuine correction, derived from retained fill history.
  • An explicit venue fill-void event: process it under the OrderFillVoided contract, including its identity, quantity, and ordering checks. It does not depend on inferring a correction from a snapshot total.

Timestamp meaning matters when establishing coverage. The Derive adapter reports an order-update timestamp; Betfair's matchedDate describes the last match, not the time of a snapshot or correction. Response arrival time, equal timestamps, or timestamps from different clocks do not by themselves prove that a snapshot includes a fill.

Rejecting a genuine correction as stale can leave local filled quantity and exposure overstated until later reconciliation resolves the discrepancy. Conversely, a stale snapshot carrying a misleadingly newer timestamp can still cause a false void if freshness checks trust that timestamp.

The execution engine applies mass-status filled-quantity decreases to retained fills even when the snapshot contains no companion trades. It automatically skips an order snapshot when a cached fill or fill void has a local initialization timestamp at or after collection starts (ExecutionMassStatus.ts_init). This skips all changes from that order report, including status, quantity, and price updates. The engine still publishes the raw report and processes companion trades through normal deduplication. It does not queue the skipped snapshot. A later snapshot can apply a genuine correction once collection starts after the cached fill activity. This requires no configuration and does not suppress explicit fill-void events.

This protection applies to runtime mass-status handling in ExecutionEngine. Startup reconciliation uses ExecutionManager, which does not apply this timestamp boundary. For runtime protection, adapters must capture the mass-status timestamp before collecting reports, using the same local clock as fill events. This boundary protects against overlapping local activity; it cannot detect venue state that is already stale when collection starts.

Order-only fill projection

During startup reconciliation, a bounded historical fill does not change a position or the portfolio when that instrument has no in-scope explicit position report. Omission is not a flat report. The order still reaches the reported status and filled quantity. An explicit position report, open or flat, is the quantity target: the engine applies the available fills, generates the difference when configured, or leaves the position unresolved.

This projection applies only to reconciliation recovery. Raw reports remain available. Setting filter_position_reports makes bounded historical fills order-only, even when the mass status contains position reports. See Bounded history safety for the undeclared-window exception and retained-fill rules.

External order creation

When a report references an order that is absent from the cache, the engine creates an external order. This covers venue-initiated ADL, liquidation, or settlement, orders placed by another process, and orders not yet observed locally.

The naming distinguishes configuration intent from live ownership state:

  • external_order_instrument_ids is the serializable strategy configuration intent. It names the instruments whose external orders should be assigned to the strategy when it is registered.
  • An external order claim is an active cache entry that maps one InstrumentId to one StrategyId. The code uses external_order_claims for the collection of these live entries.

Live strategy registration materializes the configured instrument IDs with register_external_order_claims. This operation is additive and strict: it rejects a repeated instrument or any instrument that already has a claim, including a claim for the same strategy.

The strategy method set_external_order_instrument_ids(...) delegates to the cache operation set_external_order_claims. This operation treats its input as the strategy's complete desired active set. It can retain or release that strategy's existing claims and acquire unclaimed instruments, but it cannot take a claim from another strategy. Validation covers the complete input before changing the cache, so a conflict leaves every existing claim unchanged.

The ExecutionManager and ExecutionEngine read the same canonical claim map from the cache when they process external reports. They assign an external order to:

  • The strategy identified by the active claim for the report's instrument.
  • The EXTERNAL strategy as a default fallback.

An active-claim update is therefore visible to both components without a coordination message. The claim present when an external order is created determines the assignment. Existing cached orders keep their assigned StrategyId; changing a claim does not reassign them.

Transferring an instrument between strategies requires the current owner to release it before the new owner claims it. There is no atomic handoff across strategies. A report processed between the release and acquisition has no active claim and is assigned to EXTERNAL. Cache resets preserve active claims so registered routing remains configured, while retiring a strategy clears its claims.

The external order uses the report's client_order_id when present and otherwise derives one from the venue_order_id. The engine adds the order to the cache, registers its venue order ID, and emits the applicable OrderAccepted, OrderFilled, OrderCanceled, or OrderExpired events. Positions then update through the normal event pipeline.

See Claiming external orders for strategy configuration and runtime updates.

Reducing external positions

A strategy can use reduce-only fills to reduce inherited EXTERNAL inventory under NETTING.

Position selection

Existing cached position links remain authoritative. Without a cached link, a reduce-only fill uses the strategy's own open position when available. If that position is absent or closed, the engine looks for positions that meet all of these conditions:

  • Belong to EXTERNAL and use NETTING.
  • Are open on the opposite side of the fill.
  • Match the fill's instrument and account.

The engine selects a fallback only when exactly one position matches. The fill quantity must not exceed that position's quantity, though the order's remaining quantity can be larger. If no safe fallback exists, an otherwise valid fill updates the order but neither opens nor updates a position.

Ownership and events

After a successful reduction, the engine links the order to the external position so subsequent fills use the same target. The position retains EXTERNAL ownership:

  • OrderFilled keeps the reducing strategy's ID and identifies the external position.
  • PositionChanged and PositionClosed use the EXTERNAL strategy's event topic.

Linked reduction checks

When applying position economics, each linked reduction must match the external position's account and reduce its open quantity without flipping or reopening it. If a fill violates these checks, the engine rejects it before changing the order or position.

Order-only fill projection bypasses these reduction checks because it repairs order history without changing the position.

Reconciliation configuration

Unless reconciliation is set to false, the live node runs startup reconciliation for each execution client. The reconciliation_lookback_mins parameter controls how far back it requests history through the execution engine. Startup enablement and polling intervals belong to the node's LiveExecutionEngineConfig; the manager receives the thresholds, retry limits, filters, and lookbacks used to make reconciliation decisions.

:::tip Leave reconciliation_lookback_mins unset to use the adapter's documented default. Many adapters request the maximum execution history the venue provides, while others use a bounded default to match venue retention and request limits. See the integration guide for the selected venue. :::

:::warning A bounded history window can begin after the fill that opened a position. Some venues also filter or drop older execution data. That does not change the position-report rule above: an explicit report is still the quantity target. :::

Each strategy can configure external_order_instrument_ids as its intent to claim venue-sourced external orders and materialized reconciliation activity for specific instruments. Live strategy registration materializes that intent as active claims, which the strategy can replace at runtime. This lets a strategy resume managing open orders and positions when no cached state exists.

Unclaimed external orders use strategy ID EXTERNAL with tag VENUE. Unclaimed orders generated during position reconciliation use strategy ID EXTERNAL with tag RECONCILIATION. Claimed orders and fills use the claiming strategy ID and have no external/reconciliation tag, so the strategy can continue managing the recovered state.

:::tip To detect unclaimed external orders in your strategy, check order.strategy_id.value == "EXTERNAL". Ownership does not exclude these orders from position tracking or portfolio calculations. Historical fills still follow the bounded history safety rules when applicable. :::

For all live trading options, see the LiveExecutionEngineConfig API reference.

Instrument availability

Adapters parse reconciliation reports using the instrument, so every instrument a report references must already be loaded. Adapters do not fetch missing instruments from the venue during reconciliation.

Instrument scope comes from the adapter's provider config rather than the engine. InstrumentProviderConfig.load_ids decides which instruments the adapter holds, while reconciliation_instrument_ids filters reports only after the adapter has produced them.

Reports for instruments outside an explicit load_ids scope are expected: they are dropped at debug level, so a node scoped to one instrument stays quiet about the rest of the venue. An in-scope instrument that does not resolve means something is wrong, whether it was named in load_ids or covered by load_all=True, and the outcome depends on what the report describes:

  • An open order or position report fails reconciliation, so the system does not start. A live position that cannot be priced is never silently dropped.
  • A closed or historical record logs a warning instead of aborting startup. When the adapter declares a bounded history, the record also marks the report set incomplete, applying the bounded history safety rules. Expiries routinely retire instruments that older fills still reference.

Reconciliation procedure

All adapter execution clients follow the same reconciliation procedure, calling three methods to produce an execution mass status:

  • generate_order_status_reports
  • generate_fill_reports
  • generate_position_status_reports
flowchart TD
    Start[Startup Reconciliation] --> Fetch[Fetch venue reports<br/>orders, fills, positions]
    Fetch --> Dedup[Deduplicate reports<br/>log warnings for duplicates]
    Dedup --> Orders[Order Reconciliation<br/>align order states, generate missing events]
    Orders --> Fills[Fill Reconciliation<br/>verify fills, generate missing OrderFilled events]
    Fills --> Pos[Position Reconciliation<br/>compare net positions per instrument]
    Pos --> Match{Positions<br/>match venue?}
    Match -->|Yes| Done[Reconciliation complete<br/>system ready for trading]
    Match -->|No| Gen[Generate missing orders<br/>strategy: EXTERNAL, tag: RECONCILIATION]
    Gen --> Recovered{In-scope reports match<br/>within tolerances?}
    Recovered -->|Yes| Done
    Recovered -->|No| Abort[Startup fails<br/>actors and strategies do not start]

These reports represent external reality. The procedure processes them in the order shown so each position check builds on reconciled order and fill state.

Mass-status history contract

An ExecutionMassStatus can declare the provenance of its historical reports:

  • lookback_start=None means that the adapter has not declared an explicit lower time bound.
  • lookback_start=Some(timestamp) means that historical order and fill reports exclude venue activity before that timestamp.
  • reports_complete=true means that every order, fill, and position source needed to interpret the bounded history completed and all required records were mapped successfully.

An adapter can still return authoritative active orders and position reports when a historical source fails. It marks the mass status incomplete to record the missing history. Incompleteness does not prevent recovery from an explicit position report.

Report deduplication

  • Deduplicates order reports within the batch and logs warnings.
  • Logs duplicate trade IDs as warnings for investigation.

Order reconciliation

  • Generates and applies events to move orders from cached state to current state.
  • Generates external order events for unrecognized client order IDs or reports missing a client order ID.

Fill reconciliation

  • Infers OrderFilled events for missing trade reports.
  • Verifies fill report data consistency with tolerance-based price and commission comparisons.

Position reconciliation

  • Matches the net position per account and instrument against venue position reports using the account's quantity tolerance.
  • Generates external order events when order reconciliation leaves a position that differs from the venue.
  • When generate_missing_orders is enabled (default: True), generates orders with strategy ID EXTERNAL and tag RECONCILIATION to align discrepancies.
  • Logs a warning when NETTING ownership is split across multiple strategies for the same account and instrument, since venue position reports are account-level net positions.

Reconciliation generates synthetic MARKET order reports and fills with a known price:

  • Opening from flat uses the reported avg_px_open.
  • Increasing an existing position uses a calculated price targeting the reported entry average.
  • Reducing an existing position uses the reported entry average, falling back to the cached average when the report omits it. A reduction does not change the remaining position's entry average.
  • Closing to flat uses the cached entry average.
  • Reversing direction closes the cached position, then opens the reported position at its entry average.

The engine skips quantity differences that round to zero at instrument size precision. Startup validation still checks the remaining difference against the account's quantity tolerance.

Startup position validation

After applying startup reports, the live node checks each in-scope explicit position report, including flat reports, against the cache. An unresolved position stops startup before actor or strategy on_start. The error identifies the account, instrument, venue quantity, and recovery failure.

Position caseCache identityQuantity requirement
HEDGING with venue position IDsExact venue position IDExact quantity
NETTING with or without venue position IDsAccount and instrumentWithin account tolerance
Net reports without position IDsAccount and instrumentWithin account tolerance

When the venue reports both long and short positions, both side totals must also match; equal net quantities alone are insufficient. A residual difference outside the account tolerance remains unresolved even if it rounds to zero at the instrument size precision. Differences within the tolerance remain acceptable, including tiny residuals around zero.

For an open position with a reported avg_px_open, startup also checks the entry average using the fill-adjustment relative tolerance of 0.01%. Average entry prices can fall between instrument price ticks; the comparison does not round them to instrument price precision. NETTING reports use quantity-weighted entry averages for each reported side. If a side spans several reports, all contributing reports must supply an average to establish that side's price target.

Matching quantity alone does not resolve a reported entry-price mismatch. When quantities already match, position reconciliation does not generate a correction solely to change the entry average; the remaining price mismatch fails startup. When it corrects quantity, startup still fails if the resulting average remains outside tolerance. Synthetic recovery does not establish historical realized PnL.

Recovery prerequisites and filters

Creating a position in an empty cache from a report without order or fill history requires avg_px_open. The engine does not invent an entry price.

  • Missing-order generation: disabling generate_missing_orders does not bypass startup validation.
  • Report scope: position-report and instrument filters still apply; excluded reports are not checked.
  • Reconciliation disabled: disabling reconciliation skips the check entirely.

Fill adjustment

The engine can analyze zero-crossings, remove closed lifecycles, and generate a synthetic fill when the reported fills do not explain the current venue position. A declared lookback_start does not skip this adjustment. Without a reported avg_px_open, the engine preserves the original orders and fills instead of inventing a price for synthetic fill adjustment. Setting filter_position_reports skips fill adjustment for both declared and undeclared windows.

When generate_missing_orders is disabled, the engine still processes raw venue order reports. It filters completed lifecycles when the current lifecycle explains the venue position, but it does not add or replace synthetic reports to align a fill window with the venue position or materialize an order for a fill group that has no order report.

Adapters that apply a history cutoff should still declare it through the mass-status history contract. The declaration records provenance. It does not authorize leaving an explicit position report unmatched.

Bounded history safety

An explicit position report is authoritative for the position, whether open or flat. The engine either aligns to that report within reconciliation tolerances or fails closed. It does not replay the bounded window to decide whether the report is authoritative. It applies the available orders and fills, and by default generates the orders and fills required to reach the report. Disabling generate_missing_orders does not allow a mismatch to stand.

A bounded fill for an instrument with no in-scope explicit position report does not open, close, or change a position, and it does not update portfolio economics. The order still reaches the reported status and filled quantity. Raw reconciliation reports remain available.

For compatibility, a mass status without a declared lookback_start can still apply historical fills when there is no position report. This exception does not treat a missing report as flat. Retained-fill deduplication and projection of older lifecycles continue to apply in both paths.

Reports with an explicit venue_position_id follow the position-specific reconciliation path and do not require NETTING lifecycle inference.

Failure handling

  • An adapter can preserve successful report legs after an individual source failure. Explicitly bounded mass statuses must mark the result incomplete. Incompleteness does not veto an explicit position report. A bounded fill with no explicit position report does not change a position.
  • Fill reports arriving before order status reports are deferred until order state is available.

Commission failures

An adapter fill commission that cannot be calculated or represented fails the report request under the adapter contract. The adapter does not drop that fill or replace its commission with zero or a generic formula. Startup stops before applying that client's mass status.

When the engine asks the responsible execution client to calculate an inferred-fill commission, a failure defers the inferred quantity and dependent terminal transition until a later reconciliation cycle succeeds. Valid explicit fills from the same report set can still apply. For an external order, the engine resolves the commission before adding the order to the cache or publishing its initial event, so a failure defers the entire external order. An unavailable responsible execution client has the same fail-closed result.

An inferred-fill commission failure while applying an otherwise successful mass status does not by itself stop startup. Startup still fails if an in-scope explicit position report remains unresolved. Otherwise, the unresolved work remains pending for a later reconciliation cycle.

If startup reconciliation fails for any other reason, the system logs an error and does not start.

Common reconciliation scenarios

The tables below cover startup reconciliation (mass status) and runtime checks (in-flight order checks, open-order polls, own-books audits).

Startup reconciliation

ScenarioDescriptionSystem behavior
Order state discrepancyLocal state differs from venue (e.g., local SUBMITTED, venue REJECTED).Updates local order to match venue state, emits missing events.
Missed fillsComplete venue history contains a fill the engine missed.Generates the missing OrderFilled event and applies its economics.
Multiple fillsA complete, coherent report set contains several fills for an order.Reconstructs the reported fill history in event order.
Incomplete bounded historyA required order, fill, or position source failed or could not be mapped.Aligns an explicit position report, or leaves it unresolved. Fills with no report stay order-only.
Ambiguous bounded lifecycleThe bounded reports do not prove one coherent NETTING position transition.An explicit position report is still the quantity target. Fills with no report stay order-only.
External ordersOrders exist on venue but not in local cache.Creates unclaimed orders with strategy ID EXTERNAL and tag VENUE.
Missing client originA cached order in the mass status has no recorded execution-client origin.Logs one aggregated warning with a count and sample IDs; reconciles against the reporting client.
Conflicting client originA cached order's origin differs from the client that supplied the report.Logs one aggregated deprecation warning; reconciliation proceeds during the compatibility period.
Partially filled then canceledOrder partially filled then canceled by venue.Updates state to CANCELED, preserves fill history.
Different fill dataVenue reports different fill price/commission than cached.Preserves cached data, logs discrepancies.
Filtered ordersOrders marked for filtering via config.Skips based on filtered_client_order_ids or instrument filters.
Unresolved instrumentA report references an in-scope instrument the adapter has not loaded.Fails startup for open order and position reports; warns and marks bounded history incomplete otherwise.
Fill commission failureAn adapter cannot represent a required fill commission while building reports.Fails mass-status generation and stops startup before applying that client's reports.
Inferred-fill commission failureThe responsible execution client cannot calculate a required commission.Defers inferred work; an external order remains absent, while valid explicit fills can still apply.
Duplicate order reportsMultiple orders share the same identifier.Deduplicates with warning logged.
Position quantity mismatch (long)Internal long position differs from venue (e.g., 100 vs 150).Generates BUY LIMIT with calculated price when generate_missing_orders=True.
Position quantity mismatch (short)Internal short position differs from venue (e.g., -100 vs -150).Generates SELL LIMIT with calculated price when generate_missing_orders=True.
Position reductionVenue position smaller than internal (e.g., internal 150 long, venue 100 long).Generates opposite-side LIMIT order with calculated price.
Position side flipInternal position opposite of venue (e.g., internal 100 long, venue 50 short).Generates LIMIT order to close internal and open external position.
Internal reconciliation ordersOrders generated to align position discrepancies.Uses a claim when configured; otherwise EXTERNAL + RECONCILIATION.

Runtime checks

Continuous reconciliation starts after startup reconciliation completes. It:

  • Monitors in-flight orders for delays exceeding a configured threshold.
  • Reconciles open orders with the venue at configured intervals.
  • Checks position status with the venue at configured intervals.
  • Audits internal own order books against the venue's public books.

The loop waits for startup reconciliation to finish before starting periodic checks. The reconciliation_startup_delay_secs parameter adds a further delay after startup reconciliation completes, giving the system time to stabilize.

ScenarioDescriptionSystem behavior
In-flight submit timeoutSUBMITTED remains unconfirmed beyond retry exhaustion.Resolves to REJECTED with INFLIGHT_TIMEOUT.
In-flight cancel/update timeoutPENDING_CANCEL or PENDING_UPDATE exceeds the retries.Resolves to CANCELED through reconciliation.
Open orders check discrepancyPeriodic poll detects a venue state change.Confirms status and applies transitions.
Position check discrepancyPeriodic poll detects a position mismatch.Generates reconciliation events when eligible.
Commission construction failureA required fill commission cannot be represented.Defers the affected work to a later cycle.
Own books audit mismatchOwn order books diverge from venue public books.Audits and logs inconsistencies.

The in-flight checker produces the submit and cancel/update timeout results after exhausting the configured retries. Terminal reconciliation provenance distinguishes these local policy resolutions from venue-reported outcomes.

A missing open-order report does not by itself prove a pending modify or cancel outcome, so the consistency checks below leave those states unresolved until another check can determine the venue state.

Order consistency checks (when cache state differs from venue state):

:::info[Full-history checks] The Not found rows apply only in full-history mode (open_check_open_only=False); open-only mode is the default. :::

Cache statusVenue statusResolutionRationale
SUBMITTEDNot foundREJECTEDOrder never confirmed by venue (e.g., lost during network error).
ACCEPTEDNot foundREJECTEDOrder doesn't exist at venue, likely was never successfully placed.
ACCEPTEDCANCELEDCANCELEDVenue canceled the order (user action or venue-initiated).
ACCEPTEDEXPIREDEXPIREDOrder reached GTD expiration at venue.
ACCEPTEDREJECTEDREJECTEDVenue rejected after initial acceptance (rare but possible).
PENDING_UPDATENot foundUnresolvedModification outcome remains unknown.
PENDING_CANCELNot foundUnresolvedCancellation outcome remains unknown.
PARTIALLY_FILLEDCANCELEDCANCELEDOrder canceled at venue with fills preserved.
PARTIALLY_FILLEDNot foundCANCELEDOrder doesn't exist but had fills (reconciles fill history).

Runtime reconciliation caveats:

  • Open-only mode: venue "open orders" endpoints exclude closed orders by design, making it impossible to distinguish missing orders from recently closed ones. Pending cancel/update orders remain unresolved when a missing-order check cannot prove the final venue state.
  • Recent order protection: the engine skips reconciliation for orders whose last event falls within the open_check_threshold_ms window. This prevents false positives from race conditions where the venue is still processing.
  • Targeted query safeguard: before applying a terminal "not found" resolution, the engine issues a single-order query to the venue. This catches false negatives from bulk query limitations or timing delays.
  • Position report failures: if a venue position query fails, the engine skips cached positions for that venue during the cycle instead of treating missing reports as flat.
  • Completed orders: FILLED orders that are "not found" at the venue are silently ignored. Venues commonly drop completed orders from their query results.

Retry coordination. The in-flight loop increments its own per-order retry count against inflight_check_retries and mirrors that value into missing-order tracking. The open-order loop increments the missing-order count against open_check_missing_retries. Each loop applies its own limit; neither setting automatically overrides the other.

When the open-order loop exhausts retries, the engine issues one targeted GenerateOrderStatusReport probe before applying a terminal state or leaving an ambiguous pending cancel/update unresolved. If the venue returns the order, reconciliation proceeds and missing-order tracking clears. If a pending state remains unresolved, the engine also resets the in-flight count before checking again after the configured threshold.

Position checks use separate retry counters per instrument and account. A successful position match clears the counter, while repeated unresolved discrepancies stop active reconciliation for that pair until the discrepancy clears.

Single-order query throttling. The engine caps single-order queries per cycle via max_single_order_queries_per_cycle. Remaining orders are deferred to the next cycle. single_order_query_delay_ms spaces out consecutive queries to avoid rate limits. This handles bulk query failures across hundreds of orders without overwhelming the venue API.

Common reconciliation issues

  • Missing trade reports: Some venues filter out older trades. Increase reconciliation_lookback_mins or persist all events locally. Explicitly bounded adapters mark incomplete history so unsupported fills do not change positions or portfolio economics.
  • Position mismatches: External orders that predate the lookback window cause position drift. Increase the window, restore retained state, or let an authoritative position report reconcile the current quantity. Flatten the account only as a deliberate operational recovery step.
  • Split NETTING ownership: Multiple strategies can hold cached positions for the same account and instrument, but venues report a single account-level net position. Prefer one claiming strategy per NETTING account/instrument pair when resuming external state.
  • Duplicate order IDs: Deduplicated with warnings logged. Frequent duplicates may indicate venue data integrity issues.
  • Unresolved instruments: A report references an instrument the adapter never loaded. Add it to load_ids or set load_all=True. Reports outside an explicit load_ids scope are dropped by design and need no action.
  • Precision differences: Reconciliation tolerances absorb small quantity and entry-price differences. Large discrepancies may indicate missing orders.
  • Out-of-order reports: Fill reports arriving before order status reports are deferred until order state is available.

:::tip For persistent issues, inspect the venue reports and cached ownership before dropping state or flattening an account. :::

Reconciliation invariants

The reconciliation path preserves these invariants for the reports and positions it processes:

  1. Order state: authoritative reports recover the exact order status and filled quantity even when historical fills apply only to order state.
  2. Explicit position report: an explicit open or flat report is the quantity target. The engine aligns to it, generating orders and fills when configured, or leaves it unresolved. A missing report is not flat.
  3. Missing position report: an explicitly bounded historical fill with no in-scope position report updates order state only, without changing positions or portfolio economics.
  4. Position quantity: reconciled positions match authoritative venue reports within the applicable quantity tolerance.
  5. Entry price: reported entry averages match within relative tolerance before startup proceeds. Synthetic fills use reported or calculated prices; they do not reconstruct missing historical PnL.
  6. ID determinism: synthetic trade_id and venue_order_id values are deterministic functions of the logical event, so replay deduplicates them across restarts.

Bounded history without a position report recovers the order record and leaves historical economics unapplied. An explicit open or flat report is aligned within tolerance or remains unresolved.

Fill adjustment scenarios

These scenarios apply whether or not the mass status declares a lookback_start:

ScenarioDescriptionSystem behavior
Complete lifecycleAll fills from opening to current state are captured.No adjustment.
Incomplete single lifecycleReports miss opening fills, with no zero-crossings.Adds a synthetic opening fill with calculated price.
Multiple lifecycles, current matchesZero-crossings separate earlier and current lifecycles.Filters out old lifecycles and retains the current one.
Multiple lifecycles, current mismatchThe current lifecycle differs from the venue position.Generates a synthetic position fill and preserves reported orders for replay deduplication, projecting their earlier fills onto order state only.
Flat positionThe venue reports flat regardless of fill history.Makes no adjustment.
No fillsThe report set contains no fills.Returns the empty fill set.

Concepts:

  • Zero-crossing: position quantity crosses through zero (FLAT), marking a lifecycle boundary.
  • Lifecycle: a sequence of fills between zero-crossings representing one open-close cycle.
  • Synthetic fill: a calculated fill report representing missing activity, priced to achieve the correct average position.
  • Tolerance: fill adjustment uses a relative entry-price tolerance of 0.0001 (0.01%) to absorb minor calculation differences. Startup validation uses the same price tolerance and the account's separate quantity tolerance.

Bounded history scenarios

Position reportHistorical reportsSystem behavior
Explicit openComplete, incomplete, or ambiguous bounded history.Applies available reports and attempts recovery to the reported quantity and entry average; unresolved differences block startup.
Explicit flatComplete, incomplete, or ambiguous bounded history.Applies available reports and closes residual exposure when generation is enabled; unresolved quantity differences block startup.
MissingBounded history, with or without a cached predecessor.Recovers order state only; fills do not change positions or portfolio economics.
FilteredPosition-report or instrument filters exclude the report.Position-report filtering makes bounded fills order-only; instrument filtering excludes both orders and positions.