Designing a Low-Latency Pre-Trade Risk Check System for Trading

The Challenge of In-Path Risk Checks

Placing risk checks directly in the order path (the "hot path") introduces a latency tax on every order. The primary engineering goal is to minimize this overhead while ensuring the checks are robust and accurate. This involves carefully segmenting checks into those that *must* happen synchronously (e.g., position limits, margin, fat-finger bounds) and those that can run asynchronously or inform limits without blocking the order flow (e.g., portfolio analytics, reporting). The core question for in-path checks is: "May this order be sent right now?" Anything else is a design mistake if it blocks the order.

Hot Path vs. Off-Path Checks

• In the hot path (<1ms latency requirement): Position and exposure limits, margin, fat-finger bounds, instrument/account state, kill switch, duplicate/self-trade guards. These operate on current, in-memory state.
• Off the hot path (asynchronous/informational): Portfolio risk analytics, margin model re-rating, surveillance, reporting, reconciliation, credit review. These inform the limits that the hot path enforces but do not block individual orders.

Achieving Sub-Millisecond Latency with In-Memory State

To meet stringent latency requirements, the risk check system maintains all necessary state (positions, orders, margin, limits) directly in the memory of the processing service. This avoids network calls and database queries, which are orders of magnitude slower. The rationale isn't just speed but also correctness; a database represents past state, whereas the hot path needs the absolute current view, including unacknowledged orders.

• Single-Writer per Account: Accounts are sharded across risk instances, ensuring that an account's state is never contended and eliminating locks on the order path.
• Flat, Pre-allocated Structures: Fixed-size arrays indexed by account and instrument ID are used for lookups, avoiding costly hash lookups on strings.
• Zero Allocation/I/O on Path: No memory allocation, I/O, or blocking logging is permitted on the decision path. Audit records are handed off to another thread via a queue.
• Immutable Configuration Snapshots: Configuration changes are swapped in as whole immutable snapshots to prevent inconsistent reads during updates.

Incremental Recalculation for Scalability

Full recalculation of an account's exposure and margin on every order would be prohibitively expensive, especially for active traders. The hot path employs incremental recalculation, where incoming orders apply a delta to running aggregates (net/gross exposure, used margin). This ensures the work is proportional to the order, not the portfolio size.

• Reservations: On order send, the worst-case effect is reserved against aggregates to prevent over-commitment.
• Adjustments: Reservations are released on reject/cancel/expiry, or replaced by realized position changes on fills.
• Periodic Full Recalculation: A full recomputation runs off-path periodically or on parameter changes to verify incremental results and identify discrepancies. This provides a crucial self-check against state drift.

Robustness and Failure Handling

A kill switch is implemented as a separate, atomic path, independent of the main risk machinery to ensure it functions even if the primary system is compromised. It fails closed, stopping new orders if state validity cannot be established, and supports granular scope (firm, desk, account). For state recovery, all risk-altering events are journaled locally. On restart, the journal is replayed to rebuild aggregates, followed by reconciliation against external sources (venue drop copy) before trading resumes for an account. Mismatches lead to alerts, not silent resolution.