Design a Reinforcement Learning Trading System — System Design Interview Practice

Interview prompt

Design a research-to-execution platform for reinforcement-learning trading agents that uses realistic backtests, strict risk controls, paper trading, and low-latency order execution.

• Define immutable market-data versions, point-in-time features, action/reward contracts, experiment lineage, and reproducible agent artifacts.
• Model fees, slippage, latency, liquidity, partial fills, limits, market regimes, and leakage-resistant train/validation/test splits.
• Separate offline training and simulation from a guarded paper/live execution path with idempotent orders and kill switches.
• Explain risk limits, policy approval, drift, rollback, broker failure, auditability, observability, and safe degradation.

Requirements and scale assumptions

• Ingest and validate market data, build point-in-time datasets, train agents, run reproducible backtests, and compare risk-adjusted results.
• Support paper trading and approved live execution with portfolio state, pre-trade risk checks, order status reconciliation, and kill switches.
• Version policies and data, audit every decision/order, detect regime changes, and recover safely from broker or market-data outages.
• Keep execution decisions within the configured trading latency budget and require statistically robust, leakage-free evaluation before promotion.
• Scale parallel training and backtests across many instruments without unbounded synchronous work or unsafe live rollout.
• Do not lose committed state; make retries and duplicate events safe.
• Degrade safely when downstream workers, caches, or external dependencies fail.
• Years of tick or bar data across thousands of instruments
• Partition by the primary tenant, user, item, or geographic key and isolate hot partitions.
• Keep serving state bounded; retain raw events or durable records for replay and auditing.

• Peak scale: 1k instruments; 10TB historical data — Capacity assumption that drives partitioning and backpressure.
• Latency target: p99 decision < 20ms; 100% orders risk-checked — User-facing budget for the primary request or read path.
• Durable boundary: Committed before async — Versioned market data, policies, and broker order records are authoritative and auditable.
• Async boundary: At-least-once workers — Keep Use PPO or SAC algorithms for continuous action spaces, Implement realistic market simulators with slippage and fees, Use multi-agent RL for portfolio optimization off the synchronous path.

Key entities

• MarketSnapshotinstrumentId, eventTime, sequence, price, volume, source

  Replayable market evidence used by reinforcement learning trading system.

• StrategyVersionstrategyId, version, artifactUri, features, riskPolicy, status

  Versioned strategy or policy with reproducible inputs.

• OrderIntentintentId, accountId, instrumentId, side, quantity, status, idempotencyKey

  Risk-checked reinforcement learning trading system order intent.

• PositionLedgeraccountId, instrumentId, quantity, averagePrice, realizedPnl, version

  Reconciled position and PnL state.

Data flow

1. 1. Ingest timestamped market evidenceCollectors normalize reinforcement learning trading system market events, preserve sequence gaps, and expose data quality before decisions use them.
2. 2. Evaluate a versioned strategyThe strategy service joins point-in-time features and policy versions without allowing future data leakage.
3. 3. Enforce risk before routingPre-trade risk checks exposure, limits, liquidity, and kill-switch state before creating an order intent.
4. 4. Route and reconcile fillsVenue adapters normalize acknowledgements and fills; the order manager updates positions idempotently.
5. 5. Simulate and monitor safelyBacktests use immutable evidence and remain isolated from live effects while operations tracks feed, risk, and position drift.

Deep dives and trade-offs

• No future data and reproducibilityPin market sequence, feature window, strategy artifact, and policy for each decision. Separate simulation clocks and datasets from live order routing. Record the exact evidence and model version behind every order intent.
• Unknown execution outcomesUse venue client order IDs and query-after-timeout before retrying a reinforcement learning trading system order. Treat partial fills, rejects, cancels, and out-of-order events as explicit state transitions. Reconcile positions from fills and account statements rather than trusting one callback.
• Risk and kill switchesEvaluate account, instrument, and global limits on the hot path with bounded latency. Make kill switches versioned, scoped, expiring, and independently observable. Alert on feed gaps, stale features, order latency, exposure, PnL, and reconciliation drift.
• Model sophistication versus controlKeep a simple auditable baseline and add complex policy only behind simulation, canary, and kill-switch controls. A more accurate model is not safe if its input, rollout, or failure behavior is opaque.
• Low latency versus durable auditUse an in-memory hot path backed by an append-only decision and execution log. Dropping audit events to shave latency makes incident reconstruction and reconciliation impossible.
• Single venue versus routingStart with one venue adapter and add routing once failure and reconciliation semantics are proven. Multi-venue routing multiplies duplicate, partial-fill, and fee edge cases.