Design a System to View Latest Stock Prices Worldwide — System Design Interview Practice

Interview prompt

Design a globally distributed market-data service that normalizes exchange feeds, serves fresh quotes over APIs and streams, and clearly communicates market status and price freshness.

• Define exchange feed adapters, symbol identity, quote/trade sequencing, corrections, trading sessions, entitlements, and freshness contracts.
• Separate durable raw feeds and normalized events from hot latest-price projections, historical queries, and fan-out to subscribed clients.
• Handle feed gaps, duplicate/out-of-order ticks, market halts, regional failover, connection churn, and millions of concurrent subscriptions.
• Explain delayed-data labeling, replay, rate limits, auditability, observability, and degraded stale-quote behavior.

Requirements and scale assumptions

• Ingest authorized feeds from multiple exchanges, normalize quotes and trades, and expose latest prices, candles, and market status.
• Support REST and WebSocket subscriptions with symbol entitlements, sequence/freshness metadata, reconnect replay, and throttling.
• Persist raw and corrected events, rebuild projections after gaps, and audit feed transformations and customer access.
• Propagate an accepted feed update to subscribed clients within one second and serve latest-price reads with p95 under 100ms.
• Scale to millions of concurrent subscriptions and bursty market opens without a single hot key or unbounded synchronous work.
• Do not lose committed state; make retries and duplicate events safe.
• Degrade safely when downstream workers, caches, or external dependencies fail.
• 5M concurrent subscriptions and 100k normalized ticks per second
• Partition by the primary tenant, user, item, or geographic key and isolate hot partitions.
• Keep serving state bounded; retain raw events or durable records for replay and auditing.

• Peak scale: 5M subscriptions; 100k ticks/s — Capacity assumption that drives partitioning and backpressure.
• Latency target: fan-out < 1s; latest read p95 < 100ms — User-facing budget for the primary request or read path.
• Durable boundary: Committed before async — Raw exchange events and sequence-aware corrections are authoritative; latest quotes are derived projections.
• Async boundary: At-least-once workers — Keep WebSocket for real-time updates, Market data feed integration, Time-series database for history off the synchronous path.

Key entities

• SourcePartitionsourceId, partitionId, cursor, schemaVersion, watermark, status

  Replayable system to view latest stock prices worldwide source evidence and ingestion cursor.

• SchemaVersiondatasetId, version, compatibility, owner, effectiveAt, status

  Governed system to view latest stock prices worldwide contract used to validate producers and consumers.

• ProcessingRunrunId, inputWatermark, checkpoint, qualityStatus, codeVersion, status

  Checkpointed system to view latest stock prices worldwide processing attempt with quality and lineage metadata.

• AnalyticalDatasetdatasetId, partition, watermark, schemaVersion, qualityStatus, location

  Curated system to view latest stock prices worldwide serving partition with freshness and quality state.

Data flow

1. 1. Register sources and contractsThe system to view latest stock prices worldwide catalog records owners, schemas, compatibility rules, retention, lineage, and partitioning before data is accepted.
2. 2. Ingest with backpressureConnectors checkpoint system to view latest stock prices worldwide source cursors, validate schema and deduplication keys, and slow producers when downstream capacity is exhausted.
3. 3. Process event time with checkpointsStream or batch engines compute system to view latest stock prices worldwide transformations using watermarks, late-data policy, state checkpoints, and deterministic code versions.
4. 4. Publish quality-gated datasetsOnly system to view latest stock prices worldwide outputs that pass completeness, freshness, validity, and privacy checks become visible to analytical consumers.
5. 5. Serve, replay, and reconcileConsumers read bounded partitions with freshness metadata while operators replay failed system to view latest stock prices worldwide ranges and compare output checksums.

Deep dives and trade-offs

• Schema evolution and data qualityVersion system to view latest stock prices worldwide contracts and make compatibility rules explicit for every producer and consumer. Quarantine malformed partitions instead of poisoning the whole dataset. Track row counts, null rates, duplicates, distribution changes, and policy violations by partition.
• Watermarks, late data, and exactly-once effectsUse source cursors and event-time watermarks for system to view latest stock prices worldwide progress, not wall-clock assumptions. Make checkpoints, output keys, and sink commits retry-safe under at-least-once delivery. Document how late events revise windows, aggregates, or snapshots.
• Replay, lineage, and costKeep immutable system to view latest stock prices worldwide raw evidence and code or schema versions so failed outputs can be reproduced. Separate hot serving storage from cold retention and cap replay concurrency. Measure freshness, backlog, compute cost, storage growth, and quality-gate failure rate.
• Streaming versus batchUse streaming for freshness-critical system to view latest stock prices worldwide paths and batch for backfills, compaction, and expensive recomputation. Forcing every workload into streaming makes state, replay, and cost harder to operate.
• Raw retention versus curated-only storageRetain enough immutable raw evidence for replay, audit, and correction, then tier or expire it according to policy. Without raw evidence, a bad transformation can require an unreproducible emergency fix.
• Central warehouse versus domain-owned datasetsCentralize governance and discovery while letting domain owners own contracts and quality signals. A single team owning every transformation becomes a delivery bottleneck and hides data ownership.