System Design Interview: A Real-Time Stock Price Ticker and Market Data Feed
4 views
"Design a system that shows live stock prices to millions of users" is a popular question for fintech and brokerage companies. It tests fan-out at scale, push protocols, and handling bursty data. Here is a structured approach.
1. Requirements
Functional: users see live prices for a watchlist of symbols, price charts for intraday history, and alerts when a price crosses a threshold.
Non-functional: price updates should reach users within a few hundred milliseconds; during market open, update rates can spike to hundreds of thousands of messages per second across all symbols; millions of concurrent connections; correctness of the latest price matters more than delivering every intermediate tick.
2. Data source
Prices come from exchange feeds or a market data vendor, usually as a stream of trades and quotes. An ingestion service normalizes these into a common format: symbol, price, size, timestamp, sequence number.
3. Core pipeline
- Ingestion publishes normalized ticks to a partitioned log such as Kafka, partitioned by symbol so ordering per symbol is preserved.
- A conflation service keeps the latest price per symbol and emits snapshots at a fixed interval (for example every 100 to 250 ms). Retail users don't need every tick; conflation cuts traffic drastically.
- A storage consumer writes ticks or one-second bars into a time-series database for charts.
4. Pushing to clients
Use WebSockets (or Server-Sent Events) from a fleet of stateless connection gateways. Each gateway holds thousands of connections and keeps a map of symbol to subscribed connections. When a client changes its watchlist, the gateway updates its subscriptions.
Fan-out: gateways subscribe to symbol updates through a pub/sub layer (Redis pub/sub, NATS, or Kafka consumers). Popular symbols are subscribed by every gateway; obscure ones by few. This two-level fan-out (pub/sub to gateways, gateways to clients) keeps the work balanced.
5. Initial state and reconnects
When a client connects, it first fetches a snapshot of its watchlist via a REST call served from a cache of latest prices, then applies streaming updates. Sequence numbers let clients detect gaps and request a fresh snapshot after reconnecting.
6. Price alerts
An alert service consumes the conflated stream, indexes alerts per symbol sorted by threshold, and checks only the alerts crossed by the new price. Triggered alerts go to a notification queue for push or email.
7. Scaling and failure
Scale gateways horizontally behind a load balancer that supports long-lived connections. If a gateway dies, clients reconnect to another and resync with the snapshot. Monitor end-to-end latency from exchange timestamp to client receipt.
8. Trade-offs to discuss
Conflation interval versus freshness, WebSockets versus SSE, and whether professional users need a separate unconflated tier.
Staying organized in the round
In a design round, it is easy to forget the reconnect or snapshot story. Linkjob AI (https://linkjob.ai) offers real-time help during live job interviews and technical interviews, reminding you of areas you haven't covered while you drive the discussion yourself.