Real-time Chat System — System Design Interview Practice

Interview prompt

Design one-to-one and group messaging with delivery, ordering, presence, and history so users can send a message to a conversation reliably at scale.

• Define the source of truth for message log and conversation membership and make retries idempotent.
• Use bounded, partitioned state to meet 100M daily users and 1M messages per second and message acknowledgement p95 <=150ms.
• Separate the critical request path from fanout, push notifications, search, and media.
• Explain consistency, failure recovery, authorization, observability, and a degraded mode.

Requirements and scale assumptions

• Support the core workflow to send a message to a conversation.
• Expose status, results, and freshness appropriate to one-to-one and group messaging with delivery, ordering, presence, and history.
• Support authorization, validation, updates, deletion, and recovery semantics.
• Meet message acknowledgement p95 <=150ms under normal load.
• Scale to 100M daily users and 1M messages per second without a single hot key or unbounded synchronous work.
• Do not lose committed state; make retries and duplicate events safe.
• Degrade safely when downstream workers, caches, or external dependencies fail.
• 100M daily users and 1M messages per second
• Partition by the primary tenant, user, item, or geographic key and isolate hot partitions.
• Keep serving state bounded; retain raw events or durable records for replay and auditing.

• Peak scale: 100M daily users — Capacity assumption that drives partitioning and backpressure.
• Latency target: message acknowledgement p95 <=150ms — User-facing budget for the primary request or read path.
• Durable boundary: Committed before async — The source of truth is message log and conversation membership.
• Async boundary: At-least-once workers — Keep fanout, push notifications, search, and media off the synchronous path.

Key entities

• InteractioninteractionId, actorId, objectId, type, version, occurredAt

  Canonical real time chat system interaction with an idempotency key and ordering version.

• ConnectionSessionsessionId, userId, deviceId, roomKey, lastHeartbeat, status

  Ephemeral but observable real time chat system connection registration used for routing and presence.

• FanoutCursorstreamKey, shard, offset, consumerGroup, updatedAt

  Durable progress marker for real time chat system fan-out and replay.

• DeliveryReceiptinteractionId, recipientId, channel, attempt, status, deliveredAt

  Deduplicated real time chat system delivery state for reconnects, retries, or acknowledgements.

Data flow

1. 1. Accept and commit the interactionThe real time chat system gateway authenticates the actor, validates room or object membership, applies rate limits, and conditionally commits the interaction.
2. 2. Publish an ordered eventAn outbox emits the committed real time chat system transition with an event ID, partition key, sequence, and replay retention.
3. 3. Fan out by partitionConsumers route real time chat system events to connected recipients, durable inboxes, or notification channels without making the origin write wait for every recipient.
4. 4. Resume and reconcile connectionsClients reconnect with a cursor; the real time chat system service replays missed events, deduplicates delivery, and exposes stale or degraded state.
5. 5. Measure latency and recoverOperations tracks real time chat system publish-to-deliver latency, hot partitions, reconnect storms, dropped events, and consumer lag for replay or repair.

Deep dives and trade-offs

• Ordering, idempotency, and hot keysChoose a real time chat system partition key that preserves required order while distributing high-volume rooms, users, or objects. Use event IDs, inboxes, consumer offsets, and conditional state transitions for at-least-once delivery. Split or isolate hot partitions without changing the client-visible sequence contract.
• Reconnect and replay semanticsIssue resumable real time chat system cursors with an expiry and a clear snapshot-plus-delta fallback. Bound replay windows and rebuild from durable state when a cursor is too old. Expose version and freshness so a client can distinguish current, catching up, and degraded state.
• Backpressure and presenceKeep connection heartbeats and ephemeral presence separate from durable real time chat system interactions. Coalesce safe updates, shed low-value work, and protect critical events during reconnect storms. Measure end-to-end delivery, not only broker publish latency.
• Direct fan-out versus pull-based readsUse push for latency-sensitive real time chat system deltas and pull or replay for reconnect, history, and recovery. A push-only design loses state when clients disconnect and a pull-only design wastes latency and bandwidth.
• Per-recipient queues versus shared streamsUse shared partitioned streams with per-recipient cursors where fan-out is large, and isolate exceptional high-fanout objects. A queue per recipient becomes expensive and hard to inspect at large scale.
• Strong ordering versus availabilityGuarantee ordering only within the scope the product needs, such as a room, object, or conversation. Global ordering introduces a bottleneck and still does not solve duplicate delivery or reconnect recovery.