Design a Distributed Queue like RabbitMQ — System Design Interview Practice

Interview prompt

Design a distributed message-queue service for reliable asynchronous communication, supporting acknowledgements, redelivery, dead-letter handling, ordering, priorities, and consumer backpressure.

• Define queue, exchange/topic, routing, acknowledgement, visibility timeout, retention, and delivery semantics explicitly.
• Replicate durable messages and metadata, partition hot queues, and apply producer and consumer backpressure.
• Make redelivery and duplicate consumption safe with consumer idempotency, dead-letter queues, and poison-message isolation.
• Explain ordering versus throughput, rebalancing, node loss, quotas, replay, and tenant authorization.

Requirements and scale assumptions

• Create queues and bindings, publish messages, consume with leases or acknowledgements, retry failures, and dead-letter poison messages.
• Expose queue depth, oldest-message age, consumer lag, delivery attempts, throughput, and retention state.
• Support tenant quotas, authentication, replay, message TTL, priority policy, ordering keys, and administrative purge.
• Target p95 publish acknowledgement below 20 ms for replicated durable messages and bounded consumer delivery latency.
• Handle millions of messages per second through partitioned queues, batching, replication, and per-tenant quotas.
• Never lose an acknowledged durable message; treat delivery as at-least-once and require consumer idempotency.
• Apply backpressure and preserve queued messages when consumers or replicas are degraded.
• Support 5 million messages per second, 100,000 queues, and bursts from thousands of producers.
• Partition by queue and ordering key, split hot queues, and isolate large tenants and slow consumers.
• Retain durable messages according to queue policy while keeping delivery leases and consumer presence bounded.

• Peak throughput: 5M messages/s — Drives partitioning, batching, replication, disk, network, and producer quotas.
• Publish acknowledgement: p95 <=20ms — Durable publish latency target; delivery and processing latency are separate metrics.
• Durable boundary: Committed before async — The source of truth is Publish and consume messages; Support multiple queues and topics.
• Async boundary: At-least-once workers — Keep Separate metadata and message storage, Replication for durability, Consumer groups for load balancing off the synchronous path.

Key entities

• InteractioninteractionId, actorId, objectId, type, version, occurredAt

  Canonical distributed queue interaction with an idempotency key and ordering version.

• ConnectionSessionsessionId, userId, deviceId, roomKey, lastHeartbeat, status

  Ephemeral but observable distributed queue connection registration used for routing and presence.

• FanoutCursorstreamKey, shard, offset, consumerGroup, updatedAt

  Durable progress marker for distributed queue fan-out and replay.

• DeliveryReceiptinteractionId, recipientId, channel, attempt, status, deliveredAt

  Deduplicated distributed queue delivery state for reconnects, retries, or acknowledgements.

Data flow

1. 1. Accept and commit the interactionThe distributed queue gateway authenticates the actor, validates room or object membership, applies rate limits, and conditionally commits the interaction.
2. 2. Publish an ordered eventAn outbox emits the committed distributed queue transition with an event ID, partition key, sequence, and replay retention.
3. 3. Fan out by partitionConsumers route distributed queue events to connected recipients, durable inboxes, or notification channels without making the origin write wait for every recipient.
4. 4. Resume and reconcile connectionsClients reconnect with a cursor; the distributed queue service replays missed events, deduplicates delivery, and exposes stale or degraded state.
5. 5. Measure latency and recoverOperations tracks distributed queue publish-to-deliver latency, hot partitions, reconnect storms, dropped events, and consumer lag for replay or repair.

Deep dives and trade-offs

• Ordering, idempotency, and hot keysChoose a distributed queue partition key that preserves required order while distributing high-volume rooms, users, or objects. Use event IDs, inboxes, consumer offsets, and conditional state transitions for at-least-once delivery. Split or isolate hot partitions without changing the client-visible sequence contract.
• Reconnect and replay semanticsIssue resumable distributed queue cursors with an expiry and a clear snapshot-plus-delta fallback. Bound replay windows and rebuild from durable state when a cursor is too old. Expose version and freshness so a client can distinguish current, catching up, and degraded state.
• Backpressure and presenceKeep connection heartbeats and ephemeral presence separate from durable distributed queue interactions. Coalesce safe updates, shed low-value work, and protect critical events during reconnect storms. Measure end-to-end delivery, not only broker publish latency.
• Direct fan-out versus pull-based readsUse push for latency-sensitive distributed queue deltas and pull or replay for reconnect, history, and recovery. A push-only design loses state when clients disconnect and a pull-only design wastes latency and bandwidth.
• Per-recipient queues versus shared streamsUse shared partitioned streams with per-recipient cursors where fan-out is large, and isolate exceptional high-fanout objects. A queue per recipient becomes expensive and hard to inspect at large scale.
• Strong ordering versus availabilityGuarantee ordering only within the scope the product needs, such as a room, object, or conversation. Global ordering introduces a bottleneck and still does not solve duplicate delivery or reconnect recovery.