Design a Scalable IoT Platform — System Design Interview Practice

Interview prompt

Design a multi-tenant IoT platform that securely onboards millions of devices, ingests telemetry, supports commands, processes streams, and exposes anomaly-aware device operations.

• Define device identity, provisioning, heartbeat, telemetry envelope, ordering, QoS, command acknowledgment, tenant isolation, and retention.
• Partition ingestion by tenant/device and time, handle reconnect storms, duplicates, offline buffering, backpressure, and out-of-order events.
• Separate the device gateway from durable telemetry, stream processing, time-series queries, digital-twin state, and command delivery.
• Explain certificate rotation, fleet rollouts, anomaly detection, regional failure, cost controls, observability, and degraded operation.

Requirements and scale assumptions

• Provision devices and credentials, accept telemetry, expose current and historical readings, and maintain device/twin status.
• Process windows for alerts and anomalies, provide tenant dashboards, and send authenticated commands with delivery status.
• Support firmware rollout rings, revocation, device deletion, replay, offline reconnect, and recovery from partial regional outages.
• Accept telemetry with p95 gateway acknowledgment under 200ms and surface stream alerts within 10 seconds.
• Scale to 10M connected devices and reconnect storms without a single hot key or unbounded synchronous work.
• Do not lose committed state; make retries and duplicate events safe.
• Degrade safely when downstream workers, caches, or external dependencies fail.
• 10M devices sending an average of one message per minute, with 10x reconnect bursts
• Partition by the primary tenant, user, item, or geographic key and isolate hot partitions.
• Keep serving state bounded; retain raw events or durable records for replay and auditing.

• Peak scale: 10M devices; 10x reconnect burst — Capacity assumption that drives partitioning and backpressure.
• Latency target: ack < 200ms; alerts < 10s — User-facing budget for the primary request or read path.
• Durable boundary: Committed before async — Durable telemetry and command records are authoritative; twin and alert views are rebuildable projections.
• Async boundary: At-least-once workers — Keep IoT Hub for device connectivity, Stream Analytics for real-time processing, Cosmos DB for device metadata off the synchronous path.

Key entities

• SourcePartitionsourceId, partitionId, cursor, schemaVersion, watermark, status

  Replayable scalable iot platform source evidence and ingestion cursor.

• SchemaVersiondatasetId, version, compatibility, owner, effectiveAt, status

  Governed scalable iot platform contract used to validate producers and consumers.

• ProcessingRunrunId, inputWatermark, checkpoint, qualityStatus, codeVersion, status

  Checkpointed scalable iot platform processing attempt with quality and lineage metadata.

• AnalyticalDatasetdatasetId, partition, watermark, schemaVersion, qualityStatus, location

  Curated scalable iot platform serving partition with freshness and quality state.

Data flow

1. 1. Register sources and contractsThe scalable iot platform catalog records owners, schemas, compatibility rules, retention, lineage, and partitioning before data is accepted.
2. 2. Ingest with backpressureConnectors checkpoint scalable iot platform source cursors, validate schema and deduplication keys, and slow producers when downstream capacity is exhausted.
3. 3. Process event time with checkpointsStream or batch engines compute scalable iot platform transformations using watermarks, late-data policy, state checkpoints, and deterministic code versions.
4. 4. Publish quality-gated datasetsOnly scalable iot platform outputs that pass completeness, freshness, validity, and privacy checks become visible to analytical consumers.
5. 5. Serve, replay, and reconcileConsumers read bounded partitions with freshness metadata while operators replay failed scalable iot platform ranges and compare output checksums.

Deep dives and trade-offs

• Schema evolution and data qualityVersion scalable iot platform contracts and make compatibility rules explicit for every producer and consumer. Quarantine malformed partitions instead of poisoning the whole dataset. Track row counts, null rates, duplicates, distribution changes, and policy violations by partition.
• Watermarks, late data, and exactly-once effectsUse source cursors and event-time watermarks for scalable iot platform progress, not wall-clock assumptions. Make checkpoints, output keys, and sink commits retry-safe under at-least-once delivery. Document how late events revise windows, aggregates, or snapshots.
• Replay, lineage, and costKeep immutable scalable iot platform raw evidence and code or schema versions so failed outputs can be reproduced. Separate hot serving storage from cold retention and cap replay concurrency. Measure freshness, backlog, compute cost, storage growth, and quality-gate failure rate.
• Streaming versus batchUse streaming for freshness-critical scalable iot platform paths and batch for backfills, compaction, and expensive recomputation. Forcing every workload into streaming makes state, replay, and cost harder to operate.
• Raw retention versus curated-only storageRetain enough immutable raw evidence for replay, audit, and correction, then tier or expire it according to policy. Without raw evidence, a bad transformation can require an unreproducible emergency fix.
• Central warehouse versus domain-owned datasetsCentralize governance and discovery while letting domain owners own contracts and quality signals. A single team owning every transformation becomes a delivery bottleneck and hides data ownership.