Design a Container-Based Microservices Architecture — System Design Interview Practice

Interview prompt

Design a container-based microservices platform with independent deployment and scaling, service discovery, resilient communication, centralized observability, and safe multi-tenant operations.

• Give each service clear ownership, API contracts, data boundaries, health signals, and an independently deployable artifact.
• Use service discovery, timeouts, circuit breakers, bounded retries, bulkheads, and idempotency instead of assuming a reliable network.
• Treat desired deployment state, service identity, configuration, and secrets separately from runtime metrics and logs.
• Explain rollout, schema compatibility, distributed tracing, dependency failure, noisy neighbors, and disaster recovery.

Requirements and scale assumptions

• Register services and versions, build and deploy containers, expose internal and external routes, and discover healthy endpoints.
• Scale services independently, roll out canaries, collect logs and traces, and show dependency health and saturation.
• Support mTLS identity, secrets rotation, config versioning, rollback, migrations, decommissioning, and audit history.
• Operate 100+ services and thousands of containers with p95 east-west request latency below 100 ms where possible.
• Scale discovery, telemetry, and autoscaling independently while isolating services with resource quotas and bulkheads.
• Do not lose service-owned durable state; make deploy, retry, event, and migration operations safe to repeat.
• Degrade with timeouts, cached reads, queues, and partial feature disablement when dependencies fail.
• Implement circuit breakers
• Partition by the primary tenant, user, item, or geographic key and isolate hot partitions.
• Keep serving state bounded; retain raw events or durable records for replay and auditing.

• Peak scale: Implement circuit breakers — Capacity assumption that drives partitioning and backpressure.
• Latency target: Support 100+ microservices — User-facing budget for the primary request or read path.
• Durable boundary: Committed before async — The source of truth is Deploy multiple microservices; Service discovery and routing.
• Async boundary: At-least-once workers — Keep ECS/EKS for container orchestration, ECR for container registry, Application Load Balancer for routing off the synchronous path.

Key entities

• ResourceSpecresourceId, tenantId, desiredState, version, policyVersion, updatedAt

  Versioned desired state for a container based microservices architecture managed resource.

• OperationoperationId, resourceId, requestHash, step, attempt, status

  Durable container based microservices architecture reconciliation operation with per-step progress.

• PolicyVersionpolicyId, scope, version, rules, effectiveAt, status

  Auditable container based microservices architecture policy evaluated before provisioning or mutation.

• ReconciliationCheckpointresourceId, provider, observedVersion, cursor, lastError, updatedAt

  Provider-specific container based microservices architecture observation and recovery cursor.

Data flow

1. 1. Accept a desired-state commandThe container based microservices architecture control plane authenticates the tenant, validates policy and quotas, checks the expected version, and records the desired state.
2. 2. Plan a safe operationA planner turns container based microservices architecture desired state into ordered, bounded steps with dependency checks, blast-radius limits, and rollback metadata.
3. 3. Reconcile providers asynchronouslyWorkers apply container based microservices architecture operations through provider adapters, persist checkpoints, rate-limit calls, and treat unknown outcomes as observable state.
4. 4. Publish observed healthThe serving projection joins desired and observed container based microservices architecture state with operation status, policy version, freshness, and actionable errors.
5. 5. Recover and auditRetries, dead letters, drift detection, and operator approvals repair container based microservices architecture resources without losing the original command or provider evidence.

Deep dives and trade-offs

• Desired versus observed stateKeep container based microservices architecture desired state separate from provider-observed state and show both to operators. Make every reconciliation step conditional and resumable so a worker crash does not restart unsafe effects. Version policy and resource state so old operations cannot overwrite newer intent.
• Provider failures and unknown outcomesUse provider-specific idempotency tokens and query-after-timeout behavior for container based microservices architecture operations. Bound retries with exponential backoff, circuit breakers, and per-provider quotas. Route irreconcilable drift to an approval or quarantine path instead of retrying forever.
• Blast radius and operationsPartition container based microservices architecture work by tenant, region, cluster, or resource class and cap concurrent mutations. Audit who changed desired state, which policy allowed it, and what provider evidence was observed. Alert on drift age, operation backlog, failed steps, policy denials, and stale observations.
• Push versus pull reconciliationUse event triggers for fast response and periodic scans for missed events, drift, and recovery. A push-only container based microservices architecture controller silently misses changes when a provider event is lost.
• Central control plane versus provider-native controllersKeep policy, intent, and audit centralized while isolating provider-specific application logic behind adapters. A monolithic controller becomes hard to scale and couples unrelated provider failure domains.
• Automatic repair versus approvalAutomate low-risk, reversible container based microservices architecture changes and require approval for destructive or high-blast-radius operations. Full automation without policy or blast-radius controls can turn a transient signal into a widespread outage.