Design an Anomaly Detection System for Time-Series Data — System Design Interview Practice

Interview prompt

Design a time-series anomaly platform that detects point, contextual, and collective anomalies across metrics and KPIs, adapts to seasonality, and provides actionable evidence without alert storms.

• Define series identity, sampling/time semantics, missing data, baselines, seasonality, thresholds, anomaly types, confidence, and alert lifecycle.
• Support adaptive baselines without learning incidents or maintenance as normal; distinguish noise, outages, regime changes, and collective anomalies.
• Separate ingestion and window aggregation from model scoring, alert routing, root-cause context, and exploratory analysis; make results replayable.
• Explain cardinality, late data, duplicate samples, model drift, suppression, observability, and statistical fallback.

Requirements and scale assumptions

• Ingest time-series samples, calculate seasonal/trend-aware scores, classify anomalies, and persist evidence, confidence, and model versions.
• Expose dashboards, drill-down windows, related series/root-cause hints, alert policies, maintenance windows, and acknowledged incidents.
• Support replay, baseline reset, series deletion, threshold overrides, deduplicated notifications, and recovery from processing gaps.
• Score new samples and route high-confidence anomalies within one minute while labeling incomplete windows and uncertainty.
• Process 1M concurrent series and bursty telemetry without a single hot key or unbounded synchronous work.
• Do not lose committed state; make retries and duplicate events safe.
• Degrade safely when downstream workers, caches, or external dependencies fail.
• 1M series, 5M samples/second, and 10k alert policies
• Partition by the primary tenant, user, item, or geographic key and isolate hot partitions.
• Keep serving state bounded; retain raw events or durable records for replay and auditing.

• Peak scale: 1M series; 5M samples/s — Capacity assumption that drives partitioning and backpressure.
• Latency target: score/alert p95 < 1m; evidence retained — User-facing budget for the primary request or read path.
• Durable boundary: Committed before async — Timestamped samples and versioned detector policies are authoritative; scores, alerts, and hints are derived.
• Async boundary: At-least-once workers — Keep Use STL decomposition for seasonal time-series, Implement autoencoders for unsupervised detection, Use Prophet or DeepAR for forecasting-based detection off the synchronous path.

Key entities

• DatasetVersiondatasetId, version, schemaHash, qualityStatus, lineage, createdAt

  Immutable anomaly detection system input version used for reproducible training, evaluation, or replay.

• FeatureSnapshotentityId, featureSetVersion, eventTime, values, sourceWatermarks

  Point-in-time anomaly detection system features with source watermarks so online and offline values can be compared.

• TrainingRunrunId, datasetVersion, codeVersion, metrics, artifactUri, status

  Audited anomaly detection system run that records data, code, dependency, and evaluation lineage.

• ModelVersionmodelId, version, stage, schema, qualityGates, endpoint

  A promotable anomaly detection system model version with rollout state, contract, and rollback metadata.

Data flow

1. 1. Register and validate training dataThe anomaly detection system gateway records an immutable dataset version, schema, lineage, quality status, and privacy disposition.
2. 2. Build point-in-time featuresFeature workers join anomaly detection system inputs using event-time watermarks, prevent leakage, and publish the same feature contract for training and serving.
3. 3. Train and evaluate asynchronouslyThe orchestrator schedules anomaly detection system runs with checkpointed artifacts, reproducible environments, and metrics tied to the exact input versions.
4. 4. Gate and serve a model versionA registry compares anomaly detection system quality, bias, safety, and compatibility gates before canary or production rollout with an immediate rollback pointer.
5. 5. Monitor drift and learn from feedbackOnline inference records latency, errors, drift, and delayed labels so anomaly detection system retraining is evidence-driven rather than triggered by guesswork.

Deep dives and trade-offs

• Reproducibility and leakage preventionPin anomaly detection system data, feature, code, dependency, and model versions for every run. Use point-in-time joins and quarantine failed quality or privacy checks before training. Keep raw inputs and artifacts immutable so a result can be replayed after a dependency changes.
• Safe promotion and serving contractsSeparate anomaly detection system model registration from deployment and require signed artifacts plus schema compatibility. Use shadow traffic, canaries, rollback pointers, and per-version latency/error budgets. Return model version and feature freshness so clients can explain or reproduce a prediction.
• Drift, feedback, and costMeasure feature drift, prediction drift, label delay, and segment-level quality for anomaly detection system rather than only aggregate accuracy. Sample expensive inference and cap retraining concurrency with an explicit GPU or compute budget. Keep human corrections and delayed labels linked to the original prediction and model version.
• Batch versus online featuresPrefer a shared feature contract with batch backfills and a low-latency online serving path for decisions that need freshness. Two independently defined transformations create training-serving skew and hard-to-debug regressions.
• Synchronous versus asynchronous inferenceKeep interactive anomaly detection system inference synchronous within a strict budget and queue large or expensive jobs. A request path that waits for model loading, enrichment, or retraining turns downstream slowness into an outage.
• Global model versus segment modelsStart with one versioned model and add segment-specific models only when quality or policy evidence justifies the operational cost. Many simultaneously active versions multiply monitoring, rollback, and data-lineage burden.