Design a Customer Churn Prediction System — System Design Interview Practice

Interview prompt

Design an explainable churn-prediction platform that builds leakage-resistant customer features, scores risk ahead of renewal, identifies drivers, and feeds measurable retention campaigns.

• Define customer identity, churn labels, observation/prediction windows, point-in-time features, segment policies, model versions, and attribution.
• Prevent target leakage and bias, handle censoring and delayed outcomes, calibrate probabilities, and expose confidence and data freshness.
• Separate batch training/scoring from online lookups and campaign delivery; make scores reproducible, idempotent, and rollbackable.
• Explain privacy, retention, explainability, drift, human overrides, experiment measurement, observability, and stale-score fallback.

Requirements and scale assumptions

• Ingest customer events and outcomes, create point-in-time features, train/evaluate models, publish approved scores, and calculate drivers.
• Query customer and segment risk with model version/freshness/confidence, export eligible audiences, and record campaign treatment/outcome.
• Support score correction, customer deletion, model rollback, retraining, bias/performance monitoring, and campaign deduplication.
• Publish a calibrated score at least 30 days before the prediction horizon with documented precision/recall and freshness.
• Score 100M customers and serve analyst dashboards without a single hot key or unbounded synchronous work.
• Do not lose committed state; make retries and duplicate events safe.
• Degrade safely when downstream workers, caches, or external dependencies fail.
• 100M customers, weekly scoring, and 10k analyst dashboard users
• Partition by the primary tenant, user, item, or geographic key and isolate hot partitions.
• Keep serving state bounded; retain raw events or durable records for replay and auditing.

• Peak scale: 100M customers; weekly scoring — Capacity assumption that drives partitioning and backpressure.
• Latency target: score freshness < 24h; calibrated output — User-facing budget for the primary request or read path.
• Durable boundary: Committed before async — Versioned feature snapshots, labels, and published score artifacts are authoritative; dashboards are projections.
• Async boundary: At-least-once workers — Keep Use survival analysis for time-to-churn modeling, Engineer behavioral features from usage patterns, Implement SHAP for feature attribution off the synchronous path.

Key entities

• DatasetVersiondatasetId, version, schemaHash, qualityStatus, lineage, createdAt

  Immutable customer churn prediction system input version used for reproducible training, evaluation, or replay.

• FeatureSnapshotentityId, featureSetVersion, eventTime, values, sourceWatermarks

  Point-in-time customer churn prediction system features with source watermarks so online and offline values can be compared.

• TrainingRunrunId, datasetVersion, codeVersion, metrics, artifactUri, status

  Audited customer churn prediction system run that records data, code, dependency, and evaluation lineage.

• ModelVersionmodelId, version, stage, schema, qualityGates, endpoint

  A promotable customer churn prediction system model version with rollout state, contract, and rollback metadata.

Data flow

1. 1. Register and validate training dataThe customer churn prediction system gateway records an immutable dataset version, schema, lineage, quality status, and privacy disposition.
2. 2. Build point-in-time featuresFeature workers join customer churn prediction system inputs using event-time watermarks, prevent leakage, and publish the same feature contract for training and serving.
3. 3. Train and evaluate asynchronouslyThe orchestrator schedules customer churn prediction system runs with checkpointed artifacts, reproducible environments, and metrics tied to the exact input versions.
4. 4. Gate and serve a model versionA registry compares customer churn prediction system quality, bias, safety, and compatibility gates before canary or production rollout with an immediate rollback pointer.
5. 5. Monitor drift and learn from feedbackOnline inference records latency, errors, drift, and delayed labels so customer churn prediction system retraining is evidence-driven rather than triggered by guesswork.

Deep dives and trade-offs

• Reproducibility and leakage preventionPin customer churn prediction system data, feature, code, dependency, and model versions for every run. Use point-in-time joins and quarantine failed quality or privacy checks before training. Keep raw inputs and artifacts immutable so a result can be replayed after a dependency changes.
• Safe promotion and serving contractsSeparate customer churn prediction system model registration from deployment and require signed artifacts plus schema compatibility. Use shadow traffic, canaries, rollback pointers, and per-version latency/error budgets. Return model version and feature freshness so clients can explain or reproduce a prediction.
• Drift, feedback, and costMeasure feature drift, prediction drift, label delay, and segment-level quality for customer churn prediction system rather than only aggregate accuracy. Sample expensive inference and cap retraining concurrency with an explicit GPU or compute budget. Keep human corrections and delayed labels linked to the original prediction and model version.
• Batch versus online featuresPrefer a shared feature contract with batch backfills and a low-latency online serving path for decisions that need freshness. Two independently defined transformations create training-serving skew and hard-to-debug regressions.
• Synchronous versus asynchronous inferenceKeep interactive customer churn prediction system inference synchronous within a strict budget and queue large or expensive jobs. A request path that waits for model loading, enrichment, or retraining turns downstream slowness into an outage.
• Global model versus segment modelsStart with one versioned model and add segment-specific models only when quality or policy evidence justifies the operational cost. Many simultaneously active versions multiply monitoring, rollback, and data-lineage burden.