Design a Natural Language Processing Pipeline — System Design Interview Practice

Interview prompt

Design an NLP platform that processes unstructured text through normalization, language detection, entity and relation extraction, sentiment, topic modeling, and classification with versioned outputs.

• Keep source text immutable and version processors, model artifacts, tokenization, annotations, confidence, and schema mappings.
• Support streaming and batch modes with bounded document work, tenant quotas, language-aware models, and deterministic retries.
• Return spans and relations with offsets and confidence so downstream users can inspect evidence rather than trust a label blindly.
• Explain PII redaction, model drift, long documents, malformed encodings, human correction, evaluation, and replay.

Requirements and scale assumptions

• Submit text or documents, normalize and detect language, extract entities and relations, classify content, and store versioned annotations.
• Provide batch and synchronous APIs with per-document status, confidence, model version, offsets, and processing errors.
• Support redaction, correction, reprocessing, dataset export, tenant access, retention, and evaluation feedback.
• Target entity F1 above 90% on declared evaluation sets and keep small synchronous documents below 500 ms p95.
• Process 100 million documents per day using document and tenant partitions without one long document starving the queue.
• Make document submission, model inference, annotation publication, and correction events idempotent.
• Fall back to a lightweight ruleset or return a partial result with confidence warnings when a model is unavailable.
• Process 100 million documents per day across dozens of languages and multiple domain-specific model versions.
• Partition by tenant, language, document, and model; isolate long documents and high-volume producers.
• Retain encrypted source text, token offsets, annotations, models, evaluation sets, and redaction audit records.

• Peak scale: Provide API for batch — Capacity assumption that drives partitioning and backpressure.
• Latency target: Entity extraction F1-score above 90% — User-facing budget for the primary request or read path.
• Durable boundary: Committed before async — The source of truth is Extract named entities (people, organizations, locations); Identify relationships between entities.
• Async boundary: At-least-once workers — Keep Use transformer-based models (BERT, RoBERTa) for NER, Implement SpaCy for fast entity extraction, Use fine-tuned models per domain for accuracy off the synchronous path.

Key entities

• DatasetVersiondatasetId, version, schemaHash, qualityStatus, lineage, createdAt

  Immutable natural language processing pipeline input version used for reproducible training, evaluation, or replay.

• FeatureSnapshotentityId, featureSetVersion, eventTime, values, sourceWatermarks

  Point-in-time natural language processing pipeline features with source watermarks so online and offline values can be compared.

• TrainingRunrunId, datasetVersion, codeVersion, metrics, artifactUri, status

  Audited natural language processing pipeline run that records data, code, dependency, and evaluation lineage.

• ModelVersionmodelId, version, stage, schema, qualityGates, endpoint

  A promotable natural language processing pipeline model version with rollout state, contract, and rollback metadata.

Data flow

1. 1. Register and validate training dataThe natural language processing pipeline gateway records an immutable dataset version, schema, lineage, quality status, and privacy disposition.
2. 2. Build point-in-time featuresFeature workers join natural language processing pipeline inputs using event-time watermarks, prevent leakage, and publish the same feature contract for training and serving.
3. 3. Train and evaluate asynchronouslyThe orchestrator schedules natural language processing pipeline runs with checkpointed artifacts, reproducible environments, and metrics tied to the exact input versions.
4. 4. Gate and serve a model versionA registry compares natural language processing pipeline quality, bias, safety, and compatibility gates before canary or production rollout with an immediate rollback pointer.
5. 5. Monitor drift and learn from feedbackOnline inference records latency, errors, drift, and delayed labels so natural language processing pipeline retraining is evidence-driven rather than triggered by guesswork.

Deep dives and trade-offs

• Reproducibility and leakage preventionPin natural language processing pipeline data, feature, code, dependency, and model versions for every run. Use point-in-time joins and quarantine failed quality or privacy checks before training. Keep raw inputs and artifacts immutable so a result can be replayed after a dependency changes.
• Safe promotion and serving contractsSeparate natural language processing pipeline model registration from deployment and require signed artifacts plus schema compatibility. Use shadow traffic, canaries, rollback pointers, and per-version latency/error budgets. Return model version and feature freshness so clients can explain or reproduce a prediction.
• Drift, feedback, and costMeasure feature drift, prediction drift, label delay, and segment-level quality for natural language processing pipeline rather than only aggregate accuracy. Sample expensive inference and cap retraining concurrency with an explicit GPU or compute budget. Keep human corrections and delayed labels linked to the original prediction and model version.
• Batch versus online featuresPrefer a shared feature contract with batch backfills and a low-latency online serving path for decisions that need freshness. Two independently defined transformations create training-serving skew and hard-to-debug regressions.
• Synchronous versus asynchronous inferenceKeep interactive natural language processing pipeline inference synchronous within a strict budget and queue large or expensive jobs. A request path that waits for model loading, enrichment, or retraining turns downstream slowness into an outage.
• Global model versus segment modelsStart with one versioned model and add segment-specific models only when quality or policy evidence justifies the operational cost. Many simultaneously active versions multiply monitoring, rollback, and data-lineage burden.