Social Media News Feed — System Design Interview Practice

Interview prompt

Design Design a news feed system like Facebook or Twitter that can handle billions of posts and personalized feeds. so users can Users can post updates (text, images, videos) reliably at scale.

• Define the source of truth for Users can post updates (text, images, videos); Users can follow other users and make retries idempotent.
• Use bounded, partitioned state to meet Trending topics and hashtags and Feed generation should be fast (<200ms).
• Separate the critical request path from Push vs Pull model for feed generation, Fanout strategies for celebrity posts, Machine learning for personalization.
• Explain consistency, failure recovery, authorization, observability, and a degraded mode.

Requirements and scale assumptions

• Support the core workflow to Users can post updates (text, images, videos).
• Expose status, results, and freshness appropriate to Design a news feed system like Facebook or Twitter that can handle billions of posts and personalized feeds..
• Support authorization, validation, updates, deletion, and recovery semantics.
• Meet Feed generation should be fast (<200ms) under normal load.
• Scale to Trending topics and hashtags without a single hot key or unbounded synchronous work.
• Do not lose committed state; make retries and duplicate events safe.
• Degrade safely when downstream workers, caches, or external dependencies fail.
• Trending topics and hashtags
• Partition by the primary tenant, user, item, or geographic key and isolate hot partitions.
• Keep serving state bounded; retain raw events or durable records for replay and auditing.

• Peak scale: Trending topics — Capacity assumption that drives partitioning and backpressure.
• Latency target: Feed generation should be fast (<200ms) — User-facing budget for the primary request or read path.
• Durable boundary: Committed before async — The source of truth is Users can post updates (text, images, videos); Users can follow other users.
• Async boundary: At-least-once workers — Keep Push vs Pull model for feed generation, Fanout strategies for celebrity posts, Machine learning for personalization off the synchronous path.

Key entities

• MediaAssetassetId, ownerId, sourceUri, checksum, privacy, status

  Canonical uploaded social media news feed asset and lifecycle state.

• MediaRenditionassetId, profile, codec, uri, checksum, status

  Derived social media news feed output identified by a deterministic profile and content hash.

• PlaybackSessionsessionId, assetId, viewerId, entitlementVersion, edgeRegion, expiresAt

  Short-lived social media news feed access session that binds authorization to delivery.

• ProcessingJobjobId, assetId, operation, attempt, checkpoint, status

  Retry-safe social media news feed processing job with checkpoints and per-rendition progress.

Data flow

1. 1. Reserve a resumable uploadThe social media news feed gateway authenticates the owner, reserves metadata, validates size and checksum, and returns a scoped upload URL.
2. 2. Commit and verify the sourceA completion callback verifies the social media news feed object, records an immutable checksum, and publishes a processing job only once.
3. 3. Process renditions asynchronouslyWorkers execute social media news feed transforms with deterministic profiles, checkpointing, bounded retries, and a dead-letter path for corrupt inputs.
4. 4. Publish an entitlement-aware manifestA manifest projection exposes only completed social media news feed renditions and carries policy, checksum, and freshness metadata.
5. 5. Deliver, invalidate, and recoverCDN delivery is protected by expiring URLs and revocation signals; failed social media news feed jobs and stale manifests are replayable without duplicating outputs.

Deep dives and trade-offs

• Integrity and idempotent processingUse checksums and immutable source objects for social media news feed deduplication and audit. Derive output keys from asset, profile, and transform version so retries cannot corrupt a completed rendition. Make completion callbacks and worker claims conditional on job version and attempt.
• Authorization at the edgeBind social media news feed manifests and signed URLs to the viewer, entitlement version, and expiry. Propagate takedown, privacy, and subscription changes to edge caches with bounded revocation delay. Never let a cache hit bypass the policy decision for private or paid content.
• Cost, hot assets, and backpressureSeparate interactive manifest latency from expensive social media news feed processing and encode work. Use queue priority, concurrency limits, and lifecycle policies for source and rendition storage. Measure cache hit rate, startup latency, processing backlog, failed bytes, and egress cost by profile.
• Process on upload versus on demandPrecompute common social media news feed profiles and generate rare profiles on demand with a durable job state. Generating every possible profile up front wastes storage and processing budget.
• Origin storage versus CDN cachingKeep the origin authoritative and use CDN caching for immutable or versioned outputs with explicit invalidation. A cache cannot be the only copy of a social media news feed rendition or the recovery path becomes undefined.
• Quality versus delivery costChoose profiles from device, bandwidth, and business requirements, then measure quality and egress by cohort. Maximal bitrate or resolution can make tail startup and cost unacceptable without improving viewing outcomes.