Hamropedia

Phase 8 · High-level design questions · Topic 8.5

Design a news feed (like Twitter or Instagram)

Must knowHLD

Show each user a feed of recent posts from the people they follow.

By Manoj Kumar Thapa

AI Software Engineer ·

Not started

Step 1 Understand it

Why interviewers ask this. It tests the trade-off — do the work when posting or when reading — plus and the celebrity problem.

New words in this topic (tap one to see what it means)

Watch it happen

AnimationA news feed: push to followers' lists, pull for celebrities
Read the animation as text
  1. Fan-out on write: Alice posts. The post is saved once in the post store.
  2. Workers copy the post's ID into each follower's ready-made feed list (in Redis). Writing costs one entry per follower.
  3. Reading is then trivial: Bob's feed is one list read, already in order. Post content comes from a cache, and photos from a CDN.

Requirements

Functional

  • Create posts; follow and unfollow users.
  • View a personalised feed (newest first, or ranked), with .

Non-functional

  • The feed loads fast (under about 200 ); very read-heavy.
  • is fine: a post can appear a few seconds later.

Estimates

  • 300 million daily users × 10 feed views ≈ 35,000 reads per second; 2 posts each a day ≈ 7,000 writes per second.

API

  • POST /posts
  • GET /feed?cursor=…

Data model

  • posts (post_id, author_id, content, created_at)
  • follows (follower_id, followee_id)
  • feed cache: user_id → recent post IDs ()

Start simple, then scale

  1. Version 1: fan-out on read: query the posts of everyone you follow, sorted by time.
  2. Version 2: fan-out on write into per-user feed lists in Redis.
  3. Version 3: a hybrid that handles celebrities differently.
  4. Version 4: ranking, media through a , and skipping inactive users.

Picture it

flowchart TD
  P["A normal user posts"] --> FW["Fan-out workers"]
  FW --> F1[("Followers' feed lists<br/>in Redis")]
  CEL["A celebrity posts"] --> CP[("Celebrity post store")]
  R["A user opens their feed"] --> FS["Feed service"]
  FS --> F1
  FS -->|"merge at read time"| CP
Normal posts are pushed into ' feeds when written. Celebrity posts are pulled and merged in when a feed is read.

The design, step by step

  1. Fan-out on write (push): when someone posts, add the post ID to each follower's feed list. Reads are fast.
  2. Fan-out on read (pull): build the feed at time from the followed users' recent posts. Writes are cheap; reads are slow.
  3. Hybrid: push for normal users; for celebrities with millions of followers, pull their posts at read time and merge.
  4. Post content comes from a cache and media from a CDN; a ranking service can reorder the feed.

Deep dives

  • The celebrity problem: pushing to 50 million followers for every post is far too expensive → use the hybrid.
  • Each user's feed list keeps only the latest few hundred post IDs.
  • Skip pushing to inactive users; build their feed when they in.
  • Ranking: gather candidates → score them with a model → for variety.

Common mistakes

  • Push-only fan-out that ignores celebrities.
  • Storing whole posts in every feed instead of post IDs.
  • Offset pagination on a feed that keeps changing.

Step 2 Go further (free, optional)

Optional. The notes above are written to be enough on their own. These free resources give a second explanation and extra practice.

Step 3 Practise

Try each exercise yourself first: on paper, out loud or in code. Then open the answer and compare it with yours.

  1. ExplainSay it in plain words in under a minute, with no notes.

    Explain push, pull and hybrid fan-out, and the celebrity problem.

    Show answer

    • Push (fan-out on write): when someone posts, add the post's id to the feed list of every follower straight away. Reading a feed is then just reading one ready-made list: very fast. The cost is at write time: one post makes as many writes as the author has followers.
    • Pull (fan-out on read): store each post once. When someone opens their feed, fetch the recent posts of everyone they follow and merge them. Posting is cheap, but every feed load does a lot of work, and feeds are read far more often than posts are written.
    • Hybrid: push for ordinary users, who have hundreds of followers; pull for the few accounts with millions. At read time, the pre-built feed list is merged with the recent posts of the celebrities you follow.

    The celebrity problem: with pure push, one post from an account with 50 million followers means 50 million feed writes. It swamps the fan-out and delays everyone else's posts, and most of those followers won't even open the app today. The hybrid avoids it: celebrities' posts are fetched when someone actually reads.

  2. DrawSketch the diagram (or class diagram) from memory.

    Draw the write path and the read path for the hybrid design.

    Show answer

    On write, only ordinary authors' posts are fanned out to followers' feed lists; a celebrity's post is just stored. On read, the ready-made feed list is merged with the latest posts of the few celebrities the reader follows, then hydrated (ids turned into full posts) and optionally ranked.

    text
    WRITE
Author ── POST /posts ──► Post service ──► Posts DB + post cache
                             │
                             ├─ ordinary author ──► Fan-out queue
                             │                        │ workers
                             │                        ▼
                             │        each follower's feed list (Redis)
                             │        user → [latest ~500 post ids]
                             │
                             └─ celebrity ──► only their own post list

READ
Reader ── GET /feed ──► Feed service
                          1. read my feed list (Redis)
                          2. + latest posts of celebrities I follow
                          3. merge by time (or rank)
                          4. hydrate: post cache, media via CDN
                          ──► page of posts + cursor

  3. ApplyUse it in a real design question and say exactly where it fits.

    Estimate how many writes one celebrity post causes with push fan-out, and compare with the hybrid.

    Show answer

    With pure push, one post from an account with 50 million followers is 50 million feed-list writes; at 5 posts a day, 250 million writes a day from one account.

    To put that in proportion: all ordinary posting (7,000 posts a second × about 200 followers each) is about 1.4 million feed writes a second. One celebrity post is 50 million ÷ 1.4 million ≈ 36 seconds of the whole system's normal fan-out capacity, arriving all at once, and delaying everyone else's posts behind it.

    With the hybrid: the post is written once (to the celebrity's own post list): 1 write instead of 50 million. The cost moves to reads: each follower's feed load does one extra lookup, "latest posts of the celebrities I follow", which is a few that sit in a cache.

    (Even with push, skipping inactive followers helps: if only 30% open the app today, that is 15 million writes, not 50; the others' feeds are built when they log in.)

    python
    followers = 50_000_000
posts_per_day = 5
normal_fanout_per_s = 7_000 * 200                 # every ordinary post, ~200 followers each

print(followers * posts_per_day)                  # 250000000 feed writes a day, pure push
print(round(followers / normal_fanout_per_s, 1))  # 35.7 seconds of normal fan-out, per post

  4. Trade-offsSay what it costs, when not to use it, and the alternative.

    A chronological feed or a ranked feed: what changes in the architecture?

    Show answer

    Chronological feed:
    • The feed list is just post ids in time order, so fan-out stores them sorted and reading is a range read.
    • Caching is simple, and pagination is a simple ("posts older than this one").
    • Behaviour is predictable and easy to explain.

    Ranked feed: the architecture grows several new parts:
    • : gather a few hundred possible posts (the fan-out list, plus celebrity posts, plus recommended posts from accounts you don't follow).
    • Features: a with signals per post and per user (likes so far, how often you interact with the author, the post's age), fetched at read time.
    • A ranking service that scores candidates with a model, inside the budget (tens of milliseconds).
    • Re-ranking for variety and rules (not five posts in a row from one author; remove what you have already seen).
    • Logging of what was shown and what was clicked, to train the next model, plus experiment infrastructure for .
    • Pagination gets harder: the ranked list changes between requests, so you store the ranked result for the and page through that.

    In short: chronological is a data-structure problem; ranked is that, plus a machine-learning system on every read.

Step 4 Interview-ready, then prove it

Your interview answer

About 20 to 40 seconds, said out loud. Practise it until it sounds like you, not like notes.

I'd use a hybrid fan-out. When a normal user posts, workers push the post ID into each follower's feed list in Redis, so reading is a fast cache lookup. Celebrity posts aren't fanned out; their recent posts are fetched at read time and merged in. Post bodies come from a cache and media from a CDN.

Follow-up questions

Why not fan out on write for everyone?

A celebrity with 50 million followers would cause 50 million writes per post: slow and wasteful.

How do you paginate a feed that keeps changing?

With a cursor: continue from the last post ID the user saw, so new posts don't shift the pages.

Tick each check when you can do it without notes

A topic is finished when you pass all four checks, not when you finish a video. Be strict: if you needed your notes, it is not ticked yet.

  • Explain Say it in plain words in under a minute, with no notes.
  • Draw Sketch the diagram (or class diagram) from memory.
  • Apply Use it in a real design question and say exactly where it fits.
  • Trade-offs Say what it costs, when not to use it, and the alternative.

0 of 4 checks ticked.

Saved in this browser. Back it up or move it to another device.