Design Twitter / X (the home timeline)

The problem

Twitter (now X) is a public microblogging network where 500 million people post short messages — tweets — and follow accounts whose posts they want to see. When you open the app, you get a home timeline: a ranked or reverse-chronological stream of tweets from everyone you follow, loaded in under 200 milliseconds. That timeline is the product. Everything else — likes, retweets, search — is decoration.

The mechanical challenge is deceptively simple to state: when user A posts a tweet, all of A's followers should see it in their feed. But a popular account can have 100 million followers. Delivering a single tweet to 100 million inboxes in seconds, while simultaneously handling 500 million daily users each refreshing their feeds dozens of times, produces a read load around 290,000 requests per second on average and a write fanout that dwarfs what any single database can absorb. This is the design question that made Twitter's infrastructure famous — and notorious.

The core tension is write amplification versus read latency. You can build the timeline at read time (pull from every followee on demand — cheap to write, brutal to read at scale) or you can pre-build it at write time (push the tweet into every follower's cache immediately — reads become a single cache lookup, but one tweet produces millions of writes). Neither pure approach survives full scale. The interesting engineering lives in the hybrid: fanout-on-write for ordinary users, fanout-on-read for celebrities, merged at read time. This is the canonical "design a social feed" question — variants include Facebook News Feed, Instagram, and LinkedIn — and the core trade-off is identical in all of them.

Functional requirements

• Post a tweet (text, optionally media).
• Follow / unfollow other users.
• View home timeline: tweets from people you follow, reverse-chronological (or ranked).
• View user timeline: tweets by a single user.
• (Stretch) likes, retweets, replies, search, trending.

We'll focus on post + home timeline, which is where all the interesting decisions live.

Non-functional requirements

• 500M DAU.
• p99 home-timeline load < 200ms.
• Tweets durable forever.
• 99.99% availability.

Capacity estimation

These numbers are the interview-scale model — treat them as design parameters, not live stats. (Real X/Twitter had ~238M mDAU as of the last disclosed quarterly figure, with self-reported DAU claims up to 300M in 2024, and ~500M tweets/day; interviews conventionally model a larger hypothetical to stress-test the design.)

Dimension	Estimate	How we got there
DAU	500M	Interview hypothesis
Tweet writes (real-world)	~500M/day · ~5,800/s avg	500M ÷ 86,400 ≈ 5,800/s; modeled at 1B/day (~12k/s avg, ~35k/s peak) for headroom
Timeline reads	~25B/day · ~290k/s avg · ~870k/s peak	500M users × 50 reads/day = 25B/day; 25B ÷ 86,400 ≈ 290k/s
Read:write ratio	~25:1 API-level; 1,000:1+ delivery-level	Each tweet fans out to all follower timelines; reads dominate either way
Tweet payload	~400B	280 chars text + ~100B metadata
Media per tweet	~200KB avg (20% of tweets carry media)	20% of 500M = 100M media tweets/day
Daily text storage (1B/day model)	~400 GB/day	1B × 400B = 400 GB
Daily media storage	~20 TB/day	0.1B × 200KB = 20 TB
Yearly media (no replication)	~7.3 PB/year	20 TB/day × 365 ≈ 7.3 PB
Yearly media (with 3× replication)	~22 PB/year	~7.3 PB × 3 ≈ 22 PB
Timeline cache — tweet IDs only	~3.2 TB	500M users × 800 IDs × 8B = 3.2 TB
Timeline cache — hydrated tweets	~40 TB	500M users × 800 tweets × 100B (compressed) = 40 TB

Takeaway: Media dominates storage (~20 TB/day, ~7.3 PB/year before replication); tweet text is negligible by comparison. The timeline cache (~3.2 TB of IDs) fits comfortably in Redis, while storing fully hydrated tweets (~40 TB) would not — store IDs and hydrate from a shared object cache.

Building up to the design

The full Twitter architecture has so many moving parts — fanout workers, celeb cache, ranker, snowflake IDs, multi-region replication — that it's easy to lose the thread. Start with the most obvious possible implementation and watch each piece become necessary as the user base grows.

V1: Two tables and a JOIN

tweets(id, author_id, text, created_at)
follows(follower_id, followee_id)

Loading the home timeline is one query:

SELECT t.*
FROM tweets t
JOIN follows f ON f.followee_id = t.author_id
WHERE f.follower_id = $me
ORDER BY t.created_at DESC
LIMIT 50;

This works perfectly for the first 10k users. Posting is one INSERT. Reading is one query. A CS101 student could ship this in an afternoon. The problem surfaces once a user follows 500 people: Postgres has to scan recent tweets across 500 author IDs, then sort. At 100M tweets in the table, with 25k posts/sec landing, query planning alone blows the latency budget.

V2: Cache the timeline per user

When you log in, materialize your timeline once and stash it in Redis:

ZADD timeline:user_42 score=ts member=tweet_id

Next login, read your sorted set directly — no JOIN. Timeline reads drop from a "500-way fan-in JOIN" to a single ZREVRANGEBYSCORE on one key — sub-millisecond. The new problem is cache freshness: when someone you follow posts, your cached timeline is stale. Something has to push new tweets in.

V3: Fanout on write

When a user posts, look up their followers and ZADD the new tweet ID into each follower's timeline. This is the standard "push" model.

flowchart LR
    POST[POST /tweets] --> SVC[Tweet Svc]
    SVC --> DB[(tweets)]
    SVC -->|"for each follower"| TC[(timeline:user_X)]
    style TC fill:#15803d,color:#fff

Home-timeline reads become a single sorted-set range scan — O(log N) on a bounded set. Writing to a few hundred follower timelines in the background is fine for a normal user. But then you run into Justin Bieber.

V4: The celebrity problem — hybrid fanout

Justin Bieber has 100M followers. One tweet from him triggers 100M Redis writes. At 5 tweets/day that's 500M writes for one user, with all 100M timelines spiking at the same moment.

The fix: above a follower threshold (~10k), skip the fanout entirely. Store celebrity tweets in a separate cache that gets merged at read time.

Load timeline:
  1. Read my pushed timeline (Redis)
  2. For each celeb I follow, read their recent tweets (Redis)
  3. Merge, sort, return.

Fanout cost is now bounded by non-celebrity followers. Read cost is bounded by how many celebrities you follow — in practice, a small number. Both sides stay manageable.

V5: Kafka + fanout workers

The write-fan-out relationship is now asymmetric: writing is cheap (one INSERT), but the side effects are large and slow. Kafka decouples them. The post-tweet flow becomes: write the tweet → emit a NewTweet event to Kafka → return 201. A pool of fanout workers consumes the events and does the per-follower ZADDs asynchronously.

The user sees "posted" in about 50ms. Fanout finishes in seconds. On New Year's Eve at midnight when tweet volume spikes 10×, the system queues rather than falls over.

flowchart LR
    V1[V1: SQL JOIN<br/>~10k users] --> V2[V2: + Redis timeline cache<br/>fast reads, stale on write]
    V2 --> V3[V3: + fanout on write<br/>breaks at celebs]
    V3 --> V4[V4: hybrid push+pull<br/>celebs handled]
    V4 --> V5[V5: + Kafka workers<br/>async, durable, scalable]
    V5 --> V6[V6: + ranker + sharding<br/>production Twitter]
    style V1 fill:#0e7490,color:#fff
    style V3 fill:#15803d,color:#fff
    style V5 fill:#ff6b1a,color:#0a0a0f
    style V6 fill:#a855f7,color:#fff

The rest of the article zooms in on V5–V6. If an interviewer pushes you on "but what about ranking" or "what about multi-region," you say "let's evolve from here," not "let me redesign from scratch."

API

POST /tweets             { text, media_ids? }            → 201
GET  /timeline           ?cursor=...                       → tweet list
GET  /users/:id/tweets                                     → tweet list
POST /follow             { user_id }                       → 201

The schema

-- Users
CREATE TABLE users (
  id BIGINT PRIMARY KEY,
  username TEXT UNIQUE,
  display_name TEXT,
  created_at TIMESTAMPTZ
);

-- Follow graph (this is huge — sharded heavily)
CREATE TABLE follows (
  follower_id  BIGINT,
  followee_id  BIGINT,
  created_at   TIMESTAMPTZ,
  PRIMARY KEY (follower_id, followee_id)
);

-- Tweets
CREATE TABLE tweets (
  id          BIGINT PRIMARY KEY,         -- snowflake / sortable
  author_id   BIGINT,
  text        TEXT,
  media_ids   BIGINT[],
  created_at  TIMESTAMPTZ
);

-- Timeline (cached, materialized — see fanout below)
CREATE TABLE timelines (
  user_id   BIGINT,
  tweet_id  BIGINT,
  ts        TIMESTAMPTZ
);

The core question: how do you build the home timeline?

There are two ways to approach this, and understanding why one fails for each end of the spectrum is the heart of the interview.

Fanout on read (pull)

flowchart LR
    U[User opens app] --> APP[App]
    APP -->|"1. who do I follow?"| F[(follows DB)]
    APP -->|"2. for each followee,<br/>get latest tweets"| T[(tweets DB)]
    APP -->|"3. merge + rank"| RES[Timeline]
    style F fill:#ff6b1a,color:#0a0a0f
    style T fill:#ff6b1a,color:#0a0a0f

Build the timeline at read time: get all followees, query their recent tweets, merge. Writes are trivially cheap — one row. The problem is reads. If you follow 500 people, you're fanning in 500 tweet streams on every refresh. At scale, this blows the latency budget before you even get to ranking.

Fanout on write (push)

flowchart LR
    A[Author posts tweet] --> KAFKA[Kafka:<br/>NewTweet event]
    KAFKA --> WORKER[Fanout Worker]
    WORKER -->|"for each follower,<br/>insert into their timeline"| TC[(Timeline cache:<br/>per-user list)]
    style KAFKA fill:#a855f7,color:#fff
    style TC fill:#15803d,color:#fff

When a user posts, immediately write the tweet ID into every follower's pre-built timeline. Reads become trivially cheap — just read your own timeline. The cost is write amplification: one tweet becomes N Redis writes where N is your follower count. For a normal user with a few hundred followers, this is fine. For Justin Bieber, it's 100M writes.

The hybrid (what real systems do)

Neither pure approach survives at full scale. The answer is to use fanout-on-write for the vast majority of users, and fanout-on-read for accounts above a follower threshold (~10k). When you load your timeline, the Timeline Service merges both sources:

flowchart LR
    U[User loads feed] --> APP[App]
    APP --> A1[1. Read pushed timeline<br/>from Redis]
    APP --> A2[2. Read celeb tweets<br/>fanout-on-read]
    A1 --> M[Merge + rank]
    A2 --> M
    M --> U
    style A1 fill:#15803d,color:#fff
    style A2 fill:#ffaa00,color:#0a0a0f

This is roughly what Twitter's "Manhattan + cache" stack does, and what Instagram's feed system does.

Architecture

flowchart TD
    USER[User] --> CDN[CDN / Edge]
    CDN --> APIGW[API Gateway]
    APIGW --> AUTH[Auth Svc]
    APIGW --> POST[Tweet Service]
    APIGW --> TL[Timeline Service]
    APIGW --> FOL[Follow Service]
    APIGW --> MED[Media Service]

    POST --> TWDB[(Tweets DB<br/>sharded by tweet_id)]
    POST --> KAFKA[Kafka: NewTweet]
    KAFKA --> FAN[Fanout Workers]
    FAN -->|"non-celeb writes"| TLCACHE[(Timeline Cache<br/>Redis, sharded by user_id)]

    FOL --> FOLDB[(Follow Graph<br/>sharded by follower_id)]

    TL --> TLCACHE
    TL --> TWDB
    TL --> CELEB[(Celeb Cache<br/>recent tweets per celeb)]

    MED --> S3[(S3 / blob)]
    MED --> CDN

    style POST fill:#ff6b1a,color:#0a0a0f
    style TL fill:#0e7490,color:#fff
    style FAN fill:#15803d,color:#fff
    style KAFKA fill:#a855f7,color:#fff

Storage choices

Data	Store	Why
Users	Postgres	Small (millions), needs strong consistency
Tweets	Cassandra / Manhattan / sharded MySQL	Massive scale, write-heavy, simple access pattern
Follow graph	Sharded MySQL or Cassandra	Read-heavy, denormalized
Timeline cache	Redis	In-memory, sorted-set perfect fit
Media	S3 + CDN	Big, infrequent change, cheap
Search	Elasticsearch	Full-text

Tweet IDs — Twitter's snowflake

Tweet IDs need to be globally unique, sortable by time (so timelines read in order), and fit in 64 bits. Twitter's answer is Snowflake:

| 1 bit | 41 bits       | 10 bits     | 12 bits  |
| sign  | timestamp_ms  | machine_id  | sequence |

The 10-bit machine field is typically split further: 5 bits for datacenter ID + 5 bits for worker ID within the datacenter, giving 32 datacenters × 32 workers = 1,024 unique generator nodes. timestamp_ms (milliseconds since a custom epoch) ensures sortability; machine_id ensures uniqueness without a central counter; sequence handles up to 4,096 IDs/ms per machine (2^12 = 4,096). The 41-bit timestamp field gives ~69 years of range before rollover.

Any generator node can mint an ID without coordination — there is no global counter to contend on. The sorted-by-default property means Redis sorted sets can use the tweet ID itself as the score and you still get time-ordered results. This pattern is widely adopted: Discord uses Snowflake IDs for messages, and many distributed systems use the same principle under different names.

flowchart LR
    TS["41-bit timestamp_ms<br/>~69yr range"] --> SF["Snowflake ID<br/>64 bits"]
    MID["10-bit machine ID<br/>5-bit DC + 5-bit worker<br/>= 1024 nodes"] --> SF
    SEQ["12-bit sequence<br/>4096 IDs/ms per node"] --> SF
    style SF fill:#ff6b1a,color:#0a0a0f
    style TS fill:#0e7490,color:#fff
    style MID fill:#a855f7,color:#fff
    style SEQ fill:#15803d,color:#fff

Hot path: reading your timeline

sequenceDiagram
    participant U as User
    participant API as API Gateway
    participant TL as Timeline Svc
    participant TC as Timeline Cache
    participant CC as Celeb Cache
    participant T as Tweet Cache
    U->>API: GET /timeline?cursor=...
    API->>TL: get_timeline(user_id)
    TL->>TC: ZREVRANGEBYSCORE timeline:{user_id}
    TC-->>TL: tweet_ids
    TL->>CC: read recent tweets from celebs followed
    CC-->>TL: celeb tweet_ids
    TL->>T: MGET tweets
    T-->>TL: tweet payloads
    TL-->>API: ranked, hydrated timeline
    API-->>U: JSON

Every step is in-memory. Net: ~10–50ms.

Hot path: posting a tweet

sequenceDiagram
    participant U as User
    participant API
    participant POST as Tweet Svc
    participant TWDB as Tweets DB
    participant K as Kafka
    participant FAN as Fanout Workers
    participant TC as Timeline Cache
    U->>API: POST /tweets
    API->>POST: create
    POST->>TWDB: INSERT
    POST->>K: NewTweet event
    POST-->>API: 201 Created
    API-->>U: 201
    Note over K,FAN: async
    K->>FAN: deliver
    FAN->>TC: ZADD into each follower's timeline

The user gets 201 in ~50ms. Fanout happens asynchronously and finishes within seconds for most users (longer for power users).

Edge cases & gotchas

The celebrity problem

Justin Bieber has 100M followers. Fanout-on-write means 100M writes per tweet, all hitting the timeline cache at the same moment. At 5 tweets/day, that's 500M Redis writes per day for one user.

The fix: above the follower threshold (~10k), skip the fanout. Instead, the read path queries that celebrity's recent tweets directly and merges them with the user's pushed timeline.

Inactive users

Storing full timelines for users who haven't opened the app in six months is pure waste. Only build timelines for recently active users. When a dormant user returns, compute their catch-up feed on demand with a read-time fanout — they can afford the extra latency since it's a cold session.

Fanout worker crashes mid-batch

A worker fanning out a tweet with 50k followers crashes after writing to 20k timelines. The remaining 30k followers have stale timelines.

Kafka provides at-least-once delivery: when the consumer restarts, it replays from the last committed offset and the same tweet fans out again. ZADD is idempotent — reinserting the same tweet ID with the same score is a no-op — so duplicate deliveries are safe. The worker should commit its Kafka offset only after completing the full fanout batch (or after each paginated chunk for very large followings). For very large followings, paginate in chunks of ~1,000 and checkpoint after each chunk so a crash never re-fans out more than one chunk.

Deletes

A deleted tweet must disappear from millions of timelines. The practical answer is to mark tweets deleted in the tweets DB and filter at read time — cheap, but lets the timeline cache hold IDs that resolve to nothing. For legal or CSAM takedowns, issue a ZREM from each timeline through the same fanout workers via a deletion event — expensive but necessary.

Ranking (vs. pure chronological)

Modern feeds rank by ML — engagement prediction, recency, network effects. The fanout pushes candidates into the cache; ranking reorders them at read time.

flowchart LR
    CACHE[Cached candidates] --> R[Ranker]
    R -->|"calls feature store"| FS[(features per tweet)]
    FS --> R
    R -->|"calls ML model"| MODEL[Ranker model]
    MODEL --> R
    R -->|"sorted timeline"| RES[User]
    style R fill:#ff6b1a,color:#0a0a0f

This is where Meta's Feed and TikTok's For You spend their compute budget. The fanout is the easy part.

Trade-offs to discuss in an interview

Fanout on write vs. read. Explain both, then explain the hybrid. The threshold (~10k followers) is a tunable parameter — different teams draw it differently based on their write cost vs. read latency budget.

Sharding strategy. Tweets are sharded by tweet_id — independent writes, hot tweet reads spread evenly. Timeline cache is sharded by user_id — all of a user's data lands on one shard, enabling fast O(log N) range scans on a bounded sorted set. The follow graph is sharded by follower_id so "who do I follow?" is a local lookup. Sharding by followee_id would help fanout workers (all followers of one celebrity on one shard) but creates a hot shard for celebrities — another reason the celebrity cut-off matters.

Consistency. Timelines are eventually consistent. A brand-new tweet may take 2–10 seconds to appear across all follower timelines, depending on Kafka propagation and worker load. That's fine — social feeds don't need read-after-write consistency. If you did need it (e.g., "I just posted and it should be in my own timeline immediately"), special-case the author's own timeline with a synchronous write.

Cache eviction. Timeline Redis sorted sets are capped at around 800 tweet IDs per user. When a user scrolls past that, the system falls back to the tweets DB. This bounds cache size but means deep pagination is slow — an accepted trade-off.

Cache size. 40TB of hydrated timeline cache is large. Most production systems cache only tweet IDs in the sorted set (~3TB) and hydrate payloads from a separate tweet object cache (Memcached or Redis). A popular tweet's payload is stored once and shared across all follower timelines, rather than stored once per follower.

Things you should now be able to answer

• Why is the read:write ratio of a social feed the dominant design constraint?
• Why does fanout-on-write break for celebrities, and what is the typical follower threshold?
• How does Twitter's hybrid fanout work, and how do you merge pushed + pulled timelines at read time?
• Why is Cassandra (or similar) used for tweets, not Postgres? What access patterns does it optimize?
• What goes wrong if a fanout worker dies mid-batch, and why is Kafka + ZADD a safe recovery strategy?
• Why shard tweets by tweet_id but timelines by user_id? What would break if you swapped those choices?
• How do you handle a tweet delete that must propagate to millions of cached timelines?
• At what point does ranking move from "sort by timestamp" to ML? What does the ranking stage need as inputs?

Further reading

• "How Twitter Stores 250 Million Tweets a Day Using MySQL" — Twitter eng blog (archived, 2011; describes the early MySQL + Redis architecture before Manhattan)
• "Building Timelines at Scale" — Raffi Krikorian, Twitter QCon talk (2012); the original public description of the hybrid fanout approach and the celebrity threshold
• "Manhattan: our real-time, multi-tenant distributed database for Twitter scale" — Twitter eng blog (2014); explains why they moved away from Cassandra
• "Announcing Snowflake" — Twitter eng blog (archived); the original design rationale for sortable 64-bit IDs
• Instagram engineering blog on feed ranking — describes how ML ranking operates on top of a fanout-delivered candidate set