Designing Twitter: Fan-Out Strategies, the Celebrity Problem, and the Home Timeline

Why Study Twitter?

Twitter (X) serves hundreds of millions of users posting hundreds of millions of tweets a day, but the hard part isn't storing tweets — it's the home timeline: showing every user a feed of tweets from everyone they follow, ranked roughly by recency, fast. The follower graph is wildly asymmetric — most accounts have a few hundred followers, but some have over 100 million — and that asymmetry is what makes this a genuinely hard distributed systems problem rather than a CRUD app.

Requirements Analysis

Functional Requirements

1. Post a Tweet: Users compose tweets (text, media, up to a character limit)
2. Follow/Unfollow: Users build a directed follower graph
3. Home Timeline: See a reverse-chronological (or lightly ranked) feed of tweets from followed accounts
4. Notifications: Get notified of likes, retweets, replies, and new followers
5. Engagement: Like, retweet, reply
6. Search (optional): Search tweets and users

Non-Functional Requirements

High-Level Architecture

Deep Dive: Fan-Out-on-Write vs Fan-Out-on-Read vs Hybrid

When a user tweets, that tweet needs to show up in the home timeline of every follower. There are two fundamentally different ways to make that happen, and the choice between them is the central design decision of this system.

Twitter's follower graph is the reason this matters more here than almost anywhere else: it's a directed graph with extreme in-degree skew. A typical account has a few hundred followers; a handful of accounts have over 100 million. Fan-out cost is driven entirely by follower count (in-degree), so the design has to specifically target the small number of extreme outliers rather than optimize for the "average" account.

Deep Dive: The Celebrity Problem

Any account above a follower threshold (production systems typically use a cutoff somewhere in the tens of thousands to low millions) is flagged as a "celebrity" account and is exempted from the normal fan-out path.

At read time, the Timeline Service does two things: read the user's precomputed timeline cache (populated by push fan-out from regular accounts they follow), and separately fetch recent tweets from any celebrity accounts they follow (a short, bounded list per user), then merge-sort the two sets by timestamp before returning the page.

Deep Dive: Fan-Out Sequence

The write path returns to the user as soon as the tweet is durably stored — fan-out happens asynchronously afterward via the queue, so posting latency is decoupled from follower count entirely, even for regular accounts with tens of thousands of followers.

Database Schema

Tweets Table

CREATE TABLE tweets (
    id            BIGINT PRIMARY KEY,
    user_id       BIGINT NOT NULL,
    content       VARCHAR(280) NOT NULL,
    media_urls    TEXT[],
    reply_to_id   BIGINT,
    retweet_of_id BIGINT,
    like_count    INT DEFAULT 0,
    retweet_count INT DEFAULT 0,
    created_at    TIMESTAMP DEFAULT NOW()
);
 
CREATE INDEX idx_user_created ON tweets(user_id, created_at DESC);
CREATE INDEX idx_reply_to ON tweets(reply_to_id);

Follows Table (Follower Graph)

The follower graph is a directed edge list. Two indexes are needed because the graph is queried in both directions: "who does X follow" (fan-out-on-read merge, following list) and "who follows X" (fan-out-on-write target list).

CREATE TABLE follows (
    follower_id  BIGINT NOT NULL,
    followee_id  BIGINT NOT NULL,
    created_at   TIMESTAMP DEFAULT NOW(),
    PRIMARY KEY (follower_id, followee_id)
);
 
CREATE INDEX idx_followee ON follows(followee_id, follower_id);
 
CREATE TABLE users (
    id              BIGINT PRIMARY KEY,
    username        VARCHAR(30) UNIQUE NOT NULL,
    follower_count  INT DEFAULT 0,
    is_celebrity    BOOLEAN DEFAULT FALSE,
    created_at      TIMESTAMP DEFAULT NOW()
);
 
CREATE INDEX idx_is_celebrity ON users(is_celebrity);

Timeline Cache Table (Conceptual — Redis-backed)

The precomputed timeline isn't a SQL table in production (it's a Redis sorted set or list per user), but modeling it relationally clarifies the shape of what fan-out writes:

-- Conceptual shape of what lives in Redis as a sorted set per user,
-- keyed by user_id, scored by tweet timestamp.
CREATE TABLE timeline_cache (
    user_id    BIGINT NOT NULL,
    tweet_id   BIGINT NOT NULL,
    tweet_time TIMESTAMP NOT NULL,
    PRIMARY KEY (user_id, tweet_id)
);
 
CREATE INDEX idx_user_time ON timeline_cache(user_id, tweet_time DESC);

Notification Delivery Design

Notifications (likes, retweets, replies, new followers) go through the same async queue as fan-out, but fan out to a much smaller, bounded audience (the tweet's author, or a handful of mentioned users) rather than an entire follower list — so the celebrity problem doesn't apply here in the same way.

Caching Strategy

Scaling Challenges & Solutions

Key Takeaways

1. Fan-out choice follows the graph shape: Push is cheap when follower counts are small; pull is cheap when following counts are small. Twitter's skew means neither alone works.
2. Hybrid targets the outliers: Identify the small number of extreme accounts and route only those through the expensive path (merge at read) instead of penalizing everyone.
3. Decouple posting from fan-out: A tweet is durable and visible to its author immediately; fan-out to followers happens asynchronously via a queue.
4. Notifications aren't the same problem as timelines: Bounded-audience events (likes, replies) can afford simpler delivery than unbounded-audience events (a celebrity's tweet).
5. Cache the hot path, accept eventual consistency: A timeline that's a few seconds stale is an acceptable tradeoff for sub-200ms reads at this scale.

Follow-Up Questions to Consider

1. How would you rank the timeline algorithmically instead of purely by recency?
2. How would you handle a user who follows 10,000 accounts (following-count skew, not follower-count skew)?
3. How would you support real-time timeline updates (new tweets appearing without a refresh)?
4. How would you deduplicate retweets and quote-tweets in the timeline view?
5. How would you handle deleting a tweet that's already been fanned out to millions of cached timelines?