Notifications & fan-out

The idea

A notification system turns one event into many deliveries. When you post, every one of your followers should learn about it. This one-to-many spread is called fan-out.

You can do the work eagerly at write time — copy the event straight into each follower’s inbox (fan-out on write, or push). Or you can do it lazily at read time — store the event once and let each reader assemble their feed when they open the app (fan-out on read, or pull). One makes writes cheap, the other makes reads cheap. The whole design is choosing where to pay.

See it work

How it works

On each event, decide by the author’s reach. Normal accounts push: the event is copied into every follower’s inbox right away, so their reads are instant. Huge accounts pull: the event is stored once, and readers merge in those celebrity events when they open the app. A read therefore stitches together the pushed inbox and a pull of the few celebrities you follow.

THRESHOLD = 10_000   # follower count above which we switch to pull

def on_event(author, event):
    if author.follower_count < THRESHOLD:
        for f in author.followers:        # push (fan-out on write)
            inbox[f].append(event)        # O(followers) writes
    else:
        celeb_events[author].append(event)  # pull: store once, O(1) write

def get_notifications(user):
    pushed = inbox[user]                  # already-delivered, O(1) read
    pulled = []
    for celeb in followed_celebrities(user):   # merge the few big accounts
        pulled += celeb_events[celeb]
    return merge_by_time(pushed, pulled)  # newest first, de-duplicated

Cost / trade-offs

Push trades storage and write amplification for instant reads. Pull keeps writes and storage tiny but pays on every read. Hybrid routes each author to whichever side is cheaper, so no single celebrity can blow up the write path.

Watch out for

• Celebrity write amplification. Pure push for an account with millions of followers is millions of writes per post — a hot key that can stall the queue. This single case is why the hybrid exists: cross a follower threshold and switch that author to pull.
• Ordering, dedup, and aggregation. Merged push + pull events must sort by time and collapse duplicates. Users want 3 people liked your post, not three separate rows — aggregate by target and actor count before delivery.
• Read-state across devices. Marking a notification read on your phone must clear the badge on your laptop. Read-state is shared state; store it server-side, not per-device, or counts drift.
• Editing or deleting a source event. Once an event is fanned out into thousands of inboxes, those copies are stale snapshots. Deleting or editing the original must invalidate or update every copy, or ghosts of removed posts linger.
• Unbounded inbox growth and thundering herd. Inboxes need a cap or TTL, or they grow forever. And a viral event can wake millions of clients at once — rate-limit, batch, and back off so delivery doesn’t self-DDoS.

Worked example

Normal user posts (push). Alex has 8 followers and posts once. The system writes the event into all 8 inboxes — 8 writes at post time. When each of those 8 followers later opens the app, their inbox is already built: 1 read each, O(1). Reads are instant; the cost was paid up front, once, by the writer.

Celebrity posts (pull / hybrid). Jordan has 5,000,000 followers — above threshold, so the event is stored once (1 write). Nobody’s inbox is touched. When a follower opens the app, their read assembles the pushed inbox plus a pull of the handful of celebrities they follow — say 3 — so the read does 1 + 3 = 4 lookups instead of triggering 5,000,000 writes. The 5M-write spike never happens; the small extra read cost is spread across the readers who actually show up.

Check yourself