Content-addressed deduplicating blob store

DesignHard45 minAmazon and Microsoft style

Your task

Design a content-addressed (CAS) deduplicating blob store for a backup/file-sync product holding 5 exabytes where the same byte ranges recur heavily across users (OS images, shared docs, common media). Blobs are split into variable-length content-defined chunks; identical chunks must be stored exactly once and shared across millions of users, while deletes must be safe (deleting your file must not corrupt mine). Target: write dedup at ingest, sub-100ms chunk-existence checks, and reclaiming space when the last reference to a chunk goes away.

It covers

  • Content addressed storage
  • Deduplication
  • Reference counting