Blob store with deduplication
Your task
Design a blob store with global deduplication for a backup/sync product (think Dropbox-style file storage). Constraints: petabytes stored, high duplication across users (same OS images, shared documents), block-level dedup with variable-size content-defined chunking, 10 nines durability, and you must never lose a block that is still referenced. Cover chunking, the dedup index, and reference lifecycle.
It covers
- Deduplication
- Content addressing
- Reference counting