Durable object store

DesignHard50 minAmazon and Google style

Your task

Design the storage layer of a durable object store (S3-class) targeting 11 nines of durability for exabytes of immutable blobs, optimized for cost. Objects range from tiny (KB) to huge (TB), the workload is write-once-read-many, and a fraction of objects are read constantly while most are nearly cold. Design the on-disk layout, the redundancy scheme, the metadata/index, and how you achieve the durability target affordably.

It covers

  • Erasure coding
  • Durability
  • Placement