LSM compaction for TTL data

DesignHard45 minMeta style

Your task

Design the compaction and reclamation strategy for an LSM-backed store of short-lived data: a session/event store where every row has a TTL of 24–72 hours and is then deleted, sustained ingest of 300k writes/sec, and reads only ever touch recent data (last few hours). The naive setup leaves expired rows and tombstones on disk for days, so disk usage is several times the live dataset and reads slow down wading through dead data. Design how data is laid out and compacted so expired data is reclaimed promptly and read amplification stays low.

It covers

  • Tiered vs leveled
  • Tombstones
  • Ttl reclamation