IoT time-series storage engine
Your task
Design the storage engine of a time-series database optimized for IoT telemetry: 10M devices each writing 50 metrics every 10s (~50M points/sec), almost entirely append-only with monotonically increasing timestamps, occasional late/out-of-order arrivals up to 1h late. Reads are 95% 'latest value for device D' and dashboards over the last 24h for a device group; 5% are large aggregations across all devices for a metric over a month. Design the on-disk layout, write path, compaction, and how you avoid write hot partitions.
It covers
- Lsm
- Compaction
- Hot partition