Data lake columnar storage
Your task
Design the storage layer for a data lake holding petabytes of analytical data as columnar (Parquet) files on object storage, queried by Spark/Presto-style engines. Workloads are append-heavy ingestion plus large analytical scans that read a few columns across billions of rows, with occasional row-level updates/deletes (GDPR erasure). How do you organize files, make queries fast, and support mutation on an immutable object store?
It covers
- Data lake
- Columnar
- Parquet