Columnar table mutations

DesignHard50 minDatabricks and Snowflake style

Your task

Design how a columnar lakehouse table format (think Delta/Iceberg-class, immutable column files on object storage) supports row-level UPDATEs and DELETEs and GDPR 'delete this user' without rewriting whole multi-GB files on every change. Constraint: column files are immutable on the object store, queries scan them directly, and you need ACID semantics (a reader sees a consistent table version) plus reasonable read performance after many small deletes. Design the mutation mechanism, how reads reconcile it, and the compaction that keeps reads fast.

It covers

  • Delete vectors
  • Merge on read
  • Acid on immutable files