Compute isolation for noisy neighbors

DesignHard50 minSnowflake and Databricks style

Your task

Design compute isolation for a multi-tenant query/analytics platform where 30,000 tenants share a pool of worker nodes and one tenant's runaway query (a cartesian join scanning TBs, or 10,000 concurrent requests) must NOT degrade latency for everyone else, but you also can't afford a dedicated cluster per tenant. This is performance isolation, not data isolation. Discuss the scheduling/quota model, how you detect and contain a noisy neighbor in real time, the trade-off between hard isolation (separate resources) and soft isolation (shared with fairness), and how you keep small tenants responsive.

It covers

  • Noisy neighbor
  • Fair scheduling
  • Tenant isolation