CodingLevel 6
Databases in production
Easynot started
Mediumnot started
Hardnot started
StatusFormatCompany style
Easy
Open psql session blocks order updatesOn-call incidentOn-call incident15 minInserts fail with duplicate key after a data importOn-call incidentOn-call incident15 minLogins fail as Redis hits its memory limitOn-call incidentOn-call incident15 minSearch stops updating when one node's disk fillsOn-call incidentOn-call incident15 minNew pods cannot log in to the databaseOn-call incidentOn-call incident15 minMedium
Cart items disappear on reloadOn-call incident · Shopify styleShopify styleOn-call incidentShopify style30 minSingle query latency spikeOn-call incidentOn-call incident30 minLambda spike connection rejectedOn-call incident · Amazon styleAmazon styleOn-call incidentAmazon style30 minCheckout latency spikes after deployOn-call incidentOn-call incident30 minIndex migration stalls read replicasOn-call incidentOn-call incident35 minN+1 queries after eager-load removalOn-call incident · Shopify styleShopify styleOn-call incidentShopify style30 minConnection pool exhaustion from upstream slownessOn-call incidentOn-call incident35 minIdle transaction locks after deployOn-call incident · Stripe styleStripe styleOn-call incidentStripe style30 minReplica lag stale readsOn-call incidentOn-call incident30 minSlowloris connection-exhaustion attackOn-call incidentOn-call incident35 minConsumer crash-loops on malformed messageOn-call incidentOn-call incident30 minWrites disappear on replica read-after-writeOn-call incidentOn-call incident30 minHard
Connection pool idle in transactionOn-call incident · Stripe styleStripe styleOn-call incidentStripe style40 minSplit brain writes after failoverOn-call incidentOn-call incident40 minDead tuples block invoice jobOn-call incidentOn-call incident35 minMigration floods downstream queueOn-call incident · Databricks styleDatabricks styleOn-call incidentDatabricks style40 minFile descriptor leak in new integrationOn-call incidentOn-call incident40 minDisk errors after live migrationOn-call incident · Stripe styleStripe styleOn-call incidentStripe style45 minKafka leader crash data lossOn-call incident · Databricks styleDatabricks styleOn-call incidentDatabricks style40 minSchema cache skew silent corruptionOn-call incident · Databricks styleDatabricks styleOn-call incidentDatabricks style40 minProtobuf type mismatch partition crashOn-call incidentOn-call incident40 minFile descriptor leak from connection poolOn-call incidentOn-call incident40 minEphemeral port exhaustion from disabled keep-aliveOn-call incident · Stripe styleStripe styleOn-call incidentStripe style40 minPostgreSQL data volume fills overnightOn-call incidentOn-call incident40 min