Questions

Status
Capacity and trafficLevel 6Open topic
Regional quota exhaustion on surgeHardOn-call incidentAmazon styleOn-call incidentAmazon styleHard30 minClient retry amplifies traffic surgeHardOn-call incidentGoogle styleOn-call incidentGoogle styleHard30 minNoisy neighbor starves podHardOn-call incidentOn-call incidentHard30 minShared dependency saturates on surgeHardOn-call incidentNetflix styleOn-call incidentNetflix styleHard30 minFixed-window limiter allows burstHardOn-call incidentStripe styleOn-call incidentStripe styleHard30 minGPU autoscaler CPU metric mismatchHardOn-call incidentOpenAI styleOn-call incidentOpenAI styleHard35 minS3 prefix throttling on burstHardOn-call incidentAmazon styleOn-call incidentAmazon styleHard35 minDDoS volumetric attackHardOn-call incidentOn-call incidentHard40 minTombstone read amplificationHardOn-call incidentOn-call incidentHard35 minGPU memory fragmentationHardOn-call incidentNvidia styleOn-call incidentNvidia styleHard40 minFIFO queue head-of-line blockingHardOn-call incidentAmazon styleOn-call incidentAmazon styleHard40 minGPU memory from large inputsHardOn-call incidentNvidia styleOn-call incidentNvidia styleHard35 min
Shadow traffic overloadHardOn-call incidentOpenAI styleOn-call incidentOpenAI styleHard35 minAI feature far over its daily budgetHardOn-call incidentOpenAI styleOn-call incidentOpenAI styleHard35 min
Databases in productionLevel 6Open topic
Items added to the cart vanish on reloadEasyOn-call incidentShopify styleOn-call incidentShopify styleEasy15 minOrder status updates time out for one warehouseEasyOn-call incidentOn-call incidentEasy15 minCheckout fails with duplicate key errorsEasyOn-call incidentOn-call incidentEasy10 minLogins fail while signed-in users browse normallyEasyOn-call incidentOn-call incidentEasy15 minSeller edits stop showing up in searchEasyOn-call incidentOn-call incidentEasy15 minSingle query latency spikeMediumOn-call incidentOn-call incidentMedium30 minLambda spike connection rejectedMediumOn-call incidentAmazon styleOn-call incidentAmazon styleMedium30 minIndex migration stalls read replicasMediumOn-call incidentOn-call incidentMedium35 minN+1 queries after eager-load removalMediumOn-call incidentShopify styleOn-call incidentShopify styleMedium30 minConnection pool exhaustion from upstream slownessMediumOn-call incidentOn-call incidentMedium35 minIdle transaction locks after deployMediumOn-call incidentStripe styleOn-call incidentStripe styleMedium30 minSlowloris connection-exhaustion attackMediumOn-call incidentOn-call incidentMedium35 minConsumer crash-loops on malformed messageMediumOn-call incidentOn-call incidentMedium30 minWrites disappear on replica read-after-writeMediumOn-call incidentOn-call incidentMedium30 minConnection pool exhaustedMediumOn-call incidentOn-call incidentMedium30 minSchema Registry unavailability crash loopMediumOn-call incidentOn-call incidentMedium35 minDashboard query plan regressionMediumOn-call incidentOn-call incidentMedium30 minConnection reuse regressionMediumOn-call incidentOn-call incidentMedium35 minConnection pool cold startMediumOn-call incidentOn-call incidentMedium35 minConnection pool pinned at maxMediumOn-call incidentOn-call incidentMedium30 minConnection pool saturation under loadMediumOn-call incidentOn-call incidentMedium25 minDatabase connection refusedMediumOn-call incidentOn-call incidentMedium20 minConnection pool holds idleMediumOn-call incidentOn-call incidentMedium35 minWAL directory filling diskMediumOn-call incidentOn-call incidentMedium35 minResponse schema breakMediumOn-call incidentStripe styleOn-call incidentStripe styleMedium30 minIdle connection resetsMediumOn-call incidentOn-call incidentMedium30 minConnection charset mojibake corruptionMediumOn-call incidentOn-call incidentMedium30 minBackfill join logic billing errorMediumOn-call incidentStripe styleOn-call incidentStripe styleMedium35 minPostgres replica CPU spike from unoptimized queryMediumOn-call incidentOn-call incidentMedium30 minPrepared statement errors in pgBouncerMediumOn-call incidentOn-call incidentMedium30 minSearch timeout with type mismatchMediumOn-call incidentOn-call incidentMedium30 minConnection pool idle in transactionHardOn-call incidentStripe styleOn-call incidentStripe styleHard40 minDead tuples block invoice jobHardOn-call incidentOn-call incidentHard35 minMigration floods downstream queueHardOn-call incidentDatabricks styleOn-call incidentDatabricks styleHard40 minFile descriptor leak in new integrationHardOn-call incidentOn-call incidentHard40 minDisk errors after live migrationHardOn-call incidentStripe styleOn-call incidentStripe styleHard45 min