Questions

Status
Databases in productionCoding · Level 6Open topic
Mesh sidecar cannot assign addressHardOn-call incidentOn-call incidentHard35 minQuery plan degrades after autovacuumHardOn-call incidentOn-call incidentHard35 minOrders lost after failoverHardOn-call incidentStripe styleOn-call incidentStripe styleHard40 minIndex corruption after OS upgradeHardOn-call incidentOn-call incidentHard40 minCheckpoint latency spikesHardOn-call incidentOn-call incidentHard35 minPartition pruning regressionHardOn-call incidentOn-call incidentHard35 minLong query blocking transaction poolHardOn-call incidentOn-call incidentHard40 minBackfill incomplete at code cutoverHardOn-call incidentOn-call incidentHard40 minStreaming clients thundering herdHardOn-call incidentOn-call incidentHard40 minEnum value old client rejectionHardOn-call incidentOn-call incidentHard40 minReplication lag read inconsistencyHardOn-call incidentNetflix styleOn-call incidentNetflix styleHard40 minRace condition in refactored order write pathHardOn-call incidentOn-call incidentHard45 min
Storage node rebalance cascadesHardOn-call incidentOn-call incidentHard40 minStandby promotion stalledHardOn-call incidentOn-call incidentHard45 minMigration holding exclusive lockHardOn-call incidentOn-call incidentHard40 minTransaction ID wraparound approachingHardOn-call incidentOn-call incidentHard45 minInserts fail on month boundaryHardOn-call incidentOn-call incidentHard40 minQuery model embedding mismatchHardOn-call incidentOpenAI styleOn-call incidentOpenAI styleHard35 minNAT connection timeoutHardOn-call incidentOn-call incidentHard40 minElasticsearch OOM from expensive aggregationHardOn-call incidentOn-call incidentHard40 minPlan cache parameter sensitivityHardOn-call incidentMicrosoft styleOn-call incidentMicrosoft styleHard35 minLogical replication apply lag explodesHardOn-call incidentOn-call incidentHard40 minSequence reached maximum valueHardOn-call incidentOn-call incidentHard35 minIndex larger than tableHardOn-call incidentOn-call incidentHard40 minBilling pods crash on start during a node upgradeHardOn-call incidentOn-call incidentHard25 min
Incident responseCoding · Level 7Open topic
TLS certificate expiredEasyOn-call incidentOn-call incidentEasy15 minOne pod fails every request but passes health checksEasyOn-call incidentOn-call incidentEasy15 minOne API key sends a third of all trafficEasyOn-call incidentOn-call incidentEasy15 minProduct images vanish after a storage cleanup ruleEasyOn-call incidentOn-call incidentEasy20 minHundreds of orders go through at zero dollarsEasyOn-call incidentOn-call incidentEasy15 minCache stampede on aligned expiryMediumOn-call incidentOn-call incidentMedium30 minInventory shows in stock intermittentlyMediumOn-call incidentShopify styleOn-call incidentShopify styleMedium30 minGC pauses after serialization refactorMediumOn-call incidentOn-call incidentMedium30 minJVM warmup tail latencyMediumOn-call incidentOn-call incidentMedium30 minFeature flag cache staleness after kill-switchMediumOn-call incidentOn-call incidentMedium30 minInternal certificate expiredMediumOn-call incidentOn-call incidentMedium30 minWebhook delivery delayedMediumOn-call incidentStripe styleOn-call incidentStripe styleMedium30 minSMS provider carrier-specific degradationMediumOn-call incidentOn-call incidentMedium30 minMidnight cron stampedeMediumOn-call incidentOn-call incidentMedium30 minConsumer rebalance duplicate chargesMediumOn-call incidentStripe styleOn-call incidentStripe styleMedium30 minLarge message OOM crash loopMediumOn-call incidentOn-call incidentMedium30 minMulti layer cache price flappingMediumOn-call incidentOn-call incidentMedium30 minLeaked AWS key unauthorized accessMediumOn-call incidentAmazon styleOn-call incidentAmazon styleMedium35 minCredential stuffing account takeover waveMediumOn-call incidentCoinbase styleOn-call incidentCoinbase styleMedium35 minMFA bypass impossible travelMediumOn-call incidentOn-call incidentMedium35 minDotenv file exposedMediumOn-call incidentOn-call incidentMedium35 minHardcoded API key leakMediumOn-call incidentOn-call incidentMedium35 minSIM swap account drainMediumOn-call incidentCoinbase styleOn-call incidentCoinbase styleMedium35 minConsumer lag after deployMediumOn-call incidentOn-call incidentMedium30 minDead-letter queue depth grows steadilyMediumOn-call incidentAmazon styleOn-call incidentAmazon styleMedium30 min