The library. Every incident, structured.

A growing archive of public postmortems, broken down into a consistent shape: what broke, why it cascaded, and what to take from it. New incidents added regularly.

28+
incidents
11+
years
13
organizations
/
sort

newsletter

Get the next incident.

One production failure case study every week. Read the pattern before it shows up in your own system.

3 results · filtered
topic: dns
id
incident
org
date
duration
severity
tags
FM-017
The DNSSEC Failure That Made .de Look FakeIncorrect DNSSEC signatures for Germany's .de top-level domain caused validating resolvers to reject .de answers, leading Cloudflare to temporarily bypass DNSSEC validation for the zone.
Cloudflare
2026-05-05
2h 47m
SEV-2
denicde-dnssecdns
FM-020
The DynamoDB DNS Race That Emptied US-EAST-1A DynamoDB DNS automation race emptied the US-EAST-1 regional endpoint, then cascaded into EC2 launch, NLB, Lambda, and console failures.
AWS
2025-10-19
14h 32m
SEV-1
dnsautomationdynamodb
FM-004
The Day Facebook Deleted Its Own Route to the InternetA backbone command issued to assess global capacity unintentionally took down all of Facebook's backbone. The audit tool that was supposed to block such a command had a bug, and the DNS that announced Facebook to the world withdrew itself in response.
Facebook
2021-10-04
~6h
SEV-1
bgpdnsnetworking