Library — Failure Modes — Failure Modes

/

sort

newsletter

Get the next incident.

One production failure case study every week. Read the pattern before it shows up in your own system.

4 results · filtered

GitHub

id

incident

org

date

duration

severity

tags

The Search Layer That Slowed GitHubA concentrated wave of anonymous scraping traffic saturated the load-balancing tier in front of GitHub Search, causing timeouts across issues, pull requests, repositories, Actions, packages, and Dependabot alerts.

searchscrapingload-balancing

The Silent Merge Queue Corruption That Hit 658 GitHub ReposA half-gated feature flag let an unreleased merge-base path escape into squash merge groups. Over a 4h38m impact window, GitHub's merge queue produced valid-looking commits that silently reverted prior work across 658 repositories and 2,092 pull requests.

merge-queuefeature-flagsquash-merge

The Runner Cache Bug That Queued Ubuntu CI JobsA backend cache misconfiguration after failover caused duplicate GitHub Actions job assignments, reducing Ubuntu-24 runner capacity for public repos.

cischedulerfailover

43 Seconds of Split-Brain at GitHubA 43-second network partition between GitHub's East and West Coast sites tripped automatic failover. By the time the partition healed, both coasts had taken writes and reconciling the split took most of a day.

databasemysqlreplication