FM-001Cloudflare2019-07-02impact 27mSEV-1

The Regex That Burned Every Cloudflare CPU

How a regex with one unbounded subpattern, deployed globally with no canary stage, burned every CPU core in Cloudflare's edge network — and why a global kill switch built years earlier turned a long outage into a 27-minute one.

wafregexcpuedge-network
Based on 1 sources ↓

Every Cloudflare edge server had to make the same promise on every request: inspect quickly, then forward. The Web Application Firewall was part of that promise. It matched HTTP requests against managed rules before traffic reached customer origins. As long as each rule consumed bounded CPU, the WAF behaved like a thin filter. If one rule took unbounded time on common request shapes, inspection stopped being a filter and became the thing that consumed the proxy.

On the afternoon of July 2, 2019, an engineer merged a pull request adding new managed rules aimed at a class of JavaScript-based attacks. The pipeline did what it always did. CI built the rule set, ran the test suite, and approved it for deployment. Six minutes after the build, the rule was on every edge server in the network. Unlike Cloudflare's general software releases, which first reached an internal dogfood point of presence before going global, WAF rules were treated as data and pushed everywhere at once. One of the new rules contained a regex with an unbounded inner subpattern: .*(?:.*=.*). On most strings it matched or rejected quickly. On a class of live HTTP bodies, it entered catastrophic backtracking — the engine explored exponentially many ways the pattern might match before concluding it could not. The WAF's regex engine had no per-evaluation time limit. Every core that processed HTTP traffic ran the rule on the same request shapes and stalled at nearly 100% CPU at the same time. HTTP and HTTPS traffic across the network collapsed within minutes. The blast radius reached Cloudflare's own controls. The dashboard, the API, and the status page all served through the same edge proxy that was now too busy to forward anything. Engineers correlating the CPU spike with the WAF deployment could see the cause, but the usual path to disable the rule depended on the system the rule had just taken down.

What ended the incident was a piece of tooling built years earlier for exactly this category of failure: a global terminate mechanism that disabled the WAF on every edge server without going through the regular configuration path. The team executed it at 14:07 UTC. Two minutes later, traffic and CPU were back to normal. The WAF stayed off while engineers isolated the bad rule, then came back online with the rest of the rule set. Twenty-seven minutes of global edge outage, caused by a single unbounded subpattern in a regex — and a deploy procedure that put rule changes on a fast path that code changes were not allowed to use.

From the first signal to all-clear in 27m.

13:31 UTC
Pull request merged with new WAF rule

An engineer merges a pull request adding new managed WAF rules targeting JavaScript-based attacks. One of the rules contains a regular expression with an unbounded subpattern.

13:37 UTC
CI builds and tests pass

Cloudflare's build system compiles the rule set and runs the test suite against synthetic inputs. The tests pass — checking correctness, not CPU cost — and the rule is approved for production deployment.

13:42 UTC
Rule pushed globally in one wave

The standard procedure for WAF rule changes is to deploy them to the entire network at once. The rule lands on every edge server within seconds.

13:45 UTC
First PagerDuty alert fires

Synthetic checks against the WAF begin failing. CPU usage on cores handling HTTP and HTTPS traffic climbs toward 100% as the regex enters catastrophic backtracking on live request bodies.

13:47 UTC
HTTP traffic collapses worldwide

HTTP and HTTPS error rates spike globally — an 82% traffic drop. The Cloudflare dashboard, API, and status page — all served through the same edge network — also become unreachable.

14:00 UTC
WAF identified as the cause

Engineers correlate the CPU saturation with the just-deployed WAF rule set. Standard rollback paths are slow because the dashboard used to manage rules is itself unreachable through the edge.

14:07 UTC
Global WAF disabled via kill switch

Engineers execute the global terminate — a kill switch built years earlier specifically to disable the WAF without depending on the normal deploy pipeline. The WAF stops evaluating rules on every edge server.

14:09 UTC
CPU and traffic recover

CPU usage drops back to baseline almost immediately. HTTP and HTTPS traffic returns to expected levels worldwide. Total customer impact: 27 minutes.

14:52 UTC
WAF re-enabled after verification

After isolating the bad rule and testing the rest, engineers re-enable the WAF globally with the offending rule removed.

What to take away.

01
Treat rule and config changes with the same caution as code changes.WAF rules, feature flags, and routing rules all execute on production hot paths. A pipeline that requires canary stages for binaries but pushes config globally has a gap that any operator can fall through.
deployment_parityconfiguration_as_code
02
Test regular expressions for cost, not just correctness.Regex engines that backtrack can take exponential time on inputs that look ordinary. CI should run new patterns against fuzz inputs with a hard wall-clock budget.
adversarial_testingcost_bounds
03
Bound every regex evaluation in a hot path.Use a linear-time engine like RE2, or wrap a backtracking engine in a step or wall-clock limit. A regex that exceeds the budget should fail closed, not run forever.
computational_boundfail_closed
04
Build a kill switch for the systems you operate before you need it.The fastest path to recovery was a global terminate built years earlier for exactly this class of incident. Kill switches should exist, be regularly drilled, and operate independently of the normal config pipeline.
break_glass_recoveryoperational_resilience
05
Keep incident-response surfaces off the system in outage.When the dashboard, API, and status page flow through the same proxy that is failing, operators and customers both lose visibility at the same moment. Host break-glass tooling on a path that does not share fate with the data plane.
shared_fatecontrol_plane_isolation

Read the sources.

Details of the Cloudflare outage on July 2, 2019
Cloudflare
next →
FM-003 · The S3 Command That Took Out us-east-1