Cloudflare outage caused by bug, not malicious activity

Rafi Mahmood

For awareness

• On 18 November, Cloudflare experienced a major global outage disrupting services such as ChatGPT, X, Dropbox, Shopify, and numerous critical infrastructure websites. Initial speculation suggested a cyber attack due to a spike in unusual traffic.
• Cloudflare confirmed the outage was not caused by an attack, but by a latent bug in its bot mitigation service, triggered after a routine configuration change. The bug caused cascading failures across Cloudflare’s network, resulting in widespread service degradation.
• The incident lasted several hours, with investigation starting at 11:48 UTC and a fix deployed by 14:42 UTC, though residual errors persisted. Cloudflare described the event as its worst outage since 2019 and committed to publishing a detailed postmortem and implementing safeguards to prevent recurrence. This has not been published at the current time.
• Organisations relying on Cloudflare should review business continuity plans, monitor for service disruptions, and consider multi-CDN strategies to mitigate single-provider dependency risks. While not attack-related, the outage highlights the systemic impact of configuration errors in critical internet infrastructure.

Cloudflare Status - Cloudflare Global Network experiencing issues

Dane Knecht 🦭 on X: "I won’t mince words: earlier today we failed our customers and the broader Internet when a problem in @Cloudflare network impacted large amounts of traffic that rely on us . The sites, businesses, and organizations that rely on Cloudflare depend on us being available and I" / X

Rafi Mahmood

UPDATE:

Cloudflare have released a blog post on this, reassuring that the outage was not caused by a cyber attack but by a permissions change in a database system, which led to multiple entries being written into a Bot Management feature file. This file doubled in size and was propagated across the network. The routing software, which relies on this file, had a size limit below the new file size, causing it to fail.

In response to this, Cloudflare have stated they are committed to implementing measures to prevent similar outages in the future. Namely, they are hardening ingestion of such configuration files, enabling more global kill switches for features, eliminating the ability for core dumps/error reports to overwhelm system resources, and reviewing failure modes for error conditions.