Home
 » ISP News » 
Sponsored Links

Cloudflare are Making Changes to Avoid Breaking the Internet Again in 2026

Monday, Dec 22nd, 2025 (9:13 am) - Score 3,440
Cloudflare-outage-18th-november-2025

The American Content Delivery Network (CDN) and IT service company Cloudflare has committed to make several key changes in order to avoid breaking a significant chunk of the internet again, much as they did on two occasions between November (here) and, to a lesser extent, during early December 2025.

The biggest of the two events occurred on 18th November, when a huge chunk of the internet suddenly became sporadically inaccessible for several hours after Cloudflare pushed out a “wrong configuration” (i.e. a bug in generation logic for their Bot Management feature file) that “took down our network in seconds“.

Part of the problem stems from the difference between how Cloudflare deploys different types of updates. For example, when the company releases software version updates they do so in a controlled and monitored fashion. For each new binary release, the deployment must successfully complete multiple gates before it can serve worldwide traffic (e.g. deploying to staff traffic first and then a phased roll-out).

Advertisement

If we detect an anomaly at any stage, we can revert the release without any human intervention,” said the company’s Chief Technical Officer, Dane Knecht, in a new blog (here). But Cloudflare doesn’t apply the same methodology to configuration changes, which are deployed instantly. “We give this power to our customers too: If you make a change to a setting in Cloudflare, it will propagate globally in seconds,” added Dane.

Cloudflare now acknowledges that the past two incidents have demonstrated that they “need to treat any change that is applied to how we serve traffic in our network with the same level of tested caution that we apply to changes to the software itself“. As a result, the provider has proposed to gradually make a series of changes to address this and to generally improve resilience, so that if an outage does occur again then it’s impact should be much less significant. All of this will fall under a new plan called: Code Orange: Fail Small.

Key Plans for Code Orange: Fail Small

➤ Require controlled rollouts for any configuration change that is propagated to the network, just like we do today for software binary releases.

➤ Review, improve, and test failure modes of all systems handling network traffic to ensure they exhibit well-defined behaviour under all conditions, including unexpected error states.

➤ Change our internal “break glass” procedures, and remove any circular dependencies so that we, and our customers, can act fast and access all systems without issue during an incident.

These projects aim to deliver iterative improvements as they proceed, rather than one “big bang” change at their conclusion. By the end of Q1 2026, Cloudflare expects to be in a position to ensure that all production systems are covered by Health Mediated Deployments (HMD) for configuration management (i.e. releasing config updates in the same way as software updates).

The company will also have updated its systems, by the same target date, to adhere to proper failure modes as appropriate for each product set and to ensure they have the processes in place, so the right people have the right access to provide proper remediation during an emergency.

Advertisement

We understand that these incidents are painful for our customers and the Internet as a whole. We’re deeply embarrassed by them, which is why this work is the first priority for everyone here at Cloudflare,” said Dane Knecht.

Share with Twitter
Share with Linkedin
Share with Facebook
Share with Reddit
Share with Pinterest
Tags:
Mark-Jackson
By Mark Jackson
Mark is a professional technology writer, IT consultant and computer engineer from Dorset (England), he also founded ISPreview in 1999 and enjoys analysing the latest telecoms and broadband developments. Find me on X (Twitter), Mastodon, Facebook, BlueSky, Threads.net and .
Search ISP News
Search ISP Listings
Search ISP Reviews
Comments
11 Responses

Advertisement

  1. Avatar photo Ben says:

    Has Cloudflare got too big? When Cloudflare sneezes, the internet catches a cold…

    1. Avatar photo plunet says:

      Possibly.
      But at least Cloudflare are brutally honest when they screw up, which is more than many of their competitors who get their marketing departments to make vague statements on what went wrong want what they might do to prevent it in future.
      Cloudflare do care more than most and I would think that most of the internet would give them the few months they need to sort this out.
      Most of the other cloud solutions are in the same boat, config changes do not necessarily get a gradual cadence of deployments.

    2. Avatar photo Buggerlugz says:

      Yes, definately. Much like the reliance on Amazon web services. Far too much control in such a small space.

  2. Avatar photo Phil says:

    This is the main reason why is my Cerberus Net Connect Portal Panel went blank page, they told me they have issues with it.

    Why can’t they use it in United Kingdomn cloud dasebase instead of useless United States cloud database?

    1. Avatar photo Rick says:

      Spoken like someone who hasn’t the slightest clue about how technology works. Congrats.

  3. Avatar photo Brian says:

    “If builders built houses the way programmers build programs, the first woodpecker to come along would destroy civilization.” – Gerald Weinberg.

    20 year old quote, still true to this day it seems.

    1. Avatar photo 125us says:

      I don’t think it has any relevance.

      How many people could afford houses if each house had to be designed and built to survive each and every possible combination of natural disaster, attempted burglary, criminal damage, and owner’s terrible DIY? And all of those simultaneously?

      Programmers build things, events and black hats try to break them. The last two have the advantage of infinite time whereas the programmer usually has some kind of release target.

      if you ask one builder to build a house and then give a thousand burglars unlimited time to try and break in and steal your sausage rolls, who do you think wins?

    2. Avatar photo Brian says:

      The point is the Cloudflare outages were NOT caused by hackers, they were minor accidents that crashed the entire global system – much in the way you wouldn’t expect a woodpecker to land on a house instead of a tree.

    3. Avatar photo Ivor says:

      Leaving aside the house analogies, it is probably fair to say that for every website that actually needs DDoS protection, there are many more that are never going to be a target and whose use of CF would only have decreased their overall availability.

      But the industry from which the phrase “no one ever got fired for buying IBM” originated will unnecessarily continue to place eggs in other people’s baskets.

  4. Avatar photo MikeP says:

    Who’d have thought that “configuration as code” meant you needed to test configuration changes the same way you test code changes? 🙂

  5. Avatar photo Steve Parks says:

    I’ll wait to see if this is true.
    Which is isn’t

Leave a Reply

Your email address will not be published. Required fields are marked *

NOTE: Your comment may not appear instantly (it may take several hours) due to static caching and moderation checks by the anti-spam system. Please be patient. We will reject comments that spam, troll, post via known fake IP/proxy servers or fall foul of our Online Safety and Content Policy.
Javascript must be enabled to post (most browsers do this automatically)

Privacy Notice: Please note that news comments are anonymous, which means that we do NOT require you to enter any real personal details to post a message and display names can be almost anything you like (provided they do not contain offensive language or impersonate a real person's legal name). By clicking to submit a post you agree to storing your entries for comment content, display name, IP and email in our database, for as long as the post remains live.

Only the submitted name and comment will be displayed in public, while the rest will be kept private (we will never share this outside of ISPreview, regardless of whether the data is real or fake). This comment system uses submitted IP, email and website address data to spot abuse and spammers. All data is transferred via an encrypted (https secure) session.
Cheap BIG ISPs for 100Mbps+
Community Fibre UK ISP Logo
100Mbps
Gift: None
Vodafone UK ISP Logo
Vodafone £22.00
150Mbps
Gift: None
Virgin Media UK ISP Logo
Virgin Media £23.99
264Mbps
Gift: None
Plusnet UK ISP Logo
Plusnet £24.99
145Mbps
Gift: £140 Reward Card
Youfibre UK ISP Logo
Youfibre £24.99
200Mbps
Gift: None
Large Availability | View All
Promotion
Cheap Unlimited Mobile SIMs
iD Mobile UK ISP Logo
iD Mobile £16.00
Contract: 24 Months
Data: Unlimited
Talkmobile UK ISP Logo
Talkmobile £16.95
Contract: 1 Month
Data: Unlimited
Smarty UK ISP Logo
Smarty £18.00
Contract: 1 Month
Data: Unlimited
ASDA Mobile UK ISP Logo
ASDA Mobile £19.00
Contract: 24 Months
Data: Unlimited
O2 UK ISP Logo
O2 £21.24
Contract: 24 Months
Data: Unlimited
Cheapest ISPs for 100Mbps+
Gigaclear UK ISP Logo
Gigaclear £17.00
300Mbps
Gift: None
Community Fibre UK ISP Logo
100Mbps
Gift: None
toob UK ISP Logo
toob £19.50
150Mbps
Gift: None
Vodafone UK ISP Logo
Vodafone £22.00
150Mbps
Gift: None
Beebu UK ISP Logo
Beebu £23.00
100 - 160Mbps
Gift: None
Large Availability | View All
Promotion
Sponsored

Copyright © 1999 to Present - ISPreview.co.uk - All Rights Reserved - Terms , Privacy and Cookie Policy , Links , Website Rules , Contact