Broadband ISP Zen Internet Suffers UK Network Outage UPDATE3
Some of Zen Internet’s broadband ISP customers are this afternoon being affected by a “NATIONWIDE OUTAGE“, which appears to have started at around 11:30am today and is currently still ongoing. BT also saw a small increase in complaints at around the same time, but Zen appears to be suffering from the biggest impact.
According to some of those impacted by the problem, their broadband connections are still physically live (i.e. all the router lights are green), but the actual internet connectivity itself doesn’t seem to be functioning or is only partly functional. This usually tends to suggest a problem with routing/peering or possibly a Domain Name System (DNS) issue.
A spokesperson for Zen Internet told ISPreview: “Some of our valued customers may be experiencing an internet outage. Our teams are aware of this and are working to get it resolved asap. We will keep our service alert updated and will also post again across our social media channels as we have updates. Please know that we take this very seriously and we are sorry for any inconvenience this may be causing.”
Advertisement
Zen has also opened a Service Status Page report on the fault, although at the moment it doesn’t add much to the above information.
UPDATE 1:31pm
Some customers have reported that using a Virtual Private Network (VPN) provided a temporary workaround, which further suggests that a routing or DNS issue may be the cause. We should add that the issue also appears to be impacting some of Zen’s leased line users.
UPDATE 2pm
Advertisement
Zen now states that “the cause has been identified and the issue has been resolved,” although if it was a routing or DNS issue, then it can sometimes take a bit of extra time for traffic flows to fully return to normal.
UPDATE 2:47pm
We’ve been informed that the fault was caused by a misconfiguration of a network aggregation link. Zen’s engineering teams are currently performing a deeper root cause analysis.
Mark is a professional technology writer, IT consultant and computer engineer from Dorset (England), he also founded ISPreview in 1999 and enjoys analysing the latest telecoms and broadband developments. Find me on X (Twitter), Mastodon, Facebook and Linkedin.
« Colt Expands Reach to 7 New UK Cities via CityFibre’s Network
Broadband ISP Aquiss Introduce CityFibre Based FTTP Packages »
Latest UK ISP News
- FTTP (5850)
- BT (3605)
- Politics (2656)
- Business (2372)
- Openreach (2372)
- Building Digital UK (2302)
- Mobile Broadband (2088)
- FTTC (2071)
- Statistics (1856)
- 4G (1763)
- Virgin Media (1710)
- Ofcom Regulation (1538)
- Fibre Optic (1444)
- Wireless Internet (1436)
- FTTH (1383)
Im on Zen and not down maybe because im using 8.8.88.
..that or it just didn’t impact your part of their network or the specific services you’re using. Always a bit hard to tell without more specifics from the ISP.
I’m using 1.1.1.1 hardcoded into my router and have still had the issue. It’s been weird, some devices on wifi would’t work at all, versus some devices hardwired would work. Teams calls I could type comments but not receive them.
I use Cloudflare DNS and have had no issues with Teams calls today (and other internet use).
Although, I did notice earlier that some of my downloads were stalling (using winget in PowerShell). Maybe that was related? Didn’t think too much of it as the downloads worked using the browser.
It definitely wasn’t DNS. Small requests were fine, big ones were broken, and I don’t use Zen’s DNS servers. Dropping my MTU to 1450 solved the issue for me shortly after it started.
Guessing you’ve had to increase the MTU again now?
I’ve left it for now – smaller always works, it’s just slightly less efficient. Will restore it later.
I’m also on Zen but use Cloudflare/Google DNS. I haven’t noticed any issues.
My MTU is always below 1500 and I don’t just use Zen’s DNS servers and I have had no issues.
Someone on has set the MTU on a link on a core box to 1500. If I remember rightly the recommended is at least 1546. Why they are doing this during the day is more the issue
Always frustrating to see so little information shared (e.g. on status page) for what was clearly a major outage lasting a good 2-3 hours. As a soon-to-be Zen customer the lack of transparency is worrisome.
Looks like IDNet customers experienced the same thing (based on IDNet status page):
—
Some customers are reporting no access to some sites on the internet. We are investigating.
An immediate fix is to set MTU to 1452.
*****UPDATE*****
Suppliers have advised the network fault has now been cleared. Please accept our apologies for the inconvenience caused.
—
I’d seen mention of IDNet using Zen to some extent, but wasn’t clear how widely/accurate it was. Seems confirmation enough.
Wouldnt surprise me as Zen do Wholesale services
Also use Cloudflare / Google DNS on router at home… Didnt see any issues
To be fair, there are a lot of providers in this market that are many times worse at communicating when services go wrong, including most of the largest players.
Totally agree there are many worse, but that won’t stop me from hoping for better!
Interesting Full Fibre Ltd aka Fibre Heroes experienced a similar national outage last week to the day (11/4) which lasted a couple of hours.
Symptoms were very high packet loss and what packets were getting responses were high latency impacted essentially connection was unusable.
All ISPs on the network no explanation as to what caused it.
Still intermittent – bbc.co.uk has stopped working in the last couple of hours, Yahoo has been inaccessible all day, occasional bursts of packet loss elsewhere. Not DNS, sites resolve, ping and traceroute work but HTTPS doesn’t.
I’m still seeing some issues as of 18:48 though that may be due to something cached in customer routers or changes they’ve made to try and mitigate the situation earlier. I manage a lot of Zen connections remotely and things have been very weird all day. Some services staying connected the whole time where as others on the same broadband connection appearing to be completely offline. I’ve not dug into it in any detail yet, but it certainly looks like misconfiguration somewhere.
Wouldn’t surprise me if they’re implementing anti-piracy and anti-IPTV blocking measures to appease the “Internet oppressor crisis actors E-PEE-L and MAFIAA” and in doing so it’s totally broke their entire network. As we all know, watching IPTV and free streaming movie sites are the most heinous and worst possible crimes that could be happening in the UK right now and must be fully stopped.
Luckily nothing to do with that.
I’m normally on a Manchester gateway with about 4-5ms ping, but since this I’ve been stuck on a London gateway ~18-20ms away. I hope they bring back Manchester, my normal technique of just hitting “reconnect” in the FritzBox isn’t working.
The big failure from my point of view here, is that access to their own status pages was unavailable. (From on or off their ISP service). Status pages would be better hosted via an independent network infrastructure, so they can usefully inform customers during an outage affecting their core systems.
Rant about wording time
“our valued customers”. Of course you’re valued. It goes without saying.
What it implies is, some PR person wrote the statement, not someone techy, which I’d much rather see. I also think it’s bad English to increase your word count.
Pointless rant I know, but I hate it.