MS3 Networks Back Online After Suffering Major Broadband Outage
Hull-based alternative UK network operator MS3, which has deployed their full fibre (FTTP) broadband ISP network across 234,000 premises (207k RFS) in the North of England and connected 18,000 customers (mostly in the Hull and Humber region), appears to have returned to life after suffering a major service outage yesterday. But some users may need to reboot their routers.
The outage, which initially appears to have started a little after midnight yesterday morning, was resolved a few hours later but then returned again. The cause of the issue appears to have been a piece of hardware in their London Data Centre (Telehouse North), which we suspect was probably a core network router.
According to one of MS3’s retail broadband ISPs, Squirrel Internet, “the issue followed a change on the network” that MS3 had to reverse or correct (it’s unclear if this was a configuration change or firmware update gone wrong). Several attempts were made during the day to rectify the issue, but these were not successful, until just before 9pm last night when services were finally restored for the majority of customers.
Advertisement
In the end the problematic hardware was replaced, which resolved the outage, but it took a long day of some drama to reach that point.
A Spokesperson for MS3 said:
“Engineers have restored service to the majority of our customers and are completing final checks. Please accept our sincere apologies for today’s outage, which has impacted around 25% of our partner’s end customers.
We will perform extensive reviews to ensure that the future chance of this happening is reduced.”
However, a few of the network’s customers, spread across various ISPs, were still complaining this morning about an inability to get back online. Related internet providers have been telling customers to reboot (switch on and off) both their broadband router and optical network terminals (ONT) to help unstick any remaining connectivity problems. But the latest update from MTH Networks suggests it may not always be so simple.
MTH Networks Service Update – May 15, 2025 at 8:32am
MS3 services remain stable this morning. Engineers continue to work to resolve a number of customers’ connection issues, where services have not been reinstated. Engineers are dealing with the following remaining connection issues:
– A number of connections have known config issues, engineers are actively troubleshooting and expect to have most services restored this morning.
Mark is a professional technology writer, IT consultant and computer engineer from Dorset (England), he also founded ISPreview in 1999 and enjoys analysing the latest telecoms and broadband developments. Find me on X (Twitter), Mastodon, Facebook, BlueSky, Threads.net and Linkedin.
« SpaceX Proposes Starlink Can Deliver Sat Nav to Complement GPS
Advertisement
Leave a Reply
Privacy Notice: Please note that news comments are anonymous, which means that we do NOT require you to enter any real personal details to post a message. By clicking to submit a post you agree to storing your entries for comment content, display name, IP and email in our database, for as long as the post remains live.
Only the submitted name and comment will be displayed in public, while the rest will be kept private (we will never share this outside of ISPreview, regardless of whether the data is real or fake). This comment system uses submitted IP, email and website address data to spot abuse and spammers. All data is transferred via an encrypted (https secure) session.
Ours went offline at 1:30 and didn’t return at all until about 20:45, verified via BQM.
Squirrels website couldn’t handle the relatively low levels of traffic, 500 error pages, timeouts, errors while they presumably hastily migrated to beefier hosting. It’s a simple site so I guess they were some kind of expensive/slow request for every page load of the status page, it needs caching or something. They seem to resolve it by switching to a more static page.
Trying to find updates was a bit of a pain looking at various ISPs socials, often with different updates. I’d prefer to have known about the MS3 maintenance ahead of time so could plan travel to offices etc.
Seriously ISPs if you had better service status pages and informed users of upcoming maintenance maybe you wouldn’t end up with so many users calling and overwhelming your low number of support staff.
Hopefully MS3 learn from it, it seems there was a single point of failure and they didn’t have a good plan for a rollback or how they could temporarily take the problem router out of action, didn’t seem to verify what they had done early in the morning hadn’t knocked thousands of users offline. I’m not sure they even noticed until the ISPs started telling them at 8am.
There was a post about attempting to rollback the configuration 12 hours after the issue started! Then that they’d lost remote access and had to send an engineer to Telehouse to restart and eventually reconfigure the router which started around 3pm, it wasn’t back up for a further 6 hours so presumably that didn’t work and at some point they decided they needed to replace hardware so I expect we were mostly just waiting for that to get delivered.
I have poor mobile coverage, so tethering was a bit of a struggle however there was recently an approved planning application for a mobile mast nearby, so I think I’ll add cellular failover in the future.