Customers from a number of broadband ISPs on CityFibre’s national Fibre-to-the-Premises (FTTP) network – covering 4.3 million UK premises (4.1m RFS) – appear to be experiencing sporadic service issues this morning, which range from disconnection events to problems with high network packet loss and latency.
The problems appear to have started at around 2:40am this morning and have gradually become more noticeable as people wake up, although the issue doesn’t appear to be affecting all of CityFibre’s connections (users only suffering from the latency/packet loss issue may notice it more in certain apps/services than others). But reports are currently coming in from various places across the country.
CityFibre is understood to have already engaged their “Major Incident Team“, who say they are “currently investigating an incident that is impacting high latency and packet loss across the network. We are working to identify the issue and resolve the issue as quickly as possible“. The next update is expected at around 9:30am.
Advertisement
Just for some context. The service status page for iDNET appears to indicate that 37.5% of their CityFibre connections have been disrupted by the problem, which they say reflects the fact that an “abnormally high number of customers are offline or have had a recent connection drop.”
UPDATE 9:06am
Looking at some of the feedback from customers being affected by all this, some are reporting packet loss reaching around 85%+ (upwards of 10%) and latency times have gone as high as 2000ms (2 seconds). The latter is akin to what you’d expect from an older style of GEO satellite internet connection.
But just this moment there are indications coming in that the issue may be seeing some resolution, although the feedback is still variable.
Advertisement
UPDATE 9:31am
CityFibre has informed ISPs that the issue was resolved at approximately 8:50am. “A configuration issue impacted several services, but the configuration has since been updated, resolving the issue. Thank you for your patience during this time,” said the operator. The operator will now conduct a full investigation into the root cause.
Advertisement
CityFibre’s standard response on social media is “Ask your ISP for updates”. This is not useful when the ISP may only be able provide updates during certain operating hours. I would like CityFibre to provide a public status page.
I think that’s a “you” problem for choosing to use an ISP which lacks the level of support that you require.
Your contractural relationship will be with the ISP, not CityFibre.
I guess at least part of the problem will be that some ISPs will peer at multiple CityFibre sites (like an ISP using BT Openreach would do) and some peering at just one or two with CityFibre linking traffic from other exchanges (like an ISP using BT Wholesale would do)
Some of the status updates would only affect some ISPs and not others, and some ISPs maybe more public than others regarding their network structure.
City fibre should be providing a feed of status updates to the ISP, who then could/should pass these on, maybe automatically or after staff review
Things not looking great for Cityfibre at the moment.
Failing to reach targets, running out of money, poor take up rates, big story in the telegraph yesterday and now more reliability issues.
No doubt they’ll come out and say everything’s going great but it’s clear for all to see, things are looking precarious.
It’s fine for me now, just in time for my 9am work call, so I’m not too bothered.
In absence of a CityFibre status page I found iDNET super useful – their status page just automatically showed how many of their customers had issues based on metrics, and that told me I don’t need to troubleshoot my router before any other ISP actually acknowledged a problem.
So for anything affecting CityFibre’s core, I’d suggest using the iDNET status page as a proxy 🙂
On the other hand Greg Mesch has trousered £5 million in salary in the last 7 years so I would say CityFibre has been pretty successful (if you’re Greg Mesch).
Lets hope JP that your comments are just scare mongering on behalf of interest of, most likely, BT.
I for one want them to succeed just as much as Netomnia.
BT’s legacy GPON has had its day, and a lot of us want symmetric over XGS-PON or better, which BT fail to do, and BT have service issues too and been around a lot, lot, lot longer, thank you.
@anonymous
Try reading The Telegraph’s article which link here:- https://www.telegraph.co.uk/business/2025/03/11/cityfibre-break-bt-broadband-stranglehold-struggled/
Is this the same telegraph who broke news about cityfibre being bought by VM02 and VM02 buying talktalk.
I agree about concerns about CF but talking Telegraph articles as gospel is just as worrying.
When did BT Wholesale last have a fault like this, anonymous?
There were loads of national BT 21CN outages in the early days, lasting for long periods at a time.
For example:
https://www.ispreview.co.uk/story/2012/01/05/service-problems-continue-to-impact-bts-national-uk-internet-and-phone-network.html
https://www.ispreview.co.uk/index.php/2012/12/bt-21cn-broadband-services-struck-by-major-outage-in-southern-england.html
Currently experiencing internet problems on Octaplus with CityFibre. Internet is super slow or doesn’t connect at all, but weirdly Google seems to load. Tried changing DNS servers but that didn’t work. Hopefully, it’s fixed soon.
A&A, as usual, have some interesting updates and example charts (aastatus.net). Looks like a proper network meltdown, but they’re reporting that it’s starting to normalise now.
It is certainly not resolved for Cuckoo on CityFibre.
My “ping -t” to Google DNS shows only 2 in 5 being successful, with others getting 145ms response time.
Compare that to Zen on CityFibre (a neighbour) where response times are less than 10 ms with no loss
Zen seem to be connecting to the FEX’s directly, whereas Giganet / Cuckoo seem to use the national network. I think this issue was the national network setup, so any ISP connected directly to the FEX was unaffected.
Were customers on the CityFibre local access products that hand over at the FEX affected by this? And if so, how were CF getting in the way of what’s meant to be an L2 handover.
I assume it was just National that was affected but the details are hard to come by.
Yeah national only impacting a bunch of FEX. The ISPs connecting locally didn’t have any problems.
From what I was able to see on our network it was only isp that use national interconnects to the cf network.
This means it cannot be and issue at the olt or few layer.
My spider sense is pointing in a direction of a config change that made some of the core network devices push packets via software and not hardware.
We have had an issue with this ourselves when we had a misconfigured mtu.
Just a stab in the dark.