Major Broadband Outage Hits Openreach Exchange in Glasgow UPDATE2
Customers on various Openreach based UK broadband ISPs (e.g Sky Broadband, BT, TalkTalk etc.) in parts of Glasgow (Scotland) have been left without internet and VoIP (digital phone) connectivity today after a “power incident” (fault) hit the Bridgeton exchange last night, which could take a while to fully resolve.
The city of Glasgow is actually home to around 20 Openreach exchanges and we believe that the Bridgeton site (pictured) is only responsible for connecting around 20,000 premises to various broadband services (these may not all have active / live services). But major faults can sometimes have knock-on effects for surrounding areas too.
Some ISPs, such as BT, seem to be indicating that the problem may not be fully resolved until Thursday evening this week, although Openreach themselves merely say that the fault “may take some time” to resolve (here).
An Openreach spokesperson said:
“A power incident in Bridgeton, Glasgow, last night has had a knock-on effect on our digital network. We’ve had 600 fault reports so far and believe thousands of local households are affected.
Engineers are working to restore services but the nature of the repairs, at a busy central junction, are very complex and may take some time, so please bear with us.
We’re sorry for the unexpected disruption to services meantime.”
End.
UPDATE 2:50pm
We understand that the problem relates to Scottish Power’s underground cables, rather than the exchange itself.
UPDATE 23rd Feb 2021 @ 4:31pm
Openreach has informed us that they are “continuing major repairs in Bridgeton” today after an underground power cable is said to have “exploded.” Engineers expect to have most fibre-based broadband restored by tonight, with some people already back in service (consumer routers may need a reboot too).
However, repairs to the copper network are more complicated, with each line needing to be reconnected individually. It’s still expected this work, affecting a minority of lost services and including some phone lines, won’t be fully finished until late on Thursday.
Andy Baillie, Openreach Patch Manager, told ISPreview.co.uk:
“The power cable explosion was like a hand grenade going off underground, and the damage to one of our main East End cable routes is extensive.
We’re working round the clock to get everyone back online as fast as we can. The repairs are complex but we’re making steady progress. We’re sorry for the ongoing disruption and thank residents for their patience and understanding while we work on the fix.”
Just to give readers a flavour of this..
Mark is a professional technology writer, IT consultant and computer engineer from Dorset (England), he also founded ISPreview in 1999 and enjoys analysing the latest telecoms and broadband developments. Find me on X (Twitter), Mastodon, Facebook and Linkedin.
« Vodafone UK Extend Free Mobile Data to 150,000 More School Kids
Latest UK ISP News
- FTTP (5581)
- BT (3530)
- Politics (2552)
- Openreach (2311)
- Business (2284)
- Building Digital UK (2253)
- FTTC (2050)
- Mobile Broadband (1991)
- Statistics (1799)
- 4G (1681)
- Virgin Media (1639)
- Ofcom Regulation (1472)
- Fibre Optic (1406)
- Wireless Internet (1399)
- FTTH (1382)
The joys of moving to full fibre and losing essential status of copper telephone lines as there’s no backup battery anymore either.
Pretty sure even if copper were primarily still in use, the lack of any mains power at the exchange for an extended period of time would still cause a fairly major issue…
There is definitely a backup battery for both FTTC/Gfast and exchange side incl ADSL.
The issue might be the capacity of the battery, which will inevitably be limited, and if there is a backup generator and how much fuel there is in the tank and can more be brought in.
Even if there is a generator there needs to be a UPS battery so the systems don’t die as the generator spools up then stablises and takes load. In a more modern UPS the battery is designed for peak loadings and the generator average loadings.
In the older exchanges the generators were designed to power the community in the exchange, canteen offices etc, as well as the phone lines (and old school exchange innards) so they were very, very big and there were usually three of them in the big exchanges so there was a spare to cycle for prolonged running: cold war & power/coal strike thinking. This presents a problem as these kind of units actually don’t like being run at low load and are monumentally fuel hungry by modern standards.
Imagine knocking OR, when in fact a better headline would be ‘Scotish Power Outage hits broadband customers’.
Man, I saw Glasgow and Openreach and thought it was some long awaited FTTP news 🙁
All BT exchanges have emergency generators for this kind of situation to power Essential services customers. Hospitals, police etc.
fuel costs money, Openreach probably thought : yeah but think of how much we can save by not turning the generators on. like when we lost power in london a few weeks back, I thought, no problem my laptop has a fully charged battery and so does my phone and I’ve got a PD powerbank that can power both.. except that EE was down too.
That would be the ones with diesel tanks marked DO NOT FILL?
I suspect they have been taken out of service in favour of big battery packs.
There is a mantra that the grid supply is so reliable that backups are no longer needed in some places. And even where there are backups they don’t have a very long duration.
@The Moaner Thats why i use three. they use Virgin Media for their backhaul and theyve always worked for me during a power blackout 🙂
I know certainly that in South Yorkshire the back up generators are serviced and tested regularly.
does it run on coal dust from down t’pit ?
UPDATE 2:50pm
We understand that the problem relates to Scottish Power’s underground cables, rather than the exchange itself.
Umm, this is a prime example that OR is not managing their inferstucture correctly, this failure was and is totally manageable.
How do we know when when the detail is not known?. From what I understand some how high capacity mains cables came into contact and exploded knocking out a main cable route (presumably fibre connections). Presumably the power and environment initially had to be made safe but least OR have the resource and skills to divert when required.
The issue is for all providers is that as we centralise and have longer fibre lengths the resilience of all services needs to be thought through.
Are you a telecoms infrastructure professional? Are you a Scottish Power Worker? Are you on site and know all the details? or………………are you in your parents basement playing Minecraft?
Doesn’t look like anything OR could have had any control over. https://twitter.com/WeAreOpenreach/status/1364156091211386880
Here’s some piccies of the problem folks.
https://news.stv.tv/west-central/power-cable-explosion-caused-internet-outage-for-thousands?top
An underground power cable exploded taking out nearby services including digital services cables. The quote is “its like a grenade went off”. I’m not sure what Openreach could have done.
What ? Major exchange and no capacity to engage standby-generator and isolate mains power and engage 4G router back-up.
Bet the cable’s is older than Flora MacDonald’s knitting.
More “World beating” infrastructure.
Needless to say, London started on this trail 15 years ago !
The exchange didnt loose power. The explosion damaged cables outside .
“Just to give readers a flavour of this……”
Yes, the ususal standing around with hands in pockets etc.
That was them looking busy 🙂
back in the van as soon as the camera went away.
This appears to be a trunk exchange & head end. They usually have diverse routes into the exchange building.
Granted this was out of Openreach control, but did ethernet customers, particularly those on diverse circuits get affected?
The need for resilience (especially in a post covid world) is crucial.
Think as with the VM outage around Christmas the big pain point was that this blew a bunch of access network.
Neither copper lines or the fibre feeding cabinets and FTTP customers are resilient.
There’ll be diverse routes though it wouldn’t be unknown for a ‘diverse’ circuit pair to actually be in either the same or neighbouring ducts regardless of the operator.