
Network access provider Openreach has revealed that an overnight software update at their Cowes exchange was behind this week’s broadband and phone outage on the Isle of Wight, which is an island that sits just off the south coast of Hampshire in England.
The fault, which impacted multiple broadband ISPs on the network, appears to have started on Wednesday night and continued into Thursday afternoon. Homes and business across parts of both Cowes and Newport were said to have been impacted, with some big local retailers even falling back to cash-only payment methods (Boots also stopped dispensing prescriptions at their stores as they couldn’t access the necessary database).
Openreach finally completed the repairs on Thursday afternoon, when they revealed that the fault had been caused by an unspecified software update at their exchange building in Cowes.
Advertisement
A spokesperson for Openreach said (Island Echo):
“We’re sorry to customers on the Isle of Wight who were affected by today’s broadband outage.
The problem was caused by a fault after an overnight software update at our Cowes exchange. As soon as we identified the issue, our engineers got to work finding the cause, replacing faulty equipment and restoring service as quickly as possible.
The repair is now complete, and customers should be back online.
We’re sorry for the disruption and inconvenience this caused, and we’d like to thank customers for their patience while we fixed the problem.”
Internet outages do occur from time to time and one particularly notable side of this story is how many retailers lacked adequate redundancy for such situations, which might for example have allowed them to fall back to an alternative mobile broadband network.
Advertisement
Many large retailers will have their store connectivity outsourced as a managed service, with an agreed service level including resiliency (which may not include a rapid fallback to an alternative connection). The service level may be expressed in terms of minutes of outage per year (as a figure or as a percentage of 24 hours days or trading hours: between 97.0 and 99.99–4’nines).
The outsourcers may choose to offer service credits for individual store outages instead of providing resiliency, and a retailer may choose to take the risk of outage on trade (e.g. cash only) to a single store (some POS can be configured to accept card transactions without online authorizations but this is increasingly rare).
Lack of ability to process prescriptions is more serious and indicates a potentially inadequate contracted store network service level.
The full fibre in my village seemed to be hit on Friday night by a similar weird outage. Given the Power, Pon and Lan were all lit I assumed some kind of authentication issue.
Maybe it was the same software update?