Home
 » ISP News » 
Sponsored

UPD2 Major Broadband Outage in Northampton UK After Business Park Fire

Tuesday, September 24th, 2013 (8:08 am) - Score 4,779
northampton_fire

A huge fire that began on Friday morning at the JBJ Business Park on Northampton Road (England), which took place at a PoP near to the Blisworth Telephone Exchange, has resulted in a major loss of local broadband Internet connectivity. It has also affected telecoms services to 16 other exchanges in the region.

The worst affected customers appear to be those whom typically connect via an unbundled (LLU) line supplied via O2 Wholesale, although related problems (e.g. power outages) are known to have impacted several other ISPs.

The Affected Exchanges: SMOA (OAKLEY),EMWESTO (WESTON FAVELL),EMWELLI (WELLINGBOROUGH),EMTOWCE (TOWCESTER),EMRUSHD (RUSHDEN),EMRANND (RAUNDS),EMNORTH (NORTHAMPTON),EMMOULT (MOULTON),EMKTTER (KETTERING),EMKINGS (KINGSTHORPE),EMHARRO (HARROWDEN),EMHARDI (HARDINGSTONE),EMDUSTO (DUSTON),EMDAVEN (DAVENTRY),EMCRRBY (CORBY),EMCHAPE (CHAPEL BRAMPTON).

On top of that the Fire Brigade initially restricted access to the data centre until the fire had been extinguished “due to health and safety concerns“, which means that BT are unable to access the site. But the fire has now been out for several days and yet access remains closed, which is perhaps due to the presence of “hazardous materials“.

Fire Brigade – Incident Update 20/09/2013 Factory Fire Blisworth

JBJ Business Park, Blisworth, 10:42 hours Friday 20th September 2013 – This incident was a well-developed fire in a unit of 8 industrial units, hazardous materials were involved and there was suspected asbestos in the roof.

At its height over 50 firefighters were in attendance, support was received from Buckinghamshire and Leicestershire Fire and Rescue Services. There is extensive damage to over 50% of the unit, the fire is now out, NFRS will be remaining in attendance overnight.

There were no injuries at this incident and whilst neighbouring properties were advised to keep windows and doors closed at the height of the fire due to heavy smoke, there is no further impact or risk to the local community as the fire is now out.

Resources sent include, 8 Pumping Appliances, 2 Specialist Aerial Appliances, 1 Operational Support Unit and Supporting Appliance, 1 Water Bowser and 1 Incident Command Unit.

Apparently BT is currently searching for an alternative way to re-route the affected circuits and restore service but there’s a limit to what can be done while full access is being restricted. Needless to say that O2’s Service Status update makes for some unpleasant reading.

Elsewhere the BE Usergroup reports that customers as far away as Birmingham are also being affected by problems that link back to the fire. Apparently users with a Static IP address can attempt to resolve the problem by changing to a Dynamic IP instead, although doing so will lose your previous address and you’ll also have to factory reset the router (only works for those in Birmingham).

As it stands now the situation is unchanged and nobody can get access.

UPDATE 25th September 2013

It’s now known that the PoP is Level 3’s network facility, which is used by O2 and several other providers, although Level 3 has not been able to commit to a timescale for the fix.

Meanwhile the latest update from O2 states, “We’ve identified a solution that involves re-routing traffic to an existing data centre. We’re continuing to work with our suppliers and partners to build and install additional equipment and cabling to ensure a stable service from this site.” Once again, no timescale is mentioned.

Apparently most connections have already had their traffic re-routed but Level 3 states that “a limited number of customers” are still “experiencing service disruptions“. A small amount of compensation is being organised for those worst affected.

UPDATE 26th September 2013

Sky confirms that around 3,500 customers on BE/O2’s platform have been affected by the fault and “due to the unusual circumstances, we’ve posted a complimentary 3GB wireless internet dongle to see people through until their home broadband is reactivated.” This is not actually all that unusual and Sky have done it before.

Meanwhile O2 Wholesale said that they expect service to be “fully restored by the end of the weekend“.

Add to Diigo
Mark Jackson
By Mark Jackson
Mark is a professional technology writer, IT consultant and computer engineer from Dorset (England), he also founded ISPreview in 1999 and enjoys analysing the latest telecoms and broadband developments. Find me on Twitter, , Facebook and Linkedin.
Leave a Comment
31 Responses
  1. Avatar Phil

    Pretty bad. If the telephone exchange are destroyed by fire mean they won’t be any broadband for a whole year! No ADSL/ADSL2+/LLU/FTTC/FTTP

  2. Avatar Phil

    And no phone line!

  3. Avatar Stephen Roberts

    Any businesses affected and who can’t wait for BT / O2 / BE to retify the situation might like to consider a temporary satellite broadband link installation from C&R Technologies Ltd 01909 880 798

  4. Admin-thank you for pointing this out. I have asked our web guys to ensure our address (2-3 Basset Court, Broad Street, Newport Pagnell, MK16 0JN)is shown on our web site.

  5. Avatar MikeP

    I don’t know where this idea that the fire site is “adjacent” to the local exchange came from. The online local rag is telling the same story. In reality, Bilston exchange is over 500m from the fire site, as 2 minutes with SamKnows and Google Maps will tell you.

    The truth looks like some critical equipment was installed in a standard business unit with bog-standard (i.e. non-existent) site security and clearly no data-centre-standard fire detection and control equipment. Which is quite mind-blowingly incompetent.

    • Probably O2’s status page being vague about local PoPs vs Exchanges, though the fact that 16 other exchanges have been impacted would suggest that it was rather a significant hub.

    • Avatar Marilyn Burbidge

      In reply to MikeP, how right your comments are, every word of it. We live in the house which is situated just inside this Business Park. The telephone exchange they speak of is actually in the village of Blisworth. The security, you are quite right, there is none. The units are as you put “bog standard” Perhaps I should not be saying all this but this is after having our lives disrupted since Friday, no one and I mean no one has had the decency to come and let us know what is going on. We have had a mobile exchange placed outside our home, being run by a diesel generator, we are told people cannot go back to the units because of hazardous materials! We have had sleepless nights due to noise and our dog being extremely upset with all the constant disturbances, still no one has had the courtesy to inform us of anything, we have only lived here for over 50 years, long before any units went up. We have every sympathy for the unit holders who have lost their livelihoods, but what they should now be looking to, and the worry for us, is when the water to distinguish this fire had to be fetched from Blisworth as one on the firemen told us the waterpressure on this site was not enough, is how to prevent another disaster.

    • Avatar MikeP

      Interesting, Marilyn. It seems there have been quite a few fires on this estate over the years. Does make one wonder. As you say, very worrying if there’s not enough mnains pressure for the firefighters.

      I’m still having huge difficulty understanding how a “Tier 1” (allegedly) Service Provider (Level 3, that is, not O2/Be) could have put a PoP in such a building. I can only think that is was an installation from the Fibrenet days, many years ago, which is fair enough. I can also see why Global Crossing might not have dug deep into the network they were buying, it being the days of the internet bubble and all.

      But Level 3 really ought to have carried out due diligence and also have a proper asset register. Any half-decent audit of their infrasttructure would have screamed at them that this was a major risk to them and their customers.

      It’s more grist to my “be careful of the ‘Cloud’ (whatever that may be)” message. Can you really carry out sufficient due diligence to ensure that your provider really understand their network and can back-up a guarantee that you won’t lose your data ?? Given that some of them take away your ability to get your data out again, at least in a format that you can make use of.

  6. Avatar JNeuhoff

    “It has also affected telecoms services to 16 other exchanges in the region.”

    This is pretty bad. Are there no redundancies for all the other 16 exchanges? Shows how poorly planned BTs networks really are.

    • Avatar MikeP

      This is the O2/Be network that’s borked. Since no other ISP has been affected, it seems it is only themselves with equipment in the burnt-out datacentre light industrial unit. It’s worth repeating that it isn’t a BT exchange building that’s been affected, as Blisworth exchange is around 500m from the fire site.

    • Avatar FibreFred

      Looks like your hatred is misplaced 🙂

    • Avatar JNeuhoff

      @FibreFred: Have you anything constructive to contribute then? E.g. care to explain why 16 other exchanges were affected by this? Surely built-in network redundancy could have helped to minimise the impact of the fire damage, couldn’t it?

    • Avatar FibreFred

      Just another opportunity to slate BT JNeuhoff, as already stated who else is affected? Nothing showing up on the BT service status page. If o2 have decided to build a non-resilient service then that’s their issue, not sure what BT have got to do with it other than them trying to restore some level of service.

      Looks like this was a hub for other exchanges using LLU, the hub goes pop you knock out lots of o2/BE customers.

      o2/BE should have invested in diversity, maybe worth reading up more before blaming the wrong people?

    • Avatar TheFacts

      @JN – you are looking very silly trying to blame BT. Who are actually helping the company to reprovide their service,

  7. Avatar Chris Rayner

    I’m one of those affected – and have been offline since Friday morning (currently sitting outside McDonalds using their free WiFi). The site is nowhere near Blisworth exchange, so it does indeed look like O2 have put equpiment in a “bog standard” industrial unit! No contingency, no business continuity planning – quite simply, they are unfit to be trusted with a customer network.

  8. Avatar BEmused

    Looks like the incompetents known as BE Broadband strike again.

    Seriously hasn’t everyone learned by now these clowns will blame anything and everything on anyone else to avoid taking blame for anything.

    The upside for customers is they probably have noticed little difference as their network with its rubbish latency, constantly dropping connections, lousy VoiP based phone support and general rudeness and ineptness pretty much behaves the same with or without a fire burning their clapped out equipment.

    I wonder what the BE fanboys and girls have to say about this situation?

    No wonder my connection would disconnect several times a day with them and i never had issues with other ISPs, this mob lobs equipment in industrial units and probably connects it with cheese string.

    WAKE UP PEOPLE LEAVE THEM

    WAKE UP RESELLERS STILL USING THEIR WHOLESALE CARP LIKE AAISP DID.

    Let them die a death they deserve or tell Sky to hurry up and assimilate you see you no longer have to put up with the fanboys, the rubbish support, service and equipment.

  9. Avatar MIke Switch

    BE Broadband (piggy back from O2) don’t really care – they have already sold out to SKY 5 days and counting – Single point of failure – disaster !!!

  10. Avatar ISP Network Engineer

    This is (well, was!) in fact a POP (point of presence) for a large fibre provider/carrier. We are customers of said provider and it seems that Be/O2 are too (for backhaul from regional exchanges). All our customer circuits into that POP are down, no ETR yet.

    • Avatar MikeP

      I was coming to this conclusion, with talk of it being Fibrenet/Global Crossing/Level 3 around the interwebs. Makes me wonder if any of their customers were aware of the wholly inappropriate physical environment for the PoP – and if it would even have been possible for any of them to have discovered this before using their network.

  11. Avatar Olorin

    Guys and girls, wake up and smell the bacon. When BE built their network, they were the first and only Unlimited ADSL2+ ISP in the UK. To make this work at an attractive pricepoint, it was necessary to take steps to reduce their cost. Instead of routing each of their exchanges directly to their London core, it seems they’ve routed point-to-point to this industrial estate unit and then down the country from there. Some may say they’ve cut corners, but the fact is they had to design the network this way at the time it was commissioned to charge customers a reasonable monthly fee.

    Here and now in 2013, things have changed. With the surge of Internet use and high bandwidth, large fibre links (1+Gbit) have become so much cheaper to install and run. o2/BE were getting round to making the change to exchange-direct connectivity with their “converged core” rollout, but due to the recent Sky buyout all work on moving to the converged core has stopped.

    I can understand the frustrations of customers affected by this outage and in this day and age it is unacceptable – even of a residential ISP – since many customers rely on their broadband to work from home. Although this outage is BE’s “fault” I can’t say I’m surprised at the outage based on their first offerings as a company.

    If you remember back in 2006 when TalkTalk released their “free forever” broadband, throttling and slow speeds during peak hours was the norm. When BE came along, they had no throttling, no port restrictions and no contention. Nothing has changed much since, which I guess you can call the rise and fall of what was once a ‘good’ ISP.

  12. Avatar cyclope

    @Olorin: Are you sure that Be where the fist to provide ASDL2+ LLU On a national scale ?
    As Easynet built there network on a national scale and started doing this before BE was an ISP
    http://www.samknows.com/maps/llu/23

    O2 Telefonica is what eventually spelled the demise of the BE brand, For those who never had the experience of a better ISP prior to BE /o2 will sing their praises as will those who got lucky by their geographical location and the network topology,they are the ones who got the low latency,and stable connections but the majority didn’t due to their network maping and age of LLU equipment installed at exchanges they had too many customers and not enough resources , in the time i was with them, i saw evidence of congestion and poor peering capacity (remember the BBC fiasco)? They blamed the BBC the weather, the customers lines , everything but what the underlying cause was

    @BEmused +1 had a good chuckle ,nearly choked on my cuppa, keyboard got soaked,lol

    • Avatar Olorin

      No no, there were two other LLU networks to roll out ADSL2+ before BE did. What I said was: “When BE built their network, they were the first and only Unlimited ADSL2+ ISP in the UK.”

  13. Avatar Curious

    I think you will find that Bulldog, the old supplier that Cable and Wireless bought in 2004/5, were one of, if not the fir supplier to offer LLU across a national network….

  14. Avatar Roberto

    UK Online were first to roll out beyond trials…
    http://news.bbc.co.uk/1/hi/technology/4422098.stm

    They had trials that started around the middle of 2004.

    BE broadband rolled out beyond trials about a week after UK Online.

    Bulldog is (or rather its name) is no longer C&W, Bulldog were taken over by Tiscali and later on Tiscali were taken over by Talk Talk back in 2009.

  15. Avatar zee

    Received a text message last night from Be stating “Sky have sent you a 3GB dongle to help you stay connected.”

    • Avatar Roberto

      BE the company that likes to boast its unlimited sending out 3GB dongles. Still i suppose even if you live in a area with no mobile reception it will be better than what they normally provide. 😉

  16. Avatar zemadeiran

    This type of disaster is exactly why I will bring unmetered net to the masses via my earth wide quantum communications network.

Comments RSS Feed

Javascript must be enabled to post (most browsers do this automatically)

Privacy Notice: Please note that news comments are anonymous, which means that we do NOT require you to enter any real personal details to post a message. By clicking to submit a post you agree to storing your comment content, display name, IP, email and / or website details in our database, for as long as the post remains live.

Only the submitted name and comment will be displayed in public, while the rest will be kept private (we will never share this outside of ISPreview, regardless of whether the data is real or fake). This comment system uses submitted IP, email and website address data to spot abuse and spammers. All data is transferred via an encrypted (https secure) session.

NOTE 1: Sometimes your comment might not appear immediately due to site cache (this is cleared every few hours) or it may be caught by automated moderation / anti-spam.

NOTE 2: Comments that break our rules, spam, troll or post via known fake IP/proxy servers may be blocked or removed.
Cheapest Superfast ISPs
  • Hyperoptic £18.00 (*22.00)
    Avg. Speed 30Mbps, Unlimited
    Gift: Code: HYPER19
  • Vodafone £22.00 (*24.00)
    Avg. Speed 35Mbps, Unlimited
    Gift: None
  • TalkTalk £22.45
    Avg. Speed 38Mbps, Unlimited
    Gift: None
  • Direct Save Telecom £22.95 (*29.95)
    Avg. Speed 35Mbps, Unlimited
    Gift: None
  • Origin Broadband £23.00
    Avg. Speed 35Mbps, Unlimited
    Gift: None
Prices inc. Line Rental | View All
The Top 20 Category Tags
  1. BT (2412)
  2. FTTP (2008)
  3. FTTC (1599)
  4. Building Digital UK (1550)
  5. Politics (1345)
  6. Openreach (1338)
  7. Business (1182)
  8. Statistics (1045)
  9. FTTH (972)
  10. Mobile Broadband (969)
  11. Fibre Optic (944)
  12. Ofcom Regulation (882)
  13. Wireless Internet (864)
  14. 4G (851)
  15. Virgin Media (811)
  16. Sky Broadband (576)
  17. EE (558)
  18. TalkTalk (557)
  19. Vodafone (471)
  20. Security (395)
Promotion
Helpful ISP Guides and Tips
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
Sponsored

Copyright © 1999 to Present - ISPreview.co.uk - All Rights Reserved - Terms , Privacy and Cookie Policy , Links , Website Rules , Contact