Sponsored Links

The worst UK ISP we've come across - Lumison

Hello all,

I'm not sure if any of you will of even heard of this ISP. They are called Lumison and have datacenters in both Milton Keynes and Edinburgh. We are an ITC consultancy firm and support many businesses throughout the UK and therefore manage and advise many companies ITC and data communications. We deal with many ISP's and regularly keep up with ISPreview's news and stats. Anyway, back to Lumison...

You can see from their website they claim to be one of the UKs largest ISPs but we're struggling to find any information that supports this claim. They certainly are not in the top 10 from what we can see. That's just the first and most minor point. if anyone can shed any light we'd be interested to hear from you.

http://www.lumison.net/why-lumison
Established in 1995, Lumison has grown from a local dial up provider into one of the UK’s largest, most trusted full-service ISPs with literally thousands of SME, Corporate and Government clients.

In terms of their service we have found them to be by far the poorest in terms of reliability and support.
While they claim to provide a 24/7 service in terms of connectivity and support we have recently seen them 'fall off the map, and lost ALL forms of communication after power testing in their datacenter.
http://www.lumison.net/products-solutions/hosting/colocation-and-datacentres

The outage:
Whilst testing a power circuit in business hours at their datacenter in Edinburgh a switch failed to operate once the power came back on. The affect of this caused:

The loss of all Leased Lines and broadband services to all their customers.
Their own 'single hosted' website in their datacenter was unavaliable too so no status reports.
All their voice comms were down because again all hosted in their datacenter.
All backup lines were down.
ALL VPN's down.
All voice comms provided to their business clients down.
All forms of communication to their support teams down.
All email services down.
All websites and hosted services down.
All mobile and SMS communication down so again no service status reports or notifications.
all for over 4 HOURS during business hours!

As you can imagine this caused absolute havoc for our clients that rely on their services. Many of our clients suffered significant financial losses due to their inability to trade and communicate costing many of them many thousands of pounds in lost revenue. What made matters worse was that no one was able to contact Lumison to find out what the situation was. This caused even more problems for supporting IT and Comms engineers. Businesses even started checking to see if Lumison had possibly gone into liquidation it was that bad. They fell off the map!

2 weeks later our clients have started to recieve credit notes for the loss of service. One of which kindly informed us of the actual credit amount. Bear in mind that this client pays Lumison in excess of £1000.00 per month for Leased Line and broadband services.

Credit note from Lumison:
Credit for the outage on Wednesday 28th September, covering a period of four hours of
downtime. This has been calculated on an hourly basis per service as per the terms of the Service Level
Agreement.
Total excl VAT: £4.35


You might imagine how our clients are reacting to this!


Lumison's feedback on the outage:
In response to feedback and queries from our customers, we would like to update you on the recent network outage issue at our Newbridge datacentre, the process of applying credit notes to those affected, and the measures we are taking to improve our internal processes on DR and communications.

Credit Notes

Contracted credit notes will be applied to customer accounts automatically over the coming week, in line with your SLAs across effected services. In recognition of the inconvenience caused, we have taken the longest affected service period and rounded it up to 4 hours. The majority of customers were affected for a shorter period of time but we feel this is the fairest approach.

Scheduling of the Testing

We are under regulatory obligation by Scottish Power to perform systems testing during Monday to Friday and are restricted in the timing of that testing. Since the datacentre was built over 6 years ago, these biannual tests have occurred at similar times without incident.

Telephony System

Our phone system is hosted within our Newbridge datacentre and has failovers and multiple network links to a PBX gateway. There is also a backup system which was tested within the last month. Redundancy exists through the system to deal with component and software failures but in this instance, no network traffic was getting out of Newbridge due to the issue.

If we encounter issues like those we experienced on Wednesday, our DR plan states that our telephony system is manually re-directed to our Milton Keynes datacentre to enable our main communication link to remain operational. Initially, our processes for this manual re-direction did not work and we took longer than expected to get a successful number redirection.

The Milton Keynes support line operates at a lower maximum capacity than the Newbridge site, and this affected the ability of some of our customers to access telephone support.

The level of resilience already within the telephony system is under review to expand the level of protection with automated failover.

SMS

As a business, we understand that a widespread outage could interrupt our customers’ ability to reach websites, receive email, and may affect the ability to receive telephone calls. As a result, we operate a text alerting system. This system operates via an internet-based gateway to mobile providers with a secondary gateway that does not use our network, but connects via mobile providers’ GSM network.

Despite undergoing recent testing, the backup did not function as expected during the incident which meant that our communication did not go out as normal. To address this issue, we will be reviewing the out-of-band solution and are planning to upgrade to a more advanced system with greater resiliency in the next few weeks.

Service Updates

For customer service and internal performance reasons, the Lumison website is hosted in our network and is within our Newbridge datacentre. As a result, service updates were unavailable to customers. Plans are in place to split out the service update section of our website and make it available via a secondary, separately-hosted service update facility.

Updates should have been provided through our main Twitter account and we are currently reviewing our DR communications policy to cover the use of social media channels.

We would like to apologise again to customers for the outage, which dropped our standard 100% core network service availability level to 99.95% over the last twelve months. Lumison BlueSquare remains committed to maintaining continuity to ensure our customers have the level and quality of service expected from the business.

Thank you.

Kind Regards.

Aydin Kurt-Elli
Group Chief Operating Officer


So as you can see a total failure of every part of their redundant systems and 100% failure of the Disaster Recovery procedures. FYI their SLA for their Leased Line services provides little to nothing to any of their customers should they fail to provide the service they are charging them for.

Our advice.
Quite obviously avoid Lumison, assuming you've ever heard of them.
Read and understand any SLA that comes with a service.
Assume your ISP does not have adequate Disaster Recovery procedures because what they profess and what is reality could be two entirely different things.

FYI: We have never recommended this ISP. Thankfully ^^
 
You can see from their website they claim to be one of the UKs largest ISPs but we're struggling to find any information that supports this claim. They certainly are not in the top 10 from what we can see. That's just the first and most minor point. if anyone can shed any light we'd be interested to hear from you.

Its a meaningless claim that virtually any ISP could use. If there is an ISP in the country which is smaller than them then they can claim they are "one of the largest"... It makes perfect sense also....... Its a little like the "UPTO" speed claims virtually every ISP uses. They are not claiming they are the 'single largest' only they are larger than some. Ive seen numerous businesses not just internet related use similar claims.
The outage:
Whilst testing a power circuit in business hours at their datacenter in Edinburgh a switch failed to operate once the power came back on. The affect of this caused:

The loss of all Leased Lines and broadband services to all their customers.
Their own 'single hosted' website in their datacenter was unavaliable too so no status reports.
All their voice comms were down because again all hosted in their datacenter.
All backup lines were down.
ALL VPN's down.
All voice comms provided to their business clients down.
All forms of communication to their support teams down.
All email services down.
All websites and hosted services down.
All mobile and SMS communication down so again no service status reports or notifications.
all for over 4 HOURS during business hours!

As you can imagine this caused absolute havoc for our clients that rely on their services. Many of our clients suffered significant financial losses due to their inability to trade and communicate costing many of them many thousands of pounds in lost revenue. What made matters worse was that no one was able to contact Lumison to find out what the situation was. This caused even more problems for supporting IT and Comms engineers. Businesses even started checking to see if Lumison had possibly gone into liquidation it was that bad. They fell off the map!

2 weeks later our clients have started to recieve credit notes for the loss of service. One of which kindly informed us of the actual credit amount. Bear in mind that this client pays Lumison in excess of £1000.00 per month for Leased Line and broadband services.

Credit note from Lumison:
Credit for the outage on Wednesday 28th September, covering a period of four hours of
downtime. This has been calculated on an hourly basis per service as per the terms of the Service Level
Agreement.
Total excl VAT: £4.35


You might imagine how our clients are reacting to this!

How long was the outtage??? Was it for the 4 hours you mention? If so im amazed they gave any refund for such a short period. You wouldnt had got anything from some companies, with or without service assurance agreements in place. Normally the min guaranteed fix time is something like 6 hours.
Contracted credit notes will be applied to customer accounts automatically over the coming week, in line with your SLAs across effected services. In recognition of the inconvenience caused, we have taken the longest affected service period and rounded it up to 4 hours. The majority of customers were affected for a shorter period of time but we feel this is the fairest approach.

Sounds more than fair to me, when compared to other business ISP suppliers ive came across.

4 hours downtime for a serious power outage is a very quick fix turn around.
 
Sponsored Links
@truth4free

If you read the post properly you'll see that it was simply a power test - ie turn off and on test and not as you've assumed was a serious power outage. The power was not lost at all. Even so, powering off circuits to test should be done 'out of hours'. As you suggest you you have come accross various ISPs but I assume you have little understanding of backup\failover\disaster recovery methods for datacenters?
FYI it states in the post twice that the outage was 4 hours. So yes, 4 hours.

Please note these are business Leased Line services costing customers in excess of £10,000 per year, not some home broadband services! If you think that a £4.35 credit note is adequate for the loss of £5,000 to £8,000 lost revenue due to the failure then you are entitled to your opinion. Most ISPs that provide SLAs (Service Level Agreements) for Leased Lines offer recompense in various forms.

Additionally, to claim to be one of the highest in anything surely that means above 50%? otherwise be default you are one of the lowest. i.e the mid point between High and Low is half i.e. 50%.

Did you actually read this post before replying? or are you simply trolling :)
 
What is your SLA for the leased line? What does your contract state if the service does not work?

It would appear from what you've said that they have compensated you in line with the loss of service. If you are that reliant on a single connection, paying for a basic backup ADSL line is a must.

Just because you pay £10k a year, does not mean that it'll never break - if 4 hours outage is going to cost you more than the cost of a phone line and broadband (can be picked up some places for £25 a month at an average), you should make provisions for a backup.

We have used Bluesquare, which are now owned by the Group that also own Lumison (note, MK was a Bluesquare datacentre). There have been no issues since the take over and Lumison are very responsive to any queries we may have.

Matt
 
@truth4free

If you read the post properly you'll see that it was simply a power test - ie turn off and on test and not as you've assumed was a serious power outage.

Err obviously it wasnt though and something power related went wrong or otherwise it wouldnt had been off for 4 hours. Something obviously went wrong beyond their control and they lept into action to fix it ASAP.

Problems can and do occur with any form of comms equipment, if it affects your business or any other business so much you should have backup systems in place. I gather these businesses you are referring to are not the biggest in Britain in which case even without a phone line working, let alone your lease lines the sensible thing to do if it was about to cost you hundreds, thousands, or millions of pounds would had been to nip down to the mobile phone shop and buy a prepay dongle to see you over the few hours things were not working.

The power was not lost at all. Even so, powering off circuits to test should be done 'out of hours'. As you suggest you you have come accross various ISPs but I assume you have little understanding of backup\failover\disaster recovery methods for datacenters?

I obviously have more experience than the companies you support as if i were support them id advise they have additional systems in place should a outage ever occur if its gonna cost them so much business.

FYI it states in the post twice that the outage was 4 hours. So yes, 4 hours.

A short amount of time then.

Please note these are business Leased Line services costing customers in excess of £10,000 per year, not some home broadband services!

Never said they were, even pointed out a SLA arrangement is normally a 6 hour fix.

If you think that a £4.35 credit note is adequate for the loss of £5,000 to £8,000 lost revenue due to the failure then you are entitled to your opinion. Most ISPs that provide SLAs (Service Level Agreements) for Leased Lines offer recompense in various forms.

Id like to know what their SLA actually says because i guarantee there is no way they can guarantee to fix every potential issue in under 4 hours. I can also guarantee it isnt a guaranteed 24/7 fault free service.

Additionally, to claim to be one of the highest in anything surely that means above 50%? otherwise be default you are one of the lowest. i.e the mid point between High and Low is half i.e. 50%.

Not really they say "ONE OF" the largest, if they are larger than someone else they aint the smallest. Though i fail to see what difference it makes or why you are so bothered by the claim, they could equally sound just as good to some selective readers and hearers by saying/quoting "We are one of the smallest and friendliest companies"........
Again it would be a meaningless claim......... Some obviously fall for company spiel though and cant read between the lines eh ;)

Did you actually read this post before replying? or are you simply trolling :)

Nope i read it entirely, a comms company had a short 4 hour outage.
You are claiming some businesses that use their services lost big sums of money, and those businesses disnt have the sense to have redundant systems in place to save those huge sums from such a short outage.......

On top of that you seem to want some huge refund when its highly unlikely their SLA states they will fix things in under 4 hours.

Is that about the sum of it?

Good job they dont subscribe to a BT based business service where i believe the SLA is 8 hours or Virgin Business where i think its 12 hours (may have changed recently). Or indeed one of the other numerous i can mention who dont have sub 4 hour guarantee fix times which you obviously expect.

I could not name a single company that can guarantee sub 4 hour fix times for any and all issues.
 
No ties to the company you mention, no vested interests, nor in winding you up.

I've run a hosting company before, quite small scale - 4 boxes (1 x web, 1 x sql, 1 x email + stats server + DNS1, 1 x backup/web + DNS2)

My operation was not on the scale of the company you describe.

The last of those servers was hosted at a separate data centre. Precisely because it gives secondary DNS so emails don't bounce if data centre 1 goes down, and so I could update the IP for www.mycompany.com straight away so the main site stays up keeping customers informed.

Ticketing was hosted externally, and voice was voice not VOIP so communications remained open.

If the web server goes down, at least email still works. And so on.

Right up until data centre 1 - which was in Florida - had one of the hurricanes, which uprooted a telegraph pole and brought it smashing down on the fuel line that powered the backup generators; main power was already down. About half an hour later the whole data centre went offline, albeit their own site stayed up (same approach with the DNS and backup)

It was down for something like 22 hours, fortunately over a weekend. You do however accept this because it was your own decision to put the equipment in a hurricane disaster area - hosting costs in the UK were so ridiculous back then that UK hosting just was not possible, the cost was four or five times higher.

Sadly of the two office locations (nothing to do with the data centre) one only had access to ADSL - no cable, the other had access to cable only - no ADSL but they were only two miles apart. Having two ADSL links is all well and good until the entire exchange goes tits up as happened, and the ADSL progressively failed over a week until it didn't work at all, something to do with ADSL Max deployment and the profile system up the spout, I think. The exchange is a single point of failure.

I don't think the four hours is all that unreasonable, but it does strike me that better disaster management measures might have been in place to keep at least some kit and services operational. Someone somewhere is probably analysing what "single point of failure" means. If there's a positive, I doubt the company will let this happen again.
 
Sponsored Links
Sadly, it doesnt matter how many times people read about it happening to OTHER people, they still believe their own hype until they get bitten on the arse.

I had this happen with Vispa about 2 years back, they had switched all their calls, INCLUDING STAFF EMERGENCY CALLOUT PAGERS to a voip system; when the servers went down, so did the voip, so no one could call in a fault and no one got called out to fix it; and when staff DID start arriving for work the next morning they STILL couldnt get hold of a lot of the tech people they needed due to the pager issue.
 
Much appreciate the feedback guys. A couple of you have pointed out the necessity of backup lines and of course we do advise our clients to consider DSL backup to get them by should their Leased Lines fail. Some providers we recommend actually provision ADSL failover with Leased Lines. Zen Internet are particularly good in this area.

The issue on this occasion is that Lumison lost ALL services so all backup lines were down too, DSL failover was useless so it can pay to have ADSL backup with a different ISP to cover core network outages, which again we advise.

@truth4free. Sorry but I really can't take your comments seriously as you seem to make your own assumptions and draw your own conclusions. FYI the actual fault was nothing power related as I have stated. When they carried out the power test their core network switch lost it's config and they had no form or failover/spare/redundancy/backup. For an ISP not to have fault tolerance and tested recovery procedures is laughable as is any business that relies heavily on it's datacomms. If you provide data services in any form to paying customers you have a responsibility to your customers and your own business to ensure that services remain reliable and recovery methods are TESTED and regularly. Otherwise you lose credibility, customers, money and your reputation as a good ISP is damaged. I feel you don't appreciate that the outage was due to their lack of adequate resilience and redundancy rather than an event outwith their control. ISO standards dictate this too and had they been ISO accredited this would have been identified in audits. The whole point is that this outage is due to negligence or incompetence or both and if you understand this then you may appreciate our concerns. You fail also to appreciate that tests of this nature should be carried out 'out of hours'.
Do you have any idea how much money Amazon would lose if they were down for 4 HOURS and simply because they had everything running off a single switch with no backup. Surely you're getting the idea now?

Regarding compensation. No one is expecting a huge refund. Everyone knows that outages occur and are often outwith their control. This we appreciate but this instance is due to their control or lack of it. Again you seem to have missed the point.
To make use of your suggestion we could have nipped down to PC World and bought a 3 or 4 thousand GSM dongles to get the users access to the internet XD.

4 hours is a very long time when your business charges by the hour and it's not the 4 hour outage that was the biggest problem alone but the fact that no one was able to contact the ISP - AT ALL!!! So if all your lines are down and you can't contact the ISP and have no idea when services may or may not resume, what do you do? Do you start implementing your own Disaster Recovery procedures? YES of course you do and this costs your company more money again because you have to start sending staff to other locations, get additional IT support in, get backup to the recovery center, get all staff back to work to allow your business to continue. I'm guessing your experience is home and small business sector which I suggest is out of context when relating to this post. I asked the question regarding their claim to be one of the biggest ISPs purely out of general interest, no biggie. ^^

Lastly, when was having 'all your eggs in one basket' and no backup ever a recommended model for ITC datacomms provision? LOL This may work for your clients but it certainly isn't something we recommend nor expect from an ISP. ^^ Thanks for your comments regardless.

@Captain_Cretin . I totally agree ^^

@DTMark - Yes it appears you have more than a common understanding too. We just couldn't believe they had such lack of foresight and common sense ^^

The reason I posted this in the first place was to share this experience in a hope that others may learn something from it.
 
Last edited:
It doesnt matter what the fault was, if it was fixed inside of 4 hours thats good.

What does the Lumisons SLA actually say with regards to fault time resolutions?? Do they say under 4 hours? If not they didnt even have to give any refund.

I agree with you maybe they could handle things differently, maybe they could host various things in various locations. All that is irrelevant though and has nothing to do with your complaint about how long things were down for or what the SLA may say.

EDIT: I did find this bit funny..........
"To make use of your suggestion we could have nipped down to PC World and bought a 3 or 4 thousand GSM dongles to get the users access to the internet XD."

10k seems a pretty cheap service you have there that you were moaning about also if it serves that many people.

Ten grand for a connection that serves more users than a typical telephone exchange?????? BARGAIN IMO
 
Last edited:
It's a fact of life that ISPs do, from time to time, suffer outages as an almost unavoidable consequence of the complicated networks and power reliant systems involved. Some ISPs provide better redundancy than others but if your primary connection does go down then it's always wise to have a secondary one for backup purposes. In a tiny business that could simply be a Mobile Broadband modem but in a bigger one a simple fixed line ADSL link might suffice (needs vary).
 
Sponsored Links
@Mark.J 100% agreed. Thanks for comment.

@truth4free. Am assuming you're trolling or simply missed the point entirely. I won't respond to your comments unless they are educated and relate to the specifics contained within the post. Meh.
 
Im trying to find out what exactly your complaint is because from what i read in multi long winded pages it appears.
1) You are not happy with the fault resolution time, even though 4 hours fix is good and you still refuse to tell any of us what the SLA says
2) The cost you pay in some way matters and you think is a high figure, although you have now told us that 10K serves more people than some entire telephone exchanges.
3) You are unhappy that all the systems went dead, yet the companies being provided by Lumison are so big they didnt bother to check out what redundancy Lumison have in place before signing up. Even though the net is vitally important and a 4 hour outage costs them thousands and these organisations do not have their own backup systems also. My conclusion from that is a company saying the net is important but have not acted as if it is.

Im not trolling im just trying to find out exactly (Please try to be brief) what your complaint actually is. All my posts in this thread have relative and been very specific to points you have made in your posts.
 
Top
Cheap BIG ISPs for 100Mbps+
Community Fibre UK ISP Logo
150Mbps
Gift: None
Virgin Media UK ISP Logo
Virgin Media £22.99
132Mbps
Gift: None
Vodafone UK ISP Logo
Vodafone £24.00 - 26.00
150Mbps
Gift: None
NOW UK ISP Logo
NOW £24.00
100Mbps
Gift: None
Plusnet UK ISP Logo
Plusnet £25.99
145Mbps
Gift: £50 Reward Card
Large Availability | View All
Cheapest ISPs for 100Mbps+
Gigaclear UK ISP Logo
Gigaclear £17.00
200Mbps
Gift: None
Community Fibre UK ISP Logo
150Mbps
Gift: None
Virgin Media UK ISP Logo
Virgin Media £22.99
132Mbps
Gift: None
Hey! Broadband UK ISP Logo
150Mbps
Gift: None
Youfibre UK ISP Logo
Youfibre £23.99
150Mbps
Gift: None
Large Availability | View All
Sponsored Links
The Top 15 Category Tags
  1. FTTP (6026)
  2. BT (3639)
  3. Politics (2721)
  4. Business (2439)
  5. Openreach (2405)
  6. Building Digital UK (2330)
  7. Mobile Broadband (2146)
  8. FTTC (2083)
  9. Statistics (1901)
  10. 4G (1816)
  11. Virgin Media (1764)
  12. Ofcom Regulation (1582)
  13. Fibre Optic (1467)
  14. Wireless Internet (1462)
  15. 5G (1407)
Sponsored

Copyright © 1999 to Present - ISPreview.co.uk - All Rights Reserved - Terms  ,  Privacy and Cookie Policy  ,  Links  ,  Website Rules