Home
 » ISP News » 
Sponsored

UPD BT Committed to Tackling UK Network Congestion Woes Raised by AAISP

Wednesday, February 26th, 2014 (2:10 am) - Score 2,051

BT Wholesale’s incident helpdesk manager has told the boss of broadband provider AAISP, Adrian Kennard, that they’re “fully committed to ensuring that all capacity issues are addressed” after concerns over alleged network congestion on several of BT’s 21st Century Network (21CN) telephone exchanges were raised.

The situation sprang to life after BT’s Chief Network Architect, Neil McRae, initially described AAISP’s claim to have witnessed several weeks’ worth of potentially congestion related problems (e.g. packet loss) on the operators 21CN telephone exchanges in Coventry, Hainault, Southwark, Canonbury and Loughton as “rubbish“, before later adding “there are no BT exchanges at all that have any congestion“.

Naturally it didn’t take long for Kennard to post some illustrations of the problem and blog about the issue (here), which later erupted into a lengthy Twitter row with McRae. Kennard also indicated that his ISP was willing to swap affected customers on to TalkTalk’s uncongested wholesale platform in order to avoid the problem.

Adrian Kennard, Director of ISP Andrews & Arnold, told ISPreview.co.uk:

Congestion can happen – what matters is how deal with it and BT are clearly letting us down in that area. Even paying BT extra for elevated weighting does not help. This level of loss slows data transfer significantly. On an idle line this loss shows that BT are not reliably delivering the 80 BITs per second of our monitoring packets, far below that needed for VoIP or many other protocols to work without issues.”

The problem, which follows shortly after BTWholesale’s earlier remark that “3% packet loss is not considered as a fault” on the operators hybrid fibre superfast broadband (FTTC) lines (here), has also been spotted by other ISPs that make use of the 21CN platform. But the good news is that, following their very public spat, the situation now seems to have been recognised.

According to BT, the “issue that is currently causing packet loss and slow speeds to end users is with the backhaul links being over utilised” (i.e. ports or lag groups trying to send more than 100% of their capacity). However it’s not known whether this issue affects all of the above named telephone exchange areas or only some of them and we are awaiting clarity on that (expect an update later).

But in most cases BT said that they are now working towards resolving the problem by installing new hardware (fibres and line cards) and are also investigating whether or not it’s possible to provide a temporary fix for related lines until a permanent solution can be implemented.

UPDATE 3rd March 2014

Here’s the official BT response (this actually arrived at the end of last week but it missed our end of play on Friday), albeit in the form of a more general statement than a reply to the specific issue above.

A BTWholesale Spokesperson told ISPreview.co.uk:

BT Wholesale continues to invest in and monitor its network closely. According to Ofcom, we currently deliver the best network performance in terms of throughput and minimising packet loss and jitter. We aim to maintain that position and if there is any evidence of a dip in network performance we will seek to address it as soon as possible.

We have invested in upgrading our WBC/WBMC network and have recently taken steps to further reduce the impact of short-term spikes in network traffic. We have also launched new services which deliver increased throughput SLAs for our fibre broadband products.

In order to identify ways of improving capacity management and performance, we use a wide range of monitoring mechanisms, including data from our own network and feedback from our customers. If there are some areas where network performance appears to be adversely affected, we would urge our communications provider customers to tell us and to share any end customer data which may help us to identify and address any issues as quickly as possible.”

Add to Diigo
Tags: , ,
Mark Jackson
By Mark Jackson
Mark is a professional technology writer, IT consultant and computer engineer from Dorset (England), he also founded ISPreview in 1999 and enjoys analysing the latest telecoms and broadband developments. Find me on Twitter, , Facebook and Linkedin.
Leave a Comment
29 Responses
  1. Avatar Raindrops

    Looks like Mr Mcrae has finally done the decent thing and hushed. Also looks Mr Mcraes tweets have been deleted/hidden and his tweets are now only available to followers LOL.

    I particularly liked the highlights where he basically called people “stupid” accused RevK of having no facts to back up what he was saying as well as other noteworthy gems.

    Hopefully it is the last we have heard of him on twitter and sites like this.

    At least with the other equally useless BE when AAISP also had issues, they did not make it as public as this became.

    I guess the only thing left is to see will BT actually fix the issue, or will they just shift the congestion elsewhere.

  2. Avatar Chris Conder

    Until the infrastructure has serious upgrades this problem will continue. You can’t make a silk purse out of a sow’s ear. BT are simply milking the obsolete copper assets and will do until everyone is screaming. Then they’ll move into ‘content’ and chuck the whole thing back at government to bale them out.
    you watch.
    We need fibre now. Not in another decade. Pouring millions into copper patchups and marketing it as ‘fibre broadband’ is the biggest mistake ever made. All political parties have fallen for it. Bring on the fibre. Moral and Optic.

    • Avatar Gadget

      Err – Chris I know its your “cut and paste” comment, but in this case I think the problem is with the fibre backhaul…..{“issue that is currently causing packet loss and slow speeds to end users is with the backhaul links being over utilised”] which may have escaped your notice in the article

    • Avatar Unknown101

      Lol, this article has nothing to do with copper it’s talking about back haul being over utilised. FTTP/C would only make this worse by utilising more backhaul between exchanges.

    • Avatar Somerset

      Another classic comment with zero knowledge.

    • Avatar Ignitionnet

      I have long wondered whether you actually read the articles properly or just skim them then recite the same buzzwords.

      I need wonder no more.

  3. Avatar Gadget

    Perhaps Mark J could make it easier by changing the item picture from Openreach engineer to a BTWholesale logo ???

  4. Well the thing is, ultimately Openreach are the ones that would send engineers out to fix the practical side of this issue. So an Openreach chap looking at his laptop screen seemed close enough.

  5. Avatar Matthew

    I suspect BT management would have had something to say about Neils arguing with customers in public…

    I guess how quickly this gets fixed will depend on how much work is required to put it right, if they’re lucky it might be that they already have spare unlit capacity/unused ports and they can simply add another port to the LAGG.

    If not it might take a while as it might need new kit or more fibre pulling in.

  6. Avatar DTMark

    While you could argue that broadband can never be as key a resource as, for example, electricity, that seems to follow a similar model where one company presides over the infrastructure and remains responsible for fixing issues. They may outsource but the line of accountability is clear.

    Given the essential nature of that/gas/water, I thought that the reason ‘infrastructural’ failures were fixed so quickly was because there was some sanction for continued loss of service.

    What is the sanction paced on BT for congestion issues?

    For example I was just reading a thread about a customer whose exchange is ‘congested’ and even though they’re with a top rated ISP – in such circumstances, it might not be too much of an exaggeration to say that the customer might as well be with Primus (or, BT) and save themselves money.

    This is the nature of a vertical monopoly. When there is no competition, it would seem to suggest that stronger sanctions – or indeed any sanctions – might correctly incentivise the infrastructure operator.

    • Avatar Gadget

      remembering that many CPs order backhaul from alternative suppliers such as COLT, Virgin, GEO

    • Avatar DTMark

      .. where available.

      So if an ISP is supplying over BT Wholesale and it’s letting them down, and they move customers to an alternative: leaving aside the overheads at their end, are they in any way out of pocket for doing so?

      For instance, have they already paid BT Wholesale (or, whomever) in advance?
      Do they get a refund?

    • Avatar Ignitionnet

      This is about backhaul from exchanges, not within the ISP networks so their options are limited. This is about the network carrying the PPPoE traffic to them, so whomever they purchase from needs connection to the Openreach network via CableLink.

      That leaves BT Wholesale and, when the full product is released, TalkTalk Wholesale as the two big options. I’m not sure if Vodafone/C+W offer FTTC via a wholesale product though they could if they chose.

  7. Avatar adslmax

    Does anyone have a snapshot of Neil’s tweets?

    • Avatar DTMark

      I suspect that Adrian Kennard [AAISP] may have a copy.

      It does raise one particular issue which is that of vested interests which abound in this industry.

      Whether someone posting is working for a particular company with a particular vested interest (New Londoner, for instance). If so, the name used ought to specify e.g. [BT] to indicate same. People may then weigh those comments appropriately.

      A bit like the way in which articles on the Telegraph website really ought to make clear that they have been written by Tory HQ and simply copied and pasted in with a few changes to make them look like news as opposed to propaganda.

      If people are posting personal views in a personal capacity then that also ought to be made clear.

    • Avatar JNeuhoff

      DTMark: I am sure posters like New_Londoner or FibreFred either work for BT, or are BT shareholders, or otherwise have strong business links with BT (other than being a VDSL broadband customer).

      Anyway, a similar backhaul issue for our local area has been known to us for years, and we provided evidence of it to our old ISP years ago, nothing ever happened, so eventually we gave up using BT altogether. Unfortunately, not many customers have the option, nor the knowledge, of choosing a different provider with an alternative backhaul network.

    • Avatar No clue

      Well at least one of the BT fan aliases this week admits they are a shareholder…
      http://www.ispreview.co.uk/index.php/2014/02/kc-says-71-brits-consider-broadband-speed-house-hunting.html#comment-131885

      A couple of others with a bit of googling claim they are engineers/ex-engineers (one in this item in addition to Mr Mcrae). In general though they appear to do nothing but roam sites to generally annoy and defend BT.

      The latest outburst and deletion from Mr Mcrae just demonstrates the mentality.

      You would think with such a network issues and all the bashing they have previously given Virgin over congestion on here and TBB they would have better things to do with their time.

      Then again with 2 stories on this very site recently about communities waiting over a month for complete service outages to be fixed they obviously think their time is better spent ignoring problems and roaming the internet looking for an argument.

  8. Avatar Ignitionnet

    I have a suspicion Neil won’t be commenting on this particular story.

  9. Avatar Neil McRae

    Just one clarification – I use a service to delete all my tweets every day – may people use these services. If you google you can find out more about those services.

    • Avatar No clue

      Maybe less time tweeting, forum arguing, googling and more time checking points customers make about service rather than dismissing them and calling them stupid would be time better spent.

  10. Avatar Gerarda

    I cannot see why this is a story. It just follows the normal BT style of definition. Available includes unavailable, superfast passed includes superslow passed, uncongested includes congested. Simples.

  11. Avatar Andy Wright

    I’m in the process of moving lines away from BT backhaul to TT backhaul (Plusnet to AAISP)

    I’ve been seeing packet drop as high as 20% on my line and several neighbours’ lines for about a month. (If anyone’s interested, this is a neighbour’s line yesterday – about 18% loss *when idle*! This repeats daily and is even worse over the weekend. http://www.thinkbroadband.com/ping/share/c6291ef5a970776456005511f3a75b81-26-02-2014.html)

    BT’s stubborn, written-in-stone procedures have hampered all efforts by Plusnet to get this fault fixed. Given that the packet loss is showing on numerous lines it clearly is an Exchange/Network fault.

    Despite evidence of several lines showing loss (from my own monitoring and also Plusnet’s), BT insist it’s a fault on my line. One engineer arrived, agreed it obviously wasn’t the line, but had to go through all the motions and do the line tests regardless.

    He was told by BTW to do a lift-and-shift anyway. That did nothing to cure the packet loss, but did make the line start dropping every few hours. Then a few days later my interleave depth suddenly jumped from 32 to 96 and sync speed dropped from over 5meg to under 4. Even manually tweaking the target SNR cannot bring me back to the speed I had before.

    BT then decided another engineer visit was necessary. I stayed in all day and no engineer arrived. Thanks BT, more wasted time. I never heard from BT why they didn’t show, and despite being chased by Plusnet they’ve never called to a) apologise, or b) re-arrange the visit (not that it’d fix anything anyway).

    So, Mr. McRae, if you’re reading this, please admit your network DOES have problems. 20% packet loss on several lines on the same exchange IS a fault. voip is unusable, transfer rates collapse and the service is not fit for purpose.

    Fortunately my exchange (MYTHR) recently went LLU so I now have a choice. I’m voting with my wallet and moving to TT backhaul courtesy of AAISP. I look after a lot of adsl links for clients and will be moving these away from BT wherever possible over the coming weeks also.

    I’m glad there are people around like Adrian Kennard at AAISP who actually care about quality. Yes it’s going to cost me a heck of a lot more to run 2 lines on aaisp than I’m currently paying, but at least I’ll be paying someone who cares rather than throwing money at BT for something that just plain doesn’t work.

    • Avatar James Sutherland

      I’m not sure whether to be envious or not there; like you, I ended up getting a lift and shift which didn’t solve the problem, but in my case it took a ludicrous five Openreach visits before that was done! (BT Wholesale said repeatedly in writing and then on the phone to Openreach that they were trying to get one performed, but Openreach were adamant each time that it had not actually been properly ordered.)

      The good news, after a month of evasion from BT, is that a change of LNS configuration caused my traffic to be re-routed within BT’s network – no more packet loss, since the new route avoided an apparently-faulty LAC.

      Rather absurdly, LCP echo packet loss when idle meant that an 80/20 FTTC line with perfect sync was failing to deliver even 80 bits per second reliably – yes, *bits* per second: 0.0001% of the claimed speed. Quite how that can be dismissed as “not a fault”… Still, at least the new TTW backhaul apparently avoids this, so the problem can soon be fixed even if BT keeps denying it exists. Maybe if they lose enough customers, the congestion problems will be solved anyway!

    • Avatar Andy Wright

      James,

      Had the same problem here with LCP echos getting lost. I had to increase the timeouts on the ppp client to stop the session dropping regularly because of this. Again, this even happened when the link was idle.

      Ideally I’d have run my 2 bonded lines split across TT and BT for resilience, but if BT can’t even pass LCP reliably there seems little point so both with be on TT.

      I can only dream of FTTC around here – I’m 1.7km straight-line from the exchange, but thanks to the ludicrous route the cables take, the length is 3.7km to the nearest cabinet; far too long to run vdsl. Roughly half the line length is on poles and in winter especially we all have endless faults. It must have cost BT thousands over the years sending out engineers but will they entertain re-routing the cable? Nope, even though long-term it’d save them money.

  12. Avatar Martin Pollard

    I sympathise with Neil here he’s but I reckon his PR senses need honing.

    Neil is obviously working with a budget that is under pressure to provide backhaul to an ever increasing amount of network traffic. There’s only going to be a limited amount of new fibre that can be put into the ground each year (or upgraded lighting equipment) and a need to balance traffic between demand from various exchanges.

    BT being regulated as heavily as they are are in an interesting position, where for regulated items BT is very public but for any other information the culture seems to be to treat it all as confidential. The problem seems to arise when the confidential becomes perceivable to other experts making use of the network. These external experts may not necessarily be able to pinpoint the exact location of the problem but they can pin down where it affects their service. We then end up with the situation where you know the outsiders explanation is not exactly the real issue but related and you can’t explain what’s wrong without releasing commercially sensitive information. The PR mistake here is arguing the semantics rather than just admitting there is an issue with shows X symptoms; detail a date for fix; and move on.

    BT themselves also need to bring down the hammer of doom on any help desk staff who resolve issues beyond their expertise. It might help if they allowed a limited amount of people from within ISPs to skip 1st line and go straight to 3rd line if they have the requisite network expertise. The % packet loss on an idle line was stupid, and dealt unnecessary damage to BT’s reputation that could be avoided if an expert at AA had been able to short-circuit the process and immediately escalate to a peer at BT when the 1st line response proved unsatisfactory.

    Maybe Adrian could be a bit more sympathetic, however I think he’s right to make it public if BT’s response is unsatisfactory. I’m not sure smaller ISPs really have any other weapon apart from messy legal processes.

    • Avatar Andy Wright

      I’d agree with some of this to an extent but the problems are also within BT as well as with their communications with their customers (ie the ISPs).

      The problem with packet loss is plain to see; simply setting up some TBB monitoring graphs immediately showed it affecting several connections on my exchange.

      If I can easily determine BT’s network is dropping 1 in 5 packets on several lines then why on earth can’t BT’s monitoring pick this up? They repeatedly said there was no problem at the exchange or on the network. Either their monitoring is rubbish or they are lying. I can’t see it being the former.

      So, having ignored/hidden/whatever the network problems, they send out engineers to find line faults that don’t exist. The engineer that visited me admitted he hadn’t been told what the problem was, and when I mentioned packet loss said he had no idea what that meant. To his credit he did try very hard to help, spending 3 hours on the job. Of course, this didn’t fix anything as there was nothing he could fix.

      At one point he contacted BTW and had the call on speakerphone. The way BTW spoke to him was disgusting and there seemed to be animosity between the 2 divisions. The BTW guy was totally fixated on it being a speed problem and ignored the engineer every time he mentioned packet loss on an idle line (he was being prompted by me!) and the fact it’s affecting more than just my connection.

      If BTW insist on keeping BTOR in the dark and causing engineers to waste massive amounts of time on pointless visits then what hope is there for us end-users?

      My conclusion was that there’s no hope of this being fixed any time soon so I’m moving away from BTW. Unfortunately not everyone has this choice, and those that do probably don’t realise it.

    • Avatar Ignitionnet

      These bits aren’t really Neil’s responsibility, he’s more of a core network guy rather than transport AIUI.

Comments RSS Feed

Javascript must be enabled to post (most browsers do this automatically)

Privacy Notice: Please note that news comments are anonymous, which means that we do NOT require you to enter any real personal details to post a message. By clicking to submit a post you agree to storing your comment content, display name, IP, email and / or website details in our database, for as long as the post remains live.

Only the submitted name and comment will be displayed in public, while the rest will be kept private (we will never share this outside of ISPreview, regardless of whether the data is real or fake). This comment system uses submitted IP, email and website address data to spot abuse and spammers. All data is transferred via an encrypted (https secure) session.

NOTE 1: Sometimes your comment might not appear immediately due to site cache (this is cleared every few hours) or it may be caught by automated moderation / anti-spam.

NOTE 2: Comments that break our rules, spam, troll or post via known fake IP/proxy servers may be blocked or removed.
Cheapest Superfast ISPs
  • Hyperoptic £21.00 (*25.00)
    Avg. Speed 50Mbps, Unlimited
    Gift: £50 Shopping Voucher
  • TalkTalk £21.95 (*36.00)
    Avg. Speed 38Mbps, Unlimited
    Gift: None
  • Plusnet £21.99 (*34.98)
    Avg. Speed 36Mbps, Unlimited
    Gift: £75 Cashback
  • SSE £22.00
    Avg. Speed 35Mbps, Unlimited (FUP)
    Gift: None
  • xln telecom £22.74 (*47.94)
    Avg. Speed 66Mbps, Unlimited (FUP)
    Gift: None
Prices inc. Line Rental | View All
The Top 20 Category Tags
  1. BT (2539)
  2. FTTP (2260)
  3. FTTC (1680)
  4. Building Digital UK (1618)
  5. Politics (1448)
  6. Openreach (1433)
  7. Business (1261)
  8. Statistics (1115)
  9. FTTH (1107)
  10. Mobile Broadband (1059)
  11. Fibre Optic (978)
  12. Ofcom Regulation (925)
  13. Wireless Internet (921)
  14. 4G (919)
  15. Virgin Media (871)
  16. EE (606)
  17. Sky Broadband (602)
  18. TalkTalk (587)
  19. Vodafone (535)
  20. 3G (418)
Promotion
Helpful ISP Guides and Tips
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
Sponsored

Copyright © 1999 to Present - ISPreview.co.uk - All Rights Reserved - Terms , Privacy and Cookie Policy , Links , Website Rules , Contact