Home
 » Editorial Article, ISP News » 
Sponsored

Benchmarking Broadband ISP Speed Testers On a 10Gbps Line

Sunday, August 29th, 2021 (12:01 am) - Score 7,344
Symbol max speed meter

At one point or another, most UK people will have probably attempted to run a web-based speed test on their broadband connection, but have you ever wondered how well some of the most popular speed testing services do when confronted by a fully capable 10Gbps (Gigabits per second) service? For a bit of fun, we tried to find out.

Most modern speed tests should, by now, ideally be capable of coping with today’s 1Gbps packages, although some of these services have adapted better to the demands of the gigabit era than offers. Likewise, any problems with loss of accuracy tend to grow the faster your connection goes.

Suffice to say, even if you’re fully setup to conduct the test properly (i.e. via a wired connection and with no other local network load occurring), then some testers will still struggle to deliver a reliable result for 1Gbps connections, while trying to reach the dizzy heights of 10Gbps is perhaps just asking for trouble.

All sorts of reasons can exist for this, such as the location of the server being used to conduct the test (some testers give you the option to change this), the type of test being conducted (single vs multi-thread tests can produce different results), local ISP congestion (consumer capacity is shared to keep prices affordable and thus you won’t always get the top speed) and the test server itself simply being too congested to give an accurate result.

NOTE: See our article – Why Buying Gigabit Broadband Doesn’t Always Deliver 1Gbps – for more context.

However, problems with accuracy aren’t always centred on an under-reporting of performance. For example, a number of people on 1Gbps FTTP lines who have tried Netflix’s tester – Fast.com – have boasted about getting results of 1.2-1.4Gbps, which is despite the fact that such speeds may not even be possible on their setups due to hardware limits

Testing the Testers to 10Gbps

The purpose of this article is thus less about testing the performance of a symmetric 10Gbps broadband line and more about pushing some of the internet’s most popular speed testing providers to their limits, just to see how they cope. In order to achieve this, we teamed up with the former CEO of B4RN, Barry Forde, who kindly helped to conduct the testing.

Barry is one of the very few people in this country, that we know of, to have actually installed a consumer grade 10Gbps Fibre-to-the-Premises (FTTP) service to his home. It’ll be a few more years before the rest of us can even think about joining him, although such packages are already starting to become more common in other countries.

NOTE: If you’re covered by B4RN’s network then you can get a 10Gbps package, provided you’re willing to pay £150 a month and £360 (one-off) installation.

In this setup Barry adopted ZYXEL’s AX7501 router, which was then wired (via a 10GbE LAN port) back to an Intel x550 10G Ethernet card on his PC. All the testing was done during periods of idle load on the local (home) network and over two different time periods – off-peak (around 8am) and peak time (around 8pm). A few tests were conducted a little earlier, and some slightly later, but the results followed the same trends.

Each website speed tester was tested once in the morning and once in the evening for a full week (i.e. a total of 14 tests per site / service across 7 days). The results were then averaged (mean) out below.

Results of the 10Gbps Speedtest Stress Test

➤ B4RN’s Ookla Server (Windows 10 Speedtest.net App)

  Overall Avg (Mean) Offpeak Time Avg Peak Time Avg
Ping (milliseconds) 2 2 2
Download (Mbps) 5151 4572 5730
Upload (Mbps) 1821 1714 1929

NOTE: The first “B4RN” test above is unique from those below because it uses the Windows 10 App from Ookla (Speedtest.net) against B4RN’s own Ookla server, as opposed to the standard Speedtest.net web client that we use later on. In theory the two should roughly match, but Ookla recommend the Microsoft app for high-speed systems, which they say should give better results.

➤ Fast.com (Netflix)

  Overall Avg (Mean) Offpeak Time Avg Peak Time Avg
Ping (milliseconds) 1 1 1
Download (Mbps) 5414 5371 5457
Upload (Mbps) 1443 1400 1486

➤ Speedtest.net (Ookla)

  Overall Avg (Mean) Offpeak Time Avg Peak Time Avg
Ping (milliseconds) 2 2 2
Download (Mbps) 4061 4102 4020
Upload (Mbps) 1604 1594 1615

Thinkbroadband

  Overall Avg (Mean) Offpeak Time Avg Peak Time Avg
Ping (milliseconds) 16 17 16
Download (Mbps) 2049 2052 2046
Upload (Mbps) 361 257 465

➤ Broadbandtest.which.co.uk (Which?)

  Overall Avg (Mean)
Ping (milliseconds) 11
Download (Mbps) 543
Upload (Mbps) 84

➤ Broadbandspeedchecker.co.uk

  Overall Avg (Mean) Offpeak Time Avg Peak Time Avg
Ping (milliseconds) 14 14 14
Download (Mbps) 2047 2043 2050
Upload (Mbps) 4379 4196 4562

➤ Broadbandspeedtest.org.uk

  Overall Avg (Mean) Offpeak Time Avg Peak Time Avg
Ping (milliseconds) 13 12 13
Download (Mbps) 1442 1522 1362
Upload (Mbps) 680 644 715

➤ Uswitch.com/broadband/speedtest/ (Uswitch)

  Overall Avg (Mean) Offpeak Time Avg Peak Time Avg
Ping (milliseconds) n/a n/a n/a
Download (Mbps) 801 828 774
Upload (Mbps) 346 340 352

➤ Speedof.me

  Overall Avg (Mean) Offpeak Time Avg Peak Time Avg
Ping (milliseconds) 37 36 37
Download (Mbps) 598 603 593
Upload (Mbps) 168 176 160

One obvious realisation above is that some testers (e.g. Which?, Speedof.me, Uswitch) aren’t necessarily good enough, yet, to reasonably be used for speed testing a symmetric 1Gbps line, let alone anything faster like 10Gbps. We also saw some consistently unusual results from Broadbandspeedchecker.co.uk, which always returned double the upload rate vs download (no other tester did this).

The closest we got to 10Gbps was generally via Ookla’s testers, as well as Netflix and Broadbandspeedchecker.co.uk too (provided you ignore the upload issue). We know from different testing that Barry’s line can practically hit around 7-8Gbps when the internet is able to feed capacity to his setup at that rate, but otherwise the most capable web-based testers above would typically deliver around 4-5.7Gbps (usually less on uploads).

The fastest single test was recorded on Broadbandspeedchecker.co.uk, which hit 8011Mbps, albeit on upstream rather than downstream. By comparison, the fastest single result from B4RN’s own Ookla (Speedtest.net) server was 6951Mbps (or 5131Mbps via the regular Speedtest.net site) and Fast.com (Netflix) returned 6600Mbps – both downstream. The other testers didn’t even get remotely close to these.

A Few Caveats

One key thing to be aware of above is that hardly any of the tests provided much technical information about how they work (unless you trawl the small print and even then, it can be quite ambiguous). But this is crucial because there can be significant differences between, for example, single-thread and multi-threaded testing.

Some people suggest that a single-threaded test is perhaps more representative of normal real-world use, while multi-threaded is often perceived to be a better way of pushing a connection to its limits, but without more detail it’s difficult to give the results their full context. On the other hand, whether single or multi-thread, we believe the purpose of a tester should always be to try and return a fair indication – so far as it is possible – of what your line can actually deliver, when operating at its best.

On top of that, it was rarely ever clear if the final “average” result being reported by a tester – over the duration that a single speed test was running – had been measured as a ‘median‘ or a ‘mean‘ average, which can make a difference.

Suffice to say that our past warnings about the reliability of depending upon web-based speedtests for accuracy, particularly when testing 1Gbps or faster broadband lines, should continue to be heeded. Indeed, the problem only gets worse the further above 1Gbps you go.

However, the reality is that speeds of well over 1Gbps aren’t going to become common place in the UK for a fair few years and in the meantime those speed testers will, as they’ve always done in the past, continue to evolve and get better. Hopefully, by the time most of us can actually take a 10Gbps package, they’ll all be better equipped to cope.

One final point to make is that this article is intended to be a bit of fun using a fairly anecdotal approach, thus to do it properly we’d ideally need to harness a wider variety of 10Gbps capable lines and over a greater variety of time periods. But since we don’t yet have ready access to more of those connections, then we’ll have to wait a few more years.

Leave a Comment
32 Responses
  1. Tom says:

    Thanks for highlighting this. It’s driven me mad when trying to bat back “my internet isn’t performing at 1gig” queries. Broadbandspeedchecker is one of the least reliable. The most stable for results seems to be the installable windows or Mac app from Ookla. Their website can still be wildly unreliable.

    1. AQX says:

      Agreed. Using Netflix’s website you can falsify the results by starting the test and switching to another tab, it seems to show sometimes double your actual speeds. Example, I am on VM 1Gb and so I should cap 950(max). However, I have seen speeds of 2.7-3Gb/s on Fast.com if I swap tabs. I can really only trust Speedtests desktop application.

    2. adslmax says:

      Samknows speedtest are one of the best reliable one!

    3. Damien says:

      NPerf.com and proofovh.net are also good

    4. AQX says:

      @Max – that’s true if you’re using the /realspeed irl with a Samknows capable hub, their actual Speedtest is horrific and incorrect.

  2. Lucian says:

    Meh, they didn’t test nperf.net

    1. Mark Jackson says:

      I can’t recall everything now, but there were a few reasons why we didn’t test nPerf too. One was that, according to Google’s data, fewer UK people use it than other testers. I think there were also some issues with the UK server selection, and the amount of data we had to manually collect was already getting a bit high. You have to draw a line somewhere, so we had to cut a couple of testers. Maybe next time.

    2. Sam P says:

      I find NPerf extremely reliable compared to speedtest.net and fast.com on my 1gbps connection.

    3. Damien says:

      I use it daily – as I use their app too so we can map 5G

  3. Tommy T says:

    If you put speedtest.net into Google translate you can use Google’s servers and get ridiculous speeds.

  4. chris conder says:

    Just for the record, I have tested B4RN’s 10 gig service too, and my ‘new’ laptop using a 10 gig adaptor beat Barry’s tests. Barry was using his older pc. I made a film showing it working. https://www.youtube.com/watch?v=JYva_NKO-y8 and then another customer with 10 gig got into the spirit of it all and upgraded his home kit and managed even higher tests. I agree, it ia all a bit of fun, and the internet itself has a long way to go before we can even use or test one gig properly, but hey, bragging rights and all that jazz. The ookla app is by far the best tester. Samknows hasn’t got a 10gig tester yet (I have asked them). Vodaphone Manchester server did my best test, some servers are pretty awful but you can test from a few till you find one that isn’t overworked and stick to that. Keith used vodaphone watford. This is the test result from Keith’s super machine. https://b4rn.org.uk/wp-content/uploads/2021/08/ITPS-Ltd-V8-1.jpg

  5. AT says:

    I can see it now “but I’m not getting what I’m paying for” (oh that already happens).

    Not your wifi slowing it down, not your usage, not your brand new cheap crap pc?

    Ever tried an actually download from an actual supported server? Grrr.

    Speed “faults” will never go away.

    1. Buggerlugz says:

      tbh, if I was paying an utterly ludicrous amount for a 10gig connection, that is exactly what I’d expect to receive. Anything less would be unsatisfactory.

    2. CarlT says:

      Better to pay ludicrous amounts for 100 Mbit DIA if you expect 100% all the time.

      I see loss of 10% routinely at peak, going up to 20% on occasion.

    3. Ferrocene Cloud says:

      Carl, even then I’ve still had customers complain they’re not getting full speed to a network on the other side of the globe after we hand it over to the destination network’s transit.

      There’s enough capacity for it to egress our network at the full rate, local access network is fine, and it’s taking a good route, we can’t do anything else!

      Internet routing is a tricky beast. The ISP gets blamed even if the network is fine. A while back one of my customers expected us to raise a fault with CenturyLink/Lumen because it appeared they were losing some traffic, and wouldn’t accept that it would have to be the end customer who pays them – all we could do is politely ask.

      Who would have thought the person paying a company has more rights to ask someone to raise a fault than 2 companies freely cooperating!

    4. Damien says:

      I get over 3.1Gbps on my 3Gbps line with Giganet. Okay it’s £1659 a month but still.

      I am running 151 CCTV cameras in my new venture so I sort of need it. But I can’t get any other service apart from g.fast and even then it’s 160/30 which is no good.

      Yes I also use it for personal stuff – but no one cares as No one is sharing my connection as far as I know. (they don’t care when I do 150TB in a month anyway)

    5. CarlT says:

      Genius parody, Damien!

    6. Damien says:

      Hey just because I don’t get losses and put up with them, does not meant you have to get jealous man..

      You da fool!

    7. CarlT says:

      Ah it was serious, and is the usual suspect. Okey doke.

      I genuinely thought it was parody.

  6. TBB says:

    What’s with the Think Broadband results?

  7. MM says:

    I find cloudflare’s speed testing service to be most reliable:
    https://speed.cloudflare.com/

    1. Tom says:

      Over-reporting by about double for me in the early stages but by the end of the test gives reasonable results but not as accurate as the ookla installable app.

  8. Jahbulon says:

    I would love to see a test downloading from steam/origin etc for example GTA:V which is over 80GB, or cyberpunk at something close to 100GB with all the updates now. I’ve always had 100% of my bandwidth from steam.

  9. James says:

    Pff no nperf best one out there

  10. Jam says:

    The most important test of them all is missing here, a baseline test that shows that his FTTP line, router, and PC can actually achieve 10Gbps. Without seeing this we can’t draw anything from these tests, i.e., the fastest result is simply all his PC could handle.

  11. Mark Jackson says:

    At present, there isn’t a way to establish a solid baseline on Barry’s connection and that’s the problem you have with 10Gbps. But as the article says, we have clocked it at 7-8Gbps before and so we know that capability exists at his end. But what we really need is a router with custom speed testing capabilities (firmware) or for SamKnows to produce a 10Gbps capable white box, which at present they don’t.

    What the article shows is that a lot of testers still struggle to give a fair result for even 1Gbps, let alone anything faster. As we say, this is meant as a bit of interesting fun, and hopefully a nudge for those testers to up their game.

  12. Dr John C Bullas says:

    Hey Mark, what about Star Trinity? Have you checked it out: http://startrinity.com/InternetQuality/ContinuousBandwidthTester.aspx

    Loads of customisation, available online or as an install (free for non-commercial)

    1. CarlT says:

      I’ve used this having seen it on a forum and had it raise an eyebrow. It’s a front end for iPerf and extremely unreliable, presumably in part because there’s no way of knowing what’s at the other end.

  13. Jonathan says:

    Been saying this for at least five years now. Above 1Gbps all speed testers are for jack in my experience. They have got a bit better but they are still now where near capable of actually testing the available capacity on a 10Gbps link.

    1. CarlT says:

      Indeed. While I’m not much above a gigabit I definitely see huge differences.

      Some are clearly on 1 Gbit ports, others are on contended 10 Gb/s networks so someone like Barry with that kind of bandwidth will saturate things.

  14. 10GbFTTP4WalesLot3 says:

    Great read as always @mark

    I spotted a typo you may want to correct
    Second to last paragraph :-
    “..Hopefully, buy the time most of us can..”

    I wish I could “Buy” time but instead I see time go by, bye..

    Cloudflare have a great speedtesting site/service.

    On aside note VM have just passed my front door. Finally, now i maybe able to order a service which can attain speeds higher than 56meg. Anyway, Virgin have just blown through my street i suspect BTO will be passing soon after although i doubt ill be able to order 10GbE packages as a retail consumer just yet.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Comments RSS Feed

Javascript must be enabled to post (most browsers do this automatically)

Privacy Notice: Please note that news comments are anonymous, which means that we do NOT require you to enter any real personal details to post a message. By clicking to submit a post you agree to storing your comment content, display name, IP, email and / or website details in our database, for as long as the post remains live.

Only the submitted name and comment will be displayed in public, while the rest will be kept private (we will never share this outside of ISPreview, regardless of whether the data is real or fake). This comment system uses submitted IP, email and website address data to spot abuse and spammers. All data is transferred via an encrypted (https secure) session.

NOTE 1: Sometimes your comment might not appear immediately due to site cache (this is cleared every few hours) or it may be caught by automated moderation / anti-spam.

NOTE 2: Comments that break our rules, spam, troll or post via known fake IP/proxy servers may be blocked or removed.
Cheapest Superfast ISPs
  • Hyperoptic £15.00 (*25.00)
    Speed 50Mbps, Unlimited
    Gift: None
  • Vodafone £19.50 (*22.50)
    Speed 35Mbps, Unlimited
    Gift: None
  • NOW £20.00 (*32.00)
    Speed 36Mbps, Unlimited
    Gift: None
  • Shell Energy £21.99 (*30.99)
    Speed 35Mbps, Unlimited
    Gift: None
  • Plusnet £22.99 (*38.20)
    Speed 36Mbps, Unlimited
    Gift: £65 Reward Card
Large Availability | View All
New Forum Topics
Cheapest Ultrafast ISPs
  • Hyperoptic £20.00 (*35.00)
    Speed: 150Mbps, Unlimited
    Gift: None
  • Vodafone £24.00 (*27.00)
    Speed: 100Mbps, Unlimited
    Gift: None
  • Community Fibre £25.00 (*29.50)
    Speed: 300Mbps, Unlimited
    Gift: None
  • Gigaclear £26.00 (*54.00)
    Speed: 400Mbps, Unlimited
    Gift: None
  • Virgin Media £27.00 (*51.00)
    Speed: 108Mbps, Unlimited
    Gift: None
Large Availability | View All
The Top 20 Category Tags
  1. FTTP (3489)
  2. BT (3006)
  3. Politics (1921)
  4. Building Digital UK (1914)
  5. FTTC (1882)
  6. Openreach (1817)
  7. Business (1671)
  8. Mobile Broadband (1468)
  9. Statistics (1403)
  10. FTTH (1364)
  11. 4G (1270)
  12. Fibre Optic (1164)
  13. Virgin Media (1157)
  14. Wireless Internet (1149)
  15. Ofcom Regulation (1139)
  16. Vodafone (836)
  17. EE (828)
  18. TalkTalk (760)
  19. 5G (758)
  20. Sky Broadband (744)
Promotion
Helpful ISP Guides and Tips
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
»
Sponsored

Copyright © 1999 to Present - ISPreview.co.uk - All Rights Reserved - Terms , Privacy and Cookie Policy , Links , Website Rules , Contact