At one point or another, most UK people will have probably attempted to run a web-based speed test on their broadband connection, but have you ever wondered how well some of the most popular speed testing services do when confronted by a fully capable 10Gbps (Gigabits per second) service? For a bit of fun, we tried to find out.
Most modern speed tests should, by now, ideally be capable of coping with today’s 1Gbps packages, although some of these services have adapted better to the demands of the gigabit era than offers. Likewise, any problems with loss of accuracy tend to grow the faster your connection goes.
Suffice to say, even if you’re fully setup to conduct the test properly (i.e. via a wired connection and with no other local network load occurring), then some testers will still struggle to deliver a reliable result for 1Gbps connections, while trying to reach the dizzy heights of 10Gbps is perhaps just asking for trouble.
All sorts of reasons can exist for this, such as the location of the server being used to conduct the test (some testers give you the option to change this), the type of test being conducted (single vs multi-thread tests can produce different results), local ISP congestion (consumer capacity is shared to keep prices affordable and thus you won’t always get the top speed) and the test server itself simply being too congested to give an accurate result.
However, problems with accuracy aren’t always centred on an under-reporting of performance. For example, a number of people on 1Gbps FTTP lines who have tried Netflix’s tester – Fast.com – have boasted about getting results of 1.2-1.4Gbps, which is despite the fact that such speeds may not even be possible on their setups due to hardware limits
The purpose of this article is thus less about testing the performance of a symmetric 10Gbps broadband line and more about pushing some of the internet’s most popular speed testing providers to their limits, just to see how they cope. In order to achieve this, we teamed up with the former CEO of B4RN, Barry Forde, who kindly helped to conduct the testing.
Barry is one of the very few people in this country, that we know of, to have actually installed a consumer grade 10Gbps Fibre-to-the-Premises (FTTP) service to his home. It’ll be a few more years before the rest of us can even think about joining him, although such packages are already starting to become more common in other countries.
In this setup Barry adopted ZYXEL’s AX7501 router, which was then wired (via a 10GbE LAN port) back to an Intel x550 10G Ethernet card on his PC. All the testing was done during periods of idle load on the local (home) network and over two different time periods – off-peak (around 8am) and peak time (around 8pm). A few tests were conducted a little earlier, and some slightly later, but the results followed the same trends.
Each website speed tester was tested once in the morning and once in the evening for a full week (i.e. a total of 14 tests per site / service across 7 days). The results were then averaged (mean) out below.
Results of the 10Gbps Speedtest Stress Test
➤ B4RN’s Ookla Server (Windows 10 Speedtest.net App)
Overall Avg (Mean) Offpeak Time Avg Peak Time Avg Ping (milliseconds) 2 2 2 Download (Mbps) 5151 4572 5730 Upload (Mbps) 1821 1714 1929 NOTE: The first “B4RN” test above is unique from those below because it uses the Windows 10 App from Ookla (Speedtest.net) against B4RN’s own Ookla server, as opposed to the standard Speedtest.net web client that we use later on. In theory the two should roughly match, but Ookla recommend the Microsoft app for high-speed systems, which they say should give better results.
➤ Fast.com (Netflix)
Overall Avg (Mean) Offpeak Time Avg Peak Time Avg Ping (milliseconds) 1 1 1 Download (Mbps) 5414 5371 5457 Upload (Mbps) 1443 1400 1486 ➤ Speedtest.net (Ookla)
Overall Avg (Mean) Offpeak Time Avg Peak Time Avg Ping (milliseconds) 2 2 2 Download (Mbps) 4061 4102 4020 Upload (Mbps) 1604 1594 1615
Overall Avg (Mean) Offpeak Time Avg Peak Time Avg Ping (milliseconds) 16 17 16 Download (Mbps) 2049 2052 2046 Upload (Mbps) 361 257 465 ➤ Broadbandtest.which.co.uk (Which?)
Overall Avg (Mean) Ping (milliseconds) 11 Download (Mbps) 543 Upload (Mbps) 84 ➤ Broadbandspeedchecker.co.uk
Overall Avg (Mean) Offpeak Time Avg Peak Time Avg Ping (milliseconds) 14 14 14 Download (Mbps) 2047 2043 2050 Upload (Mbps) 4379 4196 4562 ➤ Broadbandspeedtest.org.uk
Overall Avg (Mean) Offpeak Time Avg Peak Time Avg Ping (milliseconds) 13 12 13 Download (Mbps) 1442 1522 1362 Upload (Mbps) 680 644 715 ➤ Uswitch.com/broadband/speedtest/ (Uswitch)
Overall Avg (Mean) Offpeak Time Avg Peak Time Avg Ping (milliseconds) n/a n/a n/a Download (Mbps) 801 828 774 Upload (Mbps) 346 340 352 ➤ Speedof.me
Overall Avg (Mean) Offpeak Time Avg Peak Time Avg Ping (milliseconds) 37 36 37 Download (Mbps) 598 603 593 Upload (Mbps) 168 176 160
One obvious realisation above is that some testers (e.g. Which?, Speedof.me, Uswitch) aren’t necessarily good enough, yet, to reasonably be used for speed testing a symmetric 1Gbps line, let alone anything faster like 10Gbps. We also saw some consistently unusual results from Broadbandspeedchecker.co.uk, which always returned double the upload rate vs download (no other tester did this).
The closest we got to 10Gbps was generally via Ookla’s testers, as well as Netflix and Broadbandspeedchecker.co.uk too (provided you ignore the upload issue). We know from different testing that Barry’s line can practically hit around 7-8Gbps when the internet is able to feed capacity to his setup at that rate, but otherwise the most capable web-based testers above would typically deliver around 4-5.7Gbps (usually less on uploads).
The fastest single test was recorded on Broadbandspeedchecker.co.uk, which hit 8011Mbps, albeit on upstream rather than downstream. By comparison, the fastest single result from B4RN’s own Ookla (Speedtest.net) server was 6951Mbps (or 5131Mbps via the regular Speedtest.net site) and Fast.com (Netflix) returned 6600Mbps – both downstream. The other testers didn’t even get remotely close to these.
One key thing to be aware of above is that hardly any of the tests provided much technical information about how they work (unless you trawl the small print and even then, it can be quite ambiguous). But this is crucial because there can be significant differences between, for example, single-thread and multi-threaded testing.
Some people suggest that a single-threaded test is perhaps more representative of normal real-world use, while multi-threaded is often perceived to be a better way of pushing a connection to its limits, but without more detail it’s difficult to give the results their full context. On the other hand, whether single or multi-thread, we believe the purpose of a tester should always be to try and return a fair indication – so far as it is possible – of what your line can actually deliver, when operating at its best.
On top of that, it was rarely ever clear if the final “average” result being reported by a tester – over the duration that a single speed test was running – had been measured as a ‘median‘ or a ‘mean‘ average, which can make a difference.
Suffice to say that our past warnings about the reliability of depending upon web-based speedtests for accuracy, particularly when testing 1Gbps or faster broadband lines, should continue to be heeded. Indeed, the problem only gets worse the further above 1Gbps you go.
However, the reality is that speeds of well over 1Gbps aren’t going to become common place in the UK for a fair few years and in the meantime those speed testers will, as they’ve always done in the past, continue to evolve and get better. Hopefully, by the time most of us can actually take a 10Gbps package, they’ll all be better equipped to cope.
One final point to make is that this article is intended to be a bit of fun using a fairly anecdotal approach, thus to do it properly we’d ideally need to harness a wider variety of 10Gbps capable lines and over a greater variety of time periods. But since we don’t yet have ready access to more of those connections, then we’ll have to wait a few more years.
Thanks for highlighting this. It’s driven me mad when trying to bat back “my internet isn’t performing at 1gig” queries. Broadbandspeedchecker is one of the least reliable. The most stable for results seems to be the installable windows or Mac app from Ookla. Their website can still be wildly unreliable.
Agreed. Using Netflix’s website you can falsify the results by starting the test and switching to another tab, it seems to show sometimes double your actual speeds. Example, I am on VM 1Gb and so I should cap 950(max). However, I have seen speeds of 2.7-3Gb/s on Fast.com if I swap tabs. I can really only trust Speedtests desktop application.
Samknows speedtest are one of the best reliable one!
@Max – that’s true if you’re using the /realspeed irl with a Samknows capable hub, their actual Speedtest is horrific and incorrect.
Meh, they didn’t test nperf.net
I can’t recall everything now, but there were a few reasons why we didn’t test nPerf too. One was that, according to Google’s data, fewer UK people use it than other testers. I think there were also some issues with the UK server selection, and the amount of data we had to manually collect was already getting a bit high. You have to draw a line somewhere, so we had to cut a couple of testers. Maybe next time.
I find NPerf extremely reliable compared to speedtest.net and fast.com on my 1gbps connection.
If you put speedtest.net into Google translate you can use Google’s servers and get ridiculous speeds.
38 GB/s LOL
https://postimg.cc/s15khPBn
Just for the record, I have tested B4RN’s 10 gig service too, and my ‘new’ laptop using a 10 gig adaptor beat Barry’s tests. Barry was using his older pc. I made a film showing it working. https://www.youtube.com/watch?v=JYva_NKO-y8 and then another customer with 10 gig got into the spirit of it all and upgraded his home kit and managed even higher tests. I agree, it ia all a bit of fun, and the internet itself has a long way to go before we can even use or test one gig properly, but hey, bragging rights and all that jazz. The ookla app is by far the best tester. Samknows hasn’t got a 10gig tester yet (I have asked them). Vodaphone Manchester server did my best test, some servers are pretty awful but you can test from a few till you find one that isn’t overworked and stick to that. Keith used vodaphone watford. This is the test result from Keith’s super machine. https://b4rn.org.uk/wp-content/uploads/2021/08/ITPS-Ltd-V8-1.jpg
I can see it now “but I’m not getting what I’m paying for” (oh that already happens).
Not your wifi slowing it down, not your usage, not your brand new cheap crap pc?
Ever tried an actually download from an actual supported server? Grrr.
Speed “faults” will never go away.
tbh, if I was paying an utterly ludicrous amount for a 10gig connection, that is exactly what I’d expect to receive. Anything less would be unsatisfactory.
Better to pay ludicrous amounts for 100 Mbit DIA if you expect 100% all the time.
I see loss of 10% routinely at peak, going up to 20% on occasion.
Carl, even then I’ve still had customers complain they’re not getting full speed to a network on the other side of the globe after we hand it over to the destination network’s transit.
There’s enough capacity for it to egress our network at the full rate, local access network is fine, and it’s taking a good route, we can’t do anything else!
Internet routing is a tricky beast. The ISP gets blamed even if the network is fine. A while back one of my customers expected us to raise a fault with CenturyLink/Lumen because it appeared they were losing some traffic, and wouldn’t accept that it would have to be the end customer who pays them – all we could do is politely ask.
Who would have thought the person paying a company has more rights to ask someone to raise a fault than 2 companies freely cooperating!
Genius parody, Damien!
Ah it was serious, and is the usual suspect. Okey doke.
I genuinely thought it was parody.
What’s with the Think Broadband results?
I find cloudflare’s speed testing service to be most reliable:
https://speed.cloudflare.com/
Over-reporting by about double for me in the early stages but by the end of the test gives reasonable results but not as accurate as the ookla installable app.
I would love to see a test downloading from steam/origin etc for example GTA:V which is over 80GB, or cyberpunk at something close to 100GB with all the updates now. I’ve always had 100% of my bandwidth from steam.
Pff no nperf best one out there
The most important test of them all is missing here, a baseline test that shows that his FTTP line, router, and PC can actually achieve 10Gbps. Without seeing this we can’t draw anything from these tests, i.e., the fastest result is simply all his PC could handle.
At present, there isn’t a way to establish a solid baseline on Barry’s connection and that’s the problem you have with 10Gbps. But as the article says, we have clocked it at 7-8Gbps before and so we know that capability exists at his end. But what we really need is a router with custom speed testing capabilities (firmware) or for SamKnows to produce a 10Gbps capable white box, which at present they don’t.
What the article shows is that a lot of testers still struggle to give a fair result for even 1Gbps, let alone anything faster. As we say, this is meant as a bit of interesting fun, and hopefully a nudge for those testers to up their game.
Hey Mark, what about Star Trinity? Have you checked it out: http://startrinity.com/InternetQuality/ContinuousBandwidthTester.aspx
Loads of customisation, available online or as an install (free for non-commercial)
I’ve used this having seen it on a forum and had it raise an eyebrow. It’s a front end for iPerf and extremely unreliable, presumably in part because there’s no way of knowing what’s at the other end.
Been saying this for at least five years now. Above 1Gbps all speed testers are for jack in my experience. They have got a bit better but they are still now where near capable of actually testing the available capacity on a 10Gbps link.
Indeed. While I’m not much above a gigabit I definitely see huge differences.
Some are clearly on 1 Gbit ports, others are on contended 10 Gb/s networks so someone like Barry with that kind of bandwidth will saturate things.
Great read as always @mark
I spotted a typo you may want to correct
Second to last paragraph :-
“..Hopefully, buy the time most of us can..”
I wish I could “Buy” time but instead I see time go by, bye..
Cloudflare have a great speedtesting site/service.
On aside note VM have just passed my front door. Finally, now i maybe able to order a service which can attain speeds higher than 56meg. Anyway, Virgin have just blown through my street i suspect BTO will be passing soon after although i doubt ill be able to order 10GbE packages as a retail consumer just yet.
On a 500mbit connection the fastest results I only got on Vodafone Watford and Swish Fibre using speediest app on MacOS. That’s on WiFi though.. neither thinkbroadband nor others came close.