Which? Claim UK ISPs Could Overestimate Broadband Speeds by up to 62%

A new study from consumer magazine Which? has used data taken from over 700,000 speed tests conducted by Speedchecker Ltd (between January and March 2017) to claim that broadband ISPs could be overestimating their service speeds by up to 62%. Save for a few BIG caveats.

The research, which also used data collected by Ofcom for its 2016 Connected Nations report (perhaps they should have waited for the imminent 2017 update), looked at the median download speeds in each local authority area, taken from the speed tests, and compared them with the median speeds that providers say people in the same areas can get, as reported by Ofcom.

In 52% of local authority areas, people using the speed checker experienced median speeds that were at least 10% slower than the median speeds estimated by ISPs in the report. Over a third of local authority areas (35%) had speed checker medians that were at least 20% slower than estimates. So where does their headline “up to” 62% figure come from? Apparently the gap was worst in Ashfield (Nottinghamshire).

Meanwhile, people using the speed checker experienced median speeds at least 10% higher than those given by providers in 16% of local authority areas. In the remaining 32% of local authority areas, the median speeds experienced by speed checker users were less than 10% different from the median speeds specified in the report.

Alex Neill, Which? Managing Director of Home Services, said:

“Our research has shown that in some areas there can be a big gap between what people may expect versus what they actually experience in their homes.

This gap raises questions on whether people across the country are really getting the service they are paying for.

People who are unhappy with their broadband should check their speed and follow our tips on how to improve their connection at home or switch to a better service.”

Which? correctly notes that consumers usually receive slightly slower speeds than those given by providers due to factors such as line length, interference and network performance, although they should really have gone a bit further to clarify some of the context and other key limitations with this data.

In particular many people will test their speeds over WiFi rather than wired connections and wireless networks tend to produce much slower results due to local signal disruption (walls, doors, distance etc.), which means that you’re not directly testing the broadband connection itself. Ideally the best way for end-users to test would be to do so at the router level, which removes a lot of potentially disruptive factors.

Lest we forget that poor quality home copper wiring is another factor that can reduce broadband speeds, albeit particularly on the older style of ADSL and FTTC (VDSL2 / G.fast) technologies. A lot of people still don’t properly filter out interference from the old bell wire and that can have a big impact on the speed you receive, although it isn’t the ISPs fault.

The speed test itself can also suffer instability due to issues with server performance or network routing / peering, which can impact the result. Not to mention that if somebody is conducting a test while others are using their home network then that will also result in lower speeds.

We’re also unclear about the context when Which? says they compared the speed test data with Ofcom’s information on the “median speeds that providers say people in the same areas can get.” We don’t know exactly what the source of that is and if they’re using the performance based off general network availability by technology, headline advertised speeds, sync speeds or the personal user speed estimates given by ISPs.

The best comparison may be to use the personal estimates provided by ISPs because these are specific to the end-user, although accurately collating personal estimates with the data gathered from public speed tests is currently very difficult to achieve without consumers agreeing to share such data with the testing websites. Likewise not all ISPs provide a personal estimate.

We can’t stress enough how important it is to get the context of such testing correct, otherwise the results may not say what Which? think. Suffice to say that we must always take such data with a pinch of salt and it would be more reliable if they could exclude tests conducted over WiFi (very difficult to do because speed testers can’t easily identify this without more user input).

A useful approach might be for speed testing sites to give users the option of offering some extra data, which would enable them to specify if the test was conducted over wifi, how many users are on their network at the time of the test, what kind of connection technology they have, what package they chose and what personal speed estimate had been provided by the ISP. Granted many would not provide this but if some did then the data would have much more value.