BT Tops MedUX 2024 Study of Big UK Broadband ISP Performance UPDATE2

Network benchmarking firm MedUX has today published the results of a new study, which examined the fixed broadband performance and quality of three major UK ISPs – BT, Sky Broadband and Vodafone. Overall, BT ended up topping the rankings, but it’s unclear why they left out TalkTalk and Virgin Media, both of which are bigger than Vodafone.

The methodology and data gathering of the new study is a little unclear, but it claims to be using an independently developed scorecard to evaluate the performance of the networks, which is said to encompass “various benchmarks essential to fixed broadband service quality and end-user experience” (it sounds like data was crowdsourced from in-home performance measurements). The report then took a “detailed look at scores” measuring a combination of service reliability, speed compliance, accessibility, throughput, streaming, data and OTT (over-the-top) experience results.

The study found that BT – after being subject to over 900,000 network tests – not only leads in fixed broadband network responsiveness, but is also able to provide the overall finest Quality of Experience (QoE) in the United Kingdom, with a strong score of 4.59 out of 5. BT also achieved the most consistent experience (99.88% service reliability).

The research claims to show that BT’s older FTTC (Fibre-to-the-Cabinet) customers also enjoy the highest average download speed, reaching 60Mbps. Moreover, BT achieved the best performance for delivering web browsing (99.92% success rate) and an excellent gaming and streaming experience (99.78% and 99.99% success rates respectively). BT also came top for social media access, with a loading time (median) of 2.6 seconds.

However, BT did not lead in all metrics within the UK study. For example, Vodafone achieved the best average latency for online video gaming (23.7 milliseconds) and they were found to be the most likely to match their service promises with an average ratio of effective download speed versus what it advertised of 87%. The European average for that last one is 72%.

Vodafone and BT also shared the lead in network responsiveness and streaming experience. Meanwhile, Sky Broadband was named the best for web browsing, with just 8 out of 10,000 internet sessions “failing“, and they also came top for video streaming with a score of 100% for stable video playbacks – hardly surprising for a media empire.

Rafael González, CMO at MedUX, said:

“We’re pleased to be providing another first of its kind study on UK connectivity. This report builds upon the momentum from our UK and European-wide 5G quality of experience (QoE) reports, and the recently released Italian fixed broadband QoE study. It showcases the breadth of our benchmarking capabilities – providing unique insights into the state of European connectivity.

As operators, regulators and industry bodies continue to have pivotal conversations on the state of connectivity, we look forward to seeing insights from our reports being used so networks can be optimised, improving the overall experience for end users.”

Overall, we found the public summary of the new study to be quite wishy-washy and lacking in detail or useful comparisons between the named providers, as well as there being no explanation for why it ignores two of the market’s biggest providers – Virgin Media and TalkTalk.

Just to make matters worse, you have to ‘register’ in order to see the full report and must then await approval. But this should really have been provided alongside the announcement to give the correct or best context, particularly as they’ve yet to respond to our requests for access. Consequently, the somewhat random little details above are the only ones we can give, which are not particularly useful. Take this study with a pinch of salt.

UPDATE 10:10am

We’ve finally got a comparative summary of the key results, although the text is a little blurry (this is due to the original format being very pixelated).

UPDATE 1:35pm

MedUX has informed ISPreview that the data for this study was collected via “MedUX Home robots” – testing units, agnostic and all-router compatible, that are a Plug and Play solution based on non-intrusive tools. “Our MedUX Home probes perform active measurements without integration into the operators’ networks, in the home network environment, including the performance of fixed-broadband (Ethernet) and Wi-Fi (2.4GHz and 5GHz) interfaces.”

The firm added that, in the current research study, they have only focused on the most common ISP packages in the UK, with speeds between 30 and 100 Mbps, and the predominant technology being FTTC (VDSL). “We have included results for those operators with sufficient statistical representativeness and a confidence level above 82%,” which is how they explain the exclusion of TalkTalk and Virgin Media. But both providers will be included in a future report.