Posted: 03rd Aug, 2008 By: MarkJ
Samknows has released its first '
Performance Monitoring Report' (Adobe .PDF format) for UK broadband ISPs (
original news), which collects data from specially modified routers installed in peoples homes. These have been designed to track the real-world performance of Internet providers in many different ways and report back their findings.
Initially the results of just 223 trial units were aggregated to produce the findings, thus the output should be taken with a pinch of salt. Most ISPs only had under 20 units monitoring them and some providers have yet to be properly tracked (e.g. AOL UK); a basic summary of the report follows:
- In the majority of metrics there was little discernable difference between most ISPs;
- Zen Internet offered the fewest failures across all metrics;
- Virgin Medias cable services and Be/O2s services provided a consistently low latency throughout, whilst Virgin.Net (Virgins ADSL service) performed poorly;
- BT provided the fastest throughput when measured as a percentage of impliedline speed (an estimate of the potential maximum speed of the line);
- Be/O2 and Virgin Media produced the greatest raw throughput (in megabits per second), which can likely be attributed to the nature of their products;
- Virgin Medias cable throughput remained consistent on their 2, 4 and 10Mbps products, but was quite variable on their 20Mbps product;
- Testing highlighted the use of traffic shaping in the networks of BT and PlusNet, which resulted in certain classes of traffic slowing significantly during peak hours.
The report itself actually conducts a multitude of tests using many different methods, including several related to download speeds, which makes it difficult to summarise without losing context. Still, we've elected to re-order the results from Sam's latency, port 80 download and port 80 upload tests into a top 12 list.
However, it is highly recommended that readers view the report itself to gain the proper context as this data is more detailed than the usual anecdotal results:
ISP - Latency (ms / lower is better)- Virgin Media (Cable) - 29.03ms
- Be Unlimited - 30.50ms
- Karoo - 32.13ms
- Entanet - 37.14ms
- Plus Net - 37.68ms
- BT - 37.84ms
- Orange - 42.83ms
- Tiscali - 43.06ms
- TalkTalk - 43.99ms
- Zen Internet - 47.27ms
- Sky - 50.31ms
- Virgin.net (ADSL) - 62.56ms
Latency (lag) is typically a measurement of how long it takes for a server to respond and is an important consideration for online gamers where high latency can cause stutters (pauses) in play. However, it should be noted that different games have different servers and response times (ms - milliseconds) do vary.
Port 80 Download Throughput - Megabits Per Second (Mbps)- Be Unlimited - 6.32Mbps
- Virgin Media (Cable) - 5.88Mbps
- Orange - 4.22Mbps
- Entanet - 4.01Mbps
- BT - 3.94Mbps
- Sky - 3.82Mbps
- Plus Net - 3.69Mbps
- Zen Internet - 3.68Mbps
- Karoo - 3.60Mbps
- TalkTalk - 2.66Mbps
- Tiscali - 2.64Mbps
- Virgin.net (ADSL) - 2.15Mbps
Port 80 Upload Throughput - Megabits Per Second (Mbps)- Be Unlimited - 0.86Mbps
- Virgin Media (Cable) - 0.50Mbps
- Sky - 0.50Mbps
- Entanet - 0.48Mbps
- Plus Net - 0.45Mbps
- Zen Internet - 0.39Mbps
- Orange - 0.37Mbps
- BT - 0.35Mbps
- Karoo - 0.34Mbps
- Tiscali - 0.34Mbps
- Virgin.net (ADSL) - 0.33Mbps
- TalkTalk - 0.32Mbps
It's important to point out that some services, such as Be Unlimited (
O2), use more modern/robust 'up to' 24Mbps ADSL2+ technology, while others (e.g.
PlusNet) use 'up to' 8Mbps ADSL. Naturally the technology and packages offered play a part in the results, thus such results are never a fair comparison. Sams full report includes more detail.
Samknows reports that the project will continue into 2009, with improved testing metrics and a greatly increased sample size (to beyond 2000 units) being the focus points for future work. It is hoped that this will enable analysis of performance across ISP products and also across the regions.