UK ISP Website Censorship Measures Face IP Address Sharing Nightmare

An Ofcom commissioned CSMG investigation into the impact of IP address sharing upon website blocking measures, which makes such methods “technically complex” for ISPs to implement, has found that 97% of “top-level domains” (.com, .net and .org) reside on addresses that are shared with other sites. Block one and you risk hurting others.

The report comes at a time when broadband ISPs in the United Kingdom are facing mounting pressure from the government, courts and rights holders to block unlawful websites, such as those that facilitate internet piracy. This often requires the ISP to block the Internet Protocol (IP) address of a website too, which gets tricky when the IP is shared with several other legal sites (a common issue on shared web hosting packages).

Overall CSMG found 0.3 million distinct IP addresses and 5.9 million distinct websites. The ratio of websites to IP addresses for the UK top-level domain is 13:1, compared to 19:1 for the COM, NET and ORG top-level domains. The lower ratio immediately suggests that IP address sharing between websites is less significant for the UK top-level domain but still extremely common.

Even when the study discarded IP addresses that hosted more than 1,000 websites (i.e. those likely to be used for domain squatting or parking), the average number of website instances sharing a single IP address is still 7.5 for COM, NET and ORG top-level domains and rises to 8.6 for UK TLD’s.

Study Conclusions

IP address sharing is prevalent across the four major TLDs typically used by UK organisations, namely COM, NET, ORG and UK, and has significantly increased in usage since 2002. As described, for the COM, NET and ORG top-level domains, 97% of websites reside on IP addresses shared with other websites, compared to 87% in 2002. This has important implications for the use of IP address blocking to prevent access to unlawful Internet content. In particular, it is likely that some unlawful websites and lawful websites may share the same IP address.

Our analysis shows that websites are increasingly being hosted by IP addresses serving large numbers of website instances. For example, for the COM, NET and ORG top-level domains, more than 50% of website instances are now estimated to be on IP addresses hosting at least 11,580 website instances. For the UK TLD, more than 50% of website instances are hosted on IP addresses hosting at least 1,036 website instances. This is lower, mainly due to significantly fewer websites in the UK TLD space.

For the COM, NET and ORG top-level domains, the 3% of website instances which reside on unshared IP addresses are served by 64% of distinct IP addresses. This implies that for any given IP address, if the IP address hosts at least one web site, there is a 64% chance that it will only host a single website. We have not explored any qualitative differences between domains which are hosted on single IP addresses versus shared.

It should be said that some ISP filtering systems have the ability to identify whether a requested IP address is attached to a specific website (i.e. the site intended to be blocked), which prevents the wrong site being blocked, although these checks are far from perfect and can be fooled. Yet another headache for law makers and ISPs alike, although consumers already find it easy to circumvent related blocks.

CSMG Study into Websites Sharing Internet Protocol Addresses (PDF)
http://stakeholders.ofcom.org.uk/binaries/internet/websites-sharing.pdf