» ISP News » 

UPDATE3 Government Forces IP Address Matching Upon ISPs – But What is it?

Monday, November 24th, 2014 (12:19 pm) - Score 11,084

The Government’s forthcoming Counter-Terrorism and Security Bill (CTSB) will this week introduce a number of new measures including a provision to help the security services identify suspects via a computer or mobile device’s individual Internet Protocol (IP) address. But what does this actually mean for broadband ISPs and their customers?

At first glance most of the reports from newspapers and around the Internet don’t appear to fully understand the difference between what’s being proposed under the CTSB and what already exists as part of the temporary Data Retention and Investigation Powers Act 2014 (DRIP).

Under the DRIP Act ISPs are already required to maintain a voluntary and somewhat basic log of their customers Internet access including email activity (times, dates and IP addresses) for up to 12 months (note: this doesn’t include the content of your communication), which usually becomes active following a specific request to the ISP (e.g. a demand / warrant from the police).

But at best this only identifies the bill payer of the Internet connection, while the related IP address (one of these is assigned to your connection every time a device/router links to the Internet) for that service might itself also be shared between many users and lots of different devices (laptops, routers, smartphones, tablet computers etc.), such as in a family home, business, public wifi or on a mobile network etc.

According to today’s many newspaper reports, the new IP address matching measures aim to go one further by requiring ISPs to keep records “that can show which individuals have used a particular IP address at a given time” (The Telegraph). Similar quotes can be found in nearly every other report, except in traditional networks it’s impossible to do this accurately and without some sort of aggressively invasive monitoring.

For example, generally ISPs cannot track how a remote Local Area Network (LAN), which will usually be using Network Address Translation (NAT), is setup in your home (i.e. which devices are using what LAN assigned IP addresses at any one time) because that is managed by the router and generally doesn’t get communicated back to your provider. Admittedly the big ISPs might be able to add snooping code into the router that could send the logs back to your provider, but then end-users could simply buy a third-party router or hack the code to remove or fool it (we suspect that would become quite popular).

But even if you could make the above system work then it would still have no accurate way of knowing which individual is using what devices (e.g. my sister might swap her tablet to my brother and then on to our mother, none of which would ever show via basic IP logs); short of making a live webcam feed of your face available on every single device and we doubt anybody would agree to that idea (also you could fool that too). Not to mention the added fun from using non-UK based VPN, Proxy Servers and so forth.

Adrian Kennard, Director of ISP Andrews & Arnold (AAISP), said:

You cannot tell who is using a computer or mobile from an IP address. At best you can tell subscriber details, if they exist, and maybe a location where the IP is initially routed (but it may then go on to anywhere in the world). So what is being asked is impossible.”

Unfortunately we won’t find out whether or not the Government have proposed the impossible until later this week, but it’s probably much more likely that the mass media have simply got the wrong end of the stick and that what has been proposed is actually a lot more straightforward than the reports appear to claim.

A far more likely probability, and one that some of our industry sources appear to support, is that the Government will simply update the law to cater for issues like Carrier Grade NAT (CGNAT) that allows ISPs and mobile operator to share a single IP address between more than one connection / customer (an ISP level assignment, not home router level).

Mobile operators already make use of shared IP addressing and the failure to swap over to the latest IPv6 standard means that some home broadband ISPs may slowly need to do the same (i.e. old IPv4 addresses have run out but many services and hardware devices still need it and thus the old addresses may need to be shared). A few ISPs already make limited use of CGNAT, but it can cause problems and thus providers aren’t rushing to make it mandatory.

In that sense the law will most likely be updated to enhance existing ISP-side logging functions that could help them cater for CGNAT or similar networking arrangements, which is still likely to be a costly and technical tricky adjustment for ISPs. Crucially the existing DRIP Act doesn’t appear to cover CGNAT.

In other words, unless the Government has completely lost the plot and are requiring ISPs to know which end user behind a residential gateway accessed which site (we’ll find out this week), then ISPs won’t be trying to track activity on your home network after all (at least not in the way that many reports have this morning suggested).

UPDATE 1:04pm

The UK Internet Service Providers Association (ISPA) has now waded in and appears to say that the Government hasn’t even bothered to consult them, which is perhaps a worrying turn of events.

Nicholas Lansman, ISPA Secretary General, told ISPreview.co.uk:

ISPA is disappointed that the Home Office has not consulted with industry on proposals for IP matching, but we will work with our members to scrutinise and inform the legislation when it is published. IP addresses can generally only be used to identify a subscriber and not an individual. As we argued in our submission to the Anderson Review on future communications data laws, the Home Office needs to do more to consult with industry on its proposals, once again there has been a distinct lack of engagement with industry.

Government committed to a review of communications data capabilities by David Anderson QC which we supported, yet the Home Secretary appears to have pre-judged the inquiry by reemphasising the need for a new Communications Data Bill, a Bill that both relevant parliamentary committees rejected“.

UPDATE 26th Nov 2014

The Bill has now been published (here and here) and Part 3 covers the enhanced ‘Data Retention’ aspects, which broadly appears to reflect the adjustments we touched on above and crucially adds the following to existing rules:

“relevant internet data” means communications data which—

(a) relates to an internet access service or an internet communications service,

(b) may be used to identify, or assist in identifying, which internet protocol address, or other identifier, belongs to the sender or recipient of a communication (whether or not a person), and

(c) is not data which—

(i) may be used to identify an internet communications service to which a communication is transmitted through an internet access service for the purpose of obtaining access to, or running, a computer file or computer program, and

(ii) is generated or processed by a public telecommunications operator in the process of supplying the internet access service to the sender of the communication (whether or not a person);”.

UPDATE 26th Nov 2014

The following is a brief summary of what the Government’s chosen option (2) will apparently do – taken from the Impact Assessment.

Option 2: Require communication service providers to retain data necessary to attribute an IP address to a user of an internet access service and a wider range of internet services.

To protect the public, new legislation being introduced that maintains the ability of law enforcement and intelligence agencies to protect the public and support the investigation of crime in cyberspace. This will be achieved by:

* Introducing new requirements on CSPs to retain CD [Comms. Data], including beyond their own business need;
* Amending the Data Retention and Investigatory Powers Act 2014 (DRIPA) to enable communications service providers (CSPs) who provide an internet service to retain data necessary to attribute an IP address to an individual;
* Expanding DRIPA to cover a wider range of internet services.
* Providing payments to be made to CSPs in respect of costs incurred in complying with new legislation

As we’ve said before, an IP address held by an ISP can at best only be attributed to the bill payer and since most networks are share / used by more than one person then there’s no real way for broadband providers to reliably make the required link.

But the Government also, for reasons unknown, likes to call Internet content providers (e.g. Facebook) ISPs and this can sometimes confuse matters. Meanwhile Facebook may track the IP address of a user but it’s very easy to create a fake account and or hide behind a VPN etc.

Option 2 also mentions “expanding DRIPA to cover a wider range of internet services“, although so far the current changes haven’t quite clarified that aspect. The next issue is one of cost.

Costs – Option 2:

The costs are based on studies conducted by industry. The present value of costs over a 10 year period is estimated to be £99 million; this figure may change with continued development in technology and services.

In current prices, the costs of implementing IP resolution at service providers will be in the region of £27m; the costs of running and maintaining these solutions is estimated to be £96M over the 10 years.

The totals above are based on:

1. Getting the IP data from service provider systems
2. Building a solution to store the IP data at service providers
3. Running and maintaining the above

The cost estimates for the individual components above are based on:

* Studies into IP resolution conducted by industry
* Prior work with service providers and industry on similar projects

Alternative methods of investigation, such as directed surveillance and undercover officers, cost significantly more than CD, do not provide the same level of benefit and are very often more intrusive.

The report also says that “where Law enforcement agencies have accurate source information (eg IP address and accurate time) from an internet service provider they can identify which user sent that communication.” Once again no, when referencing an IP assigned to the customer by an ISP the provider can only accurately identify the bill payer who owns the connection, which on most shared networks may not be the offending user.

Meanwhile it looks like anybody running a CGNAT network of shared IPv4 addresses, especially mobile operators, are in for an unpleasant time as the job of logging and storing every session in order to make customer tracking viable will be both technically tedious and quite expensive.

Share with Twitter
Share with Linkedin
Share with Facebook
Share with Reddit
Share with Pinterest
By Mark Jackson
Mark is a professional technology writer, IT consultant and computer engineer from Dorset (England), he also founded ISPreview in 1999 and enjoys analysing the latest telecoms and broadband developments. Find me on Twitter, , Facebook and Linkedin.
Leave a Comment
20 Responses
  1. dragoneast says:

    Oh the joys of the great British press. When they’re not making it up, they’re failing to understand anything. Never mind, we all make it up to fit our agenda, don’t we?

    1. Graham Smith says:

      Not entirely the press’s fault when Theresa May’s official speech text says: “It will therefore require internet providers to retain Internet Protocol – or IP – address data to identify individual users of internet services”.

      Nor was the May 2013 Queen’s Speech Briefing Document especially illuminating: “When communicating over the Internet, people are allocated an Internet Protocol (IP) address. However, these addresses are generally shared between a number of people. In order to know who has actually sent an email or made a Skype call, the police need to know who used a certain IP address at a given point in time. Without this, if a suspect used the internet to communicate instead of making a phone call, it may not be possible for the police to identify them.
      The Government is looking at ways of addressing this issue with CSPs. It may involve legislation.”

      Finally there’s this consensus element of the Dec 2012 Draft Communications Data Bill Committee Report:

      “74. Subscriber data relating to IP addresses is the information that makes it possible to trace who is using an IP address at a given point in time. An IP address is a numerical label assigned to a device connected to the internet (e.g. a computer, smart phone or printer). The IP address of a device is not constant; it may change frequently and be shared between several devices. The originating IP address of a communication is routinely gathered in many types of internet transaction, but if the CSP does not hold information on which of its subscribers held which IP address at a particular point in time it is very hard for law enforcement authorities to prove an association between an action on the internet and a particular individual. Not all United Kingdom providers currently obtain all the data necessary to trace which subscriber is using which IP address. During the course of our inquiry we heard of various circumstances in which the lack of this data has impeded investigations. We accept that if CSPs could be required to generate and retain information that would allow IP addresses to be matched to subscribers this would be of significant value to law enforcement. We do not think that IP address resolution raises particular privacy concerns.

      75. We recommend that a narrower clause 1 should allow notices to be served on CSPs requiring them to generate and retain subscriber data relating to IP addresses.”

    2. dragoneast says:

      Journalists ought really to try and get beyond the spin. Otherwise why don’t we replace them by robots? In fact Google searches are often more informative.

      The confusion is deliberate. Security is always a game of bluff and double-bluff, to keep the enemy guessing. These days “the enemy” are all of us. And the attitude has seeped out beyond security, too.

      But overall, it’s about making us feel good, secure or reassured. Most people don’t have the time, or the inclination, to think.

  2. Martyn Dews says:

    I listened to several of these reports over the weekend and they all seemed to be missing the point as they stated that all devices would be tracked by their IP address. As stated above, once you get across the router an into the LAN, you enter the private address space.

    So unless I’m misunderstanding something here, until every device has it’s own unique IP, then how can each device be tracked?

    For example, say you have 6 people in a house with multiple Internet connected devices, all using the same router and thus, they same public IP. One of them is a terrorist. To the outside world, the suspect data entering into that house could be going to any of the occupants.

    Or have I missed something. If so, Help me out.

    1. Steve Jones says:

      Certainly in the case of things like domestic NAT routers, then it is only the public IP address that is known, although there can be other clues as to the device in the actual traffic. However, that aside, the customer for that device can be in for an uncomfortable time, even if any nefarious activity wasn’t down to them. It’s a little like your car being caught for speeding. You might not be driving, but may have to point to who was. Of course it’s a bit more complex in the case of broadband (speeding is a strict liability defence where the registered owner is responsible unless they can show who else was driving) as there is “plausible deniability”. You might claim it wasn’t me downloading those illegal images, but somebody piggy-backing on my network. If such happens, as the customer of an ISP, you could find yourself in an unpleasant investigation and civil, or even a criminal case. In that event it would be a court that would decide based on the evidence.

      In addition to the above, there are operators of public networks which require some form or registration and authentication. The networks in question might indeed use NAT behind a single IP address. However, they will be required to keep records of who was authenticated to that network at the time. Such information may not be completely definitive, but it certainly vastly narrows down possible suspects.

      So, hardly bullet-proof, but it’s very often (especially in civil cases), that there is a lot of collateral evidence that is available from the payload or access patterns.

  3. X66yh says:

    I was involved on a sidelines of a case – as a witness I hasten to add
    The police approached the owner/home address of the router whose IP address matched what they got from the ISP.
    This owner was “invited” to cooperate as to who was using the network at that time.
    Strange to report they found it very much in their interest to provide the likely name very rapidly.

    In this case it was the sending of data from the address that was the issue rather than the receiving of it.

    I guess in the future it will be like a speed camera. Police demand the name of the driver from the owner and failure to supply/cooperate is an offense.

    1. Martyn Dews says:

      Yes, that’s the only way I can see it working until each Internet connected device has a unique public facing IP, which I doubt will ever come, even with IPv6, there will still be a need to keep devices behind a router to offer a degree of security.

      So at the bill payer for my broadband, I’m likely to be responsible for all content uploaded by any device on my LAN by any person.

      The MPs clearly haven’t given this much thought have they? Knee-jerk politics. Is there an election on the horizon? 🙂

    2. TheFacts says:

      How many MPs are Engineers?

    3. Steve Jones says:

      @Martyn Drews

      Nothing really to do with MPs as such. You can already get into issues like this. One obvious example is civil action by copyright owners over illicit downloads. Ultimately any defence of “it wasn’t me” will get tested in court. Of course the standard of proof required for civil and criminal defence are very different, but it’s still a court that decides on the basis of evidence.

      However, it’s not necessary to consider just broadband. There’s a precedent should you lend your car to a friend. You have a responsibility in law to ensure that they are qualified and insured to drive your car (and you can be prosecuted under criminal law for not doing so). Further, if any of several offenses under criminal or civil law happen whilst your car was lent to your friend, you will have to show somebody else was driving, or you are held responsible (things like speeding, parking violations etc.).

      To stretch the analogy further, if you can show your car was stolen, then this would be equivalent to somebody claiming somebody had hacked your network. To be decided by a court based on the evidence at hand.

      Note, I’m not suggesting that anybody has a legal duty to guarantee that their network isn’t being used for illicit purposes, but at the very least you can have an uncomfortable time. Also, in the case of minors, it’s very likely that a court will uphold a civil case for something like breech of copyright.

      Note that ISPs will generally hold records anyway, as they need this to defend themselves. Almost certainly the civil courts can order ISPs to provide information that copyright holders demand where it’s in their reasonable control. Things like IP allocation logs fall in that area.

      So this is a tidying up. Nothing really new. Anybody engaged in something they want to hide would be wise not to rely on records not being kept, whatever the law says. Also, another good reason to keep your own network secure, and if it’s a commercial business, to retain your own logs.

  4. DTMark says:

    What’s an ISP?

    Is it simply the providers who supply your connection to the internet?

    Does this therefore mean it excludes proxy and VPN providers, because they are not ISPs? Presumably the fidelity of the data – such as the requesting MAC – is gone by the time the request reaches the destination, but the VPN provider is not required to capture same?

    Does it exclude hosting companies, so if the government one day wanted to know who has accessed a particular client’s website, I can say quite truthfully that the raw sever logs are retained for only one month?

    1. Steve Jones says:

      It doesn’t just apply to ISPs. It is any operator allocating an IP address. That’s bound to include commercial operators of WiFi networks where registration and authentication is required.

      As for proxies and the like, they are not covered by this as far as I can see, and in any case are usually located abroad for the good reason that they don’t want to be bound by UK law. Indeed, many want to be outside the scope of US law too.

      For anybody using a VPN proxy network, you are at the mercy of the integrity of those operating it, or at least supplying the software. Stories abound of back-doors in security systems.

    2. Raindrops says:

      “It doesn’t just apply to ISPs. It is any operator allocating an IP address. That’s bound to include commercial operators of WiFi networks where registration and authentication is required.”

      Good luck with that then because for BTs openzone/wifi/whatever its called now you can happily buy wireless time totally anonymously.

  5. Chris Conder says:

    Its another superfarce, it is obvious the government doesn’t get digital. The infamous washup of the digital economy act has a lot to answer for. Be sure their sins will find them out. The last two governments will go down in history for the shambles they are allowing to happen in Digital Britain. Let us hope the next one can do better and save us from the digital third world we are sliding into.

    1. TheFacts says:

      “Be sure their sins will find them out.”

      That will sort out the UK technology industry.

      How are we sliding into a digital third world?

  6. Captain Cretin says:

    Terrorists will just use Mobile Internet, which as mentioned – covers millions of users on one IP address.

    I suspect this is all a smokescreen to hide a different bill making any complaint about the government a terrorist offence.

    1. James Harkin says:

      Open Wi-fi hotspot connections. I was able to use the Internet at a mobile phone shop, no blocks I was searching Ebay for a car. Nobody bothered me.

      Also, many countries around the world have open wi-fi hotspots you don’t have to sign up to and a lot of these places don’t have security cameras or any idea what is going on. You can do anything and you could even use VPN or proxy servers if blocks do occur.

      I personally think the UK government is coming up with these rules not just for tracking terrorism offences, but to find out what people are looking at. Terrorism is merely the scapegoat. The majority of people are not terrorists and they know this, so why covertly monitor everyone. Maybe to gather data so they can either sell it to their corporate partners or use it to control us, it wouldn’t be the first time. The Elite of this world do not initiate a new control system for merely one reason. There has to be multiple reasons for this to occur.

      Nothing good can come from a small group of nefarious individuals monitoring the Internet.

  7. Random thoughts (written as a concerned internet citizen rather than any official role) – stop me when you think I’ve gone to far…

    The home office is calling for IP Matching as information not routinely collected by ISPs.

    99% of ISPs track subscribers by IP address (for various reasons) and provide subscriber data to parties via RIPA

    This implies there is a class of IP addresses where matching IP addresses to subscribers is either not happening (the 1%) or there is something that acts as a gateway device for many users (mobile networks, CGNs, 6to4 gateways, etc)

    Sounds like the home office want ISPs to record those translations in a format that can be used to “lift the veil” and identify the subscriber

    From a theoretical technical point of view that’s easy as the gateway just needs to log each translation and store it in a large database that can be accessed when the proper request is made under RIPA to identify the end user based on the external IP address and the port number used (as that’s the mapping information stored by the gateway)

    Most web servers don’t record the source port of a request by default (see http://en.wikipedia.org/wiki/Common_Log_Format)

    So either all those servers out there need their log format updating, or the ISPs are going to be required to store that information so that they can reverse engineer the information for the requestor.

    Now where did I here something like that before

  8. “even with IPv6, there will still be a need to keep devices behind a router to offer a degree of security.”

    With IPv6 there is no NAT (no matter that people are trying to put it back in) there are privacy extensions which make it harder to track individual end user but they can still be traced to a LAN level. The correct solution is a decent stateful firewall that only allows traffic back in when outbound connections have been established but they are more expensive and harder to configure than a NAT device which is very good at the illusion of security

  9. sentup.custard says:

    So, let’s see…

    I normally use an Alcatel Y800, bought new from a “reputable” supplier who may actually be the sales arm of GCHQ for all I know.
    I have an EE contract SIM, so the ISP have my details anyway.
    I always use a VPN (EE’s “content lock” is knackered, can’t switch it off, so without a VPN I’d be stuffed on a lot of sites), but we will assume that the VPN provider is in league with the Stalinists at Westminster, so it’s totally ineffective where privacy is concerned.
    The Y800 is connected by a USB cable to my ordinary desktop PC, so they doubtless know all about that and might be able to identify me by the MAC address or whatever.

    So, when I decide that the time has come to blow up the Houses of Parliament and foolishly tell all my friends the details of my plans on my Farcebook page (OK, I don’t actually have one, but never mind), they are on to me, right?


    I am, of course, thick, so it wouldn’t occur to me to use the other Y800 that I bought as a spare in case the usual one conks out from a bloke on e-bay who’s probably forgotten who I am by now, stick in the unregistered “3” PAYG 3GB SIM that I bought as a spare in case EE crashes badly, from a friend who discovered that reception is crap where he lives, connect it to the secondhand laptop that I bought (with cash not plastic) from one of those “Cash Converter” type places, and NOT use the VPN because they know who I am, would it?

    Somebody explain to Auntie Theresa, please.

  10. Captain Cretin says:

    The most secure way way would be to use snail mail; it is obvious from all the mis-directed mail I get, that no one at the Post Office knows how to read these days.


Comments are closed.

Comments RSS Feed

Javascript must be enabled to post (most browsers do this automatically)

Privacy Notice: Please note that news comments are anonymous, which means that we do NOT require you to enter any real personal details to post a message. By clicking to submit a post you agree to storing your comment content, display name, IP, email and / or website details in our database, for as long as the post remains live.

Only the submitted name and comment will be displayed in public, while the rest will be kept private (we will never share this outside of ISPreview, regardless of whether the data is real or fake). This comment system uses submitted IP, email and website address data to spot abuse and spammers. All data is transferred via an encrypted (https secure) session.

NOTE 1: Sometimes your comment might not appear immediately due to site cache (this is cleared every few hours) or it may be caught by automated moderation / anti-spam.

NOTE 2: Comments that break our rules, spam, troll or post via known fake IP/proxy servers may be blocked or removed.
Cheapest Superfast ISPs
  • Vodafone £19.50 (*22.50)
    Speed 38Mbps, Unlimited
    Gift: None
  • NOW £20.00 (*32.00)
    Speed 36Mbps, Unlimited
    Gift: None
  • Hyperoptic £20.00 (*25.00)
    Speed 50Mbps, Unlimited
    Gift: Promo Code: BIRTHDAY10
  • Shell Energy £21.99 (*30.99)
    Speed 35Mbps, Unlimited
    Gift: None
  • Plusnet £22.00 (*38.20)
    Speed 36Mbps, Unlimited
    Gift: £60 Reward Card
Large Availability | View All
Cheapest Ultrafast ISPs
  • Gigaclear £24.00 (*49.00)
    Speed: 300Mbps, Unlimited
    Gift: None
  • Vodafone £24.00 (*27.00)
    Speed: 100Mbps, Unlimited
    Gift: None
  • Community Fibre £25.00 (*27.50)
    Speed: 200Mbps, Unlimited
    Gift: None
  • Hyperoptic £25.00 (*35.00)
    Speed: 150Mbps, Unlimited
    Gift: Promo Code: BIRTHDAY10
  • Virgin Media £28.00 (*52.00)
    Speed: 108Mbps, Unlimited
    Gift: None
Large Availability | View All
The Top 20 Category Tags
  1. FTTP (3558)
  2. BT (3023)
  3. Politics (1937)
  4. Building Digital UK (1926)
  5. FTTC (1887)
  6. Openreach (1837)
  7. Business (1691)
  8. Mobile Broadband (1479)
  9. Statistics (1408)
  10. FTTH (1365)
  11. 4G (1277)
  12. Fibre Optic (1174)
  13. Virgin Media (1170)
  14. Wireless Internet (1160)
  15. Ofcom Regulation (1148)
  16. Vodafone (846)
  17. EE (834)
  18. 5G (771)
  19. TalkTalk (769)
  20. Sky Broadband (747)
Helpful ISP Guides and Tips

Copyright © 1999 to Present - ISPreview.co.uk - All Rights Reserved - Terms , Privacy and Cookie Policy , Links , Website Rules , Contact