Two UK Broadband ISPs Trial New Internet Snooping System
Two unnamed broadband or mobile ISPs are reportedly helping the UK Home Office and the National Crime Agency (NCA) to trial a new internet snooping system on their customers, which is being conducted as part of the controversial 2016 UK Investigatory Powers Act (aka – snoopers charter).
The act introduced a new power that, among many other things, could force ISPs – upon being ordered to do so by a senior judge – into logging the Internet Connection Records (ICR) of all their customers for up to 12 months (e.g. the IP addresses of the servers you’ve visited and when), which can be accessed without a warrant and occurs regardless of whether or not you’re suspected of a crime.
The Communications Data Code of Practice, which was finalised in 2018, largely indicated that an ICR would “only identify the service that a customer has been using” and this is likely to involve the retention of various different pieces of data (varying between ISPs/networks).
However, the core ICR data should include a customer’s account reference, source IP address, destination IP address + port and the date/time of the start and end of the event or its duration. Other data may additionally be added if available (e.g. volume of data transferred and partial URLs – i.e. only that which contains communications data, not content).
Simplified Interpretation of an ICR Log
| Account ID |
Date (Time) | Source IP (You) |
Destination IP:Port | Data Volume | URL |
| 1 | 19/01/2017 (12:01) | 84.56.232.71 | 123.45.62.86:80-HTTP | 800KB | omgfakeballz.com |
| 1 | 19/01/2017 (13:12) | 84.56.232.71 | 65.123.45.90:21-FTP | 0.2KB | ftp.faketest.co.uk |
| 65 | 19/01/2017 (13:14) | 84.79.130.47 | 190.45.62.86:80-HTTP | 1700KB | icanhasyourdata.net |
The IPAct effectively prohibits ISPs from talking about much of this, which makes it difficult to verify the details (it also makes it difficult for ISPs to share experiences when developing best practices for the code), but a new article on Wired has provided the first useful update on this work in some years and confirms that two ISPs are helping to develop the system (BT seem like a fair bet to be one of those, but this is not confirmed).
A spokesperson for the Investigatory Powers Commissioner’s Office (IPCO) confirmed the trial is ongoing and that it is conducting regular reviews to “ensure that the data types collected remain necessary and proportionate“. At this stage the trial is described as being “small scale,” which we’d surmise to mean that it hasn’t yet been enabled for the entire customer base of each ISP.. yet.
Recent court challenges mean that, technically speaking, such data can only be stored (or ordered to be stored) if it is considered necessary and proportionate to do so, such as in the course of helping to fight serious crime. But the Government’s definition of what is and is not a “serious” crime has sometimes been called into question, while the IPAct has also faced some related legal challenges (here).
Meanwhile it’s reported that the NCA has spent at least £130,000 on two external contracts, which are being used to commission companies to build the underlying technical systems to run trials. Assuming all goes well then the Government will want to see this system being rolled out nationally and that could be a real burden for some ISPs.
The IPAct is due for its first 5-year review in the next year, which some hope could be an opportunity to improve its transparency. On the other hand, there may be fears that, without the protection of the EU’s charter, the UK government may seek to make the law even more intrusive and thus to the detriment of personal privacy. Well.. we’re sure nobody would ever want to abuse a mass national snooping system, no not at all (*tongue firmly in cheek*).
Mark is a professional technology writer, IT consultant and computer engineer from Dorset (England), he also founded ISPreview in 1999 and enjoys analysing the latest telecoms and broadband developments. Find me on X (Twitter), Mastodon, Facebook and Linkedin.
« CWU to Ballot on National Industrial Strike Over BT Changes UPDATE
Rural ISP Airband Brings Gigabit Broadband to Oxfordshire UK »
Latest UK ISP News
- FTTP (5514)
- BT (3514)
- Politics (2536)
- Openreach (2297)
- Business (2262)
- Building Digital UK (2244)
- FTTC (2043)
- Mobile Broadband (1972)
- Statistics (1788)
- 4G (1663)
- Virgin Media (1619)
- Ofcom Regulation (1461)
- Fibre Optic (1395)
- Wireless Internet (1389)
- FTTH (1381)
Time to route everything through a decent foreign VPN provider,
you are talking about a Proxy, not a VPN. by definition, a Virtual Private Network overlays a private network over the public Internet connection, either as a site to site link, reducing the need for a dedicated tie-line, or remote access, allowing employees, contractors and partners access to corporate resources from home.
Also, Proxy servers can never be seen as secure as you cannot guarantee where your Internet connection is being routed to, or who will see your data. They won’t ever improve performance as you are introducing latency and additional hops between you and the Internet data, and also, your home ISP will always have a record of the connection, even if it is over a non-standard port, they will always see there is a tunnel traversing their network.
Calm your tits, I’m talking about routing through a VPN such as https://mullvad.net/en/
We all know how they work, I don’t care for a randos ego-led explination
Unfortunately it’s becoming a situation where a foreign shady VPN provider is more trustworthy than your own ISP (and government).
Anyone else remember Phorm?
Vpn’s and proxies can improve performance in some circumstances, usually only happens if the route between your ISP and the destination isn’t optimal or is congested then sometimes there can be better routing between your ISP and the VPN provider then onwards to the original destination.
Jesus, this is beyond scary.
So the only current option is to stay with smaller providers or ethical ones like Andrews and Arnold?
For now, that’s the only option though we have no idea how long that will last
If it’s law I doubt it will be safe under smaller ISP’s for long, they still have to abide by the law. But Scotland is about to pass a bill that means if your seen to incite a hate crime, not even commit one, even if it’s in your own home in private like your mouthing off at some news report. You can be criminally investigated for it. 1984 becoming more true by the day.
@M
Quote “if your [sic] seen to incite a hate crime, not even commit one”
I’m pretty sure that incitement *is* itself a crime!
Hate crime is a funny old subjective definition.
If some sad sack in his basement is actively running a website “I’m going to kill all the XYZs and this is how I’m going to do it” I’m all up for plod going in and having a word.
It gets a bit ridiculous at the tail end, classic example being the Count Dankula YouTube case. I mean ffs, the bloke made a pisstake video where he showed his dog watching Nazi videos and trained him to put his paw up when he said Zieg Hail (or whatever it was). The silly bender even put a similar video up training his dog to do the same to BLM.
Why on earth he should get an £800 and prison sentence is chuffing beyond me.
Will this be enforceable on foreign ISPs, such as Starlink?
Good question and I’m not 100% sure, but extraterritorial provisions at the physical network layer of a Satellite network would seem to be out of scope for the IPAct (both legally and technically). However, it’s at least conceivable that they could find a way to impose something via UK ground stations, yet the reality is they’re going to be more focused upon the big fixed and mobile operators.
In order for a foreign internet company to be exempt without chance of retaliation by the UK they’d need to have all their operations outside of the UK, and compatible end user hardware would have to be purchased separately elsewhere (like the dish).
Starlink will be automatically in scope if they have a teleport in the UK, if they don’t they could be brought into scope by the simple process of issuing them with the appropriate legal documentation, see
https://www.legislation.gov.uk/ukdsi/2018/9780111163610
So basically just a netflow/sflow collector the $gov can log in to?
I’m completely against this 100% but a genuine question, just how long will the Gov allow the smaller ISP’s not be required to do it?
Some insane backbencher would argue the smaller ISP’s are a “safe place for those wanting to evade surveillance”
Worrying times ahead.
I would imagine as soon as the tech is production ready then all ISPs would have to implement within a specific time line or face the consequences. I don’t think they have a choice about it as its law…
As mentioned by others, just use a foreign VPN service if you are worried about being snooped on.
… until those become illegal as well.
This is not going to end well.
At the moment it seems to be more of a camel’s noes under the tent move, once firmly established they will then push it on to smaller ISP’s especially as the tech becomes cheaper.
Also in China VPN’s are still available to some degree, but they also know your using them and have been known to harass those that use them (if they are politically active).
Way back in 2000 under the first version of RIPA I was working in an IT business whose director was also a director of a very small ISP. Even smaller ISPs were forced to pay for and install what I think was some sort of packet sniffing hardware (watch A Good American, a documentary about ThinThread and whistleblower William Binney it sounds exactly like what he describes) that GCHQ could directly access even then.
Hands up if you think GCHQ/MI5 wasn’t already doing this…
I have always presumed this to be the case, and to be honest it never really bothered me. While the spooks were acting illegally, this in itself was a limitation of scope. Now it’s being done so openly, they may as well put on a web front-end and pitch a free-tier + subscription upsell.
Welcome to the servalance state, did not realise we were part of the federation of Russia/China
BT and EE
Lawful intercept is nothing new, though, right?
If you’re going to go the VPN route, it might be worth considering using different card details.
Imagine how easy it would be to setup up a ‘DNS server’ that looks up, for example, a three tuple hash of yourname.youraddress.yourdebitcard details.
Require all providers to maintain such a server so the establishment can easily see who you’re paying for connectivity of any sort.
How do we even choose a VPN provider and is one provider enough?
AAISP’s L2TP VPN can already be used to shave 20ms off UK to US RTT, if you’re on VM. Others, too, possibly? How much extra latency would be added with some pimped out encryption?
You could just avoid all the big ISPs because some of the smaller outfits barely have the capability to terminate a GEA subscriber properly, let alone inspect the originating traffic.
Should we just accept we have zero privacy and forget all about it?
I use crypto to pay for my VPN and I’m pretty sure that will be the norm soon
Unless you’re using XMR, there’s not really any privacy in that.
Are there not random online credit catd generators that issue a one use authorised transaction, completely obscures all payee details? Just a thought..
I wouldn’t worry about it all you guys this will be for sex offenders like Paedophiles and threats to the uk like terrorist Communication and other illegal crimes. It will not be for general internet use.
The problem is that the system can be repurposed against the population once it’s up.
I don’t think most are too afraid of the current government but it’s about the bigger picture.
so it wont be used to curb any mass protesting then? because all the data such a system would collect would make it very easy to gauge political opinion.. history has always proved that information is power and the more of it you have the better, the biggest problem is that when it comes to privacy and use of private information that the allure of making money from it often happens very quickly, just look at every Tom, Dick and Harry that has been listed as having the ability to access current information..
The list is a bit longer than that –
Who can request access to an ICR?
Internet Connection Records can be accessed by agencies. The complete list is:
• Police and National Crime Agency (NCA)
• Intelligence agencies: MIS, MI6 and GCHQ
• HM Revenue and Customs (HMRC)
• Department of Transport
• Department for Work and Pensions
• Serious Fraud Office
• The Scottish and Welsh Ambulance Service Boards
• Local councils
• National Health Service (NHS)
• The Ministry of Defence
• Department of Health
• Ministry of Justice
• Competition and Markets Authority
• Criminal Cases Review Commission
• Food Standards Agency
Local authorities and council officials will not be permitted access to this data.
You forgot the environment agency……
Ha ha ha.
Once upon a time when the Police were granted new powers to force someone to give over the password for encrypted files, or to unlock a phone they had confiscated, we were “reassured” that it was a special power that would not be abused, because it was very specifically only ever to be used as an anti terror thing.
A few months later, it was used because when confiscating computer equipment from someone who was not suspected of (or charged with) a terrorism act, he was prosecuted because he had forgotten what password a file was encrypted with on a CD containing backups that were at least a decade old.
No, I won’t be sleeping soundly with this latest news.
If there is any modicum of hope here however, it will be that the sheer storage requirements to log every outgoing (and presumably incoming too) connection that every customer makes over a 12 month period is deemed to be insane and impractical.
(Open Wireguard for 5 minutes and log WAN requests at the router)
The only way this could possibly work would be if some third party was providing the logging service, and the data would be stored in the cloud. But I can almost guarantee that a treasure trove of the websites that every person in the UK visits over a 12 month period, is going to be hacked in the first few months, because that sort of information is extremely valuable to scammers.
Doesn’t DNS over TLS prevent this logging as you wouldn’t use your ISP DNS and the request is encrypted (along with HTTPS instead of HTTP requests for the opened comms)?
Being cynical, I would guess this is MORE about getting revenue for BPI/Media Distributors through logs as court evidence than a government caring about its population. We are all cash cows and consumers in their eyes, dispensable plebs….
They will still see where you went, the DNS query wouldn’t be visible but they would see you went to IP X.X.X.X and then could probably do a reverse lookup and figure out what site you visited
This is talking about metadata for the IP or TCP/UDP connections that you might establish after the DNS request (and those of the request itself). It wouldn’t contain an encrypted DNS request’s content, but if you make a request and then immediately open a connection to a new IP, perhaps you were looking up that website. I think TLS still provide the site name so as to determine what certificate to use, if any, too.
Of course, if you actually use your connection – and let’s face it, who doesn’t, nowadays? – they may have a lot to sort through. Bonus marks if whatever you’re doing results in a bunch of legitimate DNS requests as well as both inbound and outbound connections.
Reverse lookups only really help if the IP belongs to a server that hosts a single site. Some servers host thousands.
Nosey buggers…. a system which is likely to get abused.
Got into a habit of using a no logging VPN which has been verified by KPMG a few years back even if it is to read the news
“Likely”?
It is as certain to be abused as the sun rising. Give it time.
I’ll just start turning my VPN on all the time
Wouldn’t be easier to do like NSA and tap directly into Google and Facebook? Are we freaking about IPs now? Our society is being reconstructed by corporations but we still fear the government?
Good job “Project fear”.
Better corporations rather than your own government, am I right?
PS: I work in a tech company (Top 10 Nasdaq) so yeah…
+1
Who do I fear more? Corporations who have to persuade me to part with my money, or the government, who have guns and jails and police?
What about when the two are combined?
God that’s so 80s
The concept seems to have the ability to note the size of whatever encrypted traffic you up/down load via VPN – or what about FTP – it should note the size of either but not be able to see the content of either.
So a seedbox you empty via encrypted FTP, that grabbed it via it’s built in torrent indexer that you search instructed over encrypted instructions would only show evidence of large impenetrable lumps that arrived via FTP?
I’m altogether too wholesome for this to be of any concern. Now if you’ll excuse me, I’m due for tea at the Vicar’s, and I’m already running late. Heavens!
This is absolutely disgusting. Looks like I may have to start running a VPN full time, which is something I wanted to avoid doing.
Governments have been overstepping ever so gently for years now. This is just another step in the direction of stopping free speech and preventing organised groups that could harm a Government’s power trip
Sometimes I get the impression that everything that sucks comes from the united kingdom.
Nice, this should push more people to use TOR and the more the merrier. It sort of also makes it more anonymous for individual users, if I remember correctly.
They’d mostly see me searching for words that I don’t know how to spell so they can be my guest.
Why would anyone be surprised by this?
It’s everywhere.
How does TOR work to combat this government snooping?
Saying that, I did also consider this years census had questions in it that were probably being sold for profit by our own government.
Don’t use your isp dns, there’s plenty to choose from that are safe, and use a vpn.
Might not be perfect, but it’s better than nothing.
And anyone who is actually up to no good will be using secure tor anyway.
This should worry UK citizens.
The National Crime Agency is compared to the FBI, but it’s much worse: the NCA has complete immunity to the Freedom of Information Act, meaning it’s not possible for the public to hold them to account or judge whether their actions are proportionate. The only other agencies with this level of immunity are national security (GCHQ, MI5, MI6). Even the name of the officer who rejects the NCA’s FoI claims is redacted, as are the proceedings of the court (IPT) who citizens must use if they wish to sue the intelligence agencies.
The NCA usually justifies by citing child abuse, but they’ve been caught lying about in the past (by their own report, no less, after they falsely claimed a link between child abuse images and organized criminals to justify the merger between CEOP and SOCA that formed the NCA — the subsequent year’s report found no such link). You can also compare the NCA’s figures to actual crime statistics from police and courts, and I recommend that brave journalists reading this do so: the NCA is free to conceal inconvenient figures or cite statistics without revealing sources and methods necessary to make that support (even shampoo advertisements have to tell consumers the sample size of their group). The NCA can thus freely exaggerate the danger to the public and hide inconvenient statistics such as low crime rates and low rates of recidivism.
Naturally, if left unchecked, the purview of the NCA will expand over the next decade, as the government believes in mass surveillance as a silver bullet to prevent crime of all sorts – even petty crimes. Society at large will continue to assume that the privacy of law-abiding citizens will be protected, and the secrecy of such systems will not allow them to learn otherwise. Once in place, no major party will restore the citizens’ right to privacy for fear that it will lead to a rise in crime.
Even tech-savvy UK citizens will not be protected for long. If technology is effective at preventing government surveillance, government will simply pass a law forbidding it to the people. Tor, VPNs, DNS over HTTPS, any form of encryption not weakened by government-mandated backdoors: the government will simply outlaw these if it can’t circumvent them. Don’t forget that the government need only serve a backdoor into a few companies to have access to over 90% of website visits: Google (via Analytics and CAPTCHA used on most websites), Facebook (via Like buttons), Cloudflare (DNS over HTTPS), Amazon (hosting and CDN).
Mark my words: Internet surveillance now is a standard feature of British society, and this new power will not be rolled back once it is in place. Anyone who opposes it will be smeared by the press as a sympathizer to whichever criminals the government claims the system is in place to stop (and this will be whichever society hates the most at the time). The NCA is just GCHQ, but deployed against British citizens instead of foreign adversaries.
Sadly, I do not think the British people have the will or the way to stop this increasing erosion of liberty, privacy and human rights.
You can bet your life that one of these ISPs is BT. Time to ditch the popular ISPs and find a small niche one that’s safer.
Small ISPs are easily covered by collecting the information at wholesaler level instead (usually BT Wholesale or TalkTalk Business for most smaller ISPs using openreach lines).
And they people criticised China for spying, I think they mad because they didn’t get thier first, the government is a hypocrite
This tech is used by Sky Broadband and has been for 3 years now.
They block all users using Illegal streaming services by snooping on sky bb users actively looking for tags lines like: player_api or get.php
The company they use is Friendsmts.com based in Birmingham UK were they are paid to snoop on all users!
Now what gets me, this friendsmts not being owned by the government is allowed to watch Everything and I mean EVERYTHING You do if your a sky bb user!
Just let that sink in a moment! Sky has allowed a company that no one knows to watch EVERYTHING you do and record your data!
VPN or nothing from now on as you are being watched!
That’d be Friend MTS. The company doesn’t do what you describe, that’d be illegal in the UK.
The company goes after those rebroadcasting, not viewers, and doesn’t have a DPI feed into Sky’s network.
Lol
You are aware that Sky just resell BT Internet right?
Sky buys wholesale connectivity from Openreach and has an unbundled network into exchanges, so they’re not just a reseller for BT.
The proof of the pudding? Where are the hoards of innocent victims of government agency snooping, in court protesting their innocence? HM Govt. isn’t interested in listening in to your family chats, but any ‘free’ society needs some intel oversight. More concerning is our privacy being in jeopardy where almost every trip to a website involves giving agreement to being spied on, even if only as metadata. I’m worried about criminals finding new ways around *any* spook system, but equally concerned by open damage done by anti-vaxxers and QAnon types spreading nonsense on Twitter and Facebook under cover of free speech — or cleverly-worded editorials which foment race issues without actually crossing a legal line of ‘hate speech’ (think ex-POTUS). Many people are gullible and believe those lies. OK, we have lots of foreign nationals among us of whom a small percentage loathe and detest our culture, and they obviously need governmental surveillance if we’re not to keep hearing (too late), ‘the authorities say this man was on their radar, but was thought unlikely to carry out this dreadful terrorist crime’. We know we must pay some price for security, so the real question is, how intrusive should it be, and what are the safeguards?
It’s easy enough to set up a TOR proxy on your internal network.
Just search “setup tor proxy” on DuckDuckGo. There is loads of information out there. You can have one up and running in a matter of minutes and then you can set up your web browser (or your whole network), to route all the traffic through the proxy.
If using a private VPN provider, just make sure they are a 100% “no logs” service.
Another option is to use DNS over HTTPS so all your DNS queries are encrypted. This can be done using cloudflared. Then on your network block any outgoing traffic on port 53 to make sure no DNS queries leave your network unencrypted.
Then there is also recursive DNS. This is not encrypted but does increase privacy due to the number of network hops and the fact that it caches for local use afterwards.
Basically:
the government is watching everything you do and there is nothing you can do about it
Also reminder that London has the 4th most “surveillance” (actually just spying) cameras in the world, all the other cities are in China.
Does anyone know or suspect who the 2 ISP’s are that are running trials ?
BT definitely i think and perhaps virgin media considering their past
but no confirmation there
This is not Not new thing, officially were introduced 21years ago, All ISP required by law to install Gov BlackBox to monitor comms, people eclect politicians,once are on the thrown, firstly;they kicks people’s Asses, please read below:
https://en.wikipedia.org/wiki/Regulation_of_Investigatory_Powers_Act_2000
Ever since I saw this article 2 months ago I’ve been using a VPN, first on a cloud hosting provider, then through a paid service. Connecting to the Netherlands I don’t notice any change in connection speed or my bandwidth’s capabilities whatsoever, I really recommend that ANYONE in the UK who uses the internet for anything outside of twitter/facebook should use a VPN at all times. Don’t be a part of a GCHQ mass data collection programme.
If anyone is looking for a VPN that respects privacy, I’ve had a good experience with AirVPN so far, they are one of very few VPN companies that allow effortless port-forwarding so you can torrent while connected to a VPN (and no I’m not a spammer – I just like their service).