Broadband ISP YouFibre, which is one of the retail outlets for Netomnia’s (inc. Brsk) 8Gbps capable full fibre (FTTP) network, has worked with NFWare to deploy the “industry’s first” 400Gbps virtual Carrier Grade Network Address Translation (vCGNAT) servers to its customers. The move should help them to keep up with their rapid subscriber growth.
Netomnia’s full fibre network currently covers over 1.8 million premises, but they’re aiming to reach 2 million UK premises (homes and businesses) and 235,000 customers by the end of 2024, before then rising to 3 million premises by 2025 (inc. 1 million customers by 2028).
However, delivering an annual build rate of 1 million premises passed, while also growing their customer base by 100k in the space of just nine months, does mean they need more IPv4 addresses. YouFibre already tackles this by adopting IPv6 addresses with a mix of CGNAT (i.e. IP address sharing) on consumer plans, while also offering Static IP addresses as an extra option (£5) for those needing more flexibility.
Advertisement
The big development today is that they’ve worked with NFWare to deploy their 400Gbps vCGNAT servers too. “This vCGNAT performance utilized four 100GbE network interface cards (NIC) and is double the previous fastest server. This throughput is a result of performance improvements in the latest version of NFWare’s vCGNAT software and the use of tuned Intel® Xeon® Scalable Processor-based servers,” said the announcement.
Sam Defriez, Director of Networks at YouFibre, said:
“We’re building our network to give our customers a great internet experience even as we grow our business very rapidly. Having a 400 Gbps vCGNAT server is not just a bragging point – it provides significant value in accommodating our growth and keeping our infrastructure costs as low as possible.”
YouFibre has so far installed the 400Gbps vCGNAT setup at three of its highest volume Points of Presence (PoPs), while also using 200Gbps NFWare vCGNAT servers at less busy sites. The servers are based on 36-core, 2.4GHz Intel® Xeon® Platinum 8360Y CPUs. NFWare engineers consulted with YouFibre to tune the servers, including using non-uniform memory access (NUMA) to ensure fastest possible memory access for the CPUs. Each server features eight 100GbE connections provided by four Mellanox Connect X6 network adapters.
“Some people still think that there is a performance tradeoff when using virtualized software but it’s not true,” said Alex Britkin, co-founder and CEO of NFWare.
Advertisement
Would any of this costly energy sucking kit be needed if the industry had bothered to sort IPv6?
In short no. We also seem to be obsessed in the UK with energy sucking PPPoE.
deeply curious to know how PPPoE is “energy sucking”. Can’t begin to understand that one.
Some people are hell bent on PPP being the devil for some reason.
“deeply curious to know how PPPoE is “energy sucking”. Can’t begin to understand that one.”
It requires substantial additional processing power. On a consumer router that might only be an extra 5 watts, but it adds up over millions of customers.
yeah PPPoE seems to break a lot of the hardware NIC offloading too (the hash functions just don’t work if you use PPPoE) – something I painfully discovered when using pfsense and migrating to a provider which used PPPoE.
fortunately after over a year of complaining to my altnet provider about it, they have moved to DHCP and everything now is great.
@Ivor even if the extra processing power of wrapping and unwrapping IP packets unnecessarily into PPPoE was 0.1W per connection then, over 10 million connections, that is 1MW of power being consumed entirely unnecessarily. It’s the aggregate power draw over millions of connections that is the issue.
Think it was more that pfsense was built on FreeBSD and FreeBSD had a single threaded PPP daemon to be honest. Well known issue. Same hardware running software that could multithread PPP fixed performance problems.
What do you define as ‘necessary’ Jonathan? Migrating a wholesale network from PPPoE to IPoE isn’t trivial. Can do it in a very messy way, build up a ton of technical debt and cause huge issues down the line if it’s not done properly. At some point they can for sure stop provisioning new customers on PPP however the core has to be ready to roll before that.
Yeah PPPoE is a pain, I think at least a couple of the new altnets offer DHCP which is nice, I think part of the problem is the existing players have built up their systems over many years using PPPoE, so it would be a big shift for them to allow a different way of connecting to their network, so they keep what they got.
Assuming the cost of it will be growing together with user database, how much will it cost compared to /16 blocks or smaller? Because we have to be clear that IPv4 will never die and CGNAT is not the solution for all customers.
CGNAT definitely isn’t the solution for all customers… But it works perfectly fine for a significant number of customers.
(although probably only a minority of ISPr readers!)
Zen wanted £44+VAT per address for a /16.
“how much will it cost compared to /16 blocks or smaller”
There are no /16 blocks left and even a single /24 is getting to be a problem. Yes you can still buy some on the secondary market but that’s already not a sustainable option for consumer broadband, as this article shows. IPv4 died a decade ago, it’s just the life support systems like CGNAT are giving the patient the appearance of breathing.
Old Blue Shirt Guy: there are /16 blocks available to buy: https://auctions.ipv4.global/
@Ivor – PPPoE introduces encryption of sorts for the packets which in some cases (if not all) slows down the connection speed as the router CPU has to do a lot of extra work to encrypt and decrypt the packets on the fly and it is just not needed
not the case with capable hardware – ie not the sort of overpriced sludge that people often choose to use.
put another way – ISP supplied routers can generally do hardware accelerated PPPoE all day long and it does it while consuming a couple of tens of watts in the worst case (less when idling). If your pfsense/opnsense PC or ubiquiti dream whatever can’t cut the mustard then that’s not the ISP’s problem or one inherent to the use of PPPoE.
Similarly, at the ISP end this stuff is hardware accelerated and built into the same box that would be handling IPoE (and possibly does for other types of service, such as leased lines)
In some (even older) routers there is hardware acceleration/offloading built in which makes this overhead negligible both in terms of CPU overhead and and energy usage.
If you are using a more modern CPU on an average broadband connection even without hardware acceleration, it is no longer going to be a problem. I remember when I got the first (384kbps) ADSL connection a bad pppd configuration could eat a significant amount of CPU but I haven’t come across this problem in the last 15 years.
PPPoE (MPPE) can encrypt data between PPP points and thats all. All data leaving PPPoE is not encrypted because this is being handled by HTTPS etc. Also for MPPE to make sense it should use 128bit encryption. But who is using this in the UK? If nobody cares why not use IPoE?
No it doesn’t. PPP packets are not encrypted! I don’t think there is even a checksum from memory!
Meant to add this also..
It’s an 8 byte shim. No CPE manufactured in the past 10 years has issues running at line rate with that.
You can get into the MTU argument if you want… that is 8 precious bytes you’re taking from the payload. Most – if not all – providers support baby jumbos to give you that 8 bytes back.
It’s encapsulation rather than encryption. It doesn’t add a huge amount of work and a fair amount of consumer hardware uses chipsets that have ASIC acceleration for it. I know one of the software solutions struggled because its PPP client was single threaded but that’s on them, others were fine.
You certainly wouldn’t build a network with it now and only know one ISP that’s a fan. The biggest wholesale provider want rid but not an easy thing to switch off on an integrated network let alone a wholesale one.
99.99% of mainstream routers have no issues coping with PPPoE authentication.You can’t expect CPs to remove PPPoE based on the very tiny minority.
MIS-INFORMATION ALERT:
BT Ivor, really, the over priced slush is vastly better in most cases than an ISP supplied router; with quad core processors, memory, hardware acceleration etc. Most (not all as some ALTNETS supply decent routers) ISP routers are cheap garbage because they have to be to keep costs down for the ISP.
Mostly momentum, Name. The two biggest wholesale providers, BT and PXC, both provide services over PPPoE. ISPs built up their infrastructure around who their providers were.
One ISP it’s a major part of their product. Others it was the easiest way and made sense at the time. A bunch of engineers worked on PPP for years and it is what they know. To do it properly IPoE needs more than a regular IP network with L2TP tunnels running on it as PPP does.
PPPoE != Encryption but indeed Encapsulation and indeed very little or no overhead at all, as said.
I would still remove PPPoE from the access if I could, those 8 bytes are indeed more precious than one might think. Path MTU D is Fragile (https://datatracker.ietf.org/doc/html/rfc8900)
Thus, 1500 bytes IPv4 end to end is the optimum to try to avoid it, made possible through “PPP Max Payload Extensions” which not all CPE support, BT Wholesale BRAS/MSE DOES support PPP Max Payload Extension, but some Openreach GEA products doesn’t support the bigger lower layer frames (go figure).
One can always set the PC’s NIC to 1492 though, remove the chance the PMTUD is in play. Helpfully, IPv6 can signal the WAN MTU to the LAN in the RA packet to tune this automatically, but again, not all CPE do this either.
Hmmm. Which products do Openreach not support it on? That’s weird. Should be okay with FTTC and FTTP.