Linux BNG

Sat Jul 14 12:13:32 UTC 2018

Hello

I am investigating Linux as a BNG. The BNG (Broadband Network Gateway) 
being the thing that acts as default gateway for our customers.

The setup is one VLAN per customer. Because 4095 VLANs is not enough, we 
have QinQ with double VLAN tagging on the customers. The customers can 
use DHCP or static configuration. DHCP packets need to be option82 
tagged and forwarded to a DHCP server. Every customer has one or more 
static IP addresses.

IPv4 subnets need to be shared among multiple customers to conserve 
address space. We are currently using /26 IPv4 subnets with 60 customers 
sharing the same default gateway and netmask. In Linux terms this means 
60 VLAN interfaces per bridge interface.

However Linux is not quite ready for the task. The primary problem being 
that the system does not scale to thousands of VLAN interfaces.

We do not want customers to be able to send non routed packets directly 
to each other (needs proxy arp). Also customers should not be able to 
steal another customers IP address. We want to hard code the relation 
between IP address and VLAN tagging. This can be implemented using 
ebtables, but we are unsure that it could scale to thousands of customers.

I am considering writing a small program or kernel module. This would 
create two TAP devices (tap0 and tap1). Traffic received on tap0 with 
VLAN tagging, will be stripped of VLAN tagging and delivered on tap1. 
Traffic received on tap1 without VLAN tagging, will be tagged according 
to a lookup table using the destination IP address and then delivered on 
tap0. ARP and DHCP would need some special handling.

This would be completely stateless for the IPv4 implementation. The IPv6 
implementation would be harder, because Link Local addressing needs to 
be supported and that can not be stateless. The customer CPE will make 
up its own Link Local address based on its MAC address and we do not 
know what that is in advance.

The goal is to support traffic of minimum of 10 Gbit/s per server. 
Ideally I would have a server with 4x 10 Gbit/s interfaces combined into 
two 20 Gbit/s channels using bonding (LACP). One channel each for 
upstream and downstream (customer facing). The upstream would be layer 3 
untagged and routed traffic to our transit routers.

I am looking for comments, ideas or alternatives. Right now I am 
considering what kind of CPU would be best for this. Unless I take steps 
to mitigate, the workload would probably go to one CPU core only and be 
limited to things like CPU cache and PCI bus bandwidth.

Regards,

Baldur