Bandwidth distribution per ip

Wed Dec 20 17:16:29 UTC 2017

Denys Fedoryshchenko wrote on 12/20/2017 8:55 AM:
> National operator here ask customers to distribute bandwidth between 
> all ip's equally, e.g. if i have /22, and i have in it CDN from one of 
> the big content providers, this CDN use only 3 ips for ingress 
> bandwidth, so bandwidth distribution is not equal between ips and i am 
> not able to use all my bandwidth.
>
> And for me, it sounds like faulty aggregation + shaping setup, for 
> example, i heard once if i do policing on some models of Cisco switch, 
> on an aggregated interface, if it has 4 interfaces it will install 25% 
> policer on each interface and if hashing is done by dst ip only, i 
> will face such issue, but that is old and cheap model, as i recall.
>
> Did anybody in the world face such requirements?
> Is such requirements can be considered as legit?

Not being able to use all of your bandwidth is a common issue if you are 
provided a bonded connection (aka Link Aggregation Group). For example, 
you are provided a 4Gbps service over 4x1Gbps ethernet links. Ethernet 
traffic is not typically balanced across links per frame, because this 
could lead to out of order delivery or jitter, especially in cases where 
the links have different physical characteristics. Instead, a hashing 
algorithm is typically used to distribute traffic based on flows. This 
results in each flow having consistent packet order and latency 
characteristics, but does force a flow over a single link, resulting in 
the flow being limited to the performance of that link. In this context, 
flows can be based on src/dst MAC address, IP address, or TCP/UDP port 
information, depending on the traffic type (some IP traffic is not 
TCP/UDP and won't have a port) and equipment type (layer 3 devices 
typically hash by layer 3 or 4 info).

Your operator may be able to choose an alternative hashing algorithm 
that could work better for you (hashing based on layer 4 information 
instead of layer 3 or 2, for example). This is highly dependent on your 
provider's equipment and configuration - it may be a global option on 
the equipment or may not be an option at all. Bottom line, if you 
expected 4Gbps performance for each host on your network, you're 
unlikely to get it on service delivered through 4x 1Gbps links. 10Gbps+ 
links between you and your ISP's peers would better serve those needs 
(any 1Gbps bonds in the path between you and your provider's edge are 
likely to exhibit the same characteristics).

--Blake