Bandwidth distribution per ip

Blake Hudson blake at
Wed Dec 20 17:45:08 CST 2017

Denys Fedoryshchenko wrote on 12/20/2017 11:38 AM:
> On 2017-12-20 19:16, Blake Hudson wrote:
>> Denys Fedoryshchenko wrote on 12/20/2017 8:55 AM:
>>> National operator here ask customers to distribute bandwidth between 
>>> all ip's equally, e.g. if i have /22, and i have in it CDN from one 
>>> of the big content providers, this CDN use only 3 ips for ingress 
>>> bandwidth, so bandwidth distribution is not equal between ips and i 
>>> am not able to use all my bandwidth.
>>> And for me, it sounds like faulty aggregation + shaping setup, for 
>>> example, i heard once if i do policing on some models of Cisco 
>>> switch, on an aggregated interface, if it has 4 interfaces it will 
>>> install 25% policer on each interface and if hashing is done by dst 
>>> ip only, i will face such issue, but that is old and cheap model, as 
>>> i recall.
>>> Did anybody in the world face such requirements?
>>> Is such requirements can be considered as legit?
>> Not being able to use all of your bandwidth is a common issue if you
>> are provided a bonded connection (aka Link Aggregation Group). For
>> example, you are provided a 4Gbps service over 4x1Gbps ethernet links.
>> Ethernet traffic is not typically balanced across links per frame,
>> because this could lead to out of order delivery or jitter, especially
>> in cases where the links have different physical characteristics.
>> Instead, a hashing algorithm is typically used to distribute traffic
>> based on flows. This results in each flow having consistent packet
>> order and latency characteristics, but does force a flow over a single
>> link, resulting in the flow being limited to the performance of that
>> link. In this context, flows can be based on src/dst MAC address, IP
>> address, or TCP/UDP port information, depending on the traffic type
>> (some IP traffic is not TCP/UDP and won't have a port) and equipment
>> type (layer 3 devices typically hash by layer 3 or 4 info).
>> Your operator may be able to choose an alternative hashing algorithm
>> that could work better for you (hashing based on layer 4 information
>> instead of layer 3 or 2, for example). This is highly dependent on
>> your provider's equipment and configuration - it may be a global
>> option on the equipment or may not be an option at all. Bottom line,
>> if you expected 4Gbps performance for each host on your network,
>> you're unlikely to get it on service delivered through 4x 1Gbps links.
>> 10Gbps+ links between you and your ISP's peers would better serve
>> those needs (any 1Gbps bonds in the path between you and your
>> provider's edge are likely to exhibit the same characteristics).
>> --Blake
> No bonding to me, usually it is dedicated 1G/10G/etc link.
> Also i simulated this bandwidth for "hashability", and any layer4 
> aware hashing
> on cisco/juniper provided perfectly balanced bandwidth distribution.
> On my tests i can see that they have some balancing clearly by dst ip 
> only.

Are you claiming that your bandwidth is being equally divided 1024 ways 
(you mentioned a /22) or just that each host (IP) is not receiving the 
full bandwidth? What is the bandwidth ordered and what is the bandwidth 
you're seeing per host(IP)?

More information about the NANOG mailing list