interesting troubleshooting

Sat Mar 21 17:42:44 UTC 2020

(skipping up the thread some)

On Fri, Mar 20, 2020 at 5:58 PM Jared Mauch <jared at puck.nether.net> wrote:
> It’s the protocol 50 IPSEC VPNs.  They are very sensitive to path changes and reordering as well.
>
> If you’re tunneling more than 5 or 10Gb/s of IPSEC it’s likely going to be a bad day when you find a low speed link in the middle.  Generally providers with these types of flows have both sides on the same network vs going off-net as they’re not stable on peering links that might change paths.

a bunch of times the advice given to folk in this situation is: "Add
more entropy", which really for ipsec/gre/etc vpns means more
endpoints.
For instance, adding 3 more ips on either side for tunnel
egress/ingress will make the flows (ideally) smaller and more probable
to hash across different links in the intermediary network(s).  This
also moves the loadbalancing back behind the customer prem so ideally
perhaps even the nxM flows are now balanced a little better as well.

sometimes this works, sometimes it's hard to accomplish :(