ECN

Wed Nov 13 21:22:06 UTC 2019

On Wed, Nov 13, 2019 at 11:36 AM Saku Ytti <saku at ytti.fi> wrote:

> On Wed, 13 Nov 2019 at 18:27, Matt Corallo <nanog at as397444.net> wrote:
> > This sounds like a bug on Cloudflare’s end (cause trying to do anycast
> TCP is... out of spec to say the least), not a bug in ECN/ECMP.
>
> Not true. Hash result should indicate discreet flow, more importantly
> discreet flow should not result into two unique hash numbers. Using
> whole TOS byte breaks this promise and thus breaks ECMP.
>

Yes true.

Equal Cost MultiPath (ECMP) consistency over the life of a TCP connection
is not a promise. Anycasters would love it to be but it's not.

ECMP's only promise is that packets for a particular connection will tend
to prefer a particular path so that throughput doesn't suffer overly much
from the packet reordering you'd get by round-robining the packets on
different links. Choosing an alternate path during congestion is a
perfectly reasonable thing for ECMP to do.

Don't blame the network. This is Cloudflare choosing not to handle the
anycast spray corner case because it happens rarely enough with symptoms
obscure enough that they only occasionally get called to carpet. Their BGP
announcements make the claim they're ready for your packet at any of their
sites, but they're not.

Regards,
Bill Herrin

-- 
William Herrin
bill at herrin.us
https://bill.herrin.us/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20191113/2387222b/attachment.html>