ECN

Wed Nov 13 17:57:10 UTC 2019

It does when the split flows land in different anycast origin POPs. 
Making a few assumptions from the traceroutes, the ECMP paths are sending 
some packets to Hamburg and some to Denmark.  Each POP may be getting 
parts of what should be a single TCP stream, and I doubt they have 
anything to cope with that (another assumption).

On Wed, 13 Nov 2019, Matt Corallo wrote:

> Not ideal, sure, but if it’s only for the SYN (as you seem to indicate), splitting the flow shouldn’t have material performance degradation?
>
>> On Nov 13, 2019, at 11:51, Toke Høiland-Jørgensen <toke at toke.dk> wrote:
>>
>> 
>>
>>> On 13 November 2019 17:20:18 CET, Matt Corallo <netadmin at as397444.net> wrote:
>>> This sounds like a bug on Cloudflare’s end (cause trying to do anycast
>>> TCP is... out of spec to say the least), not a bug in ECN/ECMP.
>>
>> Even without anycast, an ECMP shouldn't hash on the ECN bits. Doing so will split the flow over multiple paths; avoiding that is the whole point of doing the flow-based hashing in the first place.
>>
>> Anycast "only" turns a potential degradation of TCP performance into a hard failure... :)
>>
>> -Toke
>
>

----------------------------------------------------------------------
  Jon Lewis, MCP :)           |  I route
  StackPath, Sr. Neteng       |  therefore you are
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________