Lossy cogent p2p experiences?

Wed Sep 6 16:28:04 UTC 2023

On 9/6/23 12:01, Saku Ytti wrote:

> Fun fact about the real world, devices do not internally guarantee
> order. That is, even if you have identical latency links, 0
> congestion, order is not guaranteed between packet1 coming from
> interfaceI1 and packet2 coming from interfaceI2, which packet first
> goes to interfaceE1 is unspecified.
> This is because packets inside lookup engine can be sprayed to
> multiple lookup engines, and order is lost even for packets coming
> from interface1 exclusively, however after the lookup the order is
> restored for _flow_, it is not restored between flows, so packets
> coming from interface1 with random ports won't be same order going out
> from interface2.
>
> So order is only restored inside a single lookup complex (interfaces
> are not guaranteed to be in the same complex) and only for actual
> flows.

Yes, this has been my understanding of, specifically, Juniper's 
forwarding complex.

Packets are chopped into near-same-size cells, sprayed across all 
available fabric links by the PFE logic, given a sequence number, and 
protocol engines ensure oversubscription is managed by a request-grant 
mechanism between PFE's.

I'm not sure what mechanisms other vendors implement, but certainly OoO 
cells in the Juniper forwarding complex is not a concern within the same 
internal system itself.

Mark.