BGP and The zero window edge

Thu Apr 22 05:11:14 UTC 2021

On 22/04/2021 02:24, Job Snijders via NANOG wrote:
> On Wed, Apr 21, 2021 at 09:22:57PM +0000, Jakob Heitz (jheitz) wrote:
>> I'd like to get some data on what actually happened in the real cases
>> and analyze it.
>>
>> [snip]
>>
>> TCP zero window is possible, but many other things could
>> cause it too.
> 
> Indeed. There could be a number of reasons that caused it.
> 
> Switchings away from TCP win=0 towards "Zombie Routes":
> 
> *RIGHT NOW* (at the moment of writing), there are a number of zombie
> route visible in the IPv6 Default-Free Zone:
> 
> One example is http://lg.ring.nlnog.net/prefix_detail/lg01/ipv6?q=2a0b:6b86:d15::/48
> 
>      2a0b:6b86:d15::/48 via:
>          BGP.as_path: 204092 57199 35280 6939 42615 42615 212232
>          BGP.as_path: 208627 207910 57199 35280 6939 42615 42615 212232
>          BGP.as_path: 208627 207910 57199 35280 6939 42615 42615 212232
>      (first announced April 15th, last withdrawn April 15th, 2021)
>      
> Another one is http://lg.ring.nlnog.net/prefix_detail/lg01/ipv6?q=2a0b:6b86:d24::/48
> 
>      2a0b:6b86:d24::/48 via:
>          BGP.as_path: 201701 9002 6939 42615 212232
>          BGP.as_path: 34927 9002 6939 42615 212232
>          BGP.as_path: 207960 34927 9002 6939 42615 212232
>          BGP.as_path: 44103 50673 9002 6939 42615 212232
>          BGP.as_path: 208627 207910 34927 9002 6939 42615 212232
>          BGP.as_path: 3280 34927 9002 6939 42615 212232
>          BGP.as_path: 206628 34927 9002 6939 42615 212232
>          BGP.as_path: 208627 207910 34927 9002 6939 42615 212232
>      (first announced March 24th, last withdrawn March 24th, 2021)
> 
> Just now, I literally rebooted the BGP speaker behind lg.ring.nlnog.net
> to make ensure that those routes are not stuck in the BGP looking glass
> itself.
> 
> 2a0b:6b86:d24::/48 was first announced on March 24th, 2021, and
> withdrawn at the end of March 24th, 2021 by the originator, and now
> almost a month later, this prefix still is visible in the default-free
> zone despite WITHDRAW messages having been sent and the AS 212232
> operator confirming they are not announcing that IP prefix anywhere.
> 
> I checked the AS 6939 Looking glass, but the d24::/48 route is not
> visible in the http://lg.he.net/ web interface. This leads me to believe
> the the route got stuck somewhere along way in either of 201701, 204092,
> 206628, 207910, 207960, 208627, 3280, 34927, 35280, 44103, 50673, 57199,
> and/or 9002.
> 
> This implies indeed might be multiple reasons a BGP route gets stuck
> ('stuck' as in - a WITHDRAW was not generated, or ignored). Perhaps on
> any one of these edges there is a very high Out Queue for one reason or
> another:
> 
>      34927 9002
>      206628 34927
>      44103 50673
>      207960 34927
>      3280 34927
>      9002 6939
>      201701 9002
>      208627 207910
> 
> I'm not sure all the these sightings of stuck routes can be pinpointed
> to one specific BGP vendor (or one bug).

I would guess that all the stuck route sightings manifest from one 
undiscovered TCP library bug that some BGP vendors are all commonly using.

-Hank

> 
> Kind regards,
> 
> Job
> 