<html><head><style>body{font-family:Helvetica,Arial;font-size:13px}</style></head><body style="word-wrap:break-word;line-break:after-white-space"><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">Hey there!</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">In our environment we generally have ASR-1000X-2s everywhere peering via iBGP/eBGP. These routers have no redundant RPs, hence cannot keep forwarding traffic while the router reboots or crashes. As such, this is a clear example of a router that's only NSF-aware (or graceful-restart-aware) but not capable.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">The reason I enabled this is because, from RFC 4724:</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">In addition, even if the speaker does not have the ability to preserve its forwarding state for any address family during BGP restart, it is still recommended that the speaker advertise the Graceful Restart Capability to its peer (as mentioned before <b>this is done by not including any <AFI, SAFI> in the advertised capability</b>). There are two reasons for doing this. The first is to indicate its intention of generating the End-of-RIB marker upon the completion of its initial routing updates, as doing this would be useful for routing convergence in general. The second is to indicate its support for a peer which wishes to perform a graceful restart.</div></blockquote><div><br></div>So what I would expect to see in the "show ip bgp neighbor <ip>" command, regarding Graceful Restart, would be something like the following:<div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div>BGP neighbor is <IP>, remote AS <AS>, internal link</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div>[...]</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div> Neighbor capabilities:</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div>[...]</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div> Graceful Restart Capability: advertised and received</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div> Remote Restart timer is 120 seconds</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div> Address families advertised by peer:</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div> <span class="Apple-converted-space"> </span><b>none</b></div></div></div></blockquote><p>Basically, GR is negotiated, but no address family is specified, effectively only using the EOR marker for routing convergence improvements.</p><div>Instead, here's what the router specifies:</div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div>BGP neighbor is <IP>, remote AS <AS>, internal link</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div>[...]</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div> Neighbor capabilities:</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div>[...]</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div> Graceful Restart Capability: advertised and received</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div> Remote Restart timer is 120 seconds</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div> Address families advertised by peer:</div></div></div><div><div style="color:rgb(0,0,0);font-family:Helvetica,Arial;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div> <b>IPv4 Unicast (was not preserved, VPNv4 Unicast (was not preserved</b></div></div></div></blockquote><p>My assumption is that the 'was not preserved' in the parentesis refers to the most recent restart of the neighbor, and it means that when the neighbor re-established the BGP connection, the GR Capability for IPv4 and VPNv4 AFIs did not set the "Forwarding bit" as specified by the GR-RFC:</p><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><p>Once the session is re-established, if the "Forwarding State" bit for a specific address family is not set in the newly received Graceful Restart Capability, or if a specific address family is not included in the newly received Graceful Restart Capability, or if the Graceful Restart Capability is not received in the re-established session at all, then the Receiving Speaker MUST immediately remove all the stale routes from the peer that it is retaining for that address family.</p></blockquote><p>Clearly the Forwarding State bit is never going to be set by this type of router, due to hardware limitations. Here's my concern though: What happens when the router reboots, and the neighboring routers keep forwarding packets to this router because the GR-capabily did specify IPv4 and VPNv4 AFI/SAFI? This would clearly cause impact as traffic would be blackholed.</p><p>I will try to simulate this and see how it behaves, I'll report back, but any info you have it would be greatly appreciated.</p><div><div id="bloop_sign_1542305153029615872" class="bloop_sign"><div>-- <br>Florin Vlad Olariu</div></div></div></body></html>