SDN Internet Router (sir)

Mike Hammett nanog at
Thu Jan 5 19:54:47 UTC 2023

I'm not concerned with which technology or buzzword gets the job done, only that the job is done. 

Communities certainly work. I could tag each of my peers (not a bad practice in the first place) with a different community, or set of communities and only allow say non-route server peers and customers to put routes into the limited-FIB boxes. However, how many routes can a given router hold in its FIB? 10K? 100k? 87,500? I'd have to make some manual choices about what gets included and what doesn't. That's not that big of a deal. 

I'm getting almost 11k prefixes from my Amazon, Cloudflare, and Google peers. How many of those 11k do I have significant bits to? 1k at most? Less than 10% of the routes I'm putting in via communities are ones I actually care about. I get 142k from my HE peer (oddly, I just noticed my HE IPV6 peer is down, so time to remedy that. See, something good does come from arguing on the Internet!). That would be at or exceeding the FIB size of many routers, yet to what gain? I of course do have to make an administration somewhere to decide what gets included and what doesn't. Of course whatever is done manually will result in sub-optimal routing. I'll be concerning myself with useless prefixes and ignoring ones from transit (excluded from this whole thing) altogether. 

Does something exist to make more intelligent choices than I can? Yes, at least two of them are in the public domain. I've had offlist responses about others home-brewing their own, similar solutions. Are they reliable? I don't know. Are they worth it? I'd assume so, but that depends greatly on if they're reliable. 

Looking briefly at the couple of things out there, they're evaluating the top X prefixes in terms of traffic reported by s-flow, where X is the number I define, and those get pushed into the FIB. One recalculates every hour, one does so more quickly. How much is appropriate? I'm not sure. I can't imagine it would *NEED* to be done all of that often, given the traffic/prefix density an eyeball network will have. Default routes carry the rest. Default routes could be handled outside of this process, such that if this process fails, you just get some sub-optimal routing until repaired. Maybe it doesn't filter properly and sends a bunch of routes. Then just have a prefix limit set on the box. Maybe it sends the wrong prefixes. No harm, no foul. If you're routing sub-optimally internally, when it does hit a real router with a full FIB, it gets handled appropriately. 

I would just be looking for solutions that influence what's in the FIB and let the rest of the router work as the rest of the router would. 

Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

----- Original Message -----

From: "Christopher Morrow" <morrowc.lists at> 
To: "Mike Hammett" <nanog at> 
Cc: "Tom Beecher" <beecher at>, "NANOG" <nanog at> 
Sent: Thursday, January 5, 2023 12:27:08 PM 
Subject: Re: SDN Internet Router (sir) 

On Thu, Jan 5, 2023 at 11:18 AM Mike Hammett < nanog at > wrote: 

Initially, my thought was to use community filtering to push just IXes, customers, and defaults throughout the network, but that's obviously still sub-optimal. 

I'd be surprised if a last mile network had a ton of traffic going to any more than a few hundred prefixes. 

I think in a low-fib box at the edge of your network your choices are: 
"the easy choice, get default, follow that" 

"send some limited set of prefixes to the device, and default, so you MAY choose better for the initial hop away" 

you certainly can do the second with communities, or route-filters (prefix-list) on the senders, or.... 
you can choose what prefixes make the cut (get the community(ies)) based on traffic volumes or expected destination locality: 
"do not go east to go west!" 

these things will introduce toil and SOME suboptimal routing in some instances... perhaps it's better than per flow choosing left/right though and the support calls related to that choice. 

In your NOLA / DFW / ATL example it's totally possible that the networks in question do something like: 
"low fib box in tier-2 city (NOLA), dfz capable/core devices in tier-1 city (DFW/ATL), and send default from left/right to NOLA" 

Could they send more prefixes than default? sure... do they want to deal with the toil that induces? (probably not says your example). 

SDN isn't really an answer to this, though.. I don't think. Unless you envision that to lower the toil ? 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the NANOG mailing list