SDN Internet Router (sir)

Fri Jan 6 18:09:28 UTC 2023

>
> What I originally linked to (and another link or two contributed since
> then) seem to be people that already built that solution.

What's been shared isn't the same as what I described. faucet / sir are
doing the decision part and the signaling part together.

I am saying separate them. BGP doesn't know how to make the *decision* you
want, but it does know how to process *signals* that you send it.

On Fri, Jan 6, 2023 at 10:58 AM Mike Hammett <nanog at ics-il.net> wrote:

> Right.
>
> Only I'm not the guy to build that solution.
>
> What I originally linked to (and another link or two contributed since
> then) seem to be people that already built that solution.
>
>
>
> -----
> Mike Hammett
> Intelligent Computing Solutions <http://www.ics-il.com/>
> <https://www.facebook.com/ICSIL>
> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
> <https://www.linkedin.com/company/intelligent-computing-solutions>
> <https://twitter.com/ICSIL>
> Midwest Internet Exchange <http://www.midwest-ix.com/>
> <https://www.facebook.com/mdwestix>
> <https://www.linkedin.com/company/midwest-internet-exchange>
> <https://twitter.com/mdwestix>
> The Brothers WISP <http://www.thebrotherswisp.com/>
> <https://www.facebook.com/thebrotherswisp>
> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
> ------------------------------
> *From: *"Tom Beecher" <beecher at beecher.cc>
> *To: *"Mike Hammett" <nanog at ics-il.net>
> *Cc: *"Mel Beckman" <mel at beckman.org>, "NANOG" <nanog at nanog.org>
> *Sent: *Friday, January 6, 2023 9:51:43 AM
> *Subject: *Re: SDN Internet Router (sir)
>
> Gotcha.
>
> Setup a Quagga/Bird box. Do your top talker analysis , use that box to
> inject the routes you deem important with communities.  On your routers ,
> create policy structure to only take a default plus those communities.
>
> Obviously lots of devils in the details of the implementation , but
> something like that is all you need to do.
>
> On Fri, Jan 6, 2023 at 10:29 Mike Hammett <nanog at ics-il.net> wrote:
>
>> Maybe?
>>
>> I don't need any additional performance tests, though. Just watching
>> which prefixes are the top talkers and leaving the rest to default.
>>
>> I'm not looking at this to do what a BGP optimizier would do and find the
>> best tested path to the top talkers and then massage BGP to get it routed
>> that way. Determine the top talkers, then let BGP do its thing for those
>> top talkers.
>>
>> I don't want to manually say X traffic from Y POP manually goes here, but
>> I don't want to just leave it to default routing either. Something in the
>> middle.
>>
>>
>>
>> -----
>> Mike Hammett
>> Intelligent Computing Solutions <http://www.ics-il.com/>
>> <https://www.facebook.com/ICSIL>
>> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
>> <https://www.linkedin.com/company/intelligent-computing-solutions>
>> <https://twitter.com/ICSIL>
>> Midwest Internet Exchange <http://www.midwest-ix.com/>
>> <https://www.facebook.com/mdwestix>
>> <https://www.linkedin.com/company/midwest-internet-exchange>
>> <https://twitter.com/mdwestix>
>> The Brothers WISP <http://www.thebrotherswisp.com/>
>> <https://www.facebook.com/thebrotherswisp>
>> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
>> ------------------------------
>> *From: *"Tom Beecher" <beecher at beecher.cc>
>> *To: *"Mike Hammett" <nanog at ics-il.net>
>> *Cc: *"Mel Beckman" <mel at beckman.org>, "NANOG" <nanog at nanog.org>
>> *Sent: *Friday, January 6, 2023 9:16:19 AM
>>
>> *Subject: *Re: SDN Internet Router (sir)
>>
>> Thanks for this example.
>>
>> It sounds like you are describing egress peer engineering, but kinda in
>> reverse.
>>
>> In 'traditional' EPE, the routers have all the routes, and you are using
>> the external controller to perform the performance tests that matter to
>> you, and signal the network where to take the traffic based on those tests.
>>
>> It seems like you want to do the same thing , but instead of having the
>> controller signal the network where to carry bits, you want the controller
>> to signal the networks what routes are present, and direct the bits that
>> way.
>>
>> Do I have this right?
>>
>> On Thu, Jan 5, 2023 at 4:12 PM Mike Hammett <nanog at ics-il.net> wrote:
>>
>>> I hesitated to get too specific in examples because someone is going to
>>> drag the conversation into the weeds.
>>>
>>> Let's take the the Dallas - New Orleans - Atlanta example where I have a
>>> connection from New Orleans to Dallas and a connection from New Orleans to
>>> Atlanta.
>>>
>>> Let's say I peer with Netflix in both markets. Netflix chooses to serve
>>> me out of Atlanta, for whatever reason. Say my default route sends my
>>> traffic to Dallas. That's not where Netflix wanted it, so now I have to go
>>> from Dallas to Atlanta, whether that's my circuit or across the public
>>> Internet. Potentially, it's on MPLS and it rides back through the New
>>> Orleans router to get back to Atlanta. That's a long trip when I already
>>> had a better path, the less-than-full-fib router just didn't know about it.
>>> Given that Netflix is a sizable amount of traffic in an eyeball ISP, that's
>>> a lot of traffic to be going the wrong way. If the website for Viktor's
>>> Arctic Plunge in Siberia was hosted in Atlanta, I wouldn't give two craps
>>> that the traffic went the wrong way because A), I'll probably never go
>>> there and B) when someone does, it won't be meaningfully enough traffic to
>>> accommodate.
>>>
>>> Someone's going to tell me to put a full-table router in New Orleans.
>>> Maybe I should. Okay, so maybe I have a POP in Ashford, Alabama. It has
>>> transport to New Orleans and Atlanta. There aren't enough grains of sugar
>>> in Ashford, Alabama to justify a current-generation, full table router. Now
>>> I'm even closer to Atlanta, but default may point to New Orleans.
>>>
>>>
>>>
>>> -----
>>> Mike Hammett
>>> Intelligent Computing Solutions <http://www.ics-il.com/>
>>> <https://www.facebook.com/ICSIL>
>>> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
>>> <https://www.linkedin.com/company/intelligent-computing-solutions>
>>> <https://twitter.com/ICSIL>
>>> Midwest Internet Exchange <http://www.midwest-ix.com/>
>>> <https://www.facebook.com/mdwestix>
>>> <https://www.linkedin.com/company/midwest-internet-exchange>
>>> <https://twitter.com/mdwestix>
>>> The Brothers WISP <http://www.thebrotherswisp.com/>
>>> <https://www.facebook.com/thebrotherswisp>
>>> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
>>> ------------------------------
>>> *From: *"Mel Beckman" <mel at beckman.org>
>>> *To: *"Mike Hammett" <nanog at ics-il.net>
>>> *Cc: *"Joe Maimon" <jmaimon at jmaimon.com>, "NANOG" <nanog at nanog.org>
>>> *Sent: *Thursday, January 5, 2023 2:54:27 PM
>>> *Subject: *Re: SDN Internet Router (sir)
>>>
>>> Mike,
>>>
>>> I’m not sure I understand what you mean by “suboptimal“ routing. Even
>>> though the Internet uses AS path length for routing,  many of those path
>>> lengths are bogus, and don’t really represent any kind of path performance
>>> value. For example, a single AS might hide many hops in an MPLS network as
>>> a single hop, obscuring asymmetric routing and other uglies. Prepending
>>> also occurs when destinations are trying to enforce their own engineering
>>>  policies, which often conflict with yours or mine.
>>>
>>> So what do you mean by “suboptimal“? Are you thinking that the “best”
>>> path in BGP tables actually meant you were getting a performance benefit?
>>> Because that’s definitely not the case in today’s Internet. Were were you
>>> thinking that you would be going along less congested paths? That’s really
>>> at the mercy of the traffic engineering of backbone providers over which we
>>> have no control.
>>>
>>> I generally populate local router FIBs to merel choose an exit point for
>>> purposes of load balancing, and nothing more.
>>>
>>>  -mel
>>>
>>> On Jan 5, 2023, at 12:38 PM, Mike Hammett <nanog at ics-il.net> wrote:
>>>
>>> 
>>> I guess I wasn't around for those days.
>>>
>>> As far as running out, again, assuming the tooling works correctly, I'd
>>> think to target fewer routes than you could hold. Maybe 1k routes is all
>>> one would need to get a significant percent of the traffic. A lot of room
>>> to mess up if you can hold 100k, 500k routes.
>>>
>>>
>>>
>>> -----
>>> Mike Hammett
>>> Intelligent Computing Solutions <http://www.ics-il.com/>
>>> <https://www.facebook.com/ICSIL>
>>> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb>
>>> <https://www.linkedin.com/company/intelligent-computing-solutions>
>>> <https://twitter.com/ICSIL>
>>> Midwest Internet Exchange <http://www.midwest-ix.com/>
>>> <https://www.facebook.com/mdwestix>
>>> <https://www.linkedin.com/company/midwest-internet-exchange>
>>> <https://twitter.com/mdwestix>
>>> The Brothers WISP <http://www.thebrotherswisp.com/>
>>> <https://www.facebook.com/thebrotherswisp>
>>> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
>>> ------------------------------
>>> *From: *"Joe Maimon" <jmaimon at jmaimon.com>
>>> *To: *"Mike Hammett" <nanog at ics-il.net>, "Christopher Morrow" <
>>> morrowc.lists at gmail.com>
>>> *Cc: *"NANOG" <nanog at nanog.org>
>>> *Sent: *Thursday, January 5, 2023 2:30:40 PM
>>> *Subject: *Re: SDN Internet Router (sir)
>>>
>>>
>>>
>>> Mike Hammett wrote:
>>> > I'm not concerned with which technology or buzzword gets the job done,
>>> > only that the job is done.
>>> >
>>> >
>>> >
>>> > Looking briefly at the couple of things out there, they're evaluating
>>> > the top X prefixes in terms of traffic reported by s-flow, where X is
>>> > the number I define, and those get pushed into the FIB. One
>>> > recalculates every hour, one does so more quickly. How much is
>>> > appropriate? I'm not sure. I can't imagine it would *NEED* to be done
>>> > all of that often, given the traffic/prefix density an eyeball network
>>> > will have. Default routes carry the rest. Default routes could be
>>> > handled outside of this process, such that if this process fails, you
>>> > just get some sub-optimal routing until repaired. Maybe it doesn't
>>> > filter properly and sends a bunch of routes. Then just have a prefix
>>> > limit set on the box. Maybe it sends the wrong prefixes. No harm, no
>>> > foul. If you're routing sub-optimally internally, when it does hit a
>>> > real router with a full FIB, it gets handled appropriately.
>>>
>>> Unless it loops.
>>>
>>> The rest sounds nice. But flow caching got a bad rap back in the early
>>> worm days. But thats because the situation was a little worse back then.
>>> Cache the wrong routes or run out of cache, router dies. So long as
>>> thats not the case automating optimization is an extremely valuable goal.
>>>
>>> >
>>> >
>>> > I would just be looking for solutions that influence what's in the FIB
>>> > and let the rest of the router work as the rest of the router would.
>>>
>>> The problem comes when the router wont work at all without the FIB
>>> routes, like in the olden days.
>>> >
>>> >
>>> >
>>> > -----
>>> > Mike Hammett
>>> > Intelligent Computing Solutions <http://www.ics-il.com/>
>>> > <https://www.facebook.com/ICSIL><
>>> https://plus.google.com/+IntelligentComputingSolutionsDeKalb><
>>> https://www.linkedin.com/company/intelligent-computing-solutions><
>>> https://twitter.com/ICSIL>
>>> > Midwest Internet Exchange <http://www.midwest-ix.com/>
>>> > <https://www.facebook.com/mdwestix><
>>> https://www.linkedin.com/company/midwest-internet-exchange><
>>> https://twitter.com/mdwestix>
>>> > The Brothers WISP <http://www.thebrotherswisp.com/>
>>> > <https://www.facebook.com/thebrotherswisp><
>>> https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
>>> >
>>> ------------------------------------------------------------------------
>>> > *From: *"Christopher Morrow" <morrowc.lists at gmail.com>
>>> > *To: *"Mike Hammett" <nanog at ics-il.net>
>>> > *Cc: *"Tom Beecher" <beecher at beecher.cc>, "NANOG" <nanog at nanog.org>
>>> > *Sent: *Thursday, January 5, 2023 12:27:08 PM
>>> > *Subject: *Re: SDN Internet Router (sir)
>>> >
>>> >
>>> >
>>> > On Thu, Jan 5, 2023 at 11:18 AM Mike Hammett <nanog at ics-il.net
>>> > <mailto:nanog at ics-il.net>> wrote:
>>> >
>>> >     Initially, my thought was to use community filtering to push just
>>> >     IXes, customers, and defaults throughout the network, but that's
>>> >     obviously still sub-optimal.
>>> >
>>> >     I'd be surprised if a last mile network had a ton of traffic going
>>> >     to any more than a few hundred prefixes.
>>> >
>>> >
>>> > I think in a low-fib box at the edge of your network your choices are:
>>> >   "the easy choice, get default, follow that"
>>> >
>>> >   "send some limited set of prefixes to the device, and default, so
>>> > you MAY choose better for the initial hop away"
>>> >
>>> > you certainly can do the second with communities, or route-filters
>>> > (prefix-list) on the senders, or....
>>> > you can choose what prefixes make the cut (get the community(ies))
>>> > based on traffic volumes or expected destination locality:
>>> >    "do not go east to go west!"
>>> >
>>> > these things will introduce toil and SOME suboptimal routing in some
>>> > instances... perhaps it's better than per flow choosing left/right
>>> > though and the support calls related to that choice.
>>> >
>>> > In your NOLA / DFW / ATL example it's totally possible that the
>>> > networks in question do something like:
>>> >   "low fib box in tier-2 city (NOLA), dfz capable/core devices in
>>> > tier-1 city (DFW/ATL), and send default from left/right to NOLA"
>>> >
>>> > Could they send more prefixes than default? sure... do they want to
>>> > deal with the toil that induces? (probably not says your example).
>>> >
>>> > SDN isn't really an answer to this, though.. I don't think. Unless you
>>> > envision that to lower the toil ?
>>> >
>>>
>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20230106/72f47f63/attachment.html>