Networks ignoring prepends?

Wed Jan 24 15:02:32 UTC 2024

On Wed, 24 Jan 2024, Jay R. Ashworth wrote:

> ----- Original Message -----
>> From: "Jon Lewis" <jlewis at lewis.org>
>
>> On Mon, 22 Jan 2024, William Herrin wrote:
>>> It gives me, your paying customer, less control over my routing
>>> through your network than if I wasn't your paying customer. That
>>> seems... backwards.
>>
>> Not at all.  Think like a service provider.
>>
>> "I've got packets to deliver.  I've got 3 different classes of paths I can
>> use.  One of them, I get paid to use.  One is cost neutral.  The last one,
>> I pay to use."
>>
>> Which path would you pick (assuming you're trying to maximize revenue
>> from your network)?
>
> And here, you nail it, Jon:
>
> The Internet stopped being an engineering construct many years ago, to its--and
> our--detriment; things work much more poorly, and harder to understand and
> diagnose and fix, because of this.
>
> His example, of packets going from Miami to Ft Lauderdale via One Wilshire,
> is a classic example.

It can be a whole lot worse.  At a previous job, running an anycast CDN, 
we had POPs originating the same prefixes all over the world.  Cogent was 
one of our transit providers in most POPs (i.e. all the POPs in North 
America and Europe).

Toward the end of my time there, Cogent started making some progress 
breaking into the transit market in Asia.  So, we saw some eyeball 
networks in Asia hitting our anycast IPs via Cogent.  Trouble was, the 
established "tier 1's" in Asia wouldn't peer with Cogent in Asia (for 
business reasons - i.e. they didn't want Cogent coming into their market 
and upsetting their apple carts).  Our Asian POPs had lots of peering (IX 
and private) and transit from established Asian tier 1's.  So this traffic 
from Cogent's Asian customers would land in our LA and San Jose POPs.  As 
you can imagine, the RTT from an eyeball in Tokyo is "a bit higher" when 
talking to our LA POP vs our Tokyo POP.  Cogent has some BGP community 
controls available, but nothing that says "keep this route in-region". 
IIRC, the closest to it they had was lower localpref when sharing with 
region X.  Lowering localpref doesn't matter if region X has no path other 
than the one received from an out-of-region customer session.  Our options 
were "stop advertising anycast to Cogent globally" or "connect to Cogent 
in Asia so we can serve that traffic locally from our Asian POPs."

In one of his messages, William complained that the big bad networks are 
breaking the BGP rules by ignoring as-path length.  That's nonsense.  If 
you look at the BGP best path decision algorithm, there are several 
attributes considered before as-path length.  Localpref is one of 
them...and since most networks exist to make money, it's standard practice 
to use localpref to make sure you route traffic economically rather than 
efficiently (via the shortest as-path, which may still not be the shortest 
actual path).  For traffic you care about, obviously there's a balance 
between cost and performance.  If you've made poor/cheap choices in your 
transit providers, nobody cares that your traffic takes the scenic route. 
At least not the networks carrying your traffic that you're not directly 
paying...and you're likely to find, as above, even when you are directly 
paying, their interests are likely to outweigh yours.

----------------------------------------------------------------------
  Jon Lewis, MCP :)              |  I route
  Blue Stream Fiber, Sr. Neteng  |  therefore you are
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________