Microsoft Express Routes woes...

Wed Jun 27 15:58:22 UTC 2018

This is classic.......

We have a direct BGP peering session to Microsoft using Express Routes for
the public peering session for services like Email, one drive, etc. this
uses MS Public. We also have/use MS Azure Public as well as MS Azure
Private in place for a few years now. We have had this happen a few times
already where one team at MS makes a routing change, and the other team is
either not aware of the change, or else doesn't communicate the change
properly.

So late last night, while a change was being made to a completely different
area of our network, they asked that I back out my change due to our entire
organization not being able to access share-point online, or MS One drive.

I had zero evidence it was our change. further investigation on our border
routers, revealed all four (4) of our ISP's were advertising the MS block
as a /24 prefix:

A:MY_NAME_CHANGED# show router 1053 bgp routes 13.107.136.0/24
========================================================================
 BGP Router ID:10.11.0.29       AS:122         Local AS:122
========================================================================
 Legend -
 Status codes  : u - used, s - suppressed, h - history, d - decayed, * -
valid
                 l - leaked, x - stale, > - best, b - backup, p - purge
 Origin codes  : i - IGP, e - EGP, ? - incomplete

========================================================================
BGP IPv4 Routes
========================================================================
Flag  Network                                            LocalPref   MED
      Nexthop (Router)                                   Path-Id     Label
      As-Path
-------------------------------------------------------------------------------
u*>?     13.107.136.0/24                                    250        10450
      4.49.118.153                                       None        -
      3356 8075 8068
-------------------------------------------------------------------------------
Routes : 1

HOWEVER MS Express Routes was advertising this:

A:MY_NAME_CHANGED# show router 1053 bgp routes 13.107.136.0/22
========================================================================
 BGP Router ID:10.11.0.29       AS:122         Local AS:122
========================================================================
 Legend -
 Status codes  : u - used, s - suppressed, h - history, d - decayed, * -
valid
                 l - leaked, x - stale, > - best, b - backup, p - purge
 Origin codes  : i - IGP, e - EGP, ? - incomplete

========================================================================
BGP IPv4 Routes
========================================================================
Flag  Network                                            LocalPref   MED
      Nexthop (Router)                                   Path-Id     Label
      As-Path
-------------------------------------------------------------------------------
i  13.107.136.0/22                                    300         0
      157.229.11.66                                      None        -
      12076

So another call to MS support and escalations to get this more specific
prefix fixed or if Express routes can advertise this more specific vs the
/22 block.

*Questions:*
Has anyone else ran into this with MS if you have a direct peering session
to them?
Has anyone did an audit on the route table's received from MS over express
routes, vs what they receive from their ISP's and noticed the differences?

MS needs to seriously think about:
Careful coordination of routing changes
Policies to prevent specific routes being advertised while larger blocks
are advertised over express routes.

Anyway I am tired, as I have not had much sleep, any comments on this,
would like to hear from you.

-Craig