AT&T Hiccup

Fri Sep 10 17:27:28 UTC 2010

No issues with AT&T.

Two DS3's in West MI: Qwest towards NYC and AT&T
towards Chicago.

Now I did have a Qwest issue and it apepars that others towards NYC may
have had similar based on route ages of about ~1:45hrs ago.

--- Original Message ---
From: J. Oquendo [sil at infiltrated.net]
Sent: Friday, September 10, 2010 12:49 PM
To: NANOG list

Alright, since things are mighty quiet on a Friday (either that or
everyone on NANOG is now blacklisted somehow and I'm not getting mail),
I'd figure to go the PITA route (as usual) and I'd ask a simple question
worthy of the mandatory responses of: "on topic questions are not the
topic of this list!", "this is not the outage list", "how is this
related to networking operations"

Did anyone see AT&T flake off the face of the map within the last 30-40
minutes? I know I did! (Seriously I did I did!)

Client: "We has no Internet"
Me: thinking to myself... "they need to call their provider... Due
diligence tracepath/ping/isitdownforeveryoneorjustme time... Global -->
AT&T looks sketchy"

# tracepath 12.200.xx.xx

 3:  so-1-3-1.585.ar1.JFK1.gblx.net (208.48.236.105)      asymm  4   3.416ms
 4:  po3-40G.ar2.ATL2.gblx.net (67.16.131.198)            asymm  7 141.179ms
 5:  no reply
 6:  no reply
 NO REPLY 24 more x
31:  no reply
     Too many hops: pmtu 1500
     Resume: pmtu 1500

AnotherClient (seconds later): "We has no VoIP"
Me: thinking to myself... "Damnit GBLX not again"

Another_Nother_Client (like even seconds after the first: "We has no p2p!"
Me: "time to tweet this question... No... post it to facebook... No -
make sure I update LinkedIn... wait?! NANOG"

Common denominator... All were on AT&T. So, was it only me or did one of
AT&T's cores take a long (3-5 minute) hiccup.

Post WTFH:

# tracepath 12.200.xx.xx

 3:  so-1-3-1.585.ar1.JFK1.gblx.net (208.48.236.105)      asymm  4   3.774ms
 4:  te4-4-10G.ar5.NYC1.gblx.net (67.16.134.158)          asymm  7 189.496ms
 5:  192.205.37.137 (192.205.37.137)                      asymm  7   5.979ms
 6:  cr1.n54ny.ip.att.net (12.122.81.58)                  asymm  9  20.088ms
 7:  cr84.n54ny.ip.att.net (12.122.115.94)                asymm  9  19.585ms
 8:  gar1.chsct.ip.att.net (12.122.105.117)               asymm  7  10.234ms
 9:  12.91.181.170 (12.91.18x.xxx)                        asymm  8  24.811ms
10:  12.200.42.67 (12.200.xx.xx)                          asymm  9
29.104ms reached
     Resume: pmtu 1500 hops 10 back 9

Just wanted to know if someone else saw it. All clients were spread
through the 12.x.x.x block not solely on 12.200.x.x