Comcast NOC - issues to/from AS13331 (Seattle)
Michael Loftis
mloftis at wgops.com
Tue Apr 23 05:35:27 UTC 2013
Comcast doesn't appear to have any usable NOC contacts via whois, and
this issue is apparently very widespread. Comcast obviously has
multiple saturated paths out in this area, so if you're seeing issues
getting to your customers on Comcast...well, it's probably Comcast.
Sort of an ongoing/me too on last months thread about same...
Outbound traffic to Comcast via basically any of our (AS13331)
upstreams except Spectrum (AS11404) is experiencing very high packet
loss once inside Comcast's network, we suspect a router or link very
near to us in Seattle is failing. I've marked out the destination
customer IPs but they're available to Comcast engineers if they get in
touch with me directly or as noc at metapeer.com -- TIA -- and again,
sorry to the list.
Below are three failing/high loss traces, and then same three traces
from our other DC which is routing out via spectrum. The loss is
present on Cogent and L3 outgoing paths at the least, maybe others.
As of now I've routed around the problem point for via the good path.
fission:~# mtr -s 900 --report --report-cycles 30 1.2.3.4
fission.myfreecams.com Snt: 30 Loss% Last Avg Best Wrst StDev
ve75-gw1-xmr.metapeer.com 0.0% 0.4 1.4 0.3 11.8 2.9
te0-7-0-15.ccr21.sea02.atlas.cogentco.com 0.0% 0.9 0.6 0.5 0.9 0.1
te-0-5-0-3-pe03.seattle.wa.ibone.comcast.net 6.7% 43.7 42.3 38.8 43.7 1.4
be-13-cr01.seattle.wa.ibone.comcast.net 3.3% 47.7 44.6 41.3 47.7 1.8
pos-0-7-0-0-cr01.denver.co.ibone.comcast.net 3.3% 69.4 68.7 65.0 71.3 1.5
he-5-15-0-0-cr01.350ecermak.il.ibone.comcast 6.7% 92.6 97.3 91.4 102.0 3.8
he-3-15-0-0-ar01.woburn.ma.boston.comcast.ne 10.0% 120.3 119.8 116.3 122.0 1.8
pos-0-1-0-0-ar01.needham.ma.boston.comcast.n 0.0% 122.0 120.5 117.1 123.9 1.8
po-80-ur01.deering.nh.boston.comcast.net 0.0% 121.4 121.3 118.3 133.8 2.8
po-21-ur01.concord.nh.boston.comcast.net 6.7% 119.6 130.1 119.3 272.8 31.8
te-1-0-0-ten01.concord.nh.boston.comcast.net 6.7% 118.7 120.0 117.6 121.4 1.3
??? 100.0 0.0 0.0 0.0 0.0 0.0
You have new mail in /var/spool/mail/root
fission:~# mtr -s 900 --report --report-cycles 30 5.6.7.8
fission.myfreecams.com Snt: 30 Loss% Last Avg Best Wrst StDev
ve75-gw1-xmr.metapeer.com 0.0% 0.3 0.6 0.3 5.2 0.9
te0-7-0-15.ccr21.sea02.atlas.cogentco.com 0.0% 0.8 0.6 0.5 0.8 0.1
te-0-5-0-3-pe03.seattle.wa.ibone.comcast.net 0.0% 41.4 42.6 38.5 43.8 1.3
be-13-cr01.seattle.wa.ibone.comcast.net 0.0% 45.6 44.1 40.4 46.7 1.8
pos-0-8-0-0-cr01.denver.co.ibone.comcast.net 6.7% 71.2 69.0 65.0 72.1 1.9
he-5-12-0-0-cr01.350ecermak.il.ibone.comcast 3.3% 98.1 96.3 86.4 101.2 3.9
he-0-15-0-0-ar01.pontiac.mi.michigan.comcast 10.0% 93.5 97.3 93.5 100.6 2.3
xe-11-0-0-0-sur01.rochestrhlls.mi.michigan.c 0.0% 97.2 99.1 92.9 141.8 10.4
te-17-10-cdn05.rochestrhlls.mi.michigan.comc 3.3% 118.9 113.2 104.6 118.9 3.5
??? 100.0 0.0 0.0 0.0 0.0 0.0
fission:~# mtr -s 900 --report --report-cycles 30 a.b.c.d
fission.myfreecams.com Snt: 30 Loss% Last Avg Best Wrst StDev
ve75-gw1-xmr.metapeer.com 0.0% 0.3 2.1 0.3 47.1 8.6
te0-7-0-15.ccr21.sea02.atlas.cogentco.com 0.0% 0.7 0.6 0.5 0.8 0.1
te-0-5-0-3-pe03.seattle.wa.ibone.comcast.net 6.7% 40.6 42.4 39.7 45.2 1.3
be-15-cr01.seattle.wa.ibone.comcast.net 0.0% 47.3 48.2 41.3 54.8 4.3
68.86.92.34 3.3% 43.7 45.5 39.5 101.7 11.6
be-18-ur06.bellevue.wa.seattle.comcast.net 0.0% 43.8 43.0 40.2 44.3 1.4
te-3-0-0-ten15.bellevue.wa.seattle.comcast.n 6.7% 42.7 44.1 42.1 45.7 1.1
??? 100.0 0.0 0.0 0.0 0.0 0.0
mloftis at phobos:~$ mtr -s 900 --report --report-wide --report-cycles 30 1.2.3.4
HOST: phobos Loss% Snt
Last Avg Best Wrst StDev
1. 207.229.74.1 0.0% 30
0.4 5.1 0.3 132.4 24.1
2. agg1-sea-t7-8.bb.spectrumnet.us 0.0% 30
1.5 14.0 1.5 206.4 39.8
3. 23.30.206.33 0.0% 30
1.8 2.0 1.7 3.5 0.3
4. be-17-cr01.seattle.wa.ibone.comcast.net 0.0% 30
12.6 5.3 2.1 12.6 2.7
5. pos-0-4-0-0-cr01.denver.co.ibone.comcast.net 0.0% 30
28.5 29.2 27.2 31.4 1.3
6. he-5-12-0-0-cr01.350ecermak.il.ibone.comcast.net 0.0% 30
49.9 52.2 49.9 56.7 2.8
7. he-3-5-0-0-ar01.woburn.ma.boston.comcast.net 0.0% 30
78.5 79.5 78.3 81.0 1.0
8. pos-1-12-0-0-ar01.needham.ma.boston.comcast.net 0.0% 30
80.2 80.1 78.2 81.7 1.1
9. po-80-ur01.deering.nh.boston.comcast.net 0.0% 30
80.0 80.1 79.9 80.4 0.1
10. po-21-ur01.concord.nh.boston.comcast.net 0.0% 30
80.7 85.1 80.7 199.3 21.6
11. te-1-0-0-ten01.concord.nh.boston.comcast.net 0.0% 30
80.5 80.5 80.5 80.7 0.0
12. ??? 100.0 30
0.0 0.0 0.0 0.0 0.0
mloftis at phobos:~$ mtr -s 900 --report --report-wide --report-cycles 30 5.6.7.8
HOST: phobos Loss%
Snt Last Avg Best Wrst StDev
1. 207.229.74.1 0.0%
30 0.3 3.6 0.3 96.3 17.5
2. agg1-sea-t7-8.bb.spectrumnet.us 0.0%
30 10.2 10.4 1.5 58.5 15.1
3. 23.30.206.33 0.0%
30 2.4 2.0 1.8 2.4 0.1
4. be-14-cr01.seattle.wa.ibone.comcast.net 0.0%
30 3.6 4.4 1.7 10.6 2.4
5. pos-0-3-0-0-cr01.denver.co.ibone.comcast.net 0.0%
30 31.1 29.5 27.2 31.6 1.3
6. he-5-12-0-0-cr01.350ecermak.il.ibone.comcast.net 0.0%
30 52.3 55.4 50.3 61.5 3.6
7. he-0-14-0-0-ar01.pontiac.mi.michigan.comcast.net 0.0%
30 58.9 58.0 56.1 60.1 1.2
8. xe-11-0-0-0-sur01.rochestrhlls.mi.michigan.comcast.net 0.0%
30 71.2 58.9 56.6 83.6 6.4
9. te-17-10-cdn05.rochestrhlls.mi.michigan.comcast.net 0.0%
30 71.2 72.6 67.6 78.0 3.2
10. ??? 100.0
30 0.0 0.0 0.0 0.0 0.0
mloftis at phobos:~$ mtr -s 900 --report --report-wide --report-cycles 30 a.b.c.d
HOST: phobos Loss% Snt
Last Avg Best Wrst StDev
1. 207.229.74.1 0.0% 30 0.7
5.1 0.3 143.4 26.1
2. agg1-sea-t7-8.bb.spectrumnet.us 0.0% 30 1.5
3.1 1.5 30.1 5.6
3. 23.30.206.33 0.0% 30 1.9
2.0 1.9 2.2 0.1
4. be-12-cr01.seattle.wa.ibone.comcast.net 0.0% 30 6.9
5.1 1.7 12.3 3.1
5. 68.86.94.70 0.0% 30 1.8
2.4 1.8 17.3 2.8
6. be-18-ur06.bellevue.wa.seattle.comcast.net 0.0% 30 3.5
3.4 3.3 3.5 0.1
7. te-3-0-0-ten15.bellevue.wa.seattle.comcast.net 0.0% 30 3.3
3.2 3.2 3.4 0.0
8. ??? 100.0 30 0.0
0.0 0.0 0.0 0.0
mloftis at phobos:~$
--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler
More information about the NANOG
mailing list