Help me make sense of these traceroutes please

Tue Dec 24 23:16:59 UTC 2013

Hello Nanog community,
I would like to enlist your help with understanding this latency I'm 
seeing.

First some background,
I have Level3 circuits in the US and some services in Europe. From 
Comcast to the US level3 IPs the performance is excellent. The same 
traceroute to Europe is terrible. The strange part is the problem 
appears to begin stateside on the same infrastructure that carriers the 
us traffic.

Here is a trace to one of my IPs in the US from Comcast

Tracing route to 4.30.x.x over a maximum of 30 hops

   1     3 ms     1 ms     1 ms  10.1.1.1
   2    30 ms    29 ms    29 ms  71.62.150.1
   3     9 ms     9 ms     9 ms  
xe-0-1-0-32767-sur01.winchester.va.richmond.comc
ast.net [68.85.71.165]
   4     9 ms    14 ms    10 ms  
xe-9-0-3-0-ar02.staplesmllrd.va.richmond.comcast
.net [68.86.125.149]
   5    32 ms    30 ms    34 ms  68.86.91.153
   6    36 ms    38 ms    53 ms  23.30.207.98
   7    34 ms    28 ms    33 ms  vlan51.ebr1.Atlanta2.Level3.net 
[4.69.150.62]
   8    29 ms    28 ms    20 ms  ae-63-63.ebr3.Atlanta2.Level3.net 
[4.69.148.241]

   9    27 ms    29 ms    30 ms  ae-2-2.ebr1.Washington1.Level3.net 
[4.69.132.86]

  10    24 ms    30 ms    24 ms  ae-71-71.csw2.Washington1.Level3.net 
[4.69.134.1
34]
  11    29 ms    31 ms    39 ms  ae-41-90.car1.Washington1.Level3.net 
[4.69.149.1
95]
  12    30 ms    30 ms    29 ms  ae-2-23.edge7.Washington1.Level3.net 
[4.68.106.2
38]
  13    38 ms    44 ms    43 ms  4.79.x.x
  14     *        *        *     Request timed out. (My firewall)
  15    39 ms    39 ms    39 ms  4.30.x.x

Trace complete.

Now here is the same computer tracing to a level3 circuit in Ireland.

Tracing route to xxx.yyy.ie [193.1.x.x]
over a maximum of 30 hops:

   1     1 ms     1 ms     1 ms  10.1.1.1
   2    38 ms    33 ms    25 ms  71.62.150.1
   3    10 ms     9 ms     9 ms  
xe-0-1-0-32767-sur01.winchester.va.richmond.comc
ast.net [68.85.71.165]
   4    14 ms    15 ms    15 ms  
xe-9-0-3-0-ar02.staplesmllrd.va.richmond.comcast
.net [68.86.125.149]
   5    28 ms    30 ms    30 ms  68.86.95.65
   6    37 ms    37 ms    37 ms  23.30.207.98
   7   118 ms        *   218 ms  vlan51.ebr1.Atlanta2.Level3.net 
[4.69.150.62]
   8   119 ms   218 ms   119 ms  ae-63-63.ebr3.Atlanta2.Level3.net 
[4.69.148.241]

   9   221 ms   119 ms   119 ms  ae-2-2.ebr1.Washington1.Level3.net 
[4.69.132.86]

  10   118 ms   119 ms   118 ms  ae-91-91.csw4.Washington1.Level3.net 
[4.69.134.1
42]
  11   119 ms   119 ms   119 ms  ae-92-92.ebr2.Washington1.Level3.net 
[4.69.134.1
57]
  12   117 ms   126 ms   120 ms  ae-43-43.ebr2.Paris1.Level3.net 
[4.69.137.57]
  13   128 ms   118 ms   120 ms  ae-6-6.car1.Dublin3.Level3.net 
[4.69.148.53]
  14   122 ms   225 ms   124 ms  4.69.148.58
  15   124 ms   118 ms   120 ms  ae-11-11.car1.Dublin1.Level3.net 
[4.69.136.93]

Notice that the hop from 23.30.207.98 to 4.69.150.62 seems very 
respectable at around 30ms for US bound traffic. However when I'm 
tracing from the same Comcast network to an IP that is in Europe the 
very same hope produces 100ms of latency and about 12% packet loss. Why 
does this hop treat traffic differently based on it's destination? Is 
this some weird result of complex asymmetrical routing or something 
else?

I can route around this problem, but it does seem strange and I want to 
understand it

Thanks,
Sam Moats