RTT of ICMP "TTL exceeded" messages in Level3 network remains the same throughout the network

Wed Aug 13 15:25:11 UTC 2014

Hi,

if I make a traceroute to a host in San Jose in Level3 network from
DigitalOcean server in Amsterdam, then in Level3 network(hop 6 in
example below) the RTT remains the same:

# traceroute -q 1 -I ZYNGA-INC.edge1.SanJose3.Level3.net
traceroute to ZYNGA-INC.edge1.SanJose3.Level3.net (4.53.208.114), 30
hops max, 60 byte packets
 1  5.101.103.253 (5.101.103.253)  0.265 ms
 2  95.85.0.229 (95.85.0.229)  0.236 ms
 3  ix-4-2-0-0.tcore1.AV2-Amsterdam.as6453.net (195.219.194.25)  0.275 ms
 4  if-7-2.tcore1.AD1-Amsterdam.as6453.net (195.219.194.46)  0.630 ms
 5  4.68.63.41 (4.68.63.41)  0.635 ms
 6  vl-3603-ve-227.csw2.Amsterdam1.Level3.net (4.69.162.153)  155.309 ms
 7  ae-56-221.ebr2.Amsterdam1.Level3.net (4.69.153.201)  155.627 ms
 8  ae-46-46.ebr2.London1.Level3.net (4.69.143.74)  153.470 ms
 9  *
10  ae-61-61.csw1.NewYork1.Level3.net (4.69.134.66)  148.972 ms
11  *
12  ae-2-2.ebr1.SanJose1.Level3.net (4.69.135.185)  147.881 ms
13  ae-91-91.csw4.SanJose1.Level3.net (4.69.153.14)  149.632 ms
14  ae-4-90.edge1.SanJose3.Level3.net (4.69.152.208)  151.107 ms
15  ZYNGA-INC.edge1.SanJose3.Level3.net (4.53.208.114)  154.431 ms
#

In other words, one sees the RTT of the end-host as a RTT for all the
hops in Level3 netwotk. If I make the traceroute to penultimate hop
ae-4-90.edge1.SanJose3.Level3.net, then RTT is as expected:

root at vserver:~# traceroute -q 1 -I ae-4-90.edge1.SanJose3.Level3.net
traceroute to ae-4-90.edge1.SanJose3.Level3.net (4.69.152.208), 30
hops max, 60 byte packets
 1  5.101.103.254 (5.101.103.254)  0.228 ms
 2  95.85.0.237 (95.85.0.237)  0.217 ms
 3  ix-4-2-0-0.tcore1.AV2-Amsterdam.as6453.net (195.219.194.25)  0.276 ms
 4  if-7-2.tcore1.AD1-Amsterdam.as6453.net (195.219.194.46)  0.656 ms
 5  4.68.63.41 (4.68.63.41)  0.607 ms
 6  vl-3604-ve-228.csw2.Amsterdam1.Level3.net (4.69.162.157)  0.696 ms
 7  ae-56-221.ebr2.Amsterdam1.Level3.net (4.69.153.201)  0.677 ms
 8  ae-45-45.ebr2.London1.Level3.net (4.69.143.70)  7.059 ms
 9  ae-44-44.ebr1.NewYork1.Level3.net (4.69.137.78)  76.311 ms
10  ae-81-81.csw3.NewYork1.Level3.net (4.69.134.74)  76.265 ms
11  ae-82-82.ebr2.NewYork1.Level3.net (4.69.148.41)  76.820 ms
12  ae-2-2.ebr1.SanJose1.Level3.net (4.69.135.185)  149.101 ms
13  ae-91-91.csw4.SanJose1.Level3.net (4.69.153.14)  150.557 ms
14  ae-4-90.edge1.SanJose3.Level3.net (4.69.152.208)  162.022 ms
root at vserver:~#

All the ICMP "TTL exceeded" messages except the first and the
penultimate one in Level3 network have MPLS extensions
header(s24.postimg.org/4z9at9z45/ICMP_echo_reply_MPLS_extensions.png)
which is always the same except the tag value changes.

How does this technically work? What are the advantages of such setup?

thanks,
Martin