RTT of ICMP "TTL exceeded" messages in Level3 network remains the same throughout the network

Wed Aug 13 15:41:24 UTC 2014

>How does this technically work? What are the advantages of such setup?

http://forums.juniper.net/t5/Routing/what-does-quot-icmp-tunneling-quot-mean-in-mpls-vpn/td-p/164284
http://www.juniper.net/techpubs/en_US/junos12.1/topics/usage-guidelines/mpls-configuring-icmp-message-tunneling.html

...and from 
https://www.nanog.org/meetings/nanog49/presentations/Sunday/mpls-nanog49.pdf:

> Some networks also run MPLS-only cores, which carry no IP routes.
> • This presents a problem, since if they did want to show the hops in
> traceroute, the router can’t do IP routing to return the ICMP TTL 
> Exceed.
> • To solve this problem, an “icmp tunneling” feature was implemented.
> • If an ICMP message is generated inside an LSP, the ICMP message is
> carried all the way to the end of the LSP before being routed back.
> • This can make traceroute look really weird, since you see all the hops
> along the LSP, but they all appear to have the same latency as the final
> hop. This causes much end-user confusion.

--
Hugo


On Wed 2014-Aug-13 18:25:11 +0300, Martin T <m4rtntns at gmail.com> wrote:
>Hi,
>
>if I make a traceroute to a host in San Jose in Level3 network from
>DigitalOcean server in Amsterdam, then in Level3 network(hop 6 in
>example below) the RTT remains the same:
>
># traceroute -q 1 -I ZYNGA-INC.edge1.SanJose3.Level3.net
>traceroute to ZYNGA-INC.edge1.SanJose3.Level3.net (4.53.208.114), 30
>hops max, 60 byte packets
> 1  5.101.103.253 (5.101.103.253)  0.265 ms
> 2  95.85.0.229 (95.85.0.229)  0.236 ms
> 3  ix-4-2-0-0.tcore1.AV2-Amsterdam.as6453.net (195.219.194.25)  0.275 ms
> 4  if-7-2.tcore1.AD1-Amsterdam.as6453.net (195.219.194.46)  0.630 ms
> 5  4.68.63.41 (4.68.63.41)  0.635 ms
> 6  vl-3603-ve-227.csw2.Amsterdam1.Level3.net (4.69.162.153)  155.309 ms
> 7  ae-56-221.ebr2.Amsterdam1.Level3.net (4.69.153.201)  155.627 ms
> 8  ae-46-46.ebr2.London1.Level3.net (4.69.143.74)  153.470 ms
> 9  *
>10  ae-61-61.csw1.NewYork1.Level3.net (4.69.134.66)  148.972 ms
>11  *
>12  ae-2-2.ebr1.SanJose1.Level3.net (4.69.135.185)  147.881 ms
>13  ae-91-91.csw4.SanJose1.Level3.net (4.69.153.14)  149.632 ms
>14  ae-4-90.edge1.SanJose3.Level3.net (4.69.152.208)  151.107 ms
>15  ZYNGA-INC.edge1.SanJose3.Level3.net (4.53.208.114)  154.431 ms
>#
>
>In other words, one sees the RTT of the end-host as a RTT for all the
>hops in Level3 netwotk. If I make the traceroute to penultimate hop
>ae-4-90.edge1.SanJose3.Level3.net, then RTT is as expected:
>
>root at vserver:~# traceroute -q 1 -I ae-4-90.edge1.SanJose3.Level3.net
>traceroute to ae-4-90.edge1.SanJose3.Level3.net (4.69.152.208), 30
>hops max, 60 byte packets
> 1  5.101.103.254 (5.101.103.254)  0.228 ms
> 2  95.85.0.237 (95.85.0.237)  0.217 ms
> 3  ix-4-2-0-0.tcore1.AV2-Amsterdam.as6453.net (195.219.194.25)  0.276 ms
> 4  if-7-2.tcore1.AD1-Amsterdam.as6453.net (195.219.194.46)  0.656 ms
> 5  4.68.63.41 (4.68.63.41)  0.607 ms
> 6  vl-3604-ve-228.csw2.Amsterdam1.Level3.net (4.69.162.157)  0.696 ms
> 7  ae-56-221.ebr2.Amsterdam1.Level3.net (4.69.153.201)  0.677 ms
> 8  ae-45-45.ebr2.London1.Level3.net (4.69.143.70)  7.059 ms
> 9  ae-44-44.ebr1.NewYork1.Level3.net (4.69.137.78)  76.311 ms
>10  ae-81-81.csw3.NewYork1.Level3.net (4.69.134.74)  76.265 ms
>11  ae-82-82.ebr2.NewYork1.Level3.net (4.69.148.41)  76.820 ms
>12  ae-2-2.ebr1.SanJose1.Level3.net (4.69.135.185)  149.101 ms
>13  ae-91-91.csw4.SanJose1.Level3.net (4.69.153.14)  150.557 ms
>14  ae-4-90.edge1.SanJose3.Level3.net (4.69.152.208)  162.022 ms
>root at vserver:~#
>
>All the ICMP "TTL exceeded" messages except the first and the
>penultimate one in Level3 network have MPLS extensions
>header(s24.postimg.org/4z9at9z45/ICMP_echo_reply_MPLS_extensions.png)
>which is always the same except the tag value changes.
>
>How does this technically work? What are the advantages of such setup?
>
>
>thanks,
>Martin