impossible circuit

Jon Lewis jlewis at
Tue Aug 12 09:37:41 CDT 2008

On Tue, 12 Aug 2008 list-nanog at wrote:

> Are dups generated on traffic going over that DS3 from (rather than to) 
> the Ocala side?

The dupes are only generated in the Orlando->Ocala direction.

> Does the DS3 cross Sprint's network?

The DS3 enters an Embarq (the telco formerly known as Sprint) central 
office.  AFAIK, the only portion of the circuit handled by Embarq is where 
it's handed to them in the CO where our gear is colo'd.

> What would happen if you pinged the Ocala router such that the TTL was 1 
> when travelling over the DS3? From your traceroute it seems it travelled 
> two IP hops that did not send ICMP error messages, but it might just be 
> that the ICMP errors from the Ocala router are arriving first.

Based on where the dupes are coming from, I assume pinging across the DS3 
with TTL tuned to expire at the Ocala side would result in TTL exceeded 
messages from both Ocala and the Sprint router where the packets are 
injected into Sprint's network.  It doesn't look as if IOS gives the 
option to set TTL on I'd try this from a Linux machine in our 
data center.

>> traffic was actually jumping off our network and coming back in via
>> Level3, I could see/block at least some of that using an ACL on our
>> interface to Level3.  How do you explain it, when you ping the remote end
>> of a DS3 interface with a single echo request packet and see 5 copies of
>> that echo request arrive at one of your transit provider interfaces?
> Just clarifying: 5 duplicates were being generated for every packet that 
> crossed the DS3, not just 1 packet that looped causing 5 duplicates?

Yes.  With the ACL on our Level3 transit, I blocked 5 dupes for each echo 
request sent from the Orlando end of the DS3 to the Ocala end.

>>  9  * * *
>> 10 (  80.774 ms  81.030 ms
>> 81.821 ms
> Was the first visibile IP hop of the dups always that Sprint router?

No.  That's one of the wild things about it.  Depending on who's network 
you trace from (we did traces from a bunch of route servers and looking 
glasses.  Some traces would show a pair of private IP hops before the 
Sprintlink IPs.  Some would simply show a different Sprint router as the 
first off-net hop.  If I break it again some night, I'll collect a few 
different examples.

> Level3 is your circuit provider?

Yes.  Originally it was a Progress Telecom circuit...but Level3 borged 

  Jon Lewis                   |  I route
  Senior Network Engineer     |  therefore you are
  Atlantic Net                |
_________ for PGP public key_________

More information about the NANOG mailing list