impossible circuit

Wed Aug 13 16:29:24 UTC 2008

On Wed, 13 Aug 2008, Justin Shore wrote:

> Are you sure that they are not crossing some channels in the middle and 
> accidentally handing them to a different customer?  You mention above that

I can't be sure of anything like that.  I don't have access.  Those who do 
haven't been willing to look harder...just repeating "this can't happen, 
it's a private line DS3" and I'm sure thinking I'm nuts or an idiot.

> I would be curious to hear if Sprint is having any problems with a circuit 
> connected to sl-bb20-dc-6-0-0.sprintlink.net, what the router is and if any

I'm very curious too, because at this point, I expect they must be seeing 
lots of errors (not sure what kind you get when a line running HDLC 
encounters PPP encapsulated packets...but I bet they're causing input 
errors of some sort).  I'm going to try to make contact again with the guy 
at Sprint I spoke with last week.

> Are you sure that the traffic being received by each of the T1s is their's? 
> Do you have any way to getting flows or packets off of individual T1s and not 
> the bundle as a whole?

Yeah...I was looking at the output of "show ip cache flow" on the 
customer's router.  The traffic was all expected traffic.  The problem 
was, they were receiving several copies of every expected packet.

> Can you have them put the circuit into maintenance and have them test it end 
> to end?  They can't deny it when their TDR says that there's a problem.

What problem do you think they'll see if they do intrusive testing?  With 
the circuit taken down and test equipment at both ends, there's no way for 
the dupe data delivered to Sprint to get back to the circuit.  All that 
might happen is Sprint may experience even more errors on their 
circuit(s)...but testing on our circuit won't reveal that to the testers.

> Right.  By changing the encap you've basically killed the circuit.  With that 
> T1 effectively down on your end you won't be sending any packets down the 
> problem path and aren't able to see that problem anymore with your 
> traceroutes.  However your customer with the bundle of T1s is down a circuit.

You've pretty badly misunderstood.  The customer with the multiple T1s is 
irrelevant.  Their T1s aren't part of the problem.  Their T1s being full 
was just one symptom of the problem.  The problem circuit is our DS3 from 
Orlando to Ocala.  Changing the encaps on that DS3 (at both ends...with 
no OOB access...kind of scarey) from hdlc to ppp keeps the circuit 
working, and keeps the dupes from coming back at us, because the ppp 
encapsulated packets aren't understood by Sprint's routers running 
[I assume] hdlc...so they can't forward the packets.

----------------------------------------------------------------------
  Jon Lewis                   |  I route
  Senior Network Engineer     |  therefore you are
  Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________