comcast routing issue question

Thu Nov 30 20:09:54 UTC 2006

On 11/29/06, Jim Popovitch <jimpop at yahoo.com> wrote:
> On Thu, 2006-11-30 at 00:06 -0500, Jim Popovitch wrote:
> > Question:  What could cause the first trace below to succeed, but the
> > second trace to fail?
> >
> > $ mtr 69.61.40.35
> > HOST: blue                        Loss%   Snt   Last   Avg  Best Wrst
> >   1. 192.168.3.1                   0.0%     1    4.3   4.3   4.3   4.3
> >   2. 73.62.48.1                    0.0%     1   10.6  10.6  10.6  10.6
> >   3. 68.86.108.25                  0.0%     1   11.4  11.4  11.4  11.4
> >   4. 68.86.106.54                  0.0%     1    9.8   9.8   9.8   9.8
> >   5. 68.86.106.9                   0.0%     1   20.5  20.5  20.5  20.5
> >   6. 68.86.90.121                  0.0%     1   11.3  11.3  11.3  11.3
> >   7. 68.86.84.70                   0.0%     1   27.7  27.7  27.7  27.7
> >   8. 64.213.76.77                  0.0%     1   24.5  24.5  24.5  24.5
> >   9. 208.50.254.150                0.0%     1   39.4  39.4  39.4  39.4
> >  10. 208.49.83.237                 0.0%     1   46.6  46.6  46.6  46.6
> >  11. 208.49.83.234                 0.0%     1   40.7  40.7  40.7  40.7
> >  12. 69.61.40.35                   0.0%     1   43.9  43.9  43.9  43.9
> >
> > $ mtr 69.61.40.34
> > HOST: blue                        Loss%   Snt   Last   Avg  Best  Wrst
> >   1. 192.168.3.1                   0.0%     1    1.1   1.1   1.1   1.1
> >   2. 73.62.48.1                    0.0%     1    9.9   9.9   9.9   9.9
> >   3. 68.86.108.25                  0.0%     1    9.3   9.3   9.3   9.3
> >   4. 68.86.106.54                  0.0%     1    9.6   9.6   9.6   9.6
> >   5. 68.86.106.9                   0.0%     1    9.0   9.0   9.0   9.0
> >   6. 68.86.90.121                  0.0%     1   18.2  18.2  18.2  18.2
> >   7. 68.86.84.70                   0.0%     1   23.9  23.9  23.9  23.9
> >   8. ???                          100.0     1    0.0   0.0   0.0   0.0
> >
> >
> > Taking the 69.61.40.33/28 subnet a bit further, .36 drops at 68.86.84.70
> > but .37 - .39 make it.  .40 drops at 68.86.84.70, but .41 makes it.
> >
> > Crazy.
>
> Btw, the problem has now been resolved, however I'm still curious as to
> what scenario could have caused that.
>
> -Jim P.

eBGP multihop peering across a pair of 10 gigE links with
static routes pointing to the remote router loopback; one
link goes south, but the interface still shows as up/up,
and voila, depending upon the hash, your packets may
go across the good link, or they may disappear into the
black hole of oblivion.

This is why multipath is a good thing, and eBGP multihop
with static routes is a Bad Thing(tm).

Matt