Routescience?

Wed Aug 22 19:35:49 UTC 2001

Eddy, Peter,

As I say, I'm very happy to discuss our approach, and our results.  My
chief concerns are (a) not filling this list with product specs and (b)
competitive concerns.  

I'm sure you've seen the situation many times before.  NANOG folks want
details; vendors want to protect them.  We have a healthy respect for
this audience, and I would like our technology to be understood here, in
terms of its impact and implications.  I hope you can understand,
though, if I stop short of posting source code :-)

A major point to clean up first: PathControl is not a router.  The
modular chassis we use comes from the need to scale; it takes quite a
bit of CPU to do this, and we need expandability to keep up with some of
our higher end trial customers!  The device is out of line; it doesn't
even need a span port.  As one of our users has commented, you can pull
out the "rip cord" (meaning the power cable) and your network's default
behaviour will immediately take over again.

"E.B. Dreger" wrote:
>
> Let me go out on a limb:
> 
> Not sure that the NW article was "detailed", but I ass-u-me that one
> measures the time from packet sent out until ACK, then uses packet
> size and timing to determine the effective bandwidth.

In isolation, that fragment wouldn't get past a patent lawyer :-) 
Certainly, folks have addressed passive data collection from TCP
before.  But the product we've announced involves the focused
integration and application of these pieces (along with some less well
known ones) in a standalone, manageable device which generates
controllable border optimization, targeted to user selectable goals. 
That's still a bit of a mouthful, so let me get down to brass tacks.

The goes-inta's and goes-outa's from PathControl are all well understood
protocols: HTTP in, BGP out.  (There's also the usual cast of characters
for a network device: SNMP, CLI, Web UI, etc.  But now I'm getting back
to product spec recitation!)  I'd like to keep some "special sauce"
defence around how we go from the input to the output, but certainly you
understand what we're basing it on.

> Perturb the routing decision by inserting a longer subnet prefix.
> Measure.  Repeat until all paths have been timed.  Repeat again to
> lower standard deviation until the paths are statistically separated,
> or bail if it's a losing battle.  Now inject the "winning" route "for
> good".

This is a common assumption, and we believe it describes some other
approaches, but it's certainly not true of PathControl.  Sending "guinea
pig" route updates is a dangerous prospect.  NLRIs with longer masks
present a number of problems, not least the risk of leaking them into
the outside world, which could be quite disastrous.  (There are some
problems within the local AS too.)

A "guinea pig" route approach also means you cannot simply report on the
relative performance of network paths.  PathControl can measure and
report on all available egress links without injecting any updates.  All
you need is to arrange that the edge routers don't apply BGP to the
measurement stream.  Folks here can probably imagine several ways to do
that ...

> Setup procedure includes making sure that the border router doesn't
> damp(en) paths learned from the route server.  Measurement statistics
> are keyed to the subnet learned from the border router, perhaps
> aggregated with adjacent CIDR blocks if the results overlap.

Aggregation would be good, but it's tricky unless you wipe out the
original, more specific advertisements, which PathControl does not do.

> Because it's futile to try tuning that occasional packet to Albania,
> there's a low-water mark that a subnet (learned from the border) must
> hit before any tuning occurs.  Maintaining state is expensive, and
> traffic follows the 80/20 rule... don't bother tuning unless many
> packets go to that destination.

Again, an approach based on "guinea pig" routes will require this; you
must establish thresholds, and the approach cannot optimize below this
built-in "noise floor", which may have to be quite high to prevent
thrashing of distant prefixes.  

But if you measure the alternate paths without sending any updates, now
you can consider any available improvement, without asking the operator
to write an essay in advance on acceptable network performance
characteristics.  PathControl can measure improvements before it sends
any routes, and it then prioritizes to ensure the only updates sent are
those worth talking about, either because of performance, or other
constraints.

> It's a route server whose adverts are
> derived from empirical measurements.  No?

That's a fair way to visualize what it does, yes.  

How precisely we've done it, what kinds of controls we offer, how the
reporting looks, and ultimately how much it pays off are all things we
love to talk about, but I'm not sure this is the forum for brick by
brick architectural inspection.  We have collected a large amount of
data on the relevant networking effects, and come up with analyses of
it; some of that might be appropriate at a NANOG meeting, perhaps.

> <cynical>
> 
> I hope that the patent-pending part isn't any of the above, as the
> above is hardly unique, innovative, or something that hasn't been done
> before.  If the above is "secret", and these boxen sell for ~$200k,
> then I should turn this company into an R&D shop and implement some of
> the ideas that we have on a much larger scale...
> 
> Shrouding something in secrecy tends to leave this crowd (NANOG-l) to
> believe that there really is nothing to it... if it's that hard to do,
> and has valuable substance, then why the hush?
> 
> </cynical>

Well, I'm not sure I'm invoking "secrecy" around the whole thing, but I
concede that we're not an open source outfit.  I'd rather not offer the
list the manufacturing plans on how to build one, but I will confirm 
your comments on what PathControl uses as a data source for decisions.

No doubt some folks reading this are implementing similar stuff; for
their benefit, I'll agree with Eddy that this is indeed quite hard to do
correctly! ;-)

> I feel that I'm probably not the only one on NANOG-l who would like to
> see some technical information ("technical" to routing geeks, not
> "technical" to an end user or a Webbie) as opposed to "it's really
> cool".

Stick me in a kill file if I ever resort to "it's really cool" :-)

Mike