Best utilizing fat long pipes and large file transfer
darren at bolding.org
Thu Jun 12 19:22:35 CDT 2008
And while I certainly like open source solutions, there are plenty of
commercial products that do things to optimize this. Depending on the type
of traffic the products do different things. Many of the serial-byte
caching variety (e.g. Riverbed/F5) now also do connection/flow optimization
and proxying, while many of the network optimizers now are doing serial-byte
I also for a while was looking for multicast based file transfer tools, but
couldn't find any that were stable. I'd be interested in seeing the names
of some of the projects Robert is talking about- perhaps I missed a few when
One thing that is a simple solution? Split the file and then send all the
parts at the same time. This helps a fair bit, and is easy to implement.
Few things drive home the issues with TCP window scaling better than moving
a file via ftp and then via ttcp. Sure, you don't always get all the file,
but it does get there fast!
On Thu, Jun 12, 2008 at 5:02 PM, Randy Bush <randy at psg.com> wrote:
> > The idea is to use tuned proxies that are close to the source and
> > destination and are optimized for the delay. Local systems can move data
> > through them without dealing with the need to tune for the
> > delay-bandwidth product. Note that this "man in the middle" may not
> > play well with many security controls which deliberately try to prevent
> > it, so you still may need some adjustments.
> and for those of us who are addicted to simple rsync, or whatever over
> ssh, you should be aware of the really bad openssh windowing issue.
-- Darren Bolding --
-- darren at bolding.org --
More information about the NANOG