Best utilizing fat long pipes and large file transfer

Thu Jun 12 23:05:50 UTC 2008

> Date: Thu, 12 Jun 2008 15:37:47 -0700
> From: Sean Knox <sean at craigslist.org>
> 
> Hi,
> 
> I'm looking for input on the best practices for sending large files over 
> a long fat pipe between facilities (gigabit private circuit, ~20ms RTT).
> I'd like to avoid modifying TCP windows and options on end hosts where 
> possible (I have a lot of them). I've seen products that work as 
> "transfer stations" using "reliable UDP" to get around the windowing 
> problem.
> 
> I'm thinking of setting up servers with optimized TCP settings to push 
> big files around data centers but I'm curious to know how others deal 
> with LFN+large transfers.

Not very fat or very long. I need to deal with 10GE over 200 ms (or more).

These should be pretty easy, but as you realize, you will need large
enough windows to keep the traffic in transit from filling the window
and stalling the flow. The laws of physics (speed of light) are not
forgiving.

There is a project from Martin Swaney at U-Delaware (with Guy Almes and
Aaron Brown) to do exactly what you are looking for.
http://portal.acm.org/citation.cfm?id=1188455.1188714&coll=GUIDE&dl=GUIDE
and
http://www.internet2.edu/pubs/phoebus.pdf
ESnet, Internet2 and Geant demonstrated it at last November's
SuperComputing Conference in Reno.

The idea is to use tuned proxies that are close to the source and
destination and are optimized for the delay. Local systems can move data
through them without dealing with the need to tune for the
delay-bandwidth product. Note that this "man in the middle" may not
play well with many security controls which deliberately try to prevent
it, so you still may need some adjustments.
-- 
R. Kevin Oberman, Network Engineer
Energy Sciences Network (ESnet)
Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab)
E-mail: oberman at es.net			Phone: +1 510 486-8634
Key fingerprint:059B 2DDF 031C 9BA3 14A4  EADA 927D EBB3 987B 3751
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 224 bytes
Desc: not available
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20080612/1443eacb/attachment.sig>