Best utilizing fat long pipes and large file transfer
Robert Boyle
robert at tellurian.com
Thu Jun 12 23:26:56 UTC 2008
At 06:37 PM 6/12/2008, you wrote:
>I'm looking for input on the best practices for sending large files
>over a long fat pipe between facilities (gigabit private circuit, ~20ms RTT).
>I'd like to avoid modifying TCP windows and options on end hosts
>where possible (I have a lot of them). I've seen products that work
>as "transfer stations" using "reliable UDP" to get around the
>windowing problem.
>
>I'm thinking of setting up servers with optimized TCP settings to
>push big files around data centers but I'm curious to know how
>others deal with LFN+large transfers.
In our experience, you can't get to line speed with over 20-30ms of
latency using TCP regardless of how much you tweak it. We transfer
files across the US with 60-70ms at line speeds with UDP based file
transfer programs. There are a number of open source projects out
there designed for this purpose.
-Robert
Tellurian Networks - Global Hosting Solutions Since 1995
http://www.tellurian.com | 888-TELLURIAN | 973-300-9211
"Well done is better than well said." - Benjamin Franklin
More information about the NANOG
mailing list