Best utilizing fat long pipes and large file transfer

Michal Krsek michal at
Fri Jun 13 06:42:57 UTC 2008

Hi Sean,
from thursday, we have copied some ~300 GB packages from Prague to San 
Diego (~200 ms delay, 10 GE flat ethernet end machines connected via 
1GE) files using RBUDP which worked great.

Each scenario needs some planning. You have to answer several questions:
1) What is the performance of storage subsystem (sometimes you need to 
connect external harddrives or tape robot)
2) How many files you need to transfer?
3) How big are these files?
4) What is the consistency scenarion (it is file consistency or package 

In example, I've sent some film data. Lot (~30.000) of 10 MB DPXes. 
Consistency was package based. Harddrives have been at the beggining 
connected via iLink (arrived on this media), then moved to eSATA (went 
to shop, bought another drive and connected it into export machine). 
Main tuning for RBUDP has been to buy another harddrive and tar these files.


> Hi,
> I'm looking for input on the best practices for sending large files 
> over a long fat pipe between facilities (gigabit private circuit, 
> ~20ms RTT).
> I'd like to avoid modifying TCP windows and options on end hosts where 
> possible (I have a lot of them). I've seen products that work as 
> "transfer stations" using "reliable UDP" to get around the windowing 
> problem.
> I'm thinking of setting up servers with optimized TCP settings to push 
> big files around data centers but I'm curious to know how others deal 
> with LFN+large transfers.
> thanks,
> Sean

More information about the NANOG mailing list