An area for operations growth - Storage Area Nets in MANS

Gordon Cook cook at
Fri May 16 23:26:55 UTC 2003

Sorry for any confusion.  Fiber channel is old tech for sure and 
there was no implied intent to evaluate the tech itself.  Roxane has 
apparently observed that on devices like the core directors the 
number of fiber channel ports being ordered is going up 
dramatically.... apparently about the only thing it makes sense to do 
with such ports is to use them for SANs.  The remark was not intended 
to be definitive or in great depth and was added almost as an after 
thought at the end of a short (35 minute) interview at the san jose 

  Certainly there is major movement in large fiber based in 
metropolitan  areas that have been built and are being built by 
banks, and fortune 500 kinds of enterprises as they move a LOT of 
data and voice traffic off the PSTN and on to new cheap equipment 
that they can get a much better return for on the dollars they 
invest.  All a part of the commoditization and decentralization of 

I am not an expert in the application of the technology being 
discussed.  However at some time in the next six month  i probably 
will want to explore this in much more depth.

>  >
>>  Although these new real time applica tions will clearly send more
>>  data over the network, the real killer application is going to be
>>  remote storage and synchronous storage.  Synchronous storage means
>>  that you have two large servers doing the exact same thing at the
>>  exact same time in two different locations.
>>  COOK Report:  Like a decentralized disk array?
>>  Googin:  Yes.  The backbone has to be incredibly fast because you
>>  cannot complete a transaction until you have acknowledgments from
>>  both disk drives.  This will happen.  Probably this year.  What they
>>  are already doing is taking fiber channel and putting that on a
>>  Cienna Core Director optical switch port.  Half of the ports being
>>  sold on the Core Director now are fiber channel.  They aren't even
>>  Ethernet.  And this is used for storage area nets (SANs).  These are
>>  corporate MANs and will have nothing to do with sales to service
>>  providers.  They are bypass business services where the storage
>>  arrays may not be more than a kilometer or two apart.  These SANs are
>>  backing up continuously terabytes of data.  We are talking huge
>>  applications that will use every bit of access to every bit of
>>  capacity they can get.
>Erm - perhaps I'm misunderstanding what Googin is trying to say here,
>but if he's talking about synchronous remote replication over fibre
>channel, this exists today. In fact, it has existed for years, and
>before it was over fibre channel, it was over ESCON.  Today, still, if
>you want to go father than a certain distance (70km maybe?) its
>generally recommended to switch over to ESCON rather than fibre channel.
>In practicality, complete remote replication is often inadvisable for
>high-performance, heavy-write applications.  The rule of thumb is that
>every KM adds another millisecond worth of I/O transaction time, so a
>few kilometers distance can add a significant overhead to writes. 
>Not to mention Fibre Channel is very unkind when it comes to recovery
>from fabric segmentation.  A poorly designed (or just very unlucky) SAN
>can be completely downed on both sides of the split, somewhat ruining
>the disaster recovery strategy if the production and DR storage networks
>are both taken down. 
>Ugh - I'll be very happy when fibre channel is dead and buried. 
>Matthew Zito
>GridApp Systems
>Email: mzito at
>Cell: 646-220-3551
>Phone: 212-358-8211 x 359

The COOK Report on Internet, 431 Greenway Ave, Ewing, NJ 08618 USA
609 882-2572 (PSTN) 703 738-6031 (Vonage) cook at  Subscription
info & prices at    Fiber 
& Wireless as
First Mile Technology - Fiber Business Models & Architecture, July- 
September 2003, 
130 pages available at

More information about the NANOG mailing list