Muni fiber: L1 or L2?

Mon Feb 11 18:54:08 UTC 2013

On 05-Feb-13 11:37, Scott Helms wrote:
> On Tue, Feb 5, 2013 at 11:30 AM, Jay Ashworth <jra at baylink.com> wrote:
>> ----- Original Message -----
>>> From: "Scott Helms" <khelms at zcorum.com>
>>>> Yes it does... It locks you into whatever is supported on the ring.
>>> I don't know how I can explain this more plainly, I can (more accurately have) taken a fiber build that was created as a ring & spoke SONET system and with the same fiber plant overlaid that with GigE and ATM (further back in time) to backhaul for PON, DSL, VOIP, and direct Active Ethernet.
>> "Overlaid"?  Could you clarify that?
> Sure, ring, hub & spoke, home run, star these are all descriptions of the physical architecture and many layer 2 technologies will happily use them all including Ethernet.  To use a specific example an existing SONET ring (OC-3 to be precise) had be in service with an ILEC for more than a decade.  This physical topology was a common one with a physical ring of fiber (32 strands, yes this was built back in the day) connected to Add/Drop
> Multiplexers (Fujitsu IIRC) along the ring as needed to deliver 25,000 or shorter copper loops either directly from the same cabinet that ADM was in or from a subtended Digital Loop Carrier off of a spur (collapsed ring) of the ring.  Now, SONET connections work off a pair of fibers, one for transmit and one for receive.  To run Ethernet (initially 100mbps but now 10G) we simply lit 2 of the remaining 30 strands to overlay an Ethernet ring on top of the SONET ring.  We then placed switches in the same remote cabinets we had the ADMs and DLCs and started trenching the fiber drops.

... but now you are dictating what technology is used, via the active
aggregation equipment (i.e. ADMs) you installed at your nodes on the
ring.  Also, the fiber provider now has to maintain and upgrade that
active aggregation equipment, as opposed to just patching fiber from one
port to another.

The point of this exercise is to design and implement a fiber plant that
can support _any_ technology, including ones that haven't even been
invented yet.

>> Owen's assertion (and mine) is that a loop architecture *requires* active equipment, suited to the phy layer protocol, at each node.  And while those loop fibers are running SONET, they can't be running anything else at the same time.
> You're confounding the physical layer topology with the layer 2 protocol.  You can't run SONET and Ethernet on the same physical fiber at the same time (unless you use WDM but that's confusing the discussion) but you'd never build a ring of fiber with only two strands.

If you're going to dictate SONET ADMs, with a fixed set of downstream
connection types, why _not_ build your ring with one pair of fiber? 
Hint: the fiber itself is a tiny fraction of the total cost.

You're optimizing the wrong variable as a result of assuming you can
predict what technology will be used 50+ years in the future.

>>> This wasn't always true because we've only had 40G and 100G Ethernet for carrier networks for a few years. In the past we were limited by how big of an etherchannel network we could use for the ring. I'd also point out that the ring architecture is optimal for redundancy since you have fewer fiber bundles to get cut in the field and any cut to your ring gets routed around the ring by ERPS (http://en.wikipedia.org/wiki/ERPS) in less than 50 milliseconds.
>> I infer from that continuation of your thought that you mean the second: active optical muxes out in the plant.
>>
>> I'm sure I've made clear why that design limits me in ways I don't want to be limited when building a fiber plant for a 50 year lifetime, but let's address your responses below.
> The only limitation you have is a limited supply of total fibers (hint, this is a big reason why its cheaper to build and run).

Exactly!  Lay enough fiber that you don't _need_ aggregation at the
local level, i.e. enough that you can patch _every_ customer connection
directly to their destination of choice without any active aggregation
equipment at all.  Every pair of fiber can be running whatever
technology the customer desires, whether that is SONET, Ethernet, or
something else that hasn't even been invented yet.

>> The vast majority of businesses don't want [dark fiber] at the price
>> they have to pay for it now -- or more to the point, the consultants
>> who do their IT don't. You have no real way, I should think, to
>> extrapolate whether that will continue as prices drop, especially if
>> sharply.
> The vast majority of businesses don't know and don't care about HOW
> their connectivity is delivered and wouldn't know the difference
> between Layer 1 and Layer 2 if it punched them in the face. Almost all
> businesses want INTERNET connectivity at the highest quality & speed
> at the lowest cost and that's it. There are a small percentage, mainly
> larger businesses, that do have special requirements, but those
> special requirements very seldom include a L1 anything.

Most customers will buy from a service provider, who lights the fiber. 
The point of dark fiber is that the service provider gets to decide how
to light the fiber to said customers, allowing competition based on
innovation.  If the fiber owner puts active aggregation equipment in the
path, though, that means the technologies available are dictated by that
equipment's capabilities--and you have introduced another point of
failure into the system.

Sure, almost nobody asks for dark fiber today because they know it costs
several orders of magnitude more than a T1 or whatever.  However, if the
price for dark fiber were the same (or lower), latent demand would
materialize.  Why would I pay through the nose for a T1 when I can light
the fiber myself with 10GE for $20/mo?

> If your customer base is primarily residential with a few businesses
> (hospitals and schools also count here) then you'll be lucky to sell a
> handful of L1 connections and some of the people who will be
> interested will want it for very low bit rate (means low price too)
> uses like RS-232 over fiber for managing SCADA nodes or other
> telemetry pieces.

Why should the fiber owner care what they use it for?  It's just dark
fiber, patched from one place to another, so the rental price is the
same whether they light it at 10Mb/s or 10x100Gb/s.

What you're missing is that in this model, _every_ connection is L1 from
the fiber owner's perspective.  Let service providers worry about L2 and
above.

> The problem I see is that you seem to think that by building the L1
> piece you're going to have ISPs that are eager to serve your
> customers. If your demographics are like most small towns in the US
> that just isn't very likely. Any ISP partner is going to have to build
> and maintain a lot of infrastructure before they can serve the first
> end user and your "no upfront engineering" is simply not true unless
> you're going to configure and run MetroE and/or GPON shelves for them.
> In any sharing scenario (L1, L2, or L3) the ISP is going to have to
> connect to you with enough bandwidth to serve those end users as well.
> How many service providers have expressed interest? Have you talked
> pricing for the loops and colo space yet? 

Why would the ISP "have to build and maintain a lot of infrastructure"? 
All they need is a fiber-capable Ethernet switch in a colo to turn up
their first customer.  That's a lot simpler than trying to turn up their
first customer via an ILEC's DSLAM, for instance.

There's nothing wrong with the muni operating a L2 (or even L3) carrier
of last resort, just to ensure that _some_ useful service is available
to residents.  However, it should (a) be priced high enough to attract
competitors and (b) be a distinct entity, treated by the fiber arm as no
different from any other L1 customer.  None of the shenanigans like the
ILECs play, where the wholesale rate to competitors is higher than the
retail rate for the ILEC's own service.

> Its an admirable goal, but you're never going to have CCIEs (probably
> not even CCNAs) doing installs. Installation is, has been, and will in
> all likelihood continue to be done by people with limited skill sets.
> You building your own fiber plant and making it easier for ISPs to
> connect isn't going to change that.

You're missing the simplicity of dark fiber.  The carrier orders a L1
circuit from a customer to their facility.  The L1 provider just patches
one fiber pair to another fiber pair, which can be done by a trained
monkey.  Then the carrier connects their own equipment to the fiber at
their own facility and at the customer site, everything lights up and
the spice^Wdata flows.  Again, that can be done by a trained monkey. 
You don't need a CCIE or even a CCNA to do this.  Heck, it's even
simpler than what's required today for DSL, cable or satellite installers.

(Note that inside wiring is a completely separate issue, and carriers
_will_ have to train techs on how to do that since few are familiar with
fiber, but that is an optional service they can charge extra for.  The
L1 provider's responsibility ends at the NIU on an outside wall, same as
an ILEC's, so it's not their problem in the first place.)

S

-- 
Stephen Sprunk         "God does not play dice."  --Albert Einstein
CCIE #3723         "God is an inveterate gambler, and He throws the
K5SSS        dice at every possible opportunity." --Stephen Hawking

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2381 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20130211/41d69983/attachment.bin>