Muni fiber: L1 or L2?

Tue Feb 5 18:07:46 UTC 2013

----- Original Message -----
> From: "Scott Helms" <khelms at zcorum.com>

> > "Overlaid"? Could you clarify that?
> 
> Sure, ring, hub & spoke, home run, star these are all descriptions of the
> physical architecture and many layer 2 technologies will happily use them
> all including Ethernet. To use a specific example an existing SONET ring
> (OC-3 to be precise) had be in service with an ILEC for more than a decade.

Yup; with you so far; I was an OC-12 tail circuit off of L3/telcove's
Pinellas County ring at an earlier job.  (And I had a fault on one side,
because an...

> This physical topology was a common one with a physical ring of fiber (32
> strands, yes this was built back in the day) connected to Add/Drop
> Multiplexers (Fujitsu IIRC) 

ADM at a site adjacent to me was in a business that had closed down, and
L3 couldn't get it out of the loop, or hadn't, or what have you, so I was
unprotected the entire 2.5 years I was there.  Only went out once or
twice, though.  Mine was a Lucent DMXplore, delivering 6 DS1s and a 10BaseT.

>                              along the ring as needed to deliver 25,000
> or shorter copper loops either directly from the same cabinet that ADM
> was in or from a subtended Digital Loop Carrier off of a spur (collapsed
> ring) of the ring. Now, SONET connections work off a pair of fibers, one for
> transmit and one for receive. To run Ethernet (initially 100mbps but now
> 10G) we simply lit 2 of the remaining 30 strands to overlay an Ethernet
> ring on top of the SONET ring. We then placed switches in the same remote
> cabinets we had the ADMs and DLCs and started trenching the fiber drops.

Surely.

You *put active equipment out in the physical plant*.

I'm sure that there are some physical plant design criteria that permit
that decision, but mine isn't one of them, for reasons I believe I've made
fairly clear.

You disagree with some of those as well, of course, but you understand
*that* I have made them, and I would expect, therefore, also why this
entire subthread isn't germane to the problem I'm trying to solve, right?

> > Owen's assertion (and mine) is that a loop architecture *requires* active
> > equipment, suited to the phy layer protocol, at each node. And while those
> > loop fibers are running SONET, they can't be running anything else at the
> > same time.
> 
> You're confounding the physical layer topology with the layer 2 protocol.
> You can't run SONET and Ethernet on the same physical fiber at the same
> time (unless you use WDM but that's confusing the discussion) but you'd
> never build a ring of fiber with only two strands.

Certainly not.

But a ring a) requires *some kind* of active equipment between the MDF
and the ONT, and b) does not support PtP at all.

So, *for my stated purposes*, it's not an acceptable alternative.

> > > There is nothing about a hub & spoke architecture is this harmful
> > > or even suboptimal for doing Gig-E directly to end users today.
> >
> > You propose to run a ring *for each subscriber*? Or put active gear
> > in the field to mux the subscriber AE loops into a SONET ring?
> >
> > Or some other approach I don't know is possible?
> 
> SONET is simply the legacy (and expensive) way that telco's used to build
> rings. I'd neither use it nor recommend it for much of anything today.
> Calix, Occam(also Calix now), Adtran, and all the other guys who play
> in this space will happily construct a Gig/10G/40G Ethernet ring in the
> same shelf you're going to be buying to put your GPON or AE line cards in.

I'm sure, but it's still a ring.

If I ever want to upgrade it, I have to do a lot more than rack new gear
in my "CO", and then move patch cords one at a time.

> > I infer from that continuation of your thought that you mean the
> > second: active optical muxes out in the plant.
> >
> > I'm sure I've made clear why that design limits me in ways I don't
> > want to be limited when building a fiber plant for a 50 year lifetime,
> > but let's address your responses below.
> >
> 
> The only limitation you have is a limited supply of total fibers
> (hint, this is a big reason why its cheaper to build and run).

Nope, that is, in fact, not the only limitation; the others have been
expressed or implied, but are left as an exercise for the student.

> > > > Lower the price per instance and you very likely find new
> > > > demands.

> > > The vast majority of business don't WANT that kind of
> > > connectivity.

> > The vast majority of businesses don't want it at the price they have to
> > pay for it now -- or more to the point, the consultants who do their IT
> > don't.
> >
> > You have no real way, I should think, to extrapolate whether that
> > will continue as prices drop, especially if sharply.

> The vast majority of businesses don't know and don't care about HOW their
> connectivity is delivered and wouldn't know the difference between Layer 1
> and Layer 2 if it punched them in the face. 

No one in this conversation, Scott, has ever suggested that *subscribers*
care how the ISP delivers the service, as long as it's fast -- though the
percentage who *do* is likely still higher than you think it is.

I care; don't you? :-)

>                                              Almost all businesses want
> INTERNET connectivity at the highest quality & speed at the lowest
> cost and that's it. There are a small percentage, mainly larger businesses,
> that do have special requirements, but those special requirements very seldom
> include a L1 anything.

Yes, but now we're into Whorf's Hypothesis: your vocabulary limits the
things you're *able* to think about; it hasn't been practical to *supply*
MAN L1 fiber at reasonable prices until about now.

> > Sure; most small businesses don't need MPLS.
> 
> Nor medium businesses, and that's where knowing your (potential) customer
> base matters more than anything I can tell you. If you're in an area with
> lots of technology jobs and/or financial companies your network will look
> differently than an average small town muni build. If your customer base
> is primarily residential with a few businesses (hospitals and schools also
> count here) then you'll be lucky to sell a handful of L1 connections and
> some of the people who will be interested will want it for very low bit
> rate (means low price too) uses like RS-232 over fiber for managing SCADA
> nodes or other telemetry pieces.

Sure, and I don't expect to sell a lot of it up front, unless my launch
ISP wants to use their own L2 gear.  *But this does not preclude my building
to enable it*, which is a different thing from how much of it I expect to
sell.

> > But there are some that do, and there are some that it doesn't matter
> > *where they are at*. "Fiber on your wall with no upfront engineering
> > charge" is a pretty strong call, in some markets, and I won't have to
> > do most of the publicity myself; it'll make the news.
> 
> You'll get some publicity, especially if you do a little self
> promoting.

Yep, and PR is not foreign to me; it will actually be in the project
budget, along with outside counsel.

> The problem I see is that you seem to think that by building the L1 piece
> you're going to have ISPs that are eager to serve your customers. If your
> demographics are like most small towns in the US that just isn't very
> likely. Any ISP partner is going to have to build and maintain a lot of
> infrastructure before they can serve the first end user and your "no
> upfront engineering" is simply not true unless you're going to configure
> and run MetroE and/or GPON shelves for them. In any sharing scenario (L1,
> L2, or L3) the ISP is going to have to connect to you with enough bandwidth
> to serve those end users as well. How many service providers have
> expressed interest? Have you talked pricing for the loops and colo
> space yet?

Oh, Jesus; Scott.  I've been clear from in front that this is a 36 months to
shovel project for me; no, I haven't talked to providers yet; I'm still
angling for the *job*.

> > > What vendors? ISPs don't.
> >
> > And your assertion here is based on what? How many places have ISPs
> > had a *choice* as to whether to take a L1 optical or L2 aggregated
> > handoff?

> I've done nothing but work with ISPs, often in the situation you're
> describing, for the past 15 years. I also know how ISPs, especially at
> the size you might attract operate.

And yet, you didn't answer the question: *how many places has the choice
about which you express an opinion *actually been made* by an ISP
serving subs on a fiber plant they don't own, if any*?

> > > That's called home running, but as I've said that's ok in some
> > > scenarios, its just that in most cases there is no benefit.
> >
> > Today. Neither you nor I know how that will change in 20, 30, or 50
> > years. But that's the horizon I'm planning not to block.
> 
> You're betting money against Ethernet's (not to mention any new technology)
> ability to keep up as a technology. You're statement is true, but its a
> lot like buying insurance against meteor strikes. Today on a normal ring
> topology most people are installing hundreds of pairs of physical fiber so
> even without adding Wave Division Multiplexing in you can easily build
> a ring with several terabits per second capacity. Even easier(and cheaper)
> is to build several 10G rings on top of each other with the exact same
> shelves you're going to be using to push out GPON or AE.

I never bet against Ethernet; I'm not stupid, and this is not my first
rodeo.  (Well, ok, the first one with bulls this big... :-)

> Its an admirable goal, but you're never going to have CCIEs (probably
> not even CCNAs) doing installs. Installation is, has been, and will in all
> likelihood continue to be done by people with limited skill sets. You
> building your own fiber plant and making it easier for ISPs to connect
> isn't going to change that.

But that, Scott, is orthogonal to *whether they can work with my L1 guy(s)
to find a problem*; a CC?? is not necessary for that, that I can see.

And anybody big enough to want to come in and do L1 fiber *will already
have guys who are fiber-trained*.

> > When you say "normal installers", do you mean "employees of the ISP",
> > "employees of the muni", or "subcontractors of one of those two"?
> > And why is that pertinent?
> 
> Whoever does the install at the home or office. Who they work for really
> doesn't matter since the skill set will be largely the same. Its pertinent
> because you can have the greatest plant on the planet but if your install
> isn't done correctly the end user (the guy paying you or your partner)
> won't be happy. The majority of your day to day problems will be driven by
> installs. The fiber was improperly terminated, the drop was in the wrong
> place, the ONT didn't work correctly, etc. Please, go join the LinkedIn
> FTTx group and read some of the things there:
> 
> You probably can't follow that link until you join but its a good
> discussion of some of the problems.
> 
> http://www.linkedin.com/groupItem?view=&gid=83578&type=member&item=208400553&qid=379425dd-6f05-4caa-b2d4-f4fbf2aeb9cc&trk=group_most_popular-0-b-ttl&goback=%2Egmp_83578

I've been avoiding LinkedIN like bubonic plague.

> > Your assertion seems to be that it will be necessary to have "abnormal"
> > installers in the field in order for them not to dump problem tickets
> > off to the muni and fail to help meaningfully in fixing them.
> >
> > First, I think this unlikely since, in most cases, we'll have 3pr available
> > at each address. If we think there's a problem with the pair, we can
> > "cut to clear" *temporarily*, and if the second pair is ok, then the
> > sub is back online while we test the first pair and clear the problem.
> >
> > (GTE's failure, for all that I give them shit about CtC is that they
> > never *worked* the dead pairs; as long as you do, it's not a problem.)
> 
> That's great, and I'm glad to hear you've worked out that part of the drop
> but most of the problems occur from the drop into the house or office.

That jack box will be on the building, either as an exterior ONT, or as an 
exterior or interior jackbox that an interior ONT plugs into; I'm still
working out the tradeoffs on that.

> The last 20 yards are the most problematic and most changed. This is where
> the installer matters most and why even good plant has bad installs.

Sure.  But see way above about "install contractor testing to the jacks".

Or, immediately below.  :-}

> > Second, since we'll be terminating all 3 pairs to a jackbox, the
> > installation contractor will be able to perform and document whatever
> > acceptance testing we instruct them to. Sure, that will cost some
> > more money, but again, if your capital plant is tested good when
> > installed, this reduces markedly the opex maintenance cost over time. Where
> > that breakeven point will be depends on what they want to charge me to do
> > the testing, just as it does with Cat6.

No comment there?

> > [ Scott: ]
> > > >> Sure it does, even in greenfield and whats more it costs more
> > > >> over the
> > > >> long term UNLESS you know where every home and business will be
> > located 10
> > > >> years from now.
> >
> > A luxury I do have, since my city is nearly 100.0% built; it's
> > certainly 100% platted.
> >
> That's good, and again my argument isn't that you shouldn't home run all
> your connections, but rather using the idea of L1 handoffs as a reason to
> do so is flawed and that hub and spoke does not limit your options.

I am using H&S.  Just with only one hub.

> > [ Owen: ]
> > > >> More yes, much more, I'm not so convinced.
> >
> > And Owen isn't the only one who thinks that, and I think I know Rob
> > Seastrom well enough from the list to think he wouldn't concur
> > unless he had some data from which to work.
> 
> Again, its not (I think I've said this enough) a problem in certain
> scenarios. The issue I take is that hub & spoke doesn't limit your
> future use and that L1 connectivity as a reason to do home runs is seldom
> worth the expense.

Perhaps.  But we seem to have established that homerun vs, say, GPON with
splitters in the plant, is merely delta; maybe 5-8% more CAPEX.  And the
other alternative you present, ring with active muxes in the field, I
have already ruled out for the reasons above.

Until I get to the point where I have real, current, quoted numbers on 
the plant install in various approaches, I suppose we ought to let
this thread drop for now.

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra at baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover DII
St Petersburg FL USA               #natog                      +1 727 647 1274