Muni fiber: L1 or L2?

Tue Feb 5 17:37:57 UTC 2013

On Tue, Feb 5, 2013 at 11:30 AM, Jay Ashworth <jra at baylink.com> wrote:

> ----- Original Message -----
> > From: "Scott Helms" <khelms at zcorum.com>
>
> > > Yes it does... It locks you into whatever is supported on the ring.
> >
> > I don't know how I can explain this more plainly, I can (more accurately
> > have) taken a fiber build that was created as a ring & spoke SONET system
> > and with the same fiber plant overlaid that with GigE and ATM (further
> back
> > in time) to backhaul for PON, DSL, VOIP, and direct Active Ethernet.
>
> "Overlaid"?  Could you clarify that?
>

Sure, ring, hub & spoke, home run, star these are all descriptions of the
physical architecture and many layer 2 technologies will happily use them
all including Ethernet.  To use a specific example an existing SONET ring
(OC-3 to be precise) had be in service with an ILEC for more than a decade.
 This physical topology was a common one with a physical ring of fiber (32
strands, yes this was built back in the day) connected to Add/Drop
Multiplexers (Fujitsu IIRC) along the ring as needed to deliver 25,000 or
shorter copper loops either directly from the same cabinet that ADM was in
or from a subtended Digital Loop Carrier off of a spur (collapsed ring) of
the ring.  Now, SONET connections work off a pair of fibers, one for
transmit and one for receive.  To run Ethernet (initially 100mbps but now
10G) we simply lit 2 of the remaining 30 strands to overlay an Ethernet
ring on top of the SONET ring.  We then placed switches in the same remote
cabinets we had the ADMs and DLCs and started trenching the fiber drops.

>
> Owen's assertion (and mine) is that a loop architecture *requires* active
> equipment, suited to the phy layer protocol, at each node.  And while those
> loop fibers are running SONET, they can't be running anything else at the
> same time.
>

You're confounding the physical layer topology with the layer 2 protocol.
 You can't run SONET and Ethernet on the same physical fiber at the same
time (unless you use WDM but that's confusing the discussion) but you'd
never build a ring of fiber with only two strands.

>
> > There is nothing about a hub & spoke architecture is this harmful or even
> > suboptimal for doing Gig-E directly to end users today.
>
> You propose to run a ring *for each subscriber*?  Or put active gear in
> the field to mux the subscriber AE loops into a SONET ring?
>
> Or some other approach I don't know it possible?
>

SONET is simply the legacy (and expensive) way that telco's used to build
rings.  I'd neither use it nor recommend it for much of anything today.
 Calix, Occam(also Calix now), Adtran, and all the other guys who play in
this space will happily construct a Gig/10G/40G Ethernet ring in the same
shelf you're going to be buying to put your GPON or AE line cards in.

>
> >                                                         This wasn't
> always
> > true because we've only had 40G and 100G Ethernet for carrier networks
> for
> > a few years. In the past we were limited by how big of an etherchannel
> > network we could use for the ring. I'd also point out that the
> > ring architecture is optimal for redundancy since you have fewer fiber
> > bundles to get cut in the field and any cut to your ring gets routed
> around
> > the ring by ERPS (http://en.wikipedia.org/wiki/ERPS) in less than 50
> > milliseconds.
>
> I infer from that continuation of your thought that you mean the second:
> active optical muxes out in the plant.
>
> I'm sure I've made clear why that design limits me in ways I don't want
> to be limited when building a fiber plant for a 50 year lifetime, but let's
> address your responses below.
>

The only limitation you have is a limited supply of total fibers (hint,
this is a big reason why its cheaper to build and run).

>
> > > Lower the price per instance and you very likely find new demands.
> > >
> >
> > The vast majority of business don't WANT that kind of connectivity.
>
> The vast majority of businesses don't want it at the price they have to
> pay for it now -- or more to the point, the consultants who do their IT
> don't.
>
> You have no real way, I should think, to extrapolate whether that will
> continue as prices drop, especially if sharply.
>

The vast majority of businesses don't know and don't care about HOW their
connectivity is delivered and wouldn't know the difference between Layer 1
and Layer 2 if it punched them in the face.  Almost all businesses want
INTERNET connectivity at the highest quality & speed at the lowest cost and
that's it.  There are a small percentage, mainly larger businesses, that do
have special requirements, but those special requirements very seldom
include a L1 anything.

>
> > How many MPLS connections get purchased by SMBs? That's the same kind of
> > connectivity at layer 3 and that's a market that is almost entirely
> > used by large corportations.
>
> Sure; most small businesses don't need that.
>

Nor medium businesses, and that's where knowing your (potential) customer
base matters more than anything I can tell you.  If you're in an area with
lots of technology jobs and/or financial companies your network will look
differently than an average small town muni build.  If your customer base
is primarily residential with a few businesses (hospitals and schools also
count here) then you'll be lucky to sell a handful of L1 connections and
some of the people who will be interested will want it for very low bit
rate (means low price too) uses like RS-232 over fiber for managing SCADA
nodes or other telemetry pieces.

>
> But there are some that do, and there are some that it doesn't matter
> *where they are at*.  "Fiber on your wall with no upfront engineering
> charge" is a pretty strong call, in some markets, and I won't have to
> do most of the publicity myself; it'll make the news.
>

You'll get some publicity, especially if you do a little self promoting.
 The problem I see is that you seem to think that by building the L1 piece
you're going to have ISPs that are eager to serve your customers.  If your
demographics are like most small towns in the US that just isn't very
likely.  Any ISP partner is going to have to build and maintain a lot of
infrastructure before they can serve the first end user and your "no
upfront engineering" is simply not true unless you're going to configure
and run MetroE and/or GPON shelves for them.  In any sharing scenario (L1,
L2, or L3) the ISP is going to have to connect to you with enough bandwidth
to serve those end users as well.  How many service providers have
expressed interest?  Have you talked pricing for the loops and colo space
yet?

>
>
> > > But the vendors do and it makes a huge difference to the barrier to
> entry
> > > price for competing
> > > vendors offering different services. (I'm talking about more than
> > > just IP at this point).
> >
> > What vendors? ISPs don't.
>
> And your assertion here is based on what?  How many places have ISPs
> had a *choice* as to whether to take a L1 optical or L2 aggregated handoff?
>
>
I've done nothing but work with ISPs, often in the situation you're
describing, for the past 15 years.  I also know how ISPs, especially at the
size you might attract operate.

> > > What I'm proposing is a hub and spoke architecture. It's just a much
> > > larger hub with much longer spokes.
> >
> > That's called home running, but as I've said that's ok in some
> > scenarios, its just that in most cases there is no benefit.
>
> Today.  Neither you nor I know how that will change in 20, 30, or 50
> years.  But that's the horizon I'm planning not to block.
>

You're betting money against Ethernet's (not to mention any new technology)
ability to keep up as a technology.  You're statement is true, but its a
lot like buying insurance against meteor strikes.  Today on a normal ring
topology most people are installing hundreds of pairs of physical fiber so
even without adding Wave Division Multiplexing in you can easily build a
ring with several terabits per second capacity.  Even easier(and cheaper)
is to build several 10G rings on top of each other with the exact same
shelves you're going to be using to push out GPON or AE.

>
> > > You're assuming the current business model of incumbent-provider owned
> > > fiber. In a case where you have service providers not allowed to own
> fiber
> > > and a fiber provider not allowed to provide services, the incentives
> all
> > > work towards cooperation and the conflicts of interest between them are
> > > eliminated. I understand what you're saying about field technicians and
> > > their motivations, but, again those are based largely on the current
> > > business models and compensation schemes. In the proposed arena,
> there's no
> > > reason management at the service provider and management at the fiber
> > > provider cannot work together to address these issues. Further, the
> > > technician that blames the fiber plant for everything rather than
> > > cooperating to resolve said issues together will inherently have his
> > > installations take longer than the ones that cooperate, so he is
> actually
> > > already automatically incentivized in the correct direction.
>
> This is my goal.
>
>
Its an admirable goal, but you're never going to have CCIEs (probably not
even CCNAs) doing installs.  Installation is, has been, and will in all
likelihood continue to be done by people with limited skill sets.  You
building your own fiber plant and making it easier for ISPs to connect
isn't going to change that.

> > > Admittedly,
> > > without some education, that may not be intuitively obvious to him,
> > > but I find that education is usually possible when attempted.
> >
> > You need to understand that I've built the exact network your describing
> > several times and in all those case this was for a muni network in a
> > relatively small town (<25,000 residents). I also know who the installers
> > are in that sized community (as a group, not personally) and even if
> > you get the best ISP partners on the planet they're going to have normal
> > installers doing much of the work.
>
> When you say "normal installers", do you mean "employees of the ISP",
> "employees of the muni", or "subcontractors of one of those two"?
>
> And why is that pertinent?
>

Whoever does the install at the home or office.  Who they work for really
doesn't matter since the skill set will be largely the same.  Its pertinent
because you can have the greatest plant on the planet but if your install
isn't done correctly the end user (the guy paying you or your partner)
won't be happy.  The majority of your day to day problems will be driven by
installs.  The fiber was improperly terminated, the drop was in the wrong
place, the ONT didn't work correctly, etc.  Please, go join the LinkedIn
FTTx group and read some of the things there:

You probably can't follow that link until you join but its a good
discussion of some of the problems.

http://www.linkedin.com/groupItem?view=&gid=83578&type=member&item=208400553&qid=379425dd-6f05-4caa-b2d4-f4fbf2aeb9cc&trk=group_most_popular-0-b-ttl&goback=%2Egmp_83578

>
> Your assertion seems to be that it will be necessary to have "abnormal"
> installers in the field in order for them not to dump problem tickets
> off to the muni and fail to help meaningfully in fixing them.
>
> First, I think this unlikely since, in most cases, we'll have 3pr available
> at each address.  If we think there's a problem with the pair, we can
> "cut to clear" *temporarily*, and if the second pair is ok, then the sub
> is back online while we test the first pair and clear the problem.
>
> (GTE's failure, for all that I give them shit about CtC is that they never
> *worked* the dead pairs; as long as you do, it's not a problem.)
>

That's great, and I'm glad to hear you've worked out that part of the drop
but most of the problems occur from the drop into the house or office.  The
last 20 yards are the most problematic and most changed.  This is where the
installer matters most and why even good plant has bad installs.

>
> Second, since we'll be terminating all 3 pairs to a jackbox, the
> installation contractor will be able to perform and document whatever
> acceptance testing we instruct them to.  Sure, that will cost some
> more money, but again, if your capital plant is tested good when installed,
> this reduces markedly the opex maintenance cost over time.  Where that
> breakeven point will be depends on what they want to charge me to do the
> testing, just as it does with Cat6.
>
> [ Scott: ]
> > >> Sure it does, even in greenfield and whats more it costs more over the
> > >> long term UNLESS you know where every home and business will be
> located 10
> > >> years from now.
>
> A luxury I do have, since my city is nearly 100.0% built; it's certainly
> 100% platted.
>
> That's good, and again my argument isn't that you shouldn't home run all
your connections, but rather using the idea of L1 handoffs as a reason to
do so is flawed and that hub and spoke does not limit your options.

> [ Owen: ]
> > >> More yes, much more, I'm not so convinced.
>
> And Owen isn't the only one who thinks that, and I think I know Rob
> Seastrom well enough from the list to think he wouldn't concur unless he
> had some data from which to work.
>

Again, its not (I think I've said this enough) a problem in certain
scenarios.  The issue I take is that hub & spoke doesn't limit your future
use and that L1 connectivity as a reason to do home runs is seldom worth
the expense.

>
> Cheers,
> -- jra
> --
> Jay R. Ashworth                  Baylink
> jra at baylink.com
> Designer                     The Things I Think                       RFC
> 2100
> Ashworth & Associates     http://baylink.pitas.com         2000 Land
> Rover DII
> St Petersburg FL USA               #natog                      +1 727 647
> 1274
>
>

-- 
Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000
--------------------------------
http://twitter.com/kscotthelms
--------------------------------