IPv6 Deployment for the LAN

Wed Oct 21 21:34:26 UTC 2009

I am replying to several people here in one message.  I think most
issues were covered fairly well, but I obviously like to hear myself
talk, and I think there are a few things that need to be said more
plainly (I hope).

On Sat, Oct 17, 2009 at 08:55:28PM -0400, Ray Soucy wrote:
> Looking for general feedback on IPv6 deployment to the edge.

Having read the entire thread, I am surprised at how few answers and
feedback there are to the actual questions you have posed.  It seems
people are much more interested in an opportunity to redesign your
network for you or grind old hatchets...

> Given that historically we have relied on DHCP for a means of NAC and
> host registration, like many academic institutions, the idea of
> sweeping changes to accommodate IPv6 was just not going to happen in
> the near future.

Unfortunately, it's a tiny bit worse than that.  The IETF adopted the
DUID for client identification; it promises to be able to identify a
client uniquely across interfaces and NIC swap-outs (the MAC address
changes, the DUID does not).  This means you can't use the MAC address
portion of a DUID reliably.

Unfortunately, this completely circumvents all existing typical work
flows for user registration or inventory accounting.

There is still some work going on to try and reintroduce MAC based
accounting to DHCPv6, and there are some proofs of concept available
in source form today (but nothing standardized yet).

Of course there is no way RA could reliably provide this, and the
folks on this mailing list who have proposed you can predict SLAAC
addresses based on prefix and MAC are confused; they are not taking
into account the many clients that use temporary addresses by default
when the A flag is set (these are intentionally not cryptographically
predictable), or that clients may need to re-iterate their SLAAC
algorithm if interfered with by a peer (not even RA-Guard can save
you from that, it is a DAD exploit) and you can't necessarily reliably
detect iterations of the algorithm...

> Our current IPv6 allocation schema provides for a 64-bit prefix for
> each network.  Unfortunately, this enables SLAAC; yes, you can
> suppress the prefix advertisement, and set the M and O flags, but that
> only prevents hosts that have proper implementations of IPv6 from
> making use of SLAAC.  The concern here is that older hosts with less
> than OK implementations will still enable IPv6 without regard for the
> stability and security concerns associated with IPv6.
> 
> Needless to say, the thought of being able to enable IPv6 on a
> per-host basis is met with far less resistance than opening up the
> floodgates and letting SLAAC take control.

Ostensibly the solution to this problem is "per host RA", which has
seen many drafts at recent IETF's.  That is to implement RA in your
switches rather than your routers such that router advertisements may
be crafted in response to router solicitations on a per-switch-port
basis (which might track to per-user).

This is of course a completely scalable and well thought out plan
showing an obvious foresight to the structure of network configuration
management.  Any initial assessment that this is some kind of "kludge"
or "hack" is completely unfounded, and if managing RA configuration
across your entire switch fabric isn't scalable for you, then you just
need to rethink your network architecture and buy more tools.  After
all, your switches are in a better position to know more about prefix
and router information than your routers are anyway, so there's no
reason to force us all into top-down configuration management models.

Things really have become that desperate.

On the other hand, as you say, you could "just use DHCPv6."

> Ultimately, the best solution that I've been able to come up with is
> to preserve the IPv6 allocation schema and reserve a 64-bit prefix for
> each network, but for the initial deployment use an 80-bit one in its
> place with the extra 16 bits given a value of 1.  The advantages of
> this: Guarantee that SLAAC will not be initiated  for the prefix;
> Allow for a migration path to 64-bit prefixes in the future; and, Make
> it easy to identify a network that us making use of an 80-bit prefix
> by setting the extra bits to a value of 1.

I can think of something you may like to know beforehand;

There is a problem with the "Both RA and DHCPv6 Model" currently
proposed by IETF iconoclasts.  Should RA fail, for any reason from
router, system, software, network path, security, or user error,
then the simplest networks imaginable (where hosts and mail/Intranet/
work servers are all co-located on the same broadcast domain) will
not be able to talk to one another, despite having valid IPv6
addresses.  The problem is that they no longer know that the prefix
for their address is locally connected.

To work around this problem, some DHCPv6 software implementers have
elected to temporarily apply a fixed /64 bit prefix to all applied
addresses until a prefix length can be made available in-band through
some means.  This does neatly work around the problem; the hosts may
now speak to one another.  But it may complicate your designs if you
are assigning /80's.

IPv6 does not have 'broadcast addresses' like IPv4 does, it uses a
separate multicast address space instead, so there should not be
similar non-interoperability issues with mixed prefix lengths as we
have had in IPv4.

I don't actually know if these circumstances will directly change your
current plans, but it could have future ramifications if anyone
believed they could segregate clients in separate prefixes under the
covering /64.

Anyway, I thought that this was the sort of thing you'd like to know.

I doubt that many people try to use /80 prefixes or smaller prefixes
with anything other than routers (and there, manual address
assignment) for point to point links, so you are probably forging new
territory here.

> Windows XP has a poor implementation of IPv6 but has the option of
> using Dibbler or some other 3rd party DHCPv6 client.

Dibbler is a solid software package.

> Mac OS X is a
> challenge; it currently has no option for DHCPv6, though newer
> releases provide for manual configuration of IPv6 addressing.

According to the release notes of ISC DHCP 4.1.0, ISC 'dhclient -6'
compiles and runs on Mac OSX.  I don't actually own a Mac, so I have
never used it myself, and our release notes even go into detail about
the limitations of the current 'dhclient-script' for the Darwin
platform (apparently their configuration system is somewhat opaque).

But if there are ways our dhclient-script can be improved (perhaps by
including other C-based tools to interface with OSX's configuration
schemas?), we absolutely accept patches.

(any followup to dhcp-users at isc.org and dhcp-bugs at isc.org please, no
 need to bore NANOG)

> Does anyone know if Apple has plans to release a DHCPv6 client for Mac OS X?

No...but I have heard plans of the exact opposite.

At one of the last IETF's I attended, there was an experiment during
the Plenary session; IPv6 single-stack was provided.  At the same
time, an escape was provided for those who were unable to use IPv6
single-stack (for whatever reason).  This was provided via two
competing implementations of "4-6-4 NAT", an experimental technology
at the time.

According to statistics (DHCPv4 PRL fingerprinting), the most popular
client on the 4-6-4 NAT network was OSX.  It is supposed that Macs
could not get name servers (without manually configuring them), and so
could not even start to get net on IPv6 single-stack.  When people at
the microphone asked why Apple did not include a DHCPv6 client
implementation, even a stateless one, I believe Bernard Aboba
addressed the concern at the microphone saying, and I am paraphrasing
from memory here, "We were told by the IETF that RA was all we would
ever need, implementing DHCPv6 is optional, so RA is all we are
doing."

So, no, I wouldn't say there's been any indication they have plans for
a DHCPv6 implementation.  It seems they are steadfastly against it,
which is disappointing.

I don't want to say that queries to Apple's support infrastructure to
ask that DHCPv6 be implemented is misplaced, but I think those trying
may be "paddling upstream."

If there is some change in the direction the water is flowing, I'd be
more than happy to lend Apple any assistance I can offer, especially
with testing a new DHCPv6 software package; there has not been a
DHCPv6 vendor bake-off in years now, Apple has missed the time of
opportunity when there was interest in testing implementations against
each other, so that sort of thing has to be synthesized anew.

If there isn't, then I'm more than happy to provide a locus for
coordinating the development of DHCPv6 client services for Macs.

> default behavior.  Is this likely to happen or am I being too
> optimistic?

There will be one of three outcomes, with complete certainty.

The first is that DHCPv6 becomes feature-complete and widely adopted
to finally fulfill the global requirements of host configuration, and
not merely those of the IETF iconoclasts' homes and laptops while
visiting IETF meetings (where DHCPv4+RA is sufficient).

The second is that RA is extended, with new messages as well as new
options for host configuration, until it finally becomes DHCP.

The third is that some new protocol will be designed, by people who
are unapologetic about fulfilling specifically the global requirements
of host configuration, which supplants both RA and DHCPv6.

On Sun, Oct 18, 2009 at 02:17:58PM -0400, TJ wrote:
> Um, DHCPv6 does configure the client - perhaps not until the +M or +O option
> is recieved.

This, the 'M and O bits', is a very pernicious mistake that I can't
resist commenting on (I think TJ you might not have meant it, so this
is not necessarily directed at you, but it's a theme I've seen in this
thread that "A, M, and O bits are completely reliable").

To summarize, RA+DHCPv6 implementers are posed with problems:

1) Can I trust that RA will be there independently of DHCPv6?

2) Provided it is there, can I trust that the M and O bits will be
   consistent across all routers on the segment?

3) Provided it is not consistent, which one should I believe?  Should
   DHCPv6 services be turned on and off as the M and O bits toggle?
   Should it stay on so long as any M or O bit is set in any router
   (so M and O states must be kept on a per-router basis, leading to
   yet another problem in that an attacker can attempt to create an
   infinite number of M-and-O states in the client)?

4) What do you do if both M and O are set?  What if O is set but A
   isn't?

5) It takes time for RS->RA to complete, and then time again for
   DHCPv6's Solicit->Advertise->Request->Reply to complete.  If I
   run these two in parallel, I get on the net faster and my user is
   happy about that.  Why wouldn't I do that?

There are possibly some others I've forgotten.  There is for example
the entire set of corner cases; e.g. a DHCPv6 client returning to a
network segment will "Confirm" to determine if their old binding is
still valid for the network they're attached to (laptop returning from
hibernate for example) - it'll certainly do this before it receives
any RA's.  If the DHCPv6 configuration is valid and the Confirm
succeeds, are you still supposed to wait (possibly forever) for an RA
before installing configuration?

This is why the RA RFC's are extremely waffle-mouthed on the subject
of what a client should do when it encounters M and O bits.  SHOULD
and MAY and all that.  That waffle-mouthedness is the reason why there
are all kinds of bad behaviour in clients receiving RA's; some will
start a DHCPv6 client every time M or O toggles 1->0->1->0...and
ultimately crash.

A RA and DHCPv6 software implementor from Beijing (I think) wrote a
draft awhile ago asking for clarity (and from which I'm cribbing his
problems list from memory):  "I am a software implementor.  Tell me
clearly what to do here."  The official IETF consensus was, in my
observation of it, "we can't, we don't even know ourselves, just go
make something up.  It will work out.  Don't write an RFC, no one will
ever agree."

So we don't have an RFC.

But the answer is that a DHCPv6 software implementor's best bet is to
simply run DHCPv6 statefully all the time, and run DHCPv6 stateless
whenever that fails and SLAAC addresses are successfully configured.
That is simply the most reliable thing to do.

The RA RFC's are completely permissive of this most liberal
interpretation.  So as it turns out, whether or not a router is on
link has nothing to do with whether or not a DHCPv6 server is on link.

So although you may find some implementations that still try and
guess something from M and O bits, I suspect these will slowly become
fewer and fewer in number.

> And RA Guard will fix it for IPv6.  Did we have DHCP Guard @ day 1?

DHCPv4 you mean?  Basically yes, it's just that at the time there
wasn't as much need for that sort of thing.  It was a simpler time.

Also, in my opinion RFC 3118 is more more believable than SEND, and
DHCPv* snooping is a more reliable way to enforce switch fabric RPF
than anything RA/ND based.

There is another issue in the shadows I want to speak to here however.

When I worked as an operator, all those years ago, we had a vendor.  I
don't think their name matters, so I won't share it.  They sold server
hardware.

The details of what would go wrong with their hardware and the ways we
would have preferred to work with failing systems aren't important;
what's important is their approach to our every complaint that the
device they sold us wasn't meeting our expectations, or didn't fit in
our network architecture.

Every answer to every question was to buy more hardware, or to take a
hammer to our network.

So we stopped coming to them for solutions, we used folks who answered
the question on the first try.  It is easier to patronize someone who
genuinely wants to solve your problems than to fight them for
everything you need.

Similarly, with RA, every answer to every question is to buy more
software add-ons that just need one more RFC, or to re-architect your
network.  Twiddle the bits just the right way, set your frob to zero
and your woznit to 1 and it's all good, right?

I am extremely skeptical that we'll ever reach where we're going if
we get there one millimeter at a time all the while redrafting our
plans over and over.

Someone has forgotten that the reason IPv4 deployed so pervasively is
because it was so very trivially simple.  You didn't have to know the
names of single-bit fields in the headers of ICMPv4 Router Discovery
(which really is not very different from IPv6 RA).

> > egos from tripping over other egos, each camp kept on their own turf:
> > dhcp6 was hobbled to the extent that it couldn't feasibly be called a
> > host
> > configuration protocol (no default route, no address assignment and no
> 
> Incorrect, DHCPv6 can assign addresses.

I think in the spirit he is speaking at the time, he is speaking to
the intention of the IETF iconoclasts that, realistically, SLAAC would
be the only addressing mechanism used because it is the only universal
addressing mechanism, and DHCPv6 would only be used statelessly, if at
all.  Stateful DHCPv6 service would be relegated to the outlier
"unusual" networks.

I think perhaps people are starting to learn that networks that use
the network management features of DHCPv4 are not quite as unusual as
had first been imagined, and that IPv4 will not reach these networks
until one can reliably expect the same management features from
DHCPv6.

> > - there were two protocols required for stateless network management
> > instead of one
> > - we already had a really good working model in ipv4
> 
> Perhaps, But I submit that "good" and "working" do not mean "ideal".

SLAAC set out to solve a problem that no one had.

It chose to complicate the host implementation (SLAAC address
generation and the subsequent DAD and SLAAC-iterating is much more
complicated than copying a field from DHCP into another field) as the
expense for simplifying the router implementation (which now had to
just send flat RA messages).  Was that really a problem we had before?

Looking at just one network (broadcast domain) at a time...

How many routers do you have?  How many software implementations and
versions?

How many clients do you have?  How many software implementations and
versions?

Only one of these two things is "ideal" to simplify at the expense of
the other.

It actually turns out that having a simplistic DHCPv6 server
implementation that dumbly creates SLAAC-like addresses and spits them
at the client with no state-keeping at all fulfills a superset of the
requirements SLAAC solves...and keeps clients simple.  SLAAC then
becomes a proper subset of network operations requirements, obviated
by the existence of a DHCP-like analogue.

Eventually we'll get there.

> With the addition of RFC5006, you can actually choose just one (once hosts
> implement it).
> Just not the one you seem to favor.

DNS servers and search paths are not the alpha and omega of host
configuration.

> And I am OK with all that as well, although THAT also complicates scenarios
> for implementers.
> (Now hosts will need to support two discrete host-configuration protocols
> that actively step on each others' capabilities).

When we migrated from walking around the office setting our users'
computers' IP addresses and configurations manually to BOOTP or RARP,
we bled for awhile; we still had to do the dirty work while we waited
for implementation support.

When we migrated from BOOTP to DHCP, we also had to bleed for awhile,
reserving addresses for BOOTP-only clients while migrating systems to
DHCP service...and those systems had to cope with the problem where
DHCP may not have been available initially, and fall back to BOOTP (of
all the devices on the Internet today, the only one I've seen that
actually still does this that you can buy new are APC UPS's management
interfaces...kind of astounding really).

When migrating from RA to DHCPv6, there will be pain, but the
difference is that there is a light at the end of a tunnel; we do not
run BOOTP anymore.  There will come a day when we can turn off RA at
the routers, and hosts won't need to carry SLAAC implementations.

It seems the thing history teaches us is how to repeat it.

> Well, obviously not _fully_ standardized as we are still adding stuff ...
> but I would argue the sanity part is OK.
> And again, it still (esthetically and architecturally) seems to make a lot
> of sense for the router to send out information about the router.
> With the added bonus of "it can and does work today", regardless of the
> arguments for/against it.

Unfortunately this "IETF horse sense" doesn't track into actual
networks.  It is not a question of "what device knows more", but
rather a question of what person knows more about what the
configuration for a given host should be (even its specific IP
address and other network related configuration).

That person is not typically your router operator.  It is typically
a systems operator shackled to a short desk in the basement.  These
two people are commonly two discrete entities, serving different
masters in the umbrella of a bureaucracy that hates itself.

For the "RA+DHCPv6" model to succeed, you will first have to force
those people to come into good enough terms with one another to
design, build, and debug services together in peace and harmony.

When you're done with that, I think they can use your help in the
middle east.  Should be easy.

On Sun, Oct 18, 2009 at 12:15:47AM -0500, Clue Store wrote:
> >Since the goal for this initial wave is to make IPv6 available to
> >those who request it or have a need for it, we feel its acceptable
> >that there will need to be some user participation in enabling IPv6
> >for a host.
> 
> To me, from a small ISP perspective, this is where the largest delima is....
> what 'vendor' is already depolying end user equipment that is ipv6 ready??
> Then there's the 'delivering the customer' their ipv6 block (hopefully
> alongside their ipv4 block). Dual stack seems the way to go...

For even a small ISP, dual stack is not incredibly obvious to me as
a selectable solution.

As the IPv4 shortfall comes, realistically most content on the
Internet will continue to be on the IPv4 network.  Any IPv6 content
will also be available on the IPv4 network; being IPv4-single stack
will not deprive the customers of content.

What it does deprive them of, with increasing layers of NAT or proxy
service, is "dial-in" access.  Many do not require this feature.  The
cost of providing it is increased support costs; debugging two
networks and three or four protocols.  Today, even debugging IPv4
problems with customers is problematic and costly enough.

That doesn't seem like a winning combination to me; more cost for
no real benefit.

A few NANOG's ago, Alain Durand from Comcast spoke about their plans
to use IPv6 as a new L2, so their infrastructure can all be IPv6
addressed, and their customers traffic will be carried (by tunnel and
NAT) and delivered by IPv4 riding on top of it.

Having converted the infrastructure to IPv6, this puts them in a very
good position to start deploying IPv6 in a limited capacity to the
customer premise (for their own equipment, or for customers who elect
to be IPv6 addressed, possibly as haven from being NAT'd).

So far this is the best story I've heard for incremental IPv6
deployment.  If you can make the charge straight out to dual
stack, that's great, but I'm happy to see even large networks with
more realistic, incremental goals.

> To me, there's still a lot of wiggle room on how this should be deployed to
> the absolute edge.
> 
> What's folks experience in rolling this out the the customer ... be it DHCP
> or SLAAC?? Also from a BBA perspective??

I have only worked with IPv6 directly in "office" networks, not in
traditional service provider networks, but there my experience with
SLAAC has been extremely poor; back to the days of manually
configuring your clients kind of poor.  DHCPv6 and in particular
dynamic DNS is really the only solution in the office.

I can suppose however that giving your customers IPv6 prefixes, and as
a result gibberish IPv6 addresses, is going to give rise to a greater
need for dynamic DNS services; they don't pass the phone test, for
one thing.

However it is perfectly acceptable in this "absolute edge" (and in
fact every IETF meeting network does it this way) to use SLAAC to
give IPv6 lip-service addresses while still using DHCPv4 to assign
domain name servers, domain-search paths, NETBIOS configuration,
so on, so forth.

In this sense, IPv6 right now needs DHCPv4 as a crutch in order to
bootstrap.  And there's nothing wrong with that; DNS can resolve
AAAA addresses using IPv4 addresses for your name servers just as
easily as if you had supplied IPv6 addresses for them.  Your DNS
bits are not tastier if delivered by IPv6.

When you move closer to the core...

DOCSIS3 cable modems typically are assigned addresses (and
configuration) by DHCPv6 rather than being left to their own default
or customer configuration.

PPP is not going to be extended to assign IPv6 addresses; over the
PPP channel one will use either RA or DHCPv6 again.  Because like any
other network, an operator must ensure the client on this link is not
sending from bogus (neighbor) source addresses, they will need a way
to assign an IPv6 address to match filter rules, so then that means
DHCPv6.

Finally, I have something very abstract to say on the general subject
of the underlying SLAAC-vs-Stateful-DHCPv6 debate;

I submit the remainder of the debate over RA or DHCPv6 for address
assignment is not technical.  It is not religious or political.  It
is philosophical.

With RA, you have a very Marxist-turned-Communist philosophy of
design.  All hosts are equal, everyone gets the same thing.  From each
according to their ability, to each according to their need.  You want
to be a router?  Go ahead!  Send an RA.  The hosts are all allowed to
freely roam and operate within their own limited capacity; but not
really, eventually you need papers (along comes SEND and all the
future development), and a bureaucracy of enforcement.

DHCPv6 on the other hand embodies the philosophies of fascist
dictators.  Everything within the state, nothing outside the state.
The will of the network over all.  Hosts do what they are told, if
they don't then results can't be guaranteed.  You might just get hung
out to dry unless you step in line.

Although you might fail at building a social structure around both
Communist and Fascist ideology, or perhaps some of you might debate
that, I submit that when applying a philosophy of design to building a
network for money - not as some social service, experiment or hobby -
then the only philosophy worth applying is absolutely Fascism.

Anything less, despite the nobilities espoused by their protractors,
and you will bleed hidden costs.

-- 
David W. Hankins	"If you don't do it right the first time,
Software Engineer		     you'll just have to do it again."
Internet Systems Consortium, Inc.		-- Jack T. Hankins
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20091021/18028a48/attachment.sig>