Failover IPv6 with multiple PA prefixes (Was: IPv6 fc00::/7 - Unique local addresses)

Tue Nov 2 23:52:36 UTC 2010

In message <CC14FCD0-1924-425A-8879-0C1FA6ADE941 at delong.com>, Owen DeLong write
s:
> 
> On Nov 2, 2010, at 3:08 AM, Mark Smith wrote:
> 
> > On Mon, 1 Nov 2010 18:04:28 -0700
> > Owen DeLong <owen at delong.com> wrote:
> >=20
> >>>>>=20
> >>>> He may or may not be. I don't think it's such a bad idea.
> >>>>=20
> >>>=20
> >>> How about algorithmically generating these addresses, so that
> >>> they're near unique, instead of having the overhead of a central
> >>> registry, and a global routability expectation?
> >>>=20
> >> Why not just keep a low-overhead central registry and start accepting
> >> that PI !=3D global routability. Routability is a discussion between =
> the
> >> resource holder and their ISP(s).
> >>=20
> >> ULA is the algorithmically generated problem and I think it's a =
> generally
> >> bad idea. It's basically an expectation of uniqueness where it may or
> >> may not exist and the potential to fudge the level of routability =
> into whatever
> >> strange definition long-term creativity may evolve.
> >>=20
> >> I think it's better to make GUA easy to get and remove the =
> expectation
> >> that GUA =3D=3D Routable. (Ideally, we'd restore that eventually with =
> a
> >> scalable routing paradigm).
> >>=20
> >=20
> > Is it possible to get GUA from RIRs without making it globally =
> visible?
> > I think the only current exception is IXes. A couple of years back I'd
> > have liked to have gotten a globally unique IPv4 allocation that =
> wasn't
> > being announced globally for use with customer facing DNS servers,
> > reducing their DoS attack surface significantly. Wasn't able to.
> >=20
> Depends on what you mean by globally visible.
> 
> If you mean routed, sure. There is no requirement whatsoever to route
> your address space and there are specific provisions for non-connected
> networks to get space. There always have been.
> 
> If you mean visible in whois, then, IXes are not exempt and there are
> no actual exceptions for RIR-isssued addresses.
> 
> If you weren't able to get the space you describe, most likely there
> was some other problem with your application, or, you did not apply
> under the correct policy for your situation. (assuming this was an
> ARIN application. I am not as familiar with the other RIRs policies
> in this regard).
> 
> 
> >>>>> Recently we've seen somebody (on either nanog or ipv6-ops) =
> proposing to
> >>>>> set valid lifetimes of 24 hours on ISP GUA prefixes. While a 24 =
> hour
> >>>>> outage is unusual for a always connected broadband service, it =
> isn't
> >>>>> for intermittently connected nodes and networks.
> >>>>>=20
> >>>> The upstream valid lifetime doesn't have a lot to do with what =
> happens on
> >>>> the internal network if you're smart.
> >>>>=20
> >>>=20
> >>> Residential end-users aren't "smart" and aren't network engineers - =
> they
> >>> pay people like us so that they don't have to be.
> >>>=20
> >> That still doesn't have a lot to do with enterprise failover which is =
> what we
> >> were talking about.
> >>=20
> >> As to residential, residential end users mostly don't care if their =
> network
> >> goes down when their provider goes away. However, for those that do,
> >> it still shouldn't be hard for the provider to uncouple circuit =
> viability from
> >> address presence.
> >>=20
> >=20
> > In some markets, CPE are sold at the local electronics/white goods
> > store.
> >=20
> So?
> 
> >>>>> In effect people who suggest using PA GUAs or PI for all IPv6 =
> addressing
> >>>>> are saying you can't run IPv6 unless you have an available IPv6 =
> ISP
> >>>>> connection or you must be able to afford to be able to thrust upon =
> the
> >>>>> world occupation of a global route table slot. They're not =
> realistic
> >>>>> requirements for all potential users of IPv6.=20
> >>>>>=20
> >>>> No...PI does not require an available IPv6 ISP connection at all. =
> This
> >>>> is a misstatement that does not become any less false no matter how
> >>>> many times you repeat it.
> >>>>=20
> >>>=20
> >>> What if you don't have an IPv6 ISP connection? Where do you get your =
> PA
> >>> from? Link local isn't good enough, because it can't span more than =
> a
> >>> single link. Homes in the future are likely to have multiple =
> networks -
> >>> visitor segments, multicast segments for video, children
> >>> segments, 6LowPAN for home sensor networks etc.
> >>>=20
> >> It gets configured in your router.
> >=20
> > By whom?
> >=20
> Ideally automation. Potentially either the ISP or the end user.
> 
> >> Why is that such a difficult concept?
> >=20
> > For me it isn't, but a lot of things are simple to people with the
> > right expertise. For residential customers who don't know what an IP
> > address, I'm sure it is not only difficult but probably for most an=20
> > impossible concept.
> >=20
> This was intended as a suggestion for enterprise customers that
> cared about reliable failover between providers. If a residential
> customer is multi-homing and cares about faling over between
> two providers, I'm pretty sure they have some expertise.
> 
> However, this could still be automated even in the case where
> no expertise is present.
> 
> >> Your home gateway that talks to your internet connection can either
> >> get it via DHCP-PD or static configuration. Either way, it could =
> (should?)
> >> be set up to hold the prefix until it gets told something different, =
> possibly
> >> even past the advertised valid time.
> >=20
> > That breaks the IPv6 spec. Preferred and valid lifetimes are there for
> > a reason.
> >=20
> Sure... However, if you have no upstream connectivity there is no harm
> in hanging on to the prefix until connectivity is restored and you =
> receive
> an indication that you should do something different from what you are
> currently doing.
> 
> >> It can delegate subnets using
> >> DHCP-PD, but, the point is that the top level router at the site can
> >> easily be coded/configured to keep the prefix regardless of the state =
> of the
> >> external link.
> >>=20
> >>> You've stated you use link locals for this sort of thing, yet you'd =
> be
> >>> specifying the interface to use as well. That isn't much different =
> to
> >>> using a subnet number, embedded in the address, to specify either
> >>> directly attached or remotely reachable subnets. The nice thing =
> about
> >>> doing it that way is that IPv6 applications are addressing scope
> >>> agnostic - they just use the address supplied, and ask the
> >>> underlying OS, which uses the local route table, to
> >>> direct where the packets go and therefore select the outgoing =
> interface.
> >>> Link locals + interfaces is more complicated, because now socket
> >>> options have to be invoked, and only in the case of when a link =
> local
> >>> address is specified, which also means performing an address type =
> check
> >>> for the interface option. This code has to be present in ever
> >>> application, instead of letting the underlying OS taking care of how
> >>> application packets are directed towards their destinations, and the
> >>> applications not having to care.
> >>>=20
> >> No, I've stated that you could. I have stated that I use link locals =
> for
> >> a variety of things.
> >>=20
> >> Usually for this type of thing, I'd use a legitimate GUA prefix =
> whether
> >> PI or PA.
> >=20
> > That doesn't mean there they only options that will work. I'm
> > guaranteed to have a ULA prefix to solve this problem, because I can
> > generate one myself, and know that it won't collide with a GUA prefix
> > if I get one at a latter date. If I follow the ULA algorithm, it
> > probably also won't collide with any other ULA domains I may
> > interconnect with. I can't be assured of those attributes for a GUA
> > prefix, and may not be able to wait to establish a commercial
> > relationship with an ISP before I get a GUA for use with this purpose.
> >=20
> Huh? You are absolutely guaranteed that a GUA prefix won't collide
> with anyone else's GUA prefix. You are absolutely guaranteed that it
> will not collide with anyone else's ULA prefix. You can't generate a GUA
> yourself, you need to get it from an ISP, and LIR, or an RIR, but, =
> that's
> the only way to get guaranteed uniqueness.
> 
> >>=20
> >>=20
> >>>>> For the most common and scalable case of PA, external addressing
> >>>>> dependencies reduce reliability, because you can't control them.
> >>>>> Completely relying on external connectivity and addressing for =
> your
> >>>>> internal networks reduces their reliability and availability.
> >>>>>=20
> >>>> This is also false if you use any form of sanity in applying the =
> assigned
> >>>> PA prefix to your network.
> >>>>=20
> >>>=20
> >>> I suppose since they don't have the expertise, you could consider
> >>> residential end-users insane. You can't make the insane sane just by
> >>> telling them to be so. Preventing their "insanity" from breaking =
> their
> >>> Internet service, causing them to call your helpdesk, is the sane
> >>> thing to do. That is achieved by making their Internet service work
> >>> with the absolute least operational intervention on their part. It's
> >>> hard enough to get them to enter their username/password via an
> >>> embedded web server - to the point where some vendors supply setup =
> CDs
> >>> to automate the discovery of the device, avoiding the end user =
> having
> >>> to type an IP address URL into their browser.
> >>>=20
> >> Sure, but, why can't you set it up so that you either ship them =
> pre-configured
> >> hardware, have their hardware download it's configuration once each =
> time
> >> the CONFIGURATION MASTER RESET button is pushed, or, having the
> >> hardware learn the configuration via DHCP-PD, but, keeping the =
> configuration
> >> until a newer configuration is received?
> >>=20
> >=20
> > As I said earlier, CPE isn't always sold or operated by through the =
> ISP
> > the customer will get Internet services from.
> >=20
> So? That doesn't mean what I have proposed can't work. Shipping
> pre-configured hardware was one of three options I gave above
> for solving this problem. The other two do not depend on sold or
> operated by ISP.
> 
> >> All of these provide zero-user-intervention ways to configure their
> >> equipment such that they will have a valid PA address locally that
> >> survives a link failure.
> >>=20
> >>>>> In this common case of PA, how are you going to justify that "no =
> IPv6
> >>>>> without an IPv6 ISP" view to people who are very remote, such that =
> even
> >>>>> intermittent Internet access is very occasional; to people who run =
> IPv6
> >>>>> sensor networks and don't ever want them connected to the =
> Internet; or
> >>>>> 3rd world countries where just local connectivity provides a very
> >>>>> significant benefit, when global connectivity just isn't =
> affordable?
> >>>>> These and similar are cases where only ISP PA or PI aren't =
> acceptable.
> >>>>>=20
> >>>> Nobody is trying to. This is a fallacy of logic that you keep =
> pushing,
> >>>> but, it's still false. If I wire a PA prefix into my router, it =
> doesn't go
> >>>> away just because the ISP does. All that happens is that I can't
> >>>> reach the internet from it, which is kind of true regardless of the
> >>>> address space used at the point where your ISP goes away.
> >>>>=20
> >>>=20
> >>> You haven't ever tried to get a majority of residential end-users
> >>> to update their firmware have you? You'll have the same luck getting =
> a
> >>> majority of them to "wire a PA prefix into" their routers.=20
> >>>=20
> >> Why do they have to wire it in? Why can't I wire it in for them?
> >>=20
> >> I know lots of companies that maintain control of the top-level CPE =
> router
> >> for just this reason.
> >>=20
> >=20
> > as before.
> >=20
> Yes... It's not a one-size fits all solution, it's one way to address =
> one particular
> problem.
> 
> I proposed others that are workable in the other situations as well.
> 
> >>>>> Permanent connectivity to the global IPv6 Internet, while common,
> >>>>> should not be essential to being able to run IPv6, and neither =
> should
> >>>>> PI. All you should need to run IPv6 reliably is stable internal
> >>>>> addressing. Global connectivity should be optional, and possibly =
> only
> >>>>> occasional.
> >>>>>=20
> >>>> Why shouldn't PI if it was easy to get? I still don't understand =
> the
> >>>> perceived advantage of ULA vs. PI other than the perception that
> >>>> it is easier to get. If PI is just as easy to get, why is it a =
> problem?
> >>>>=20
> >>>=20
> >>> It seems to me your main criticism of ULAs is that people would =
> expect
> >>> it to be globally routed if they paid enough money. Now you're =
> saying
> >>> that if PI is really easy to get, people *won't* have a global =
> routing
> >>> expectation of PI routability? I certainly would if I was given PI.
> >>> What would be worse is that this "non-routable" PI won't come out of =
> a
> >>> specific prefix so that it can easily filtered, unlike ULAs.
> >>>=20
> >> If you find a provider that will route your PI, no harm done. You're =
> paying
> >> enough to get someone to listen to your routes and the internet is
> >> accepting them for the time being.
> >>=20
> >> At least your PI is subject to policy and the will of the community.
> >>=20
> >> ULA, on the other hand, has no community oversight, no policy body,
> >> and no restrictions whatsoever on who else uses "your ULA". Yet,
> >> through creativity and luck, ULA will eventually get routed across
> >> more and more of the internet until it starts to look like cheap easy
> >> policy free GUA. At that point, the harm is not about your =
> expectations,
> >> the harm is about the failed expectations of the rest of the internet
> >> with respect  to ULA.
> >>=20
> >=20
> > I think you under estimate the value of free. With GUA addresses you =
> get
> > free global routability. With ULAs you don't. Why would a network
> > choose to only use ULAs and then try to force upstreams to route it by
> > writing out big cheques (checks), when instantly and
> > persistently globally routable GUA either comes for free with the
> > Internet service, or at a much lower cost than trying to make
> > non-globally routable ULA address space routable - that isn't ever
> > assured of being anywhere near as successful in providing
> > global reachability as using GUAs is.
> >=20
> I don't know. If it weren't for an NDA, I'd give you the names of =
> several
> companies that you could ask why they chose to try and do this with
> invalid IPv4 addresses. (IANA reserved free-pool addresses, not even
> RFC-1918).
> 
> >=20
> >>>>>> 2) ULA brings with it (as do any options that include multiple
> >>>>>> addresses) host-stack complexity and address-selection issues... =
> 'do I
> >>>>>> use ULA here or GUA when talking to the remote host?'
> >>>>>>=20
> >>>>>=20
> >>>>> There's an app for that (or rather a library routine called
> >>>>> getaddrinfo() and an optional table it consults), and there's soon =
> going
> >>>>> to be a way to distribute it via DHCPv6 if the defaults don't suit =
> -
> >>>>>=20
> >>>>> http://tools.ietf.org/html/draft-fujisaki-dhc-addr-select-opt-09
> >>>>>=20
> >>>> Sure, now, how many applications have been coded to actually
> >>>> pay attention to what getaddrinfo is telling them about address
> >>>> selection order?
> >>>>=20
> >>>=20
> >>> All the ones I use - they all seem to use the first getaddrinfo()
> >>> response. They should be attempting to successively connect() to all
> >>> responses in the order that getaddrinfo() returns as connect()
> >>> failures occur. I don't know if they are (as destination =
> reachability
> >>> is usually good), however if they aren't, then the application
> >>> developers haven't used getaddrinfo() correctly. That behaviour
> >>> wouldn't be exclusive to IPv6 though - IPv4 applications should also =
> be
> >>> attempting to connect() to successive addresses when multiple are
> >>> returned. IOW, applications coping with multiple responses to
> >>> getaddrinfo() is not an exclusive issue to IPv6.
> >>>=20
> >> There are well behaved and not so well behaved applications out
> >> there with respect to getaddrinfo. I agree with you about the ideal,
> >> but, counting on that is sort of like counting on the user to =
> configure
> >> something correctly... Not likely to reduce your help desk calls.
> >>=20
> >=20
> > With IPv4, the history of applications on the Internet is that they've
> > only tried one connection attempt, due to the nature of
> > gethostbyname(). That has worked very reliably. IOW, that
> > means destination hosts are usually available the majority of the =
> time.
> > So, as I said, ideally applications should try to connect to multiple
> > addresses in turn if multiple addresses are returned. IPv4 history has
> > shown that it isn't actually as important as it may appear to be.
> > Therefore, attempting to connect to each returned address is more of
> > an an optional robustness and resilience mechanism. The need for it
> > in IPv6 might be increased due to the possibility of multiple paths to
> > the destination host, but I don't think it is an all that significant
> > increase. The likelyhood is the first returned address will work most
> > of the time, as it has in IPv4. For highly available services, people
> > choose to put in place mechanisms such as HSRP or VRRP to provide
> > further availability for announced addresses.
> >=20
> Actually, gethostbyname returns a linked-list and applications should
> try everything in the list until successfully connecting. Most do.
> 
> However, the long timeouts in the connection attempt process make
> that a less than ideal solution. (In fact, this is one of the main =
> reasons
> that Google does not publish AAAA records generally today).
> 
> However, that isn't the issue above. The issue above is about whether
> or not:
> 	getaddrinfo() always returns the addresses to be tried in proper
> 		order.
> 	Applications are always well behaved in attempting connections
> 		in the order returned by getaddrinfo()
> 	Whether the deployment of the gal.conf file to hosts in order to
> 		give getaddrlinfo() the correct hints about ordering is
> 		likely to occur correctly and reliably.
> 	etc.
> 
> There are many dependencies to making source address selection
> in IPv6 work correctly. They are exacerbated in a ULA environment.
> If you thought putting a single address (or prefix) into a CPE router
> by hand was hard, do you really expect the customer to manage
> a gal.conf file on all their hosts? Seems to me this is much harder
> than the router configuration.

You do realise that it is easy to do completly automate this as ULA
come from a well defined address block.  A simple tool can generate
this for the older machines which haven't been updated to know about
ULAs

If you have a interface configured with a ULA address.  Take that
address, generate two entries.  One for /48 and one for the /64.

Preference the ULA/64 addresses first (link). 
Preference the ULA/48 addresses next (site).
Preference the PA/PI/6to4/64 addresses next (link).
Preference the PA/PI/6to4/48 addresses next (site).  (a RA would be a good way
to distribute the site size other than /48 for PA/PI).
Preference 2000::/3 next. 
Preference 2002::/16 next.
[2000::/3 2002::/16 reverse order if you don't have any non-ULAs outside of
 2002::/16]
Preference fc00::/7 last.

For ULA/64 destination select a source address from the corresponding ULA/64.
For ULA/48 destination select a source address from the corresponding ULA/48.
For PA/PI/6to4/64 destination addresses select a source address from the corresponding PA/PI/6to4/64.
For PA/PI/6to4/48 destination addresses select a source address from the corresponding PA/PI/6to4/48.
For 6to4 destination addresses not already handled select a 6to4 address if available then a PA/PI source address and ULA address last.
For 2000::/2 destination addresses not already handled select a PA/PI source address then 6to4 addres and ULA address last.
For ULA destination addresses not already handled select a PA/PI source address then 6to4 addres and ULA address last.

Now is that really so hard?

I'm not sure where the IETF is with revising the default address
selection rules but ULA came out after the first set of rules was
published so it needs to be taken into account if it hasn't already
been.

If you are merging two sites you just extend the ULA of one to cover
the other as well then slowly deprecate the other or tweak the rules
above and distribute them via DHCP.

> >>> I actually override the current default IPv6 address rules. Here's
> >>> my /etc/gai.conf, which makes ULAs override GUAs as that currently
> >>> isn't in the default address selection rules, and makes tunnelled =
> IPv6
> >>> preferred over native IPv4, as I don't currently have native IPv6. =
> The
> >>> MRS entries are the non-defaults, the rest are from the gai.conf =
> manual
> >>> page.
> >>>=20
> >> You do this for your residential customers? It's fun to watch how =
> this
> >> discussion gets steered back to enterprise for places where it works
> >> better for you as an enterprise, but, residential customers are =
> suddenly
> >> the topic when I give an answer that solves the enterprise problem =
> but
> >> may not work for residential.
> >>=20
> >=20
> > The problem is you never seem to qualify your statements by saying
> > "For enterprises, ". You make broad statements without qualification,
> > which can only be interpreted to mean that you think they apply to all
> > IPv6 situations. I know of situations, which I think are relevant
> > to this mailing list, where they don't, which is why I point them out.
> >=20
> Fair point. However, so do others in this discussion. I've attempted
> to align my statements with the users being discussed in the
> earlier parts of the thread. Qualifying every statement with the
> covered userbase would be an impractical additional overhead
> for most people.
> 
> Owen
> 
> 
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: marka at isc.org