Deploying IPv6 in a datacenter (Was: Awful quiet?)

Wed Dec 21 17:54:03 UTC 2005

On Dec 21, 2005, at 10:13 AM, Kevin Loch wrote:

>
> Kevin Day wrote:
>
>> 9) Once we started publishing AAAA records for a few sites, we  
>> started getting complaints from some users that they couldn't  
>> reach the sites.
>
> It is possible that a broken 6to4 relay somewhere was causing  
> problems.
> Running your own local 6to4 relay (rfc3068) will improve  
> performance and
> reduce the chances of going through a broken one.
>
> FWIW, I have been running some AAAA records for almost a year on some
> revenue generating sites without any reachability complaints or
> drop in traffic.   I do run a local 6to4 relay though.

I will admit, the number of complaints was very small. But, I don't  
know how many it was affecting who didn't contact us, so it was  
decided it wasn't worth the risk with no perceived gain at this point.

>> I know this is still a hot topic and several proposals are being  
>> passed around to resolve some of these issues, but it seems like I  
>> *lose* functionality with IPv6 that I have with IPv4, mostly due  
>> to the "don't deaggregate your allocation" mantra, and how far the  
>> bar was raised to get PI space.
>
> It sounds like you are an existing ISP in the ARIN reigon. If so  
> then you qualify for a /32 allocation.  Let us know if you have any  
> problems getting one.  For non-ISP's, the policy is still being  
> worked out.  See the ARIN ppml list for discussion.  As for  
> deaggregation, it is not
> recommended because some filter (/48's) and some don't which results
> in sub-optimal paths, but it can be done depending on what your
> peers will accept.

Correct, we're in the ARIN region. We did qualify for a /32, but just  
barely, and only because of (hopeful) future growth and some new  
products we're launching. We do managed hosting for a small number of  
specialized clients, and act sort of as a content delivery network.

Not long ago we were a little smaller, and qualified for a /20 of our  
own in IPv4. /20 = 4096 addresses. We were able to pretty easily  
qualify for that between a few racks of servers, some IP based  
vhosting(necessary for a few applications), etc.  In the IPv6 world,  
nothing we could do under that business model would qualify us for a  
direct allocation.

Back then we wouldn't have qualified for a /32 because we wouldn't  
have met the requirements. We wouldn't have met the proposed 2005-1  
requirements for a /44 (we don't come close to 100,000 devices), and  
lose functionality if we're required to advertise it through a single  
aggregated address.

Just for the sake of argument though, let's say we didn't get a /32  
and had to either get provider assigned space, or a /44 through the  
2005-1 proposal.

1) We've got separate POPs in different cities, with no dedicated  
connectivity between them. They act as entirely independent networks.  
We don't even have the same transit providers in each city. Some  
content is replicated to each POP, so we can dump content directly on  
peers where they want it, or for redundancy. Some content is only  
available on one POP (dynamic sites, etc), so traffic destined for  
that POP can only go to that POP.

IPv4: We split our /20 into a /23 for each city, and announce only that.

IPv6: We have to announce the entire /44, and aren't allowed to  
deaggregate it. Where do we announce it? If I use transit provider Y  
only in city #1, and I announce to them our /44 even though I only  
really want a /48 worth coming there, I'm going to end up receiving  
traffic for city #2 at city #1. I can't even bounce it back off my  
router onto transit again, because I might see provider Y as the best  
path for it and it'll loop.  Don't say we should ask for a /44 in  
each city, we don't need that much space in each location, and we're  
allocating space only to avoid deaggregating it, which is even more  
wasteful. :)

2) We use anycast to select the closest server on our network to a  
viewer, anycasted DNS for quicker lookups, and a few other anycast  
services.

IPv4: We have a /24 dedicated for anycast, and announce that block in  
each city.

IPv6: ??? I honestly have no idea how to do this with IPv6, and still  
announce only a single block, without effectively anycasting  
everything we do.

3) We're far past the size where we can renumber every time we change  
transit providers.

IPv4: We were able to qualify for PI allocations as small as a /22,  
because we were multihomed. a /20 if we weren't. This is within the  
range of feasibility of any small/medium hosting company to reach in  
a very short time.

IPv6: Even forgetting multihoming issues, we're beyond the size where  
we can do a renumbering without it becoming a serious ordeal. If  
we're forced to take provider assigned space, we're locked into that  
transit provider unless we want to leave them so badly that we're  
willing to go through the hassle. I can't even easily transition from  
one provider to another by having the new provider announce my PA  
space from my old provider temporarily, since my provider might be  
forced to announce everything as a single aggregate as well.

The allocation and advertisement policies work great for small end- 
sites (they get many subnets, and more addresses than they'll ever  
need), and great for large providers (they have no problem justifying  
the need for a /32, and are able to break things up if needed). If  
you fit somewhere between those two though, I don't think the  
policies really address our needs.

Small to medium sized hosting providers, content networks and other  
non-bandwidth-supplying companies generally aren't providing transit  
to others, so they can't get a /32. They ARE used to being able to  
deaggregate when necessary though, anycast as needed, multihome  
easily, and not go through renumbering hell every time you change  
providers. In the case of hosting companies, a renumbering is  
especially painful when you have to coordinate renumbering with  
thousands of customers who are going to see it as a hassle with no  
benefit.

I completely understand the need to reduce table growth, slow ASN  
allocations and not waste IPv6 space. But, (and I honestly don't mean  
this as an attack or flamebait, just a different perspective) it  
feels like a lot of the big providers are saying "We can't keep up  
with table growth, we need policies to stop this." Totally  
understandable. Policies got written to slow the table growth and ASN  
allocation, but for the most part they're painless for the big  
providers(easy allocations, and no loss of functionality compared to  
IPv4), and all the concessions are being made by those on the small- 
medium side, with very little benefit shown to them.

I don't mean to sound so resistant to change, but IPv6 is really a  
step backwards for people in our position if we're giving up so much  
just to get a bigger block of addresses.

-- Kevin

<waits for flame war to erupt> :)