Wisdom of using 100.64/10 (RFC6598) space in an Amazon VPC deployment

Mon Feb 23 18:52:18 UTC 2015

Hi, Eric -

Bill already described the salient points. The "transition" space is 
meant to be used for cases where there are multiple stacked NATs, such 
as CGN with CPE-based NAT. In theory, if the NAT implementations support 
it, one could use it repeatedly by stacking NAT on top of NAT ad 
nauseum, but the wisdom of doing so is questionable. If one uses it like 
additional RFC1918 space then routing could become more difficult, 
specifically in the case where hosts (e.g. VPC servers) are numbered 
with it. This is true because, in theory, you don't need the transition 
space to be routed on the "internal" network which avoids having NAT 
devices hold conflicting routes etc. Even if the edge NAT devices don't 
currently see conflicting routes to 100.64/10, if that changes in the 
future then client hosts may find themselves unable to reach the VPC 
hosts at that time.

That being said, if you understand the risks that I described above, 
then it may work well for a "community of interest" type of 
inter-network that hosts non-global resources. From your description it 
sounds like that might be the situation you find yourself in. To be 
clear, it's not unwise to do so, but it does carry risk that needs to be 
evaluated (and documented).

Cheers,
-Benson

> William Herrin <mailto:bill at herrin.us>
> February 23, 2015 at 12:58 PM
>
> Hi Eric,
>
> The main risk is more or less as you summarized it. Customer has no
> firewall or originates the VPN directly from their firewall. Customer
> buys a non-hosting commodity Internet product that uses carrier NAT to
> conserve IP addresses. The customer's assigned address AND NETMASK
> combined overlap some of the hosts you're trying to publish to them.
>
>
>
> Mitigations for that risk:
>
> Can you insist that the customer originate connections from inside
> their firewall (on RFC1918 space)?
>
> Most service providers using 100.64/10 either permit customers to opt
> out (getting dynamic globally routable addresses) or offer customers
> the opportunity to purchase static global addresses for a nominal fee.
> Are you comfortable telling impacted customers that they have to do
> so?
>
>
> A secondary risk comes in to play where a customer may wish to
> interact with another service provider doing the same thing as you.
> That essentially lands you back in the same problem you're having now
> with RFC1918.
>
>
> One more question you should consider: what is the nature of your
> customer's networks? Big corps that tend to stretch through 10/8 won't
> let their users originate VPN connections in the first place. They
> also don't touch 100.64/10 except where someone is publishing a
> service like yours. Meanwhile, home and SOHO users who are at liberty
> to originate VPNs might currently hold a 100.64/10 address. But they
> just about never use the off-bit /16s in 10/8. By off-bit I mean the
> ones with 4 or 5 random 1-bits in the second octet.
>
>
> My opinion: The likelihood of collisions in 100.64/10 increases
> significantly if you use them on servers. I would confine my use to
> client machines and try to put servers providing service to multiple
> organizations on globally unique IPs. Confining 100.64/10 to client
> machines, you're unlikely to encounter a problem you can't readily
> solve.
>
> Regards,
> Bill Herrin
>
>
> Eric Germann <mailto:ekgermann at cctec.com>
> February 23, 2015 at 10:02 AM
> Currently engaged on a project where they’re building out a VPC 
> infrastructure for hosted applications.
>
> Users access apps in the VPC, not the other direction.
>
> The issue I'm trying to get around is the customers who need to 
> connect have multiple overlapping RFC1918 space (including overlapping 
> what was proposed for the VPC networks). Finding a hole that is big 
> enough and not in use by someone else is nearly impossible AND the 
> customers could go through mergers which make them renumber even more 
> in to overlapping 1918 space.
>
> Initially, I was looking at doing something like (example IP’s):
>
>
> Customer A (172.28.0.0/24) <—> NAT to 100.127.0.0/28 <——> VPN to DC 
> <——> NAT from 100.64.0.0/18 <——> VPC Space (was 172.28.0.0/24)
>
> Classic overlapping subnets on both ends with allocations out of 
> 100.64.0.0/10 to NAT in both directions. Each sees the other end in 
> 100.64 space, but the mappings can get tricky and hard to keep track 
> of (especially if you’re not a network engineer).
>
>
> In spitballing, the boat hasn’t sailed too far to say “Why not use 
> 100.64/10 in the VPC?”
>
> Then, the customer would be allocated a /28 or larger (depending on 
> needs) to NAT on their side and NAT it once. After that, no more NAT 
> for the VPC and it boils down to firewall rules. Their device needs to 
> NAT outbound before it fires it down the tunnel which pfSense and 
> ASA’s appear to be able to do.
>
> I prototyped this up over the weekend with multiple VPC’s in multiple 
> regions and it “appears” to work fine.
>
> From the operator community, what are the downsides?
>
> Customers are businesses on dedicated business services vs. consumer 
> cable modems (although there are a few on business class cable). 
> Others are on MPLS and I’m hashing that out.
>
> The only one I can see is if the customer has a service provider with 
> their external interface in 100.64 space. However, this approach would 
> have a more specific in that space so it should fire it down the 
> tunnel for their allocated customer block (/28) vs. their external side.
>
> Thoughts and thanks in advance.
>
> Eric
>
>