Carrier Grade NAT

Tue Jul 29 22:19:31 UTC 2014

Thanks for sharing your experience; it's very unusual to get the
perspective of an operator running CGN (on a broadband ISP; wireless has
always had it).

On 7/29/14 5:28 PM, "Tony Wicks" <tony at wicks.co.nz> wrote:

>OK, as someone with experience running CGNAT to fixed broadband customers
>in
>general, here are a few answers to common questions. This is based on the
>setup I use which is CGNAT is done on the BNG (Cisco ASR1K6).
>
>1. APNIC ran out of IPv4 a couple of years ago, so unless you want to pay
>USD $10+ per IP then CGNAT is the only option.

Eh, a bit over US$7 now, but whatever. Higher in APNIC.

>2. IPv6 is nice (dual stack) but the internet without IPv4 is not a viable
>thing, perhaps one day, but certainly not today (I really hate clueless
>people who shout to the hills that IPv6 is the "solution" for today's
>internet access)

It's viable, it's just not a substitute for IPv4 yet.
Except for specific scenarios.  For instance, you mention gaming below; if
two users are playing on Xbox ONE, they can use IPv6 and they're off the
CGN.  Or if a bank has blacklisted an IPv4 address on the CGN, but the
bank is dual-stack, some users can still get there.
Of course, that snowballs.

>3. 99.99% of customers don't notice they are transiting CGNAT, it just
>works.

Surprised it's that high.

>4. You need to log NAT translations for LI purposes. (IP
>source/destination,
>Port source/destination, time) Surprisingly this does not produce that
>big a
>database burden. However as Cisco's Netflow NAT logging is utterly useless
>you need to use syslog and this ramps up the ASR CPU a bit.

Can you quantify?
The log entry has to be at least:
32 bits	source address
16 bits source port
32 bits destination address
16 bits destination port
64 bits? timestamp
---
160 bits = 20 bytes per flow
You have to log the end of the flow, too, right?  Another 20 bytes?
40 bytes per flow.  Not including syslog severity and message text.

As I recall, a site like cnn.com opens 80 flows, so 3200 bytes of log data.
If, as you say in #6, 10,000 customers = 200,000 active translations,
that's 8,000,000 bytes of syslog. . . per second?  Not sure if "active"
indicates how fast those sessions churn.
180 days of log retention would be. . . 124TB of data.  Per 10,000 users.

By the way, if that's 8MB of syslog, that's 32Mbps just of logging data.
Average, not peak.

Maybe the actual log rate is 8MB per five minutes?  That's only 400GB for
six months.

I'm really interested in what your actual log rate is.

>5. NAT translation timeouts are important, XBOX and PlayStation suck.

At least Xbox ONE prefers IPv6.
PS4 can, it just doesn't yet.
Maybe Kiwis don't play enough games for Sony to care?

>6. 10,000 customers= approximately  200,000 active translations and 1-2
>/24's to be comfortable

So you've cut your address expense to US$0.50 per user.  Definitely better.
(500*$10/10000)

>7. CGNAT protects your customers from all sorts of nasty's like small DDOS
>attacks and attacks on their crappy CPE
>8. DDOS on CGNAT pool IP's are a pain in the rear and happen often.

Between #7 and #8, do they balance out?

>9. In New Zealand we are not a state of the USA so spammed DCMA emails can
>be redirected to /dev/null. If a rights holder wishes to have a potential
>violation investigated (translation logs) they need to pay a $25 fee, so
>in
>general they don't bother. Police need a search warrant so they generally
>only ask for user info when they actually can justify it, so it's not a
>big
>overhead.

As long as you have a tool to query your logging system, should be fine.

>10. It is not uncommon for people who run some game servers and websites
>(like banks) to be completely clueless/confused about cgnat and randomly
>block IP's as large numbers of users connect from  single IP. This is not
>a
>big issue in practice.

Really?  Seems like those would be some of the loudest users.

I've always suggested adding IPv6 as an outlet, so that if someone
complains about something not working through CGN, you can tell them to
deploy IPv6.  

Thanks again for this perspective.

Lee