Carrier Grade NAT

Tue Jul 29 23:23:27 UTC 2014

>>3. 99.99% of customers don't notice they are transiting CGNAT, it just 
>>works.

Surprised it's that high.

So was I to be honest, but in general "It Just Works".

>>4. You need to log NAT translations for LI purposes. (IP 
>>source/destination, Port source/destination, time) Surprisingly this 
>>does not produce that big a database burden. However as Cisco's Netflow 
>>NAT logging is utterly useless you need to use syslog and this ramps up 
>>the ASR CPU a bit.
>
>Can you quantify?
>The log entry has to be at least:
>32 bits	source address
>16 bits source port
>32 bits destination address
>16 bits destination port
>64 bits? timestamp

The issue with the Cisco NAT Translation flow is that as soon as you set the
nat mode to CGN it no longer sends the Pre Nat IP (100.64.x.x), which makes
it useless for matching against radius to identify the user. Several weeks
of arguing with TAC engineers got nowhere. TAC said, no that can't be done,
but could not explain why it worked fine with syslog translation logging.

---
160 bits = 20 bytes per flow
You have to log the end of the flow, too, right?  Another 20 bytes?
40 bytes per flow.  Not including syslog severity and message text.

As I recall, a site like cnn.com opens 80 flows, so 3200 bytes of log data.
If, as you say in #6, 10,000 customers = 200,000 active translations, that's
8,000,000 bytes of syslog. . . per second?  Not sure if "active"
indicates how fast those sessions churn.
180 days of log retention would be. . . 124TB of data.  Per 10,000 users.

That is 200,000 active translations, not 200,000 per second. The ESP40 can
handle 2,000,000 active translations. 

>By the way, if that's 8MB of syslog, that's 32Mbps just of logging data.
>Average, not peak.
>
>Maybe the actual log rate is 8MB per five minutes?  That's only 400GB for
six months.
>
>I'm really interested in what your actual log rate is.

Per 10,000 customers we are getting about 2,000,000 records per day in the
database real world. We first in first out these after three months. How
much bandwidth ? Don't know, I have not actually looked.

>>5. NAT translation timeouts are important, XBOX and PlayStation suck.
>
>At least Xbox ONE prefers IPv6.
>PS4 can, it just doesn't yet.
>Maybe Kiwis don't play enough games for Sony to care?

Few CPE routers support native v6 (we are a low cost, BYO router ISP)

>>7. CGNAT protects your customers from all sorts of nasty's like small 
>>DDOS attacks and attacks on their crappy CPE 8. DDOS on CGNAT pool IP's 
>>are a pain in the rear and happen often.
>
>Between #7 and #8, do they balance out?

Yes, you just need to treat DDOS mitigation a little differently, you can't
just upstream block your destination ip as that can randomly nuke thousands
of customer translations. You need to remove the target IP from your CGANT
pool first. 

>>9. In New Zealand we are not a state of the USA so spammed DCMA emails 
>>can be redirected to /dev/null. If a rights holder wishes to have a 
>>potential violation investigated (translation logs) they need to pay a 
>>$25 fee, so in general they don't bother. Police need a search warrant 
>>so they generally only ask for user info when they actually can justify 
>>it, so it's not a big overhead.
>
>As long as you have a tool to query your logging system, should be fine.

Yes, it doesn't take a lot to develop the tool. Most of the work is in
educating the authorities that they need to supply the exact
source/destination ip, destination port and timestamps if they want any data
back .

>>10. It is not uncommon for people who run some game servers and 
>>websites (like banks) to be completely clueless/confused about cgnat 
>>and randomly block IP's as large numbers of users connect from  single 
>>IP. This is not a big issue in practice.
>
>Really?  Seems like those would be some of the loudest users.
>
>I've always suggested adding IPv6 as an outlet, so that if someone
complains about something not working through CGN, you can tell them to
deploy IPv6.  

Yes, there are only been a few websites that have caused some issues over
the last two years, nowhere near as bad as I expected it to be.

>Thanks again for this perspective.
>
>Lee

Happy to help. People tend to panic about the unknown. And in this case it's
really not as scary as people think, in general it just works and pretty
much no standard residential customers notice.