TCP time_wait and port exhaustion for servers

Thu Dec 6 00:49:09 UTC 2012

In message <201212052325.qB5NPrZe005631 at xs8.xs4all.nl>, "Miquel van Smoorenburg"
 writes:
> In article <xs4all.20121205220127.7F6F12CA0F17 at drugs.dv.isc.org> you write:
> >
> >In message <CAP-guGW6oXo=UfTfg+SDiFjB4=qxPShO+YfK6vxnLkCC58PvgQ at mail.gmail.co
> m>,
> > William Herrin writes:
> >> The thing is, Linux doesn't behave quite that way.
> >> 
> >> If you do an anonymous connect(), that is you socket() and then
> >> connect() without a bind() in the middle, then the limit applies *per
> >> destination IP:port pair*. So, you should be able to do 30,000
> >> connections to 192.168.1.1 port 80, another 30,000 connections to
> >> 192.168.1.2 port 80, and so on.
> >
> >The socket api is missing a bind + connect call which restricts the
> >source address when making the connect.  This is needed when you
> >are required to use a fixed source address.
> 
> William was talking about the destination address. Linux (and I would
> hope any other network stack) can really open a million connections
> from one source address, as long as it's not to one destination address
> but to lots of different ones. It's not the (srcip,srcport) tuple that
> needs to be unique; it's the (srcip,srcport,dstip,dstport) tuple.
> 
> Anyway, you can actually bind to a source address and still have a
> dynamic source port; just use port 0. Lots of tools do this.
> 
> (for example, strace nc -s 127.0.0.2 127.0.0.1 22 and see what it does)
> 
> Mike.

Eventually the bind call fails.  Below was a 

counter: dest address in hex

16376: 1a003ff9
16377: 1a003ffa
bind: before bind: Can't assign requested address
16378: 1a003ffb
connect: Can't assign requested address
bind: before bind: Can't assign requested address

and if you remove the bind() the connect fails

16378: 1a003ffb
16379: 1a003ffc
connect: Can't assign requested address
16380: 1a003ffd

this is with a simple loop

	socket()
	ioctl(FIONBIO)
	bind(addr++:80)
	connect()

I had a firewall dropping the connection attempts

-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: marka at isc.org