"Does TCP Need an Overhaul?" (internetevolution, via slashdot)

Kevin Day toasty at dragondata.com
Sat Apr 5 10:34:30 UTC 2008

On Apr 4, 2008, at 8:51 PM, Paul Vixie wrote:
> 	What is really necessary is to detect just the flows that need to  
> slow
> 	down, and selectively discard just one packet at the right time, but
> 	not more, per TCP cycle. Discarding too many will cause a flow to
> 	stall -- we see this when Web access takes forever.
> 	...
> i wouldn't want to get in an argument with somebody who was smart  
> and savvy
> enough to invent packet switching during the year i entered  
> kindergarden,
> but, somebody told me once that keeping information on every flow  
> was *not*
> "inexpensive."  should somebody tell dr. roberts?

I don't claim to understand this area more than Dr. Roberts either,  
but to paraphrase George Santayana:

"Those who do not understand SACK are condemned to re-implement  
it." (or fight it)

Years ago I worked on a project that was supposed to allow significant  
over-subscription of bandwidth to unmodified clients/servers by  
tracking the full state of all TCP traffic going through it. When the  
output interface started filling up, we'd do tricks like delaying the  
packets and selectively dropping packets "fairly" so that any one flow  
wasn't penalized too harshly unless it was monopolizing the bandwidth.  
To sum up many months of work that went nowhere:

As long as you didn't drop more packets than SACK could handle  
(generally 2 packets in-flight) dropping packets is pretty ineffective  
at causing TCP to slow down. As long as the packets are making it  
there quickly, and SACK retransmits happen fast enough to keep the  
window full... You aren't slowing TCP down much. Of course if you're  
intercepting TCP packets you could disable SACK, but that strikes me  
as worse off than doing nothing.

If you are dropping enough packets to stall SACK, you're dropping a  
lot of packets. With a somewhat standard 32K window and 1500 byte  
packets, to lose 3 non-contiguous packets inside a 32K window you're  
talking about 13% packet loss within one window. I would be very  
annoyed if I were on a connection that did this regularly. You get  
very little granularity when it comes to influencing a SACK flow - too  
little loss and SACK handles it without skipping a beat. Too much loss  
and you're severely affecting the connection.

You've also got fast retransmit, New Reno, BIC/CUBIC, as well as host  
parameter caching to limit the affect of packet loss on recovery time.  
I don't doubt that someone else could do a better job than I did in  
this field, but I'd be really curious to know how much of an effect a  
intermediary router can have on a TCP flow with SACK that doesn't  
cause more packet loss than anyone would put up with for interactive  

The biggest thing we learned was that end user perceived speed is  
something completely different from flow speed. Prioritizing UDP/53  
and TCP setup packets had a bigger impact than anything else we did,  
from a user's perspective. If either of those got delayed/dropped,  
pages would appear to stall while loading, and the delay between a  
click and visible results could greatly increase. This annoyed users  
far more than a slow download.

Mark UDP/53 and tcpflags(syn) packets as high priority. If you wanted  
to get really fancy and can track TCP state, prioritize the first 2k  
of client->server and server->client of HTTP to allow the request and  
reply headers to pass uninterrupted. Those made our client happier  
than anything else we did, at far far less cost.

-- Kevin

More information about the NANOG mailing list