"Does TCP Need an Overhaul?" (internetevolution, via slashdot)
toasty at dragondata.com
Sat Apr 5 10:34:30 UTC 2008
On Apr 4, 2008, at 8:51 PM, Paul Vixie wrote:
> What is really necessary is to detect just the flows that need to
> down, and selectively discard just one packet at the right time, but
> not more, per TCP cycle. Discarding too many will cause a flow to
> stall -- we see this when Web access takes forever.
> i wouldn't want to get in an argument with somebody who was smart
> and savvy
> enough to invent packet switching during the year i entered
> but, somebody told me once that keeping information on every flow
> was *not*
> "inexpensive." should somebody tell dr. roberts?
I don't claim to understand this area more than Dr. Roberts either,
but to paraphrase George Santayana:
"Those who do not understand SACK are condemned to re-implement
it." (or fight it)
Years ago I worked on a project that was supposed to allow significant
over-subscription of bandwidth to unmodified clients/servers by
tracking the full state of all TCP traffic going through it. When the
output interface started filling up, we'd do tricks like delaying the
packets and selectively dropping packets "fairly" so that any one flow
wasn't penalized too harshly unless it was monopolizing the bandwidth.
To sum up many months of work that went nowhere:
As long as you didn't drop more packets than SACK could handle
(generally 2 packets in-flight) dropping packets is pretty ineffective
at causing TCP to slow down. As long as the packets are making it
there quickly, and SACK retransmits happen fast enough to keep the
window full... You aren't slowing TCP down much. Of course if you're
intercepting TCP packets you could disable SACK, but that strikes me
as worse off than doing nothing.
If you are dropping enough packets to stall SACK, you're dropping a
lot of packets. With a somewhat standard 32K window and 1500 byte
packets, to lose 3 non-contiguous packets inside a 32K window you're
talking about 13% packet loss within one window. I would be very
annoyed if I were on a connection that did this regularly. You get
very little granularity when it comes to influencing a SACK flow - too
little loss and SACK handles it without skipping a beat. Too much loss
and you're severely affecting the connection.
You've also got fast retransmit, New Reno, BIC/CUBIC, as well as host
parameter caching to limit the affect of packet loss on recovery time.
I don't doubt that someone else could do a better job than I did in
this field, but I'd be really curious to know how much of an effect a
intermediary router can have on a TCP flow with SACK that doesn't
cause more packet loss than anyone would put up with for interactive
The biggest thing we learned was that end user perceived speed is
something completely different from flow speed. Prioritizing UDP/53
and TCP setup packets had a bigger impact than anything else we did,
from a user's perspective. If either of those got delayed/dropped,
pages would appear to stall while loading, and the delay between a
click and visible results could greatly increase. This annoyed users
far more than a slow download.
Mark UDP/53 and tcpflags(syn) packets as high priority. If you wanted
to get really fancy and can track TCP state, prioritize the first 2k
of client->server and server->client of HTTP to allow the request and
reply headers to pass uninterrupted. Those made our client happier
than anything else we did, at far far less cost.
More information about the NANOG