Subject: Re: [Tsvwg] new I-D for Smart Fragmentation of TCP segment

From: "Claudio E. Casetti" <[email-masked]>
Date: Tue, 20 Nov 2001 11:52:06 +0100

Joe,
    thanks for biting :)

>> This is somewhat related to the Increasing TCP's Initial Window issue
>> and to the ABC scheme. 

>Related to or made redundent by?

We do not think there's much redundancy in these schemes, they simply address
different issues. Let's go over them:

* TCP-SF vs. TCPIW

- TCP-SF's main goal is to enable Fast Recovery (FR) even in the small window 
  regime, and to avoid the long first-window RTO, without increasing the amount 
  of application data sent into the network. As a secondary goals, it 
  eliminates the delayed-ACK timeout for the first segment and speeds up the 
  convergence of the RTT estimate.

  If we look at the TCPIW proposal, we see that it increases the initial
  congestion window and, with it, the amount of application data sent into the
  network; this results in increased source burstiness, and it raises issues
  of aggressiveness toward 'Classic TCP' sources. Instead, TCP-SF achieves the
  same level of aggressiveness as 'Classic TCP'. 

- Moreover, while TCPIW allows the increase of only the Initial and Restart 
  Window, TCP-SF consider every beginning of slow start in an identical way 
  (i.e., including the Loss Window).

- Another key feature of TCP-SF is that it is better suited to the current TCP 
  traffic (HTTP...), i.e., a collection of small file transfers. Since today's
  most common SMSS is about 1460 bytes, most TCP connections 
  are completed in very few segment transmissions... Simply allowing a large 
  window (as in TCPIW) will therefore cut the completion time for HTTP traffic, 
  but in case  of segment loss, doesn't improve the completion time, as simply 
  there won't be enough segments to trigger FR.

- We also believe that TCP-SF and TCPIW can work coupled; here's a possible
  implementation: while in the first slow start CWND = 4SMMS, then no small 
  segment option (as suggested by TCP-SF) is needed; in case of RTO, being 
  the restart window set to 1, the small size segments option can be turned 
  on and allow the sender to eventually trigger FR...
  This solution, however, doesn't allow Short Lived flows to benefit from 
  the effect of TCP-SF...

* TCP-SF vs. ABC

- It seems to us that ABC aims at removing the unfairness due to the 
  i) delayed ack policy eventually adopted by the receivers and 
  ii) cope with acks lost in the network. It doesn't interact with 
  the loss recovery (either RTO or FR) mechanisms.
  In our opinion, TCP-SF can greatly be simplified by an appropriate 
  implementation of the ABC policy, and indeed TCP-SF in its Variable Size 
  implementation needs to explicitly consider the number of acked bytes to 
  perfectly mimic a 'Classic TCP' sender...
  Thus we believe ABC and TCP-SF can easily coexist.

> And how does it interact with ACK compression and Nagle, in particular?

- About ACK compression: actually we don't see any new problem caused by 
  ACK compression to TCP-SF that does not affect any other TCP variants... 
  What do you think might be problem? Can you be more specific?

- Nagle's algorithm must indeed be taken into account in the TCP-SF 
  implementation and we plan to incorporate it.  At first glance, a 
  possible (simple) solution may go like this:
  1) if full size segments are used (i.e., cwnd>=4SMMS), then the standard Nagle's 
  algorithm is run.
  2) if smaller size segments are used (i,e., cwnd <4SMMS), then  Nagle's algorithm 
  MUST consider the CURRENT value of the segment size as its activation threshold. 
  Thus, rephrasing RFC 1122 it could boil down to the following:
    "The Nagle algorithm is generally as follows:
     If there is unacknowledged data, then the 
     sending TCP buffers all user data (regardless 
     of the PSH bit), until the outstanding data has been 
     acknowledged or until the TCP can send the CURRENT full-sized 
     segment."

  Is this a possible solution to the problem?

Thanks,
Claudio, Marco, Michela

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Joe Touch <[email-masked]>
Date: Tue, 20 Nov 2001 09:49:22 -0800

>   same level of aggressiveness as 'Classic TCP'. 

Agreed, with the exception that you are sending more headers. Some 
systems are dependent on header rate (e.g., when processing limited), 
rather than link bandwidth. And the headers also add overhead - 
splitting a 500 byte packet into 4 100 byte packets results in 30% 
overall network and transport overhead (add link to that and it grows to 
35%). Contrast that with 7% (or 10% with link) for Classic TCP.

However, my point is that TCPIW is going to proposed standard. That 
makes doing your version moot, IMO.

>   (i.e., including the Loss Window).

Can you elaborate? Are you doing this every RTT? It seems you'd want to 
do TCPIW exactly where you do this (and vice-versa)

>   there won't be enough segments to trigger FR.

If you're sending only a few segments, then the probability of a segment 
loss during your 'lifetime' should be very small, in which case, again, 
this technique is moot.

If you're losing 10% of your packets (which you'd have to to make that 
sort of impact), you have bigger problems than this will necessarily 
fix. However, I agree, when there is HIGH LOSS, sending more smaller 
packets is better (it avoids fate-sharing). It's not clear that's 
warranted, though, in general implementations.

>   This solution, however, doesn't allow Short Lived flows to benefit from 

This would be a very minor contribution if that's all that's left at the 
end of the day. Again, if RTO occurs, maybe you should be sending one 
segment, period.

>   Thus we believe ABC and TCP-SF can easily coexist.

OK (might be useful to add that to the draft)

>   What do you think might be problem? Can you be more specific?

You'll get more ACKs back; if you're not doing ABC, you might interpret 
the result as opening the window more quickly (see Savage's papers).

>      acknowledged or until the TCP can send the CURRENT full-sized 

How do you know whether the returned segments were sent during the 
TCP-SF phase, or due to lack of data at the sender? For Nagle, it 
matters, and the response differs. That seems not fully solved by the above.

Joe

---

From: "Claudio E. Casetti" <[email-masked]>
Date: Fri, 23 Nov 2001 15:36:10 +0100

On Tuesday 20 November 2001 18:49, Joe Touch wrote:
> > We do not think there's much redundancy in these schemes, they simply
> > address different issues. Let's go over them:
> >
> > * TCP-SF vs. TCPIW
> >
> > - TCP-SF's main goal is to enable Fast Recovery (FR) even in the small
> > window regime, and to avoid the long first-window RTO, without increasing
> > the amount of application data sent into the network. As a secondary
> > goals, it eliminates the delayed-ACK timeout for the first segment and
> > speeds up the convergence of the RTT estimate.
> >
> >   congestion window and, with it, the amount of application data sent
> > into the network; this results in increased source burstiness, and it
> > raises issues of aggressiveness toward 'Classic TCP' sources. Instead,
> > TCP-SF achieves the same level of aggressiveness as 'Classic TCP'.
>

We agree with you that this is a delicate issue, and also Sally Floyd pointed 
out this issue. However

- most of the current IP stacks use a 1500 MTU, which translate into about 
10% overhead for 375 small size MTU. This can be explicitely taken into 
account into the TCP-SF draft, which may be enabled only id SMMS > 
thresholds...

- Moreover, TCP-SF will switch back to full size MSS as soon as CWND>=4SMSS. 
In particular, considering the variable size version, at the beginning of 
each slow start, the number of segments used to send _exaclty_ the same 
payload is 4+4+4 instaed of 1+2+4 before switching back to full size SMMS... 
that is only 5 headers more than classic TCP. The fized size implementation 
will send 4+8+(12+1) segments...

- ... But notice that in case of segment drop, the retransmission of smaller 
segments is more efficient than having to resend a full-sized SMSS (again, 
this is a bit blunt put in this way, but in the rejected Infocom paper we 
analytically evaluated such gain). In a nutshell, the throughput gain of 
SF-TCP over Classic TCP in the small-window regime grows with the segment 
loss probability, thanks also to the smaller bandwidth waste due to 
retransmission.

- (ok, this is straighforward) smaller-sized segments exploit pipelined 
transmission especially in long-latency, slow links (such as modems)

> makes doing your version moot, IMO.

They can work coupled and coexist... Why ruling out proposing TCP-SF as a 
future enhancement?

> > - Moreover, while TCPIW allows the increase of only the Initial and
> > Restart Window, TCP-SF consider every beginning of slow start in an
> > identical way (i.e., including the Loss Window).

Let us rephrase this: TCPIW allows the Initial and Restart window to be equal 
to i.e. 4 SMMS. The Loss Window must be set to 1 SMMS. On the contrary, a 
TCP-SF sender must use small-sized segments EVERY time the current cwnd is 
smaller than 4SMMS, including then the Loss Window. This enables Fast 
Recovery for senders who experienced an RTO, alleviating the sender latency.

> > - Another key feature of TCP-SF is that it is better suited to the
> > current TCP traffic (HTTP...), i.e., a collection of small file
> > transfers. Since today's most common SMSS is about 1460 bytes, most TCP
> > connections
> >   are completed in very few segment transmissions... Simply allowing a
> > large window (as in TCPIW) will therefore cut the completion time for
> > HTTP traffic, but in case  of segment loss, doesn't improve the
> > completion time, as simply there won't be enough segments to trigger FR.
>

True, but for such short lived flows, waiting for 3-second RTO is a HUGE 
time... That's the main reason why TCP-SF outperforms 'Classic TCP' when the 
completion time is used as performance figure, even if the dropping 
probablility is small. We feel that avoiding this penalty is important, 
probably more than guaranteeing larger throughput to long-lived flows...

> warranted, though, in general implementations.

Actually, models and simulations show that TCP completion time already 
improves when the dropping probablity is 1 %, or if the bottleneck 
utilization is over 50% ...

> > - We also believe that TCP-SF and TCPIW can work coupled; here's a
> > possible implementation: while in the first slow start CWND = 4SMMS, then
> > no small segment option (as suggested by TCP-SF) is needed; in case of
> > RTO, being the restart window set to 1, the small size segments option
> > can be turned on and allow the sender to eventually trigger FR...

One segment or one SMSS-worth of data split into more than one segment? :-)

> >   implementation of the ABC policy, and indeed TCP-SF in its Variable
> > Size implementation needs to explicitly consider the number of acked
> > bytes to perfectly mimic a 'Classic TCP' sender...

Ok.

> >   ACK compression to TCP-SF that does not affect any other TCP
> > variants... What do you think might be problem? Can you be more specific?

Ok, but again, this is exactly the same problem all non-ABC-TCP versions face.
Actually, having ABC and TCP-SF coexist can also simplify the variable-size 
implementation.

> >   1) if full size segments are used (i.e., cwnd>=4SMMS), then the
> > standard Nagle's algorithm is run.
> >   2) if smaller size segments are used (i,e., cwnd <4SMMS), then  Nagle's
> > algorithm MUST consider the CURRENT value of the segment size as its
> > activation threshold. Thus, rephrasing RFC 1122 it could boil down to the
> > following: "The Nagle algorithm is generally as follows:

Correct us if we're wrong, but it doesn't seem to us that the sender needs to
have knowledge of WHEN the acked segments were sent. What Nagle really cares 
about is the amount of data buffered at the sender and when to send new segments.
New segments can be sent when all old data is acknowledged (in that case,
sending one not full-sized segment is allowed), or a new segment can be sent as
soon as the buffered data is larger or equal to the current full-sized segment,
provided the cwnd allows it (note that by "current full-sized" we mean a segment 
whose size is either SMSS, or a fraction of it as defined by TCP-SF).
Furthermore, when Nagle's algorithm is activated, congestion control becomes of
secondary importance, and the add-ons of Classic TCP are ineffective.
So, even keeping the standard Nagle's algorithm (and threshold) to SMSS does not
impact the effectiveness of TCP-SF.

Marco, Michela, Claudio

PS: we'll be presenting a paper on TCP-SF at Globecom next week. If someone
from this mailing list is at the conference and is interested in exchanging
their opinion, it would be most welcome.

---

From: Joe Touch <[email-masked]>
Date: Fri, 23 Nov 2001 10:19:40 -0800

> account into the TCP-SF draft, which may be enabled only id SMMS > 

IP's largest common MTU is 576. Microsoft uses 576 for PPP (the vast 
majority) of connections.

Back in 1997, there were stats posted at NLANR that 85% of the packets 
were 576 or smaller. http://www.nlanr.net/NA/Learn/packetsizes.html

Newer measurements from CAIDA reinforce this point 
http://www.caida.org/analysis/AIX/plen_hist/.

If you have other data to back up an alternate view, please let us know.

> will send 4+8+(12+1) segments...

This should be in the draft, at least.

> loss probability, thanks also to the smaller bandwidth waste due to 

This assumes that the loss is helped by using smaller retransmission 
packets. If queues are RED per-packet or tail drop, this will not matter.

> transmission especially in long-latency, slow links (such as modems)

But they also increase overhead. Since these links are from the host, 
packets aren't dropped entering the links, which means the packet size 
is irrelevant unless the link is so noisy that's where the drops are (in 
which case you have much larger problems), or you're running a true 
router at home.

> They can work coupled and coexist... Why ruling out proposing TCP-SF as a 

Unless there is a substantive need, TCP (and most protocols) are better 
off when simple, rather than having every possible 'feature'. While 
TCP-SF might be interesting for future research, there isn't a need, 
esp. in light of TCPIW.

> Recovery for senders who experienced an RTO, alleviating the sender latency.

That could be perceived as violating the Loss Window intent - to inject 
only ONE packet into the network. TCPSF's loss window should be one 
small packet, in that case.

> probably more than guaranteeing larger throughput to long-lived flows...

Do you have stats on how many of these short lived connections suffer 
RTO timeouts, and the quantitative improvement of sending small packets 
on that probability?

> utilization is over 50% ...

1% is fairly high too. I'm not clear what the bottleneck utilization 
here refers to - how many flows were aggregated over the bottleneck in 
the simulation?

> One segment or one SMSS-worth of data split into more than one segment? :-)

That's a good question - as above, it depends on whether you're packet 
or bandwidth limited. With tail-drop or packet-drop RED, you SHOULD be 
sending one segment. If you can ensure E2E byte-drop RED, then it's not 
clear which is better. However, you SHOULD be sending 1 SMSS total - 
i.e., taking 1000 bytes + 40 byte header into 4 packets SHOULD result in 
a total of 1040 bytes, i.e., 250+40, 250+40, 250+40, and 130+40 
(actually, you SHOULD subtract off your link headers as well). You 
SHOULD NOT be sending more data into the network due to the headers.

> Actually, having ABC and TCP-SF coexist can also simplify the variable-size 

I'm not suggesting that ABC reacts more poorly non-ABC would react to 
TCP-SF. The ACK compression issue isn't related to ABC, as I proposed it.

> impact the effectiveness of TCP-SF.

If this is true, then it would need to be validated and incorporated 
into the ID. I suspect that there will be problems when TCP-SF 
fractioning is activated during a pending-small-segment episode.

Joe