Subject: Re: [tcpm] some comments on draft-gomez-tcpm-ack-rate-request-02.

From: "Carles Gomez Montenegro" <[email-masked]>
Date: Mon, 21 Mar 2022 18:56:50 +0100

Hi Jonathan,

Many thanks for your message, and apologies for such a long delay in my
response.

Please find below my inline comments:

>>>    For example, one of my naive questions is how often we want to
>>> change the frequency of ACKs.
>>>    In 'small' or 'large' scenarios in the draft, it seems to me that it
>>> might be sufficient to use the same ACK frequency during the
>>> connection.
>>>    It would be good if the draft describes the needs of changing
>>> frequency.
>>
>> Agreed, we will add some guidance in this regard.
>>
>> Yes, in many scenarios it may make sense to just set the ACK frequency
>> once during the connection.
>>
>> We'll think further about whether there could be particular scenarios
>> where the ACK frequency might need to be changed during the connection
>> lifetime.
>
> It seems to me that the ack frequency could depend mostly on the cwnd
> value used by the sender, which can change over the connection's lifetime
> for many reasons, and is generally not directly known by the receiver
> (though it might be inferred by estimated RTT, segment size, and observed
> delivery rate).
>
> In particular, cwnd will start at a low value and grow rapidly during the
> slow-start phase, then settle into a reasonably consistent range for the
> congestion-avoidance phase - assuming the underlying BDP remains constant.
>  Shifts in routing, or the addition or removal of significant load from
> the path the flow shares, may change the underlying BDP significantly; the
> cwnd should be expected to change accordingly, and this may prompt a
> change in ack rate.
>
> So we should expect some changes to the ack rate to be requested over the
> lifetime of the connection, but very frequent updates seem unlikely if the
> ack rate is derived from the cwnd.  This does not seem difficult to me.
>
> Conversely, the ack frequency may be controlled by "ack congestion
> control", alone or in combination with a derivation from the cwnd.  That
> is, the sender may notice that acks it receives cover more segments than
> the ack rate requested, indicating ack decimation en route, and decide to
> reduce the ack frequency to reduce network load up to the ack decimation
> point.  Future updates may also permit CE marks to appear on pure acks.
> This might involve more frequent ack rate updates, perhaps once an RTT, as
> the algorithm adjusts and probes around an operating point.

Thank you very much for providing such detailed explanations.

In our last update [1], we have added a new section (Section 5) that
describes reasons why the requested ACK rate might change over the
lifetime of a TCP connection. We have tried to capture your input above in
this new section.

>> Well, yes, the phrase "as long as packet reordering does not occur"
>> might
>> not be clear enough here. The intent was to say that one ACK will be
>> sent
>> every R full-sized received data segments, but if there is reordering,
>> then a different amount/rate of ACKs may be sent for a while.
>
> Isn't this sort of thing covered by existing Delayed Ack specifications?
> They are just cases in which an immediate ack is called for, overriding
> the Delayed Ack algorithm and resetting the counter and timer associated
> with it.  You should just be able to make a normative reference to that.

Agreed.

We added a reference to RFC 2581, which states the following:

  “A TCP receiver SHOULD send an immediate duplicate ACK when an out-
   of-order segment arrives.”

>>>   Also, since N looks finite number, I'm wondering how to set R=0 for
>>> the
>>> entire connection. Or, we shouldn't do such a thing?
>>
>> Good point!
>>
>> Perhaps we can define a special value for N meaning "infinity".
>
> An R value of 1 or 0 should produce a similar effect, should it not?  I
> think this falls into the category of choosing the ack frequency based on
> the sender's cwnd, which for the resource-constrained sender is a
> permanently small value.  I think it's important to describe with
> precision what the meanings of R=1 and R=0 are in any case; it's one of
> those pesky little corner cases.

Agreed.

Pending the discussion of how we represent the value of R in the TARR
option format, we now state in -03 that R=0 is not allowed. A TARR option
with R=1 requests immediate ACKs from the receiver.

> Conversely, the N field would be most useful for advanced congestion
> control algorithms which have a need to probe latency statistics at high
> fidelity, but only for a short period at a time.  We shouldn't use the N
> field to duplicate functionality that can adequately be described by the R
> field.

Agreed as well. After some consideration, we found that N actually
duplicates functionality, as you suggested. In consequence, in -03 we
removed the N field from the TARR option, and we understand that it is
possible to produce the same effect by just using the R field.

Cheers,

Carles

>  - Jonathan Morton

[1]
https://datatracker.ietf.org/doc/html/draft-gomez-tcpm-ack-rate-request-03

---

From: "Carles Gomez Montenegro" <[email-masked]>
Date: Mon, 21 Mar 2022 19:13:55 +0100

Hi Bob,

Many thanks for all your feedback, and apologies for such a long delay in
my response.

Please find below my inline responses:

> Carles,
>
> On 05/03/2021 12:34, Carles Gomez Montenegro wrote:
>> Hi Yoshi,
>>
>> Thanks a lot for your insightful comments!
>>
>> Also, sorry for the late response.
>>
>> Please find inline responses to your comments below:
>>
>>> Hi,
>>> I've read the draft. I like the basic concept of the draft and think
>>> it's
>>> useful in some ways. I put some comments belows.
>>>
>>> 1: In general, I would like to see some more example usages or
>>> guidances
>>> of
>>> the features described in the draft.
> [BB] Here's a couple of reasons why the DelACK ratio would change during
> the lifetime of a flow.
>
> 1/ I am interested in being able to use a facility like TARR to suppress
> delACKs in order to measure the RTT of each packet, particularly during
> flow start-up, but then release that constraint once the flow has started.
>
> A Linux receiver has a heuristic to detect slow start and suppress
> DelACKs just for that period.Â  But variants of slow-start at the sender
> such as hystart, hystart++ etc. alter the ending of slow-start, so they
> can confuse the heuristics at the receiver. Similarly we've been working
> with a technique called paced chirping, which confuses the heuristics.
> Or put another way, if someone invents a better approach than slow
> start, but it confuses the heuristics at the receiver, it means that the
> receiver's heuristics have ossified the existing slow-start behaviour at
> the sender. In contrast, an explicit signal between the peers like TARR
> allows for evolution.
>
> 2/ In the high cwnd scenarios, the sender wants the DelACK ratio to
> still ensure a minimum number of ACKs per RTT. Obviously cwnd is not
> instantly large - it grows from the start of the flow onwards, and when
> the flow starts, the sender doesn't know what it will reach. Also, the
> window changes as other traffic comes and goes or the link capacity
> changes. So clearly the sender will need to keep updating the DelACK
> ratio.

These detailed explanations are very much appreciated. We have tried to
capture your input in (the new) Section 5.

> ==Units==
>
> This last example raises the question of whether the number of packets
> between ACKs is the correct unit to use for the DelACK ratio. If the
> sender is trying to ensure there are say 4 ACKs per RTT, you might
> naively think it would be better to tell the receiver that. I worked
> through this question of units here:
> https://github.com/quicwg/base-drafts/issues/1978#issuecomment-436553339
> pasted below
>
>>>> Packets? bytes? time? fraction of RTT?
>>>>
>>>> Think of this question in two stages:
>>>> Q1. what units the sender starts from
>>>> Q2. what units the message is in, which the sender has to derive from
>>>> Q1
>>>>
>>>> If the message was a fraction of 1 RTT, the receiver would have to
>>>> calulate the sender's RTT. It can do that, using ACKs of ACKs, but
>>>> that implies work and delay.
>>>>
>>>> In all cases that I could see in #1428
>>>> <https://github.com/quicwg/base-drafts/issues/1428>, the sender had
>>>> all the info to translate from one unit to another (it knows its own
>>>> SMSS, its window, its RTT). This leaves Q2, to which a good enough
>>>> answer is packets (which also compresses the message size).

Thanks a lot for this insight!

> more...
>
>>
>>> 2:  "a TARR-option-capable receiving TCP MUST modify its ACK rate to
>>> one
>>> ACK every R full-sized received data segments from the sender, as long
>>> as
>>> packet reordering does not occur"
>>>      -> Do this mean if recorder happens, TARR option will be ignored?
>>>          Let's say if you send packet 1 with TARR and packet 2 without
>>> TARR,
>>> if packet 2 arrives earlier than packet1, the TARR option in packet1
>>> will
>>> be ignored?
>>>          How about storing the highest seqno that carries TARR and
>>> compare
>>> it when you receive packets with TARR?
>> Anyway, your suggestion is very useful, since the behavior in a scenario
>> like the one in your example needs to be clear as well.
>>
>> Therefore, we take two action points from this comment: clarifying the
>> text, and also incorporating your suggestion.
>>
>>> 3:  "If all bits of this field are set to 0, the field indicates a
>>> request
>>> by the sender for the receiver to trigger one or more ACKs immediately.
>>> "
>>>      -> For me, this might be read as if it can request to send back
>>> multiple ACKs immediately. How about changing it 'without delay' or
>>> something?
>> Agreed, we will edit the text.
>>
>>> 4:  "Otherwise, the field carries the binary encoding of the number of
>>> full-sized segments"
>>>      -> I'm not sure about the meaning of binary encoding here. does it
>>> mean storing 1-255 values or else?
>> Yes, that is the intended meaning (well, for 1-127 values with the new
>> field size).
>>
>> The original intent was to be clear on how to represent the value, but
>> is
>> perhaps "binary encoding" misleading?
>>
>>> 5: I have some concerns on ignoring reorder.
>>>      In case of QUIC, even if reorder is ignored, timer-based loss
>>> detection
>>> scheme can still find packet losses.
>>>      However, in TCP, RACK might not be always supported, so it might
>>> take
>>> significant amount of time to find losses if this feature is enabled.
>>>      I think the doc needs to discuss this point and it would be good
>>> to
>>> have some guidelines here.
>> Agreed!
>
> [BB] It wasn't clear to me what problem was intended to be addressed by
> the 'ignore reordering' flag.
> If it was that the sender is using time-based detection, so it tells the
> receiver it doesn't need ACKs of out of order packets, that would make
> sense. But perhaps you had a different use-case in mind?

The Ignore Order feature was suggested earlier to us, and we thought that
it might make sense for the use case that you mention. However,
considering the  feedback regarding this feature, we plan to ask in the
tcpm meeting whether this feature is actually considered useful enough.

>>> 6: Similar to 1:, I am a bit wondering about the use case of N field in
>>> TARR option.
>>>     I can imagine this would be useful if an endpoint uses Hystart++ or
>>> similar approach.
>>>     However, is it still useful for other cases such as plain
>>> slow-start
>>> algorithm?
>>>     I think it would be better to have some usage guidelines for N as
>>> well.
>> Agreed. We will add this guidance.
>
> [BB] It seemed to me that the ability to suppress ACKs for N packets
> would be redundant. Because the sender can send a different TARR option
> after N packets to ask the receiver to change the DelACK ratio. This
> would seem more useful in 2 respects:
> 1) It doesn't consume any option space
> 2) Typically the sender doesn't know exactly when its need for quick
> ACKs will end, until it does end.

Agreed.

In -03, we have removed the N field.

>> There are also considerations similar to the earlier question on whether
>> the expected behavior would be the same for the whole connection
>> lifetime.
>> Perhaps, in some cases, setting N to "infinity" might represent an
>> improvement (since the option would only be sent once). However, if this
>> can represent any potential issue, a different setting should be chosen
>> for N.
>>
>> Let's provide guidance on this.
>
> [BB] I have some other comments on the draft, but I'll send a separate
> review email for those that don't relate directly to Yoshi's thread.

Once again, many thanks!

Cheers,

Carles

> Cheers
>
>
>
> Bob
>
>>
>> We will incorporate your comments in our next draft update.
>>
>> Thanks!
>>
>> Cheers,
>>
>> Carles
>>
>>
>>
>>> Thanks,
>>> --
>>> Yoshi
>>> _______________________________________________
>>> tcpm mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/tcpm
>> _______________________________________________
>> tcpm mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/tcpm
>
> --
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/
>
>