Subject: Re: [Tsvwg] draft-ietf-tsvwg-sctpimpguide-03.txt: Questions

From: [email-masked]
Date: Wed, 30 Jan 2002 13:21:18 -0500

Hello Sctp,

In sctpimpguide chapter *2.8 Issues with Fast Retransmit* in *New text: 
(Section 7.2.4)* paragraph 5) it reads:

"...However,  as they are marked for retransmission they will be 
retransmitted
      later on as soon as cwnd allows."

This refers to the TSN's marked for fast retransmission, but could not 
been fast retransmitted due to 
restriction in paragraph 3 (only one single packet allowed).

Question: Do we wait for T3 for retransmission, or do we *delay* the fast 
retransmission until consecutive SACKs
will open the CWND. We feel that the latter is applicable and the impguide 
needs clarification. In our implementation,
we do not reset the TSN.Missing.Report counter, if we can not FR 
immediatly, the next SACK which opens CWND,
will kick off the *delayed fast retransmission* for those TSNs.

paragraph 3) says, the cwnd SHOULD be ignored for fast retransmission. 
What about rwnd? If we receive a SACK
which kicks off fast retransmission, where the SACK advertises rwnd=0, we 
are probably blocked. The impguide
says 1 single packet according to PMTU is allowed, cwnd SHOULD be ignored, 
but does not mention rwnd.

The impguide defines max burst rules for FR. Why is cwnd/rwnd not 
sufficient? What makes a SACK acknowledging a FR different to apply
that rule?

BTW: we ran into the same problem addressed in *2.8 Issues with Fast 
Retransmit*, having high traffic, delays and losses.
 We realized, that one packet loss will cause the CWND to crumble, due to 
repeated FR's.
 We *fixed* the problem (before reading the implguide) by saving the 
number of packets in flight for that TSN
at the first FR, and decrementing the outstanding packets counter (for 
that TSN) at receiption
of each consecutive SACK. While the counter is greater zero, we did not 
apply another FR. I have to confess though, that keeping
track of the number of packets in flight is not an easy task.

Another question regarding RFC2960:

where it says:
C5) Karn's algorithm: RTT measurements MUST NOT be made using packets
       that were retransmitted (and thus for which it is ambiguous
       whether the reply was for the first instance of the packet or a
       later instance).

In case we have continous retransmissions we do not update the RTT 
measurement.
Also, above rule implies, that RTT measuremt (for DATA) is performed only 
on the *current* destination, where
current is the primary, if active. The RtoPending flag is per destination, 
where the SACK __MAY__ come
back on a different path. Shouldnt the RtoPending flag relate to the 
assoc?

Heinz Prantner

---

From: [email-masked]
Date: Wed, 30 Jan 2002 21:10:15 +0200

Hi!

	See some comments inline...

-----Original Message-----
From: ext [email-masked] [mailto:[email-masked]]
Sent: 30. January 2002 20:21
To: [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [Tsvwg] draft-ietf-tsvwg-sctpimpguide-03.txt: Questions

Hello Sctp, 

In sctpimpguide chapter *2.8 Issues with Fast Retransmit* in *New text: (Section 7.2.4)* paragraph 5) it reads: 

"...However,  as they are marked for retransmission they will be retransmitted 
      later on as soon as cwnd allows." 

This refers to the TSN's marked for fast retransmission, but could not been fast retransmitted due to 
restriction in paragraph 3 (only one single packet allowed). 

---------------------
	Exactly.
---------------------

Question: Do we wait for T3 for retransmission, or do we *delay* the fast retransmission until consecutive SACKs 
will open the CWND. We feel that the latter is applicable and the impguide needs clarification. In our implementation, 

---------------------
	The idea is the second one. Take into account that once you have marked one packet for retransmission, it is not supposed to be on flight any more, and thus no T3 timer should be running for that TSN. The problem with a Fast retransmission is that it shrinks the cwnd. Then, if several TSNs must be fast retransmitted and their total size is larger than one MTU, it is highly probable that you won't be able to send all of them...

	When the cwnd grows later on thanks to the received SACK chunks, then you will be able to retransmit it...
---------------------

we do not reset the TSN.Missing.Report counter, if we can not FR immediatly, the next SACK which opens CWND, 
will kick off the *delayed fast retransmission* for those TSNs. 

---------------------
	Uhm... :-/

	Do you mean that if the next SACK does not acknowledge those TSNs not already fast retransmitted, you make another fast retransmission?

	But the problem with this is that you will receive many SACKs. They are on the wire comming back... If you do that, you will apply the Fast Retransmission algorithm several times, and you will lower the cwnd a lot. If you make a fast retransmission you will normaly have suddenly more outstanding data than the value of cwnd. Then you will have to wait until cwnd grows (and the number of MTUs on flight lowers) before you can send anything (the chunks marked for retransmission).
---------------------

paragraph 3) says, the cwnd SHOULD be ignored for fast retransmission. What about rwnd? If we receive a SACK 

---------------------
	Well, if you have to retransmit something, the rwnd will never limit you, as you consider those TSNs to be retransmitted as not being on flight any more, and thus they do not occupy any space at the receiver... You decrease rwnd by the size of all the TSNs to be fast retransmitted, and then you add only some of them... you can't have more than before (though some politicians would say that it is possible :->)
---------------------

which kicks off fast retransmission, where the SACK advertises rwnd=0, we are probably blocked. The impguide 

---------------------
	Uhm... I get your point... So the receiver is effectively shrinking its buffers... bad buy... :-0

	Maybe it could be mentioned...
---------------------

says 1 single packet according to PMTU is allowed, cwnd SHOULD be ignored, but does not mention rwnd. 

---------------------
	Yeah... You probably have already that packet on flight...
---------------------

The impguide defines max burst rules for FR. Why is cwnd/rwnd not sufficient? What makes a SACK acknowledging a FR different to apply 
that rule? 

---------------------
	Imagine that if there is one TSN missing and the subsequent TSNs have already arrive. In the normal case, when the TSNs are delivered in order to the upper user, then the SCTP receiver will have part of its buffer full due to all those received TSNs that can't be delivered. So, once the missing TSN arrives, all those TSNs will be delivered to the upper user and rwnd will grow considerably. If rwnd was limiting the sender, and not cwnd, suddenly the data sender could be allowed to send a lot of TSNs...

	That's what Max.Burst is for...
---------------------

BTW: we ran into the same problem addressed in *2.8 Issues with Fast Retransmit*, having high traffic, delays and losses. 
 We realized, that one packet loss will cause the CWND to crumble, due to repeated FR's. 
 We *fixed* the problem (before reading the implguide) by saving the number of packets in flight for that TSN 
at the first FR, and decrementing the outstanding packets counter (for that TSN) at receiption 
of each consecutive SACK. While the counter is greater zero, we did not apply another FR. I have to confess though, that keeping 
track of the number of packets in flight is not an easy task. 

where it says: 

In case we have continous retransmissions we do not update the RTT measurement. 

---------------------
	In any case, after a retransmission you duplicate RTO, and thus the main use of RTT is somehow lost... :-/
---------------------

Also, above rule implies, that RTT measuremt (for DATA) is performed only on the *current* destination, where 
current is the primary, if active. The RtoPending flag is per destination, where the SACK __MAY__ come 
back on a different path. Shouldnt the RtoPending flag relate to the assoc? 

---------------------
	I'm not sure if I understand your question... :-/

	But what you test is the RTT of a specific address. The SACK are supposed to be sent to the source address of the DATA that originates it (note that due to the delayed SACK algorithm this is not always possible). If the SACK comes to another different address, then the RTT is not that exact, but this should be a corner case.

	In any case, you measure the RTT of the rest of addresses (not the primary one) using the heartbeats... At the end, you are measuring the RTT of the primary address once every RTO, and you measure the RTT of the other addresses once every heartbeating interval (only *one* of those addresses at a time)...

	I hope these comments are of any help...

	Bye!

	BR Iván Arias Rodríguez

---------------------

Heinz Prantner 

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: [email-masked]
Date: Wed, 30 Jan 2002 16:48:37 -0500

Hi Ivan,

thank you for your explanations, yes, your comments help :-)
please see my few more comments inline...

Heinz

<[email-masked]>
01/30/2002 02:10 PM

        To:     <[email-masked]>, <[email-masked]>
        cc:     <[email-masked]>, <[email-masked]>, 
<[email-masked]>
        Subject:        RE: [Tsvwg] draft-ietf-tsvwg-sctpimpguide-03.txt: Questions

                 Hi!

-----Original Message-----
From: ext [email-masked] [mailto:[email-masked]]
Sent: 30. January 2002 20:21
To: [email-masked]
Cc: [email-masked]; [email-masked]; 
[email-masked]
Subject: Re: [Tsvwg] draft-ietf-tsvwg-sctpimpguide-03.txt: Questions

Hello Sctp, 

---------------------
                 Exactly.
---------------------

---------------------
                 The idea is the second one. Take into account that once 
you have marked one packet for retransmission, it is not supposed to be on 
flight any more, and thus no T3 timer should be running for that TSN. The 
problem with a Fast retransmission is that it shrinks the cwnd. Then, if 
several TSNs must be fast retransmitted and their total size is larger 
than one MTU, it is highly probable that you won't be able to send all of 
them...

                 When the cwnd grows later on thanks to the received SACK 
chunks, then you will be able to retransmit it...
---------------------

---------------------
                 Uhm... :-/

                 Do you mean that if the next SACK does not acknowledge 
those TSNs not already fast retransmitted, you make another fast 
retransmission?

                 But the problem with this is that you will receive many 
SACKs. They are on the wire comming back... If you do that, you will apply 
the Fast Retransmission algorithm several times, and you will lower the 
cwnd a lot. If you make a fast retransmission you will normaly have 
suddenly more outstanding data than the value of cwnd. Then you will have 
to wait until cwnd grows (and the number of MTUs on flight lowers) before 
you can send anything (the chunks marked for retransmission).

[HP] I was not precise: we basically do what you describe above in *The 
idea is the second one*. We do not apply FR algorithm several times. 
It would shrink the cwnd a lot, as you say.
---------------------

---------------------
                 Well, if you have to retransmit something, the rwnd will 
never limit you, as you consider those TSNs to be retransmitted as not 
being on flight any more, and thus they do not occupy any space at the 
receiver... You decrease rwnd by the size of all the TSNs to be fast 
retransmitted, and then you add only some of them... you can't have more 
than before (though some politicians would say that it is possible :->)

[HP] "...have to retransmit something, the rwnd will never limit you...", 
can we have that in the impguide?
The RFC says the opposite:

6.1  Transmission of DATA Chunks

...

The following general rules MUST be applied by the data sender for
   transmission and/or retransmission of outbound DATA chunks:
                ^^^^^^^^^^^^^^^^^^^^^
 
   A) At any given time, the data sender MUST NOT transmit new data to
      any destination transport address if its peer's rwnd indicates
      that the peer has no buffer space 

---------------------

---------------------
                 Uhm... I get your point... So the receiver is effectively 
shrinking its buffers... bad buy... :-0

[HP] according to above statement, no rwnd limitation at retrans, it will 
work ... and might be the only way to help the receiver
 (the one who advertised rwnd=0) to free his buffers, since he needs that 
TSN, otherwise he is blocked.
---------------------

---------------------
                 Yeah... You probably have already that packet on 
flight...
---------------------

---------------------
                 Imagine that if there is one TSN missing and the 
subsequent TSNs have already arrive. In the normal case, when the TSNs are 
delivered in order to the upper user, then the SCTP receiver will have 
part of its buffer full due to all those received TSNs that can't be 
delivered. So, once the missing TSN arrives, all those TSNs will be 
delivered to the upper user and rwnd will grow considerably. If rwnd was 
limiting the sender, and not cwnd, suddenly the data sender could be 
allowed to send a lot of TSNs...

where it says: 

---------------------
                 In any case, after a retransmission you duplicate RTO, 
and thus the main use of RTT is somehow lost... :-/

[HP] I'm sending on primary destination, and assuming continuous fast 
retransmissions on the alternate destination. (FR does not update RTO, 
right?, and if not FR, T3 expired for the primary, ie. duplicate the RTO 
on primary). Now the alternate destination is busily used for FR, thus no 
heartbeats are sent, so no RTT measurement is done. But maybe thats ok...
---------------------

---------------------

                 But what you test is the RTT of a specific address. The 
SACK are supposed to be sent to the source address of the DATA that 
originates it (note that due to the delayed SACK algorithm this is not 
always possible). If the SACK comes to another different address, then the 
RTT is not that exact, but this should be a corner case.

                 In any case, you measure the RTT of the rest of addresses 
(not the primary one) using the heartbeats... At the end, you are 
measuring the RTT of the primary address once every RTO, and you measure 
the RTT of the other addresses once every heartbeating interval (only 
*one* of those addresses at a time)...

                 Bye!

                 BR Iván Arias Rodríguez

---------------------

Heinz Prantner

---

From: [email-masked]
Date: Wed, 30 Jan 2002 17:46:46 -0500

<SNIP>
 
In any case I think that I don't get your point. Why is it a problem? If 
you are sending FR then you can't use then to measure RTT, i.e. you 
continue sending heartbeats (look at the definition of "idle address" in 
page 93 of RFC 2960).
 
<SNIP>

Uuups, I will then update my code to the correct IDLE definition. Thanks 
for the catch!

Heinz

---

From: [email-masked]
Date: Thu, 31 Jan 2002 00:28:53 +0200

Hi again!

-----Original Message-----
From: ext [email-masked] [mailto:[email-masked]]
Sent: 30. January 2002 23:49
To: Arias-Rodriguez Ivan (NRC/Helsinki); [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: RE: [Tsvwg] draft-ietf-tsvwg-sctpimpguide-03.txt: Questions

Hi Ivan, 

Heinz 

01/30/2002 02:10 PM 

        To:        <[email-masked]>, <[email-masked]> 
        cc:        <[email-masked]>, <[email-masked]>, <[email-masked]> 
        Subject:        RE: [Tsvwg] draft-ietf-tsvwg-sctpimpguide-03.txt: Questions

                 Hi!

-----Original Message-----
From: ext [email-masked] [mailto:[email-masked]]
Sent: 30. January 2002 20:21
To: [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [Tsvwg] draft-ietf-tsvwg-sctpimpguide-03.txt: Questions

Hello Sctp, 

---------------------
                Exactly.
---------------------

---------------------

---------------------
                Uhm... :-/

[HP] I was not precise: we basically do what you describe above in *The idea is the second one*. We do not apply FR algorithm several times. 
It would shrink the cwnd a lot, as you say.
---------------------

---------------------

[HP] "...have to retransmit something, the rwnd will never limit you...", can we have that in the impguide? 
The RFC says the opposite: 

... 

[Arias-Rodriguez Ivan (NRC/Helsinki)] Yep... When I answered I didn't consider the case when the SACK sender is shrinking its buffers... Probably we should also say something about that in the Fast Retransmit section... :-/

---------------------

---------------------

[HP] according to above statement, no rwnd limitation at retrans, it will work ... and might be the only way to help the receiver 
 (the one who advertised rwnd=0) to free his buffers, since he needs that TSN, otherwise he is blocked.

[Arias-Rodriguez Ivan (NRC/Helsinki)]  Well, you always can have at least one TSN in flight. If due to rwnd limitation you can't send the datagram, you should check if you have any other TSN on flight. As always, if there is no TSN in flight rwnd does not limit.
 
If the receiver has its buffers full, but still need a previous TSN, then it should renege one of the subsequent TSNs in order to be able to deliver something to the upper user and then free some space...
 
Summarizing, I think that the best behaviour is not letting cwnd limit us, but take a look at rwnd. If rwnd is less than what we would like to send (the first TSN marked for retransmission), then we can't send anything unless there isn't any other TSN in flight (taking into account that all those TSNs marked for fast retransmission are not in flight any more)
---------------------

---------------------

---------------------

where it says: 

---------------------

[HP] I'm sending on primary destination, and assuming continuous fast retransmissions on the alternate destination. (FR does not update RTO, right?, and if not FR, T3 expired for the primary, ie. duplicate the RTO on primary). Now the alternate destination is busily used for FR, thus no heartbeats are sent, so no RTT measurement is done. But maybe thats ok...

[Arias-Rodriguez Ivan (NRC/Helsinki)] Ah, when you said that you had continuous retransmissions I thought they were triggered by T3. If those retransmissions are due to the Fast Retransmit algorith, then you don't double RTO.
 
In any case I think that I don't get your point. Why is it a problem? If you are sending FR then you can't use then to measure RTT, i.e. you continue sending heartbeats (look at the definition of "idle address" in page 93 of RFC 2960).
 
Moreover, you can only FR once a chunk. So, you do FR the TSN to another address, and then the T3 timer will expire after some time, and you will double its RTO. Meanwhile, either you are not sending anything new to the primary address (and thus the heartbeat algorithm applies) or you are measuring RTT with the new DATA chunks sent...
 
I don't know if the problem is that if you are FR a lot you would never take a measure of the RTT on that secondary address, which is not the case as that address continues being "idle"... :-/
 
Could you explain better the problem?
 
Thanks!
 
BR Iván Arias Rodríguez 
---------------------

---------------------

                Bye!

                BR Iván Arias Rodríguez

---------------------

Heinz Prantner