Subject: Re: [Tsvwg] sctp Window deadlock

From: "Mohammed Ashraf" <[email-masked]>
Date: Tue, 05 Feb 2002 20:00:38 +0500

hi arun,
	If this is the case(if we are not retransmitting ), then there is
another possibility of a deadlock, when the SACK that carries window
change get lost in the n/w.i.e sender didn't receive SACK and  believes
that window is still zero and receiver will assume that, its not getting
anymore data because sender doesn't have  any more data to send(this is
juz because there is no reliablity transmission mechanism built into for
SACK:--)))..

Thanks,
ashraf
Content-Type: multipart/alternative;
	 boundary="------------48E1687CF03AA848973D7833"

--------------48E1687CF03AA848973D7833
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

hi Ashraf,

From Rfc:
However, regardless of the value of rwnd (including if it
      is 0), the data sender can always have one DATA chunk in flight to
      the receiver if allowed by cwnd

    My understanding from the above is, trying to send the Data chunk only once
for probing, and if that chunk is not acknowledged (or acknowledged with rwnd
as zero), then even when the Rtx timer expires for that Data chunk, the data chunk
will be queued in the local Q (as the rwnd is zero, and already one data chunk
has been sent for probing...) and will not be retransimtted.....
    I dont know whether my understanding is right......  Just getting a feeling that
Iam going wrong somewhere....( might be ur understanding is right.....)
Somebody please clarify.....

Thanks
-arun

Ashraf wrote:

> Hi,
>     My understanding is ( pls correct me if I am wromg),  each time when the timer expires, the path /assoc retrans counter will be
>
> incremented and sender keeps on probing  for change in window size  by sending next fragment ( in ur caase fragment B ) and  ending
>
> up in T3 timer expiry each time. Will the Assoc will be  terminated once this count reached the threshold value.?"
>
> Thanks,
> ashraf
>
> Arun Prasad wrote:
>
> > Hi Ashraf,
> >     ya RTT timer will be started, but upon its expiry, we come to the same problem..
> > because we cant send the retransmitted packets due to rwnd shortage......
> >
> > Thanks
> > -arun
> >
> > Ashraf wrote:
> >
> > > Hi arun,
> > >     I think  for the new data chunk ( which we are sending  to probe for change in window), we start RTT timer and  give
> > > appropraite treatment, when it expires.
> > >
> > > thanks,
> > > ashraf
> > >
> > > Arun Prasad wrote:
> > >
> > > > Hi Ashraf,
> > > >     Thanks for the rsp....
> > > > ya, I agree that partial-delivery will solve the problem... But imagine an errornous
> > > > stack ( or SCTP stack which didnt implement the partial delivery API)  is on the
> > > > receiving side.... Then with the below example scenario, the Sender stack will
> > > > get stuck.... ie.,  the association will never tear down (as RTX timer is not running)
> > > > so the Sender side will be hung (with no mistake on its side....) forever.....
> > > > I think this situation should be avoided......
> > > >
> > > > Thanks
> > > > -arun
> > > >
> > > > Ashraf wrote:
> > > >
> > > > > Arun,
> > > > >     This is what RFC has to say :
> > > > > "If the data receiver  run out of buffer space while still  waiting for  for more fragments
> > > > > to complete the re-assembly, of the message, it should dispatch part of its inbound message
> > > > > through a partial deliver API, freeing some if its buffer space in order receive  rest of
> > > > > the message"
> > > > >
> > > > > guess this should take care of the deadlock  situation you are mentioning here.
> > > > >
> > > > > Thanks,
> > > > > Mohd. Ashraf
> > > > > Arun Prasad wrote:
> > > > >
> > > > > > Hi all,
> > > > > >     The question might be more of  SCTP implementation specific....   According
> > > > > > to the SCTP rfc2960, the data sender should not send any data packets if it doesnt
> > > > > > have enough Send-Windown (Peer Rwnd).... Just take the below example....
> > > > > >
> > > > > > The Peer's Rwnd (at the Init time ) is 1500 bytes.
> > > > > > The Sender receivies a data buffer of size 6000 from its upper layer to be sent
> > > > > > to the Peer.... The Sender will then fragment the packets to fragments A,B,C,D of
> > > > > > size 1500 bytes (because the local mtu size is 1500bytes)....
> > > > > > Now the Sender sends the frag A to the Peer... Upon receiving the frag A, the
> > > > > > Peer's Rwnd will be zero.... This is informed to the Sender by sending a SACK with
> > > > > > rwnd as 0. The Sender which would have queued the frags B,C,D... will then
> > > > > > send frag B as the probing data to the Peer... The peer again will respond with
> > > > > > a SACK of rwnd 0 .  Now the deadlock situation occurs,  as the
> > > > > >     * Sender cant send any more data packets as the Peer's rwnd is zero....
> > > > > >     * Peer cant increase its rwnd till it releases the data chunks which it had already
> > > > > >        received,  but to release that it needs to reassemble the frags (ABCD) and
> > > > > >       deliver it to the above layer, only then it can release the buffers..... Since its
> > > > > >       waiting for the frags C and D...  this will never release the buffers.... so
> > > > > >       will not increase the rwnd.....
> > > > > >
> > > > > > This situation can be avoided if the Sender doesnt sent Data messages of size
> > > > > > greater than the Peer's rwnd.... But the Rfc doesnt say anything about this.....
> > > > > > Or there should be some timers, to get out of the Deadlock situation.....
> > > > > >
> > > > > > Any suggestions........
> > > > > >
> > > > > > Thanks
> > > > > > -arun
> > > > > >
> > > > > > _______________________________________________
> > > > > > tsvwg mailing list
> > > > > > [email-masked]
> > > > > > https://www1.ietf.org/mailman/listinfo/tsvwg
> > > > >
> > > > >   ------------------------------------------------------------------------------------------------------------------------
> > > > >                            Name: Wipro_Disclaimer.txt
> > > > >    Wipro_Disclaimer.txt    Type: Plain Text (text/plain)
> > > > >                        Encoding: 7bit
> > > >
> > > > --
> > > > ****************************************************************
> > > > V.Arun Prasad
> > > > HCL Technologies Ltd.
> > > > 51, Jawaharlal Nehru Road,
> > > > Ekkattuthangal,
> > > > Guindy Industrial Estate,
> > > > Chennai - 600097.
> > > >
> > > > Contact # : 9144 - 2334174
> > > >             9144 - 2334181
> > > >             extn : 233
> > > > ****************************************************************
> > > >
> > > > _______________________________________________
> > > > tsvwg mailing list
> > > > [email-masked]
> > > > https://www1.ietf.org/mailman/listinfo/tsvwg
> > >
> > >   ------------------------------------------------------------------------------------------------------------------------
> > >                            Name: Wipro_Disclaimer.txt
> > >    Wipro_Disclaimer.txt    Type: Plain Text (text/plain)
> > >                        Encoding: 7bit
> >
> > --
> > ****************************************************************
> > V.Arun Prasad
> > HCL Technologies Ltd.
> > 51, Jawaharlal Nehru Road,
> > Ekkattuthangal,
> > Guindy Industrial Estate,
> > Chennai - 600097.
> >
> > Contact # : 9144 - 2334174
> >             9144 - 2334181
> >             extn : 233
> > ****************************************************************
>
>   ------------------------------------------------------------------------------------------------------------------------
>                            Name: Wipro_Disclaimer.txt
>    Wipro_Disclaimer.txt    Type: Plain Text (text/plain)
>                        Encoding: 7bit

--
****************************************************************
V.Arun Prasad
HCL Technologies Ltd.
51, Jawaharlal Nehru Road,
Ekkattuthangal,
Guindy Industrial Estate,
Chennai - 600097.

Contact # : 9144 - 2334174
            9144 - 2334181
            extn : 233
****************************************************************

--------------48E1687CF03AA848973D7833
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit

<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
hi Ashraf,
<p>From Rfc:
<br><font color="#3333FF">However, regardless of the value of rwnd (including
if it</font>
<br><font color="#3333FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; is 0), the data
sender can always have <b>one DATA chunk in flight</b> to</font>
<br><font color="#3333FF">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; the receiver if
allowed by cwnd</font>
<p>&nbsp;&nbsp;&nbsp; My understanding from the above is, trying to send
the Data chunk only once
<br>for probing, and if that chunk is not acknowledged (or acknowledged
with rwnd
<br>as zero), then even when the Rtx timer expires for that Data chunk,
the data chunk
<br>will be queued in the local Q (as the rwnd is zero, and already one
data chunk
<br>has been sent for probing...) and will not be retransimtted.....
<br>&nbsp;&nbsp;&nbsp; I dont know whether my understanding is right......&nbsp;
Just getting a feeling that
<br>Iam going wrong somewhere....( might be ur understanding is right.....)
<br>Somebody please clarify.....
<p>Thanks
<br>-arun
<br>&nbsp;
<p>Ashraf wrote:
<blockquote TYPE=CITE>Hi,
<br>&nbsp;&nbsp;&nbsp; My understanding is ( pls correct me if I am wromg),&nbsp;
each time when the timer expires, the path /assoc retrans counter will
be
<p>incremented and sender keeps on probing&nbsp; for change in window size&nbsp;
by sending next fragment ( in ur caase fragment B ) and&nbsp; ending
<p>up in T3 timer expiry each time. Will the Assoc will be&nbsp; terminated
once this count reached the threshold value.?"
<p>Thanks,
<br>ashraf
<p>Arun Prasad wrote:
<p>> Hi Ashraf,
<br>>&nbsp;&nbsp;&nbsp;&nbsp; ya RTT timer will be started, but upon its
expiry, we come to the same problem..
<br>> because we cant send the retransmitted packets due to rwnd shortage......
<br>>
<br>> Thanks
<br>> -arun
<br>>
<br>> Ashraf wrote:
<br>>
<br>> > Hi arun,
<br>> >&nbsp;&nbsp;&nbsp;&nbsp; I think&nbsp; for the new data chunk (
which we are sending&nbsp; to probe for change in window), we start RTT
timer and&nbsp; give
<br>> > appropraite treatment, when it expires.
<br>> >
<br>> > thanks,
<br>> > ashraf
<br>> >
<br>> > Arun Prasad wrote:
<br>> >
<br>> > > Hi Ashraf,
<br>> > >&nbsp;&nbsp;&nbsp;&nbsp; Thanks for the rsp....
<br>> > > ya, I agree that partial-delivery will solve the problem... But
imagine an errornous
<br>> > > stack ( or SCTP stack which didnt implement the partial delivery
API)&nbsp; is on the
<br>> > > receiving side.... Then with the below example scenario, the
Sender stack will
<br>> > > get stuck.... ie.,&nbsp; the association will never tear down
(as RTX timer is not running)
<br>> > > so the Sender side will be hung (with no mistake on its side....)
forever.....
<br>> > > I think this situation should be avoided......
<br>> > >
<br>> > > Thanks
<br>> > > -arun
<br>> > >
<br>> > > Ashraf wrote:
<br>> > >
<br>> > > > Arun,
<br>> > > >&nbsp;&nbsp;&nbsp;&nbsp; This is what RFC has to say :
<br>> > > > "If the data receiver&nbsp; run out of buffer space while still&nbsp;
waiting for&nbsp; for more fragments
<br>> > > > to complete the re-assembly, of the message, it should dispatch
part of its inbound message
<br>> > > > through a partial deliver API, freeing some if its buffer space
in order receive&nbsp; rest of
<br>> > > > the message"
<br>> > > >
<br>> > > > guess this should take care of the deadlock&nbsp; situation
you are mentioning here.
<br>> > > >
<br>> > > > Thanks,
<br>> > > > Mohd. Ashraf
<br>> > > > Arun Prasad wrote:
<br>> > > >
<br>> > > > > Hi all,
<br>> > > > >&nbsp;&nbsp;&nbsp;&nbsp; The question might be more of&nbsp;
SCTP implementation specific....&nbsp;&nbsp; According
<br>> > > > > to the SCTP rfc2960, the data sender should not send any
data packets if it doesnt
<br>> > > > > have enough Send-Windown (Peer Rwnd).... Just take the below
example....
<br>> > > > >
<br>> > > > > The Peer's Rwnd (at the Init time ) is 1500 bytes.
<br>> > > > > The Sender receivies a data buffer of size 6000 from its
upper layer to be sent
<br>> > > > > to the Peer.... The Sender will then fragment the packets
to fragments A,B,C,D of
<br>> > > > > size 1500 bytes (because the local mtu size is 1500bytes)....
<br>> > > > > Now the Sender sends the frag A to the Peer... Upon receiving
the frag A, the
<br>> > > > > Peer's Rwnd will be zero.... This is informed to the Sender
by sending a SACK with
<br>> > > > > rwnd as 0. The Sender which would have queued the frags B,C,D...
will then
<br>> > > > > send frag B as the probing data to the Peer... The peer again
will respond with
<br>> > > > > a SACK of rwnd 0 .&nbsp; Now the deadlock situation occurs,&nbsp;
as the
<br>> > > > >&nbsp;&nbsp;&nbsp;&nbsp; * Sender cant send any more data
packets as the Peer's rwnd is zero....
<br>> > > > >&nbsp;&nbsp;&nbsp;&nbsp; * Peer cant increase its rwnd till
it releases the data chunks which it had already
<br>> > > > >&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; received,&nbsp;
but to release that it needs to reassemble the frags (ABCD) and
<br>> > > > >&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; deliver it to the above
layer, only then it can release the buffers..... Since its
<br>> > > > >&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; waiting for the frags
C and D...&nbsp; this will never release the buffers.... so
<br>> > > > >&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; will not increase the
rwnd.....
<br>> > > > >
<br>> > > > > This situation can be avoided if the Sender doesnt sent Data
messages of size
<br>> > > > > greater than the Peer's rwnd.... But the Rfc doesnt say anything
about this.....
<br>> > > > > Or there should be some timers, to get out of the Deadlock
situation.....
<br>> > > > >
<br>> > > > > Any suggestions........
<br>> > > > >
<br>> > > > > Thanks
<br>> > > > > -arun
<br>> > > > >
<br>> > > > > _______________________________________________
<br>> > > > > tsvwg mailing list
<br>> > > > > [email-masked]
<br>> > > > > <a href="https://www1.ietf.org/mailman/listinfo/tsvwg">https://www1.ietf.org/mailman/listinfo/tsvwg</a>
<br>> > > >
<br>> > > >&nbsp;&nbsp; ------------------------------------------------------------------------------------------------------------------------
<br>> > > >&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Name: Wipro_Disclaimer.txt
<br>> > > >&nbsp;&nbsp;&nbsp; Wipro_Disclaimer.txt&nbsp;&nbsp;&nbsp; Type:
Plain Text (text/plain)
<br>> > > >&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Encoding: 7bit
<br>> > >
<br>> > > --
<br>> > > ****************************************************************
<br>> > > V.Arun Prasad
<br>> > > HCL Technologies Ltd.
<br>> > > 51, Jawaharlal Nehru Road,
<br>> > > Ekkattuthangal,
<br>> > > Guindy Industrial Estate,
<br>> > > Chennai - 600097.
<br>> > >
<br>> > > Contact # : 9144 - 2334174
<br>> > >&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
9144 - 2334181
<br>> > >&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
extn : 233
<br>> > > ****************************************************************
<br>> > >
<br>> > > _______________________________________________
<br>> > > tsvwg mailing list
<br>> > > [email-masked]
<br>> > > <a href="https://www1.ietf.org/mailman/listinfo/tsvwg">https://www1.ietf.org/mailman/listinfo/tsvwg</a>
<br>> >
<br>> >&nbsp;&nbsp; ------------------------------------------------------------------------------------------------------------------------
<br>> >&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Name: Wipro_Disclaimer.txt
<br>> >&nbsp;&nbsp;&nbsp; Wipro_Disclaimer.txt&nbsp;&nbsp;&nbsp; Type:
Plain Text (text/plain)
<br>> >&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Encoding: 7bit
<br>>
<br>> --
<br>> ****************************************************************
<br>> V.Arun Prasad
<br>> HCL Technologies Ltd.
<br>> 51, Jawaharlal Nehru Road,
<br>> Ekkattuthangal,
<br>> Guindy Industrial Estate,
<br>> Chennai - 600097.
<br>>
<br>> Contact # : 9144 - 2334174
<br>>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
9144 - 2334181
<br>>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
extn : 233
<br>> ****************************************************************
<p>&nbsp; ------------------------------------------------------------------------------------------------------------------------
<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Name: Wipro_Disclaimer.txt
<br>&nbsp;&nbsp; Wipro_Disclaimer.txt&nbsp;&nbsp;&nbsp; Type: Plain Text
(text/plain)
<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Encoding: 7bit</blockquote>

<p>--
<br>****************************************************************
<br>V.Arun Prasad
<br>HCL Technologies Ltd.
<br>51, Jawaharlal Nehru Road,
<br>Ekkattuthangal,
<br>Guindy Industrial Estate,
<br>Chennai - 600097.
<p>Contact # : 9144 - 2334174
<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
9144 - 2334181
<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
extn : 233
<br>****************************************************************
<br>&nbsp;</html>

--------------48E1687CF03AA848973D7833--

---

From: [email-masked]
Date: Tue, 5 Feb 2002 18:07:44 +0200

Ups! this was even clarified in the last version of the Implementors Guide... :-O I should take some more coffee...

	So the point here is that the receiving side will keep open the association as long as it wants... So, it is the upper user at the sending side the one in charge of tearing down the association if it is not advancing... In any case, it is the upper user at the receiver side the one that is not taking the data (or the SCTP implementation does not have the partial delivery API, thus taking the risk of being in this situation)...

	BR Iván Arias Rodríguez

> -----Original Message-----
> From: ext Randall Stewart [mailto:[email-masked]]
> Sent: 05. February 2002 17:50
> To: Arias-Rodriguez Ivan (NRC/Helsinki)
> Cc: [email-masked]; [email-masked]; 
> [email-masked]
> Subject: Re: [Tsvwg] sctp Window deadlock
> 
> 
> Ivan:
> 
> I agree with all your assesement except I would NOT count
> the T3 against anything towards failure (no matter if
> HB's are running or not) if we are doing widow-probes.
> 
> I run special code for this when we get in the window-probe
> mode... And no threshold managment is done... 
> 
> This is the same thing that happens in TCP (if my understanding
> is correct).. i.e. once the sender clamps the rwnd=0 you
> will continue to probe until it opens up.. as long as
> you get a SACK back that is...
> 
> R
> 
> 
> [email-masked] wrote:
> > 
> >     Hi all!
> > 
> >     Well, I think that the point here is that once the T3 
> timer expires on an address, all the TSNs that were 
> outstanding on that address are automatically marked for 
> retransmission, i.e, they are not outstanding TSNs any longer...
> > 
> >     In this situation, as you are allowed to have at least 
> one TSN outstanding no matter the value of rwnd, you will 
> send the oldest TSN marked for retransmission and restart the 
> T3 again (if possible using another different address than 
> the first time)...
> > 
> >     After several failures, you will abort the 
> association... It might take time (RTO.Max is one minute), 
> but at the end you get rid of the association...
> > 
> >     However... If the path heartbeat mechanism is enabled, 
> every received HEARTBEAT ACK chunk will erase the 
> retransmission counter... Thus we are in the same problem... 
> But this is because the receiver is not behaving in the 
> expected way...
> > 
> >     BR Iván Arias Rodríguez
> > 
> > -----Original Message-----
> > From: ext Arun Prasad [mailto:[email-masked]]
> > Sent: 05. February 2002 16:37
> > To: Ashraf
> > Cc: Tsvwg; Randall Stewart
> > Subject: Re: [Tsvwg] sctp Window deadlock
> > 
> > hi Ashraf,
> > 
> > >From Rfc:
> >     My understanding is ( pls correct me if I am wromg),  
> each time when the timer expires, the path /assoc retrans 
> counter will be
> > 
> > incremented and sender keeps on probing  for change in 
> window size  by sending next fragment ( in ur caase fragment 
> B ) and  ending
> > 
> > up in T3 timer expiry each time. Will the Assoc will be  
> terminated once this count reached the threshold value.?"
> > 
> > Thanks,
> > ashraf
> > 
> > Arun Prasad wrote:
> > 
> > > Hi Ashraf,
> > > >     I think  for the new data chunk ( which we are 
> sending  to probe for change in window), we start RTT timer and  give
> > > > > > "If the data receiver  run out of buffer space 
> while still  waiting for  for more fragments
> > > > > > to complete the re-assembly, of the message, it 
> should dispatch part of its inbound message
> > > > > > >     * Peer cant increase its rwnd till it 
> releases the data chunks which it had already
> --------------------------------------------------------------
> ----------------------------------------------------------
> --------------------------------------------------------------
> ----------------------------------------------------------
> --------------------------------------------------------------
> ----------------------------------------------------------
> Randall R. Stewart
> [email-masked] 815-342-5222 (cell phone)
>