Subject: [tsvwg] SCTP mis indication counting during Fast Recovery

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Tue, 21 Oct 2014 09:45:00 +0200

Hi,

I have come to realize that I for the SCTP TLP (SCTP Tail Loss probe
additions) and SCTP FR improvements (adoptation of certain RFC6675
nextseg() features for SCTP FR) that we are working on (as presented at
IETF 90) have made an assumption on the SCTP FR function which is neither
aligned with our

actual SCTP implementation nor with the text of RFC4960 (/RFC4460) as
written (section 7.2.4).

However in order to better understand the motivation for the text of
RFC4960 as written,

I would appreciate very much if the wg would comment on the following
formulation of RFC4960 section 7.2.4:

If an endpoint is in Fast Recovery and a SACK

   arrives that advances the Cumulative TSN Ack Point, the

   miss indications are incremented for all TSNs reported

   missing in the SACK.

Especially on why the text DOES NOT read:

If an endpoint is in Fast Recovery and a SACK

   arrives that advances the Cumulative TSN Ack Point, the

   miss indications are incremented for all TSNs still

   missing up to and including the Fast Recovery Exit Point.

The basic philosophi being that when the cum ack advances, then the newly
cum ack’ed TSN has

been assumed lost, has been retransmitted and consequently the mis
indication count should go up for

all still missing TSNs up to and including the Fast Recovery Exit Point.

It is clear that the above aproach by no means is bullet proof in the sense
that it may very well be that the cum ack advances

due to receipt of SACK without this in fact being a result of the receipt
by peer of a fast retransmitted TSN, but this is the situation

also with the approach written into RFC4960.

Many thanks in advance.

BR, Karen

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 23 Oct 2014 09:46:52 +0200

On 21 Oct 2014, at 09:45, Karen Elisabeth Egede Nielsen <[email-masked]> wrote:

>  Hi,
>  
> I have come to realize that I for the SCTP TLP (SCTP Tail Loss probe additions) and SCTP FR improvements (adoptation of certain RFC6675 nextseg() features for SCTP FR) that we are working on (as presented at IETF 90) have made an assumption on the SCTP FR function which is neither aligned with our
> actual SCTP implementation nor with the text of RFC4960 (/RFC4460) as written (section 7.2.4).
>  
> However in order to better understand the motivation for the text of RFC4960 as written,
> I would appreciate very much if the wg would comment on the following formulation of RFC4960 section 7.2.4:
>  
What are you asking for:
1. Do you want to limited the TSNs for which you want to increase the counter,
   but still consider only the ones reported in the SACK.
2. Do you want to ignore what is reported missing in the SACK and just base it
   on the senders view?
>  
>  
> The basic philosophi being that when the cum ack advances, then the newly cum ack’ed TSN has
> been assumed lost, has been retransmitted and consequently the mis indication count should go up for
I don't understand the part: "the newly cum ack’ed TSN has been assumed lost"
Could you clarify what you mean?

Best regards
Michael
> It is clear that the above aproach by no means is bullet proof in the sense that it may very well be that the cum ack advances
> due to receipt of SACK without this in fact being a result of the receipt by peer of a fast retransmitted TSN, but this is the situation

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Thu, 23 Oct 2014 11:03:55 +0200

, I would appreciate very much if the wg would
> comment on the following formulation of RFC4960 section 7.2.4:
> >
[Karen] No 100% sure what you mean. This to me sounds as the present text
of RFC4960.
> 2. Do you want to ignore what is reported missing in the SACK and just
base it
[Karen] No I don't what to ignore what it is the SACK. I want to use what
it is in SACK PLUS the senders knowledge
that the now newly cumack'ed  TSN was sent later than all TSN missing up
to and including the FR Exit point.

Do me the RFC4960 text goes only half the way. It does use what it in the
SACK and it does use the knowledge
that the now newly cumack'ed TSN TSN was sent later than some TSNs - but
here only the TSNs reported missing in the SACK.

IF there is a SACK'ed TSN after FR exit point then RFC4960 text does what
I want, but if the highest SACK'ed TSN is below the FR exit point it does
not.

I am actually much more concerned with getting mis-indications for the
TSNs below the FR exit point than above.
This is particularly important when there is a loss in the end and no new
data to drive FR. But yes we are looking at complex drop scenarios for
this to take effect

RFC6675 TCP Rescue operation (nextseg() 4)) is one way to amend FR to work
better to recover tails and we are  actually implementing this
in our SCTP SW, but for me it has still been very important to understand
when/if this is needed.

As reference then in the presentation
http://www.ietf.org/proceedings/90/slides/slides-90-tsvwg-16.pdf,
slide 30-33 (sorry for numbers not being there consistently) we presented
some analysis of the RFC6675 effects on SCTP FR.
There are a number of loss scenarios in the tables there marked with
RFC4960 recover high L (TSN lost in the end)
This marking - assuming the text of RFC4960 is to be taken literally -
should not rightfully read
RFC6674 nextseg() 4) Rescue operation recover high L (TSN lost in the
end).

> >
> >
> > The basic philosophi being that when the cum ack advances, then the
> > newly cum ack'ed TSN has been assumed lost, has been retransmitted and
> > consequently the mis indication count should go up for
> I don't understand the part: "the newly cum ack'ed TSN has been assumed
> lost"
[Karen] Let me answer by a question:
What, in your understanding, is the motivation for the following
prescription, RFC4960 section 7.2 (added by RFC4460):

          " If an endpoint is in Fast Recovery and a SACK
          arrives that advances the Cumulative TSN Ack Point, the
          miss indications are incremented for all TSNs reported
          missing in the SACK."

Is it not that the mis indications should come because the newly cumak'ed
TSN is assumed to have been fast retransmitted and
receive by peer after the other TSNs, reported as missing in the SACK, was
sent ?

BR, Karen

> > It is clear that the above aproach by no means is bullet proof in the
> > sense that it may very well be that the cum ack advances due to
> > receipt of SACK without this in fact being a result of the receipt by
peer of a
> fast retransmitted TSN, but this is the situation also with the approach
> written into RFC4960.
> >
> > Many thanks in advance.
> >
> > BR, Karen

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 23 Oct 2014 05:27:11 -0600

Karen,

Please see comments below:

On 21 Oct 2014, at 09:45, Karen Elisabeth Egede Nielsen wrote:
> 
>  Hi,
>  
> I have come to realize that I for the SCTP TLP (SCTP Tail Loss
> probe additions) and SCTP FR improvements (adoptation of certain
> RFC6675 nextseg() features for SCTP FR) that we are working on (as
> presented at IETF 90) have made an assumption on the SCTP FR
> function which is neither aligned with our actual SCTP
> implementation nor with the text of RFC4960 (/RFC4460) as written
> (section 7.2.4).
>  
> However in order to better understand the motivation for the text
> of RFC4960 as written, I would appreciate very much if the wg
> would comment on the following formulation of RFC4960 section
> 7.2.4:
>  
> If an endpoint is in Fast Recovery and a SACK arrives that
>    advances the Cumulative TSN Ack Point, the miss indications are
>    incremented for all TSNs reported missing in the SACK.
>  
> If an endpoint is in Fast Recovery and a SACK arrives that
>    advances the Cumulative TSN Ack Point, the miss indications are
>    incremented for all TSNs still missing up to and including the
>    Fast Recovery Exit Point.
>  

Would it be possible for you to describe a simple non-renegging
scenario in which the highest acknowledged TSN in a SACK that
advances the cummulative ack point would not encompass TSNs up
to and including the fast recovery exit point?

I can't think of one at the moment.

Also, the last paragraph of RFC4960/7.2.4 reads:

 "A straightforward implementation of the above keeps a counter for
  each TSN hole reported by a SACK.  The counter increments for each
  consecutive SACK reporting the TSN hole.  After reaching 3 and
  starting the Fast-Retransmit procedure, the counter resets to 0.
  Because cwnd in SCTP indirectly bounds the number of outstanding
  TSN's, the effect of TCAP Fast Recovery is acheived automatically
  with no adjustment to the congestion control window size."

I understand this to mean that no fast recovery exit point needs to
be tracked at all for SCTP.

Is there a reason would make it necessary to track the fast recovery
exit point?

--brian

> The basic philosophi being that when the cum ack advances, then
> the newly cum ack’ed TSN has been assumed lost, has been
> retransmitted and consequently the mis indication count should go
> up for all still missing TSNs up to and including the Fast
> Recovery Exit Point.
>  
> It is clear that the above aproach by no means is bullet proof in
> the sense that it may very well be that the cum ack advances due
> to receipt of SACK without this in fact being a result of the
> receipt by peer of a fast retransmitted TSN, but this is the
> situation also with the approach written into RFC4960.
>  
> Many thanks in advance.
>  
> BR, Karen

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Thu, 23 Oct 2014 13:35:51 +0200

Hi Brian,

Thanks please see below.

BR, Karen

> -----Original Message-----
> From: tsvwg [mailto:[email-masked]] On Behalf Of Brian F. G.
> Bidulock
> Sent: 23. oktober 2014 13:27
> To: Michael Tuexen
> Cc: Karen Elisabeth Egede Nielsen; tsvwg WG; Randall Stewart
> Subject: Re: [tsvwg] SCTP mis indication counting during Fast Recovery
>
> Karen,
>
> > additions) and SCTP FR improvements (adoptation of certain
> > presented at IETF 90) have made an assumption on the SCTP FR function
> > which is neither aligned with our actual SCTP implementation nor with
> > the text of RFC4960 (/RFC4460) as written (section 7.2.4).
> >
> > RFC4960 as written, I would appreciate very much if the wg would
> Would it be possible for you to describe a simple non-renegging scenario
> in
> which the highest acknowledged TSN in a SACK that advances the
> cummulative ack point would not encompass TSNs up to and including the
> fast recovery exit point?
>

[Karen] Please see the analysis presented at IETF90:

As referenced in the presentation
http://www.ietf.org/proceedings/90/slides/slides-90-tsvwg-16.pdf,
slide 30-33 (sorry for numbers not being there consistently) we presented
some analysis of the RFC6675 effects on SCTP FR.
There are a number of loss scenarios in the tables there marked with RFC4960
recover high L (TSN lost in the end) This marking - assuming the text of
RFC4960 is to be taken literally - should rightfully read
RFC6674 nextseg() 4) Rescue operation recover high L (TSN lost in the end).

> I understand this to mean that no fast recovery exit point needs to be
> tracked at all for SCTP.
>
> Is there a reason would make it necessary to track the fast recovery exit
> point?
>
[Karen] I am not sure that I understand how to implement SCTP FR without
tracking the exit point.

BR, Karen
> > newly cum ack’ed TSN has been assumed lost, has been retransmitted and
> > consequently the mis indication count should go up for all still
> > missing TSNs up to and including the Fast Recovery Exit Point.
> >
> > peer of a fast retransmitted TSN, but this is the situation also with
> > the approach written into RFC4960.
> >
> > Many thanks in advance.
> >
> > BR, Karen
>
> --
> Brian F. G. Bidulock
> [email-masked]
> http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 23 Oct 2014 06:00:42 -0600

Karen,

Karen Elisabeth Egede Nielsen wrote:  (Thu, 23 Oct 2014 13:35:51)

--X--snip--X--
> RFC6674 nextseg() 4) Rescue operation recover high L (TSN lost in the end).

I looked at the presentation before replying and I failed to
trace any scenario from the presentation.  Could you describe the
packet exchange for just one of them?  Treat me like I'm six years
old...

> [Karen] I am not sure that I understand how to implement SCTP FR without

The first three sentences describe how to implement SCTP FR without
tracking the exit point.  As I understand it, The purpose of the exit
point in TCP was to limit the number of packets in flight (because
TCP cwnd tracks bytes not packets).  Because SCTP cwnd indirectly limits
packets in flight all day long, the TCP FR cwnd adjustments are not
necessary, thus in SCTP tracking the FR exit point is not necessary.

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 23 Oct 2014 13:12:15 -0600

Karen,

Please see comments inline:

On Thu, 23 Oct 2014, Karen Elisabeth Egede Nielsen wrote:

>    HI,
> 
>    Let's assume SCTP send a batch of packets, the packets marked with L
>    below are dropped, the packets with S are sacked.
> 
>    There is no more new data to send.
> 
>    SACKs arrive resulting in the following scorebard:
> 
>    LSSSLL
> 
>    in order from low -  high TSN.
> 
>    This makes SCTP fast retransmit the first lost packet setting FR exit
>    point to last TSN.
> 
>    resulting in increase of cumack and following scoreboard (A indicate
>    cumack'ed) once this TSN is sacked (which I assume).
> 
>    AAAALL
> 
>    There is 0 miscounts of the last two lost packets even if one packet
>    send since them obviously have been Sacked.

 TSN 1 ----------------> X
 TSN 2 ----------------------------->
       <----------------------------- SACK 0, 2-2
 TSN 1 peg = 1
 TSN 3 ----------------------------->
       <----------------------------- SACK 0, 2-3
 TSN 1 peg = 2
 TSN 4 ----------------------------->
       <----------------------------- SACK 0, 2-4
 TSN 1 peg = 3
 Enter FR  exit point = TSN 1
 TSN 1 ----------------------------->
       <----------------------------- SACK 4
 CA TSN >= exit point => exit FR
 TSN 5 ----------------> X
 TSN 6 ----------------> X

 At what point is the FR exit point greater than the highest
 outstanding TSN in a SACK?  That is, at what point would pegging
 up to the exit point include more TSNs that provided in the SACK?

>    Assumed also the following loss pattern. no new packets:
> 
>    LSLSLSLL
> 
>    Then Â again this makes SCTP fast retransmit the first lost packet
>    setting FR exit point to last TSN.
> 
>    After SACK of the retransmitted packet we have:
> 
>    AALSLSLL and the first loss (TSN packet number 3) will now have 3
>    miscounts and will be allowed to be fast retransmitted following the
>    rule,
> 
>    the same again for the third L. But the ones lost in the end do not get
>    any miscounts even if they same rule that apply to the first Lost
>    packets should
> 
>    apply to the last lost ones.

 TSN 1 ----------------> X
 TSN 2 ----------------------------->
       <----------------------------- SACK 0, 2-2
 TSN 1 peg = 1
 TSN 3 ----------------> X
 TSN 4 ----------------------------->
       <----------------------------- SACK 0, 2-2, 4-4
 TSN 1 peg = 2, TSN 3 peg = 1
 TSN 5 ----------------> X
 TSN 6 ----------------------------->
       <----------------------------- SACK 0, 2-2, 4-4, 6-6
 TSN 1 peg = 3, TSN 3 peg = 2, TSN 5 peg = 1
 Enter FR  exit point = TSN 5
 TSN 1 ----------------------------->
       <----------------------------- SACK 2, 4-4, 6-6
 TSN 3 peg = 3, TSN 5 peg = 2
 TSN 3 ----------------------------->
       <----------------------------- SACK 4, 6-6
 TSN 5 peg = 3
 TSN 5 ----------------------------->
       <----------------------------- SACK 6
 CA TSN >= exit point => exit FR
 TSN 7 ----------------> X
 TSN 8 ----------------> X

 At what point is the FR exit point greater than the highest
 outstanding TSN in a SACK?  That is, at what point would pegging
 up to the exit point include more TSNs that provided in the SACK?

>    I have written this a little hastely. I hope this can give the picture.
> 
>    Perhaps with this explanation also the information in slide 30-33 of
>    the following now makes sense:
> 
>    [1]http://www.ietf.org/proceedings/90/slides/slides-90-tsvwg-16.pdf,
> 
>    BR, Karen
> 
>    From: Randall Stewart [mailto:[2][email-masked]]
>    Sent: 23. oktober 2014 16:48
>    To: Karen Elisabeth Egede Nielsen
>    Cc: Michael Tuexen; [3][email-masked]
>    Karen:
>    Note I dropped tsvwg due to the fact that this email would get locked
>    up and held/bounced since [4][email-masked] is
>    not a subscriber to tsvwg
> 
>    I like Brian am a bit confused by your request in that I don't see how
>    this could happen.
> 
>    SCTP has the ability to identify 300+ sack gaps. I think this
>    is why the wording is the way it is. We could not conceive of a situation
>    where the SACK would come back (outside of reneging) and not list what
>    was previously there. Especially if the SACK is a response to the
>    fast-retransmit.
> 
>    In theory it *should* contain all the gap acks that were present up to
>    your "fast recovery point".
> 
>    Like Brian I too don't remember ever making a "fast recovery point" in
>    SCTP.
> 
>    Now I guess you could get a situation where the fast retransmit passed the
>    last TSN sent and the sender is thus responding.. but thats still
>    pending data that *may* still arrive.
> 
>    So unless the sender purposely stops gap-acking previously gap-acked
>    packets (reneged) then I don't see how this could happen.
> 
>    Could you explain how you see this could happen .. it could be I am
>    missing something that is clear to you..
> 
>    Thanks
> 

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Thu, 23 Oct 2014 21:35:06 +0200

Hi,

The TSN are sent at the same time, not sequentially.

Assume they chunks are  ~0,6*MTU, then there would be no CWND constrains and
no HB burst constraints.
This I am saying just to avoid to get into such discussions or bundling
discussions for that matter.

BR, Karen

> From: Brian F. G. Bidulock [mailto:[email-masked]]
> Sent: 23. oktober 2014 21:12
> Cc: Randall Stewart; Michael Tuexen; [email-masked]
>        <----------------------------- SACK 0, 2-2  TSN 1 peg = 1  TSN
> 3 -------------------
> ---------->
>        <----------------------------- SACK 0, 2-3  TSN 1 peg = 2  TSN
> 4 -------------------
> ---------->
>        <----------------------------- SACK 0, 2-4  TSN 1 peg = 3  Enter FR
> exit point =
> TSN 1  TSN 1 ----------------------------->
>        <----------------------------- SACK 4  CA TSN >= exit point => exit
> FR  TSN 5 ---
> -------------> X  TSN 6 ----------------> X
>
>  At what point is the FR exit point greater than the highest  outstanding
> TSN
> in a SACK?  That is, at what point would pegging  up to the exit point
> include
> more TSNs that provided in the SACK?
>
>
> >    Then   again this makes SCTP fast retransmit the first lost packet
>        <----------------------------- SACK 0, 2-2  TSN 1 peg = 1  TSN
> 3 ---------------->
> X  TSN 4 ----------------------------->
>        <----------------------------- SACK 0, 2-2, 4-4  TSN 1 peg = 2, TSN
> 3 peg = 1
> TSN 5 ----------------> X  TSN 6 ----------------------------->
>        <----------------------------- SACK 0, 2-2, 4-4, 6-6  TSN 1 peg =
> 3, TSN 3 peg =
> 2, TSN 5 peg = 1  Enter FR  exit point = TSN 5  TSN
> 1 ----------------------------->
>        <----------------------------- SACK 2, 4-4, 6-6  TSN 3 peg = 3, TSN
> 5 peg = 2
>        <----------------------------- SACK 4, 6-6  TSN 5 peg = 3  TSN
> 5 -------------------
> ---------->
>        <----------------------------- SACK 6  CA TSN >= exit point => exit
> FR  TSN 7 ---
> -------------> X  TSN 8 ----------------> X
>
>  At what point is the FR exit point greater than the highest  outstanding
> TSN
> in a SACK?  That is, at what point would pegging  up to the exit point
> include
> more TSNs that provided in the SACK?
>
>
> >    Now I guess you could get a situation where the fast retransmit
> > passed
> the

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 23 Oct 2014 14:21:34 -0600

Karen,

Karen Elisabeth Egede Nielsen wrote:    (Thu, 23 Oct 2014 21:35:06)
> discussions for that matter.

Did you mean like this:

 TSN 1 ---> X
 TSN 2 ---------\
 TSN 3 --------\ \
 TSN 4 -------\ \ \
 TSN 5 ---> X  \ \ \
 TSN 6 ---> X   \ \ \
                 \ \ \-------------->
       <----------\-\---------------- SACK 0, 2-2
 TSN 1 peg = 1     \ \-------------->
       <------------\---------------- SACK 0, 2-3
 TSN 1 peg = 2       \-------------->
       <----------------------------- SACK 0, 2-4
 TSN 1 peg = 3
 Enter FR exit point = TSN 1
 TSN 1 ----------------------------->
       <----------------------------- SACK 4
 CA TSN >= exit point => exit FR

And like this:

 TSN 1 ---> X
 TSN 2 ------------\
 TSN 3 ---> X       \
 TSN 4 ------------\ \
 TSN 5 ---> X       \ \
 TSN 6 ------------\ \ \
 TSN 7 ---> X       \ \ \
 TSN 8 ---> X        \ \ \
                      \ \ \-------->
       <---------------\-\---------- SACK 0, 2-2
 TSN 1 peg = 1          \ \-------->
       <-----------------\---------- SACK 0, 2-2, 4-4
 TSN 1 peg = 2, 3 peg = 1 \-------->
       <---------------------------- SACK 0, 2-2, 4-4, 6-6
 TSN 1 peg = 3, TSN 3 peg = 2, TSN 5 peg = 1
 Enter FR  exit point = TSN 5
 TSN 1 ----------------------------->
       <----------------------------- SACK 2, 4-4, 6-6
 TSN 3 peg = 3, TSN 5 peg = 2
 TSN 3 ----------------------------->
       <----------------------------- SACK 4, 6-6
 TSN 5 peg = 3
 TSN 5 ----------------------------->
       <----------------------------- SACK 6
 CA TSN >= exit point => exit FR

The result is the same.

Again, at what point is the FR exit point greater than the
highest outstanding TSN in a SACK?

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Fri, 24 Oct 2014 08:23:48 +0200

Hi Randy,

I am ok with this not being in RFC4960,

but I don’t really buy the argument about not knowing or the argument about
the delays.

This is generally true for ALL mis-indication counts.

Let’s consider this discussion as closed, we/I will get what we/I want from
RFC6675 additions to SCTP.

However, getting these arguments from you, then out of curiosity, I wonder
if you have the time and

the strength to explain the motivation for the existing text of RFC4960
saying:

missing in the SACK.

Thanks J

BR, Karen

*From:* Randall Stewart [mailto:[email-masked]]
*Sent:* 23. oktober 2014 21:47
*To:* Karen Elisabeth Egede Nielsen
*Cc:* [email-masked]; Michael Tuexen; [email-masked]
*Subject:* Re: [tsvwg] SCTP mis indication counting during Fast Recovery

Karen:

Assume the TSN’s (same sequence) are sent 100ms apart..

Sure we can construct a condition where they all go at once, but I can

also construct a condition where they do not. You can’t increment FR counts

on things you have not been told.

You *can* implement one of the alternative schemes that folks have been
floating around

to recover from tail drops.. but that was never in the spec and won’t ever
be in 4960.

It may be in some other RFC, thats fine, but 4960 never had the changes in
it (however good

or bad they may be) you are asking for..

R

On Oct 23, 2014, at 12:35 PM, Karen Elisabeth Egede Nielsen <
[email-masked]> wrote:

Hi,

BR, Karen

-----Original Message-----
From: Brian F. G. Bidulock [mailto:[email-masked]
<[email-masked]>]

Karen,

  HI,

  LSSSLL

  AAAALL

  LSLSLSLL

  the same again for the third L. But the ones lost in the end do not
get
  any miscounts even if they same rule that apply to the first Lost
  packets should

  I have written this a little hastely. I hope this can give the
picture.

  BR, Karen

  Subject: Re: [tsvwg] SCTP mis indication counting during Fast
Recovery

  Karen:

  I like Brian am a bit confused by your request in that I don't see
how
  this could happen.

  SCTP has the ability to identify 300+ sack gaps. I think this
  is why the wording is the way it is. We could not conceive of a
situation
  where the SACK would come back (outside of reneging) and not list
what
  was previously there. Especially if the SACK is a response to the
  fast-retransmit.

  In theory it *should* contain all the gap acks that were present up
to
  your "fast recovery point".

  Like Brian I too don't remember ever making a "fast recovery point"
in
  SCTP.

the

  Thanks

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

--------

Randall Stewart

[email-masked]

803-317-4952

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Fri, 24 Oct 2014 08:31:30 +0200

HI,

Please see below.

> > Assume they chunks are  ~0,6*MTU, then there would be no CWND
> > constrains and no HB burst constraints.
> > This I am saying just to avoid to get into such discussions or
> > bundling discussions for that matter.
> >
> > BR, Karen
>
> Did you mean like this:
>
>  TSN 1 ---> X
>  TSN 2 ---------\
>  TSN 3 --------\ \
>  TSN 4 -------\ \ \
>  TSN 5 ---> X  \ \ \
>  TSN 6 ---> X   \ \ \
>                  \ \ \-------------->
>        <----------------------------- SACK 0, 2-4  TSN 1 peg = 3  Enter
FR exit point =
>        <----------------------------- SACK 4  CA TSN >= exit point =>
exit FR
[Karen] No I don't mean like this.

I mean

TSN 1 ---> X
 TSN 2 ---------\
 TSN 3 --------\ \
 TSN 4 -------\ \ \
 TSN 5 ---> X  \ \ \
 TSN 6 ---> X   \ \ \
                 \ \ \-------------->
exit point = TSN 6 !!

The FR exit point is TSN6 not TSN1.

BR, Karen

>        <-----------------\---------- SACK 0, 2-2, 4-4  TSN 1 peg = 2, 3
peg = 1 \--------
> >
>        <---------------------------- SACK 0, 2-2, 4-4, 6-6  TSN 1 peg =
3, TSN 3 peg =
> 2, TSN 5 peg = 1  Enter FR  exit point = TSN 5  TSN 1
----------------------------->
>        <----------------------------- SACK 2, 4-4, 6-6  TSN 3 peg = 3,
TSN 5 peg = 2
>        <----------------------------- SACK 4, 6-6  TSN 5 peg = 3  TSN 5
-------------------
> ---------->
>        <----------------------------- SACK 6  CA TSN >= exit point =>
exit FR
>
>
> The result is the same.
>
> Again, at what point is the FR exit point greater than the highest
outstanding
> TSN in a SACK?

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Fri, 24 Oct 2014 10:11:05 +0200

Hi Brian, Randy, Michael

Michael, Randy: I dont't know if you have the time to look at the
"tiresome" questions below .. :-)

Thanks.

> > [Karen] I am not sure that I understand how to implement SCTP FR
> > without tracking the exit point.
>
point in
> TCP was to limit the number of packets in flight (because TCP cwnd
tracks
> bytes not packets).  Because SCTP cwnd indirectly limits packets in
flight all
> day long, the TCP FR cwnd adjustments are not necessary, thus in SCTP
> tracking the FR exit point is not necessary.
>
[Karen]

I this really true ?  (I thought this was more about inflating/deflating
and all that TCP CC business)

You would need to maintain a FR exit point to control
the CWND handling during and after FR. Also in order to know when to take
or not
take yet a CWND/2 hit (i.e., exit and re-enter of FR).

Here one may give the side note that the RFC4960 spec is not clear on the
boundary condition when FR is existed and
CWND ramp up should commerce.

TCP RCF 5681 is more clear in its formulation that:

*CWND should not be ramped up during FR
*At exit of FR, TCP is in congestion avoidance state
*The slow start algorithm is used when cwnd < ssthresh, while the
   congestion avoidance algorithm is used when cwnd > ssthresh.  When
   cwnd and ssthresh are equal, the sender may use either slow start or
   congestion avoidance.

In RFC4960 there is no clear specification of the CC operation at exit of
FR and it thus is not
clear whether the prescription to be in CA at exit of FR is assumed to be
inherited from TCP or not.

It would be interesting to know if it is meant to be left to
implementations to start in slow start or in CA after FR exit ?
(I do realize that the CC state will transfer to CA as soon as the CWND
gest ramped up).

Still there should be no question that during FR, the CWND and
partial_bytes_ack is not increased.
Correct ?

Thanks

BR, Karen

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 24 Oct 2014 07:47:15 -0600

Karen,

Karen Elisabeth Egede Nielsen wrote:               (Fri, 24 Oct 2014 08:31:30)
> [Karen] No I don't mean like this.

Ah, you misread 7.2.4:

   6)  If not in Fast Recovery, enter Fast Recovery and mark the highest
       outstanding TSN as the Fast Recovery exit point.  When a SACK
       acknowledges all TSNs up to and including this exit point, Fast
       Recovery is exited.  While in Fast Recovery, the ssthresh and
       cwnd SHOULD NOT change for any destinations due to a subsequent
       Fast Recovery event (i.e., one SHOULD NOT reduce the cwnd further
       due to a subsequent Fast Retransmit).

TSN 5 and 6 are "in-flight" and not "outstanding".  A TSN is not
"outstanding" until a T3-timeout has occurred or it has been
gap-acked as missing.  So, the correct exit point is TSN 1.

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Fri, 24 Oct 2014 11:55:33 -0600

Karen,

Karen Elisabeth Egede Nielsen wrote:            (Fri, 24 Oct 2014 10:11:05)
> > The first three sentences describe how to implement SCTP FR
> > without tracking the exit point.  As I understand it, The
> > purpose of the exit point in TCP was to limit the number of
> > packets in flight (because TCP cwnd tracks bytes not packets).
> > Because SCTP cwnd indirectly limits packets in flight all day
> > long, the TCP FR cwnd adjustments are not necessary, thus in
> > SCTP tracking the FR exit point is not necessary.
> >
> [Karen]
> 
> I this really true ?  (I thought this was more about
> inflating/deflating and all that TCP CC business)

Fast recovery was introduced in draft-ieft-tsvwg-sctpimpguide-08.txt
published Feb 2003 as an optional procedure to reduce the halving of
cwnd.  If you do not implement fast recovery, such as described in
the last paragraph of 7.2.4, the rest of the document (as was the
case in RFC 2960 before), you are required to half cwnd on each and
every retransmission cycle: FR or not.  So, in your LSLSLS... case,
by not implementing fast recovery, the congestion control procedures
in 7.2.3 would have be performed on each of the retransmissions of
TSN 1, TSN 3 and TSN 5.  When implementing fast recovery, you are
permitted to skip 7.2.3 on retransmission of TSN 3 and TSN 5.  Fast
recovery essentially permits you to treat loss reports for TSN 1,
TSN 3 and TSN 5 as a single loss from the standpoint of 7.2.3
congestion avoidance.  (This is also why the exit point is TSN 5
and not TSN 7 or 8.)

> You would need to maintain a FR exit point to control the CWND
> handling during and after FR. Also in order to know when to take
> or not take yet a CWND/2 hit (i.e., exit and re-enter of FR).

No, you need to maintain an FR exit point to avoid a CWND/2 hit.
By not maintaining one you take a hit on each retransmitted TSN.

> Here one may give the side note that the RFC4960 spec is not clear
> on the boundary condition when FR is existed and CWND ramp up
> should commerce.
> 
> In RFC4960 there is no clear specification of the CC operation at
> exit of FR and it thus is not clear whether the prescription to be
> in CA at exit of FR is assumed to be inherited from TCP or not.
>
> (I do realize that the CC state will transfer to CA as soon as the
> CWND gest ramped up).

Again, the only thing that fast recovery allows you to do is to not
take some 7.2.3 hits.  That is, SCTP is always in congestion
avoidance.  All of the rules for slow start still apply when in fast
recovery; however, because the ack clock is still running, slow
start should not be necessary unless a T3 timeout ruins the day.
Perhaps it should have been noted that fast recovery is exited when
a T3 timeout occurs, but I think that is obvious if not explicitly
stated somewhere.
> partial_bytes_ack is not increased.

Maybe not: Second last paragraph of RFC 4960/7.2.4:

   Note: Before the above adjustments, if the received SACK also
   acknowledges new DATA chunks and advances the Cumulative TSN Ack
   Point, the cwnd adjustment rules defined in Section 7.2.1 and Section
   7.2.2 must be applied first.

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Mon, 27 Oct 2014 10:13:19 +0100

Hi Randy,

Thanks.

What are your thoughts on the following:

-          ??

P.S.: Our (Ericsson) SCTP goes to slow start and then to CA after exit  of
FR. But this is not what TCP (RFC%681) says on the other hand it

is a tiny (tiny!) step towards CubicJ

BR, Karen

*From:* Randall Stewart [mailto:[email-masked]]
*Sent:* 24. oktober 2014 20:27
*To:* Karen Elisabeth Egede Nielsen
*Cc:* [email-masked]; Michael Tuexen; tsvwg WG
*Subject:* Re: [tsvwg] SCTP mis indication counting during Fast Recovery

On Oct 24, 2014, at 1:11 AM, Karen Elisabeth Egede Nielsen <
[email-masked]> wrote:

Hi Brian, Randy, Michael

Thanks.

point in

tracks

flight all

[Karen]

Right

While you are recovering a FastRetransmit.. your not going to increase PBA
or the CWND..

R

Thanks

BR, Karen

--
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

--------

Randall Stewart

[email-masked]

803-317-4952

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Mon, 27 Oct 2014 10:42:33 +0100

*Hi Randy,*

*Thanks a lot for the willingness to continue this discussion !!*

*Please see below.*

*From:* Randall Stewart [mailto:[email-masked]]
*Sent:* 24. oktober 2014 20:24
*To:* Karen Elisabeth Egede Nielsen
*Cc:* [email-masked]; Michael Tuexen
*Subject:* Re: [tsvwg] SCTP mis indication counting during Fast Recovery

Karen:

I think the reason is because you only *know* the receivers perspective

based on the SACK.

*[Karen] This I think is the case for Highest Newly Ack’ed part of the
algorithm.*

*But this part*

(#) If an endpoint is in Fast

   Recovery and a SACK arrives that advances the Cumulative TSN Ack

   Point, the miss indications are incremented for all TSNs reported

   missing in the SACK.

*which is NOT stipulating new SACK information, necessarily addresses the
senders perspective of having retransmitted the TSN now newly cumulatively
Ack’ed.*

*The motivation for the increase of the mis indication count of the
non-SACK’ed TSNs up  until the highest’es SACK indeed is that this TSN,
i.e., the newly cum Ack’ed TSN, have been sent and received after the
non-SACK’ed TSNs up  until the highest SACK.*

*This must be the only reasonable way to interpret this sentence given we
already have the Highest Newly Ack’ed algorithm. *

*Correct ?*

Sure you may have sent two tailing TSN’s in a packet

at the same time (your case I think) as you sent the other packets.. <or>
you may

have *just* sent the trailing two TSN’s..

*[Karen] Yes you MAY just have sent them. Still you send them before
entering FR (!) *

*(My assumption is for increase of mis indication count only up to and
including FR Exit point).*

you don’t know if the receiver has had time

to get them or not..

*[Karen] Well at least you have retransmitted a lost TSN which now gets
cumack’ed,*

*and this retransmitted TSN was sent after the trailing TSNs.*

*This is the same argument used for the non-SACK’ed TSNs up  until the
highest’es SACK*

The idea is that I can only base my incrementing of the gap’s based on what
is in the SACK.. not

based on what I sent.

*[Karen] Then you should not have the prescription (#) in the first place.
Should you ?*

*Either way one is looking at an Errata – or where did I go wrong ?*

*BR, Karen*

R

On Oct 23, 2014, at 11:23 PM, Karen Elisabeth Egede Nielsen <
[email-masked]> wrote:

discussion as closed, we/I will get what we/I want from RFC6675 additions
to SCTP.

arrives that advances

--------

Randall Stewart

[email-masked]

803-317-4952

---

From: Randall Stewart <[email-masked]>
Date: Thu, 23 Oct 2014 12:47:02 -0700

Karen:

You *can* implement one of the alternative schemes that folks have been floating around
to recover from tail drops.. but that was never in the spec and won’t ever be in 4960.
It may be in some other RFC, thats fine, but 4960 never had the changes in it (however good
or bad they may be) you are asking for..

R

--------
Randall Stewart
[email-masked]
803-317-4952

---

From: Randall Stewart <[email-masked]>
Date: Fri, 24 Oct 2014 11:26:32 -0700

Right

While you are recovering a FastRetransmit.. your not going to increase PBA or the CWND..

R

--------
Randall Stewart
[email-masked]
803-317-4952