Subject: Re: [rmcat] Still looking for reviews of draft-ietf-rmcat-nada

From: "Flohr, Julius" <[email-masked]>
Date: Mon, 3 Jul 2017 10:40:36 +0000

Hi everyone,

I read the document and I don’t have any comments.
I used an earlier version of the document (I think it was 01 or 02) to successfully implement the algorithm described.
I think it’s good to go.

Kind regards,

Julius
 
> On 30. Jun 2017, at 12:42, Anna Brunstrom <[email-masked]> wrote:
> 
> Dear all,
> 
> The WGLC for NADA ended some time ago without any wg feedback. We still need a few reviews in order to progress the document. Note that even if you have no specific comments, but have read the document and find it ready for publication this is useful feedback.
> Thanks for your help,
> Anna
> 
> On 2017-04-05 11:30, Anna Brunstrom wrote:
>> This email announces a RMCAT Working Group Last Call (WGLC) on:
>> 
>> NADA: A Unified Congestion Control Scheme for Real-Time Media
>> draft-ietf-rmcat-nada
>> 
>> Due to Easter, this WGLC will run for 3 weeks, ending at midnight US Eastern Time on Tuesday, April 25.  Comments should preferably be sent to the [email-masked] list, although purely editorial comments may be sent directly to the authors at [email-masked] - if doing so, please cc: the WG chairs at [email-masked].
>> 
>> We are looking for at least a couple of reviews during WGLC, so please help review the document. 
>> Thanks,
>> Anna, Colin & Martin
>> 
> 

=====================================
Julius Flohr, M.Sc.
University of Duisburg-Essen
Computer Networking Technology Group

Room SL-408
Schützenbahn 70
D-45326 Essen/Germany

E-Mail:  [email-masked]
Phone: +49-201-183-7667
Fax:     +49-201-183-7673
=====================================

---

From: Ingemar Johansson S <[email-masked]>
Date: Tue, 4 Jul 2017 11:50:42 +0000

Hi

I have now read through draft-ietf-rmcat-nada-04 . I see that it differs from an earlier version in that there is more pseudo code added, which I believe simplifies the reading.
A few remarks :
Page 8, computation of d_tilde : I mentioned it before and it is also stated in the scream draft. I fear that the non-linear warping can be quite prone to create self-inflicted congestion and I recall that examples of this has been shown, and it was also dicussed  ?. If you agree with this, then it can be good to elaborate a bit around these risks in the draft.

Section 5.1.2 : Estimation of p_mark : I am not convinced that you will get a very meaningful metric here with a “classic” ECN marking behavior. On the one hand there can be e.g. RED based  ECN marking (and PIE) that increases the ECN mark ratio with the amount of ‘overload’, on the other hand there can be AQM algorithms that are more conservative in their ECN  marking behavior. L4S marking is a different story as L4S is here assumed to give a mark probability that scales with the link overload.

Regards
/Ingemar

From: Anna Brunstrom [mailto:[email-masked]]
Sent: den 30 juni 2017 12:43
To: [email-masked]
Subject: [rmcat] Still looking for reviews of draft-ietf-rmcat-nada

Dear all,

Thanks for your help,
Anna

NADA: A Unified Congestion Control Scheme for Real-Time Media
draft-ietf-rmcat-nada<https://datatracker.ietf.org/doc/draft-ietf-rmcat-nada/>

Due to Easter, this WGLC will run for 3 weeks, ending at midnight US Eastern Time on Tuesday, April 25.  Comments should preferably be sent to the [email-masked]<mailto:[email-masked]> list, although purely editorial comments may be sent directly to the authors at [email-masked]<mailto:[email-masked]> - if doing so, please cc: the WG chairs at [email-masked]<mailto:[email-masked]>.

Thanks,
Anna, Colin & Martin

---

From: "Flohr, Julius" <[email-masked]>
Date: Tue, 18 Jul 2017 08:59:02 +0000

> On 4. Jul 2017, at 13:50, Ingemar Johansson S <[email-masked]> wrote:
> 

Hi,

I have updated my simulation model to the latest version of the draft. This is definitely true and the recent changes have made the situation worse, especially for links with small bandwidth and long queues.

Regards,

Julius

Room SL-408
Schützenbahn 70
D-45326 Essen/Germany

---

From: Safiqul Islam <[email-masked]>
Date: Tue, 18 Jul 2017 09:04:43 +0000

Hi,

I have read through the draft. It is overall in a very good shape.

Regards,
/Safiqul

On 30 Jun 2017, at 12:42, Anna Brunstrom <[email-masked]<mailto:[email-masked]>> wrote:

Dear all,

Thanks for your help,
Anna

Thanks,
Anna, Colin & Martin

---

From: Anna Brunstrom <[email-masked]>
Date: Wed, 19 Jul 2017 00:18:52 +0200

> I have updated my simulation model to the latest version of the draft. This is definitely true and the recent changes have made the situation worse, especially for links with small bandwidth and lo…

Hi Julius,

Thanks for trying out the latest version of the draft! I am curious as 
to how different your results were between the two versions?

Thanks,
Anna

---

From: Sergio Mena <[email-masked]>
Date: Wed, 19 Jul 2017 09:01:28 +0200

Hello,

We agree with Ingemar's point, and we will address it in the version we 
are preparing: we will include a paragraph discussing the risk of being 
stuck on loss mode, due to the warping (which can be tuned by parameter 
LAMBDA).

As for Julius's tests, we thank Julius for the effort put into it, and 
would like to see how his results compare to our latest ones (see Slides 
presented by Xiaoqing in April's RMCAT interim).

Thanks,

Sergio

On 19/07/17 00:18, Anna Brunstrom wrote:
>>> On 4. Jul 2017, at 13:50, Ingemar Johansson S 
>>> <[email-masked]> wrote:
>>>
>>> Page 8, computation of d_tilde : I mentioned it before and it is 
>>> also stated in the scream draft. I fear that the non-linear warping 
>>> can be quite prone to create self-inflicted congestion and I recall 
>>> that examples of this has been shown, and it was also dicussed  ?. 
>>> If you agree with this, then it can be good to elaborate a bit 
>>> around these risks in the draft.
>> Hi,
>>
>> I have updated my simulation model to the latest version of the 
>> draft. This is definitely true and the recent changes have made the 
>> situation worse, especially for links with small bandwidth and long 
>> queues.
>
> Hi Julius,
>

---

From: "Flohr, Julius" <[email-masked]>
Date: Wed, 19 Jul 2017 11:19:24 +0000

Hi,

> On 19. Jul 2017, at 09:01, Sergio Mena <[email-masked]> wrote:
> 
> 
> As for Julius's tests, we thank Julius for the effort put into it, and would like to see how his results compare to our latest ones (see Slides presented by Xiaoqing in April's RMCAT interim).
> 

I haven’t done too much testing, but as far as I can tell there is an issue when the bottleneck bandwidth is smaller than RMAX: 
If you have a competition scenario between NADA and a loss based flow, it seems that nada gets stuck in the competitive mode because itself creates queueing delay that is larger than QTH. 

I actually made a little mistake when I said the problem has gotten worse compared to the last version of the draft. The issue actually has been introduced in version 03 with the introduction of TEXPLOSS. Before, the draft was very unclear and stated that one should warp the congestion signal IFF packet loss is present, but without stating a time span.

Therefore the implementation just used the knowledge it had from the recent observation window LOGWIN (500 ms). That worked really well because it helped the algorithm to return to the normal mode of operation more easily, because after that timespan it would stop warping regardless whether there is lots of queueing delay present or not.

I don’t know if this is an issue with my implementation or if there is an actual issue with the algorithm, but this is what I have observed.

I’ll contact Xiaoqing and Sergio directly so we can work together on this issue and post our findings to this mailing list directly.

Regards,

Julius

Room SL-408
Schützenbahn 70
D-45326 Essen/Germany

---

From: "Xiaoqing Zhu (xiaoqzhu)" <[email-masked]>
Date: Thu, 20 Jul 2017 17:48:11 +0000

Catching up on this thread. 

Thanks much to Julius, Ingmar, and Safiqul for reviewing this draft and providing your input. 

Regarding Ingmar's comments:

* Page 8, computation of d_tilde and the risk of being locked in the loss-based mode due to non-linear warping:  fully agree that this is a limitation of the algorithm we should acknowledge in the draft.  In the meanwhile would like to work with Julius to look into specific example failure cases, and hopefully derive further insights on how to better tune the algorithm parameters.  Will update both our ns3-rmcat implementation and NADA draft after that investigation. 

* Section 5.1.2 : Estimation of p_mark : as I understand your comments point to the fact that different ECN marking behaviors may occur and some (e.g., L4S) may not interact well with NADA.  It is true that we have not tested NADA extensively against different variants of ECN markings --- the one's we've tried before are RED (marking instead of dropping).  One way to help clarify that will be to point to our Appendix A.2 as the specific ECN marking behavior we expect, in Sec. 5.1.2.  Do you think that will address your concern?   

And, thanks to Julius for sharing with us issues you've observed in your implementation efforts.  We've set up a separate email thread to meet up remotely and go over that in greater technical details. 

Question back to the Chairs:  our plan now is to update both our draft (to address both sets of Ingmar's comments) and corresponding ns3 open-source implementation, within a month.  Procedure-wise, are we expecting the updated version (-05) to go through WGLC again at that point?  Just trying to understand what to expect next. 

Thanks,
Xiaoqing

________________________________________
From: Flohr, Julius <[email-masked]>
Sent: Wednesday, July 19, 2017 6:19 AM
To: Sergio Mena de la Cruz (semena)
Cc: Anna Brunstrom; Ingemar Johansson S; [email-masked]; [email-masked]
Subject: Re: [rmcat] Still looking for reviews of draft-ietf-rmcat-nada

Hi,

Regards,

Julius

Room SL-408
Schützenbahn 70
D-45326 Essen/Germany

---

From: Ingemar Johansson S <[email-masked]>
Date: Thu, 20 Jul 2017 19:55:15 +0000

Hi Xiaoqing + others

To your ECN question... This is tricky, if I understand things right you tie NADA to a specific ECN marking algorithm. For instance CoDel will display a completely different ECN marking behavior. The snag is that with classic ECN marking like this you can only indicate the existence of congestion. It is AFAIK possible to accurately indicate the degree of congestion. 
L4S is a different story as it allows to indicate the degree of congestion.

/Ingemar 

> -----Original Message-----
> From: Xiaoqing Zhu (xiaoqzhu) [mailto:[email-masked]]
> Sent: den 20 juli 2017 19:48
> To: Flohr, Julius <[email-masked]>; Sergio Mena de la Cruz (semena)
> <[email-masked]>
> Cc: Anna Brunstrom <[email-masked]>; Ingemar Johansson S
> <[email-masked]>; [email-masked]; draft-ietf-rmcat-
> [email-masked]
> * Page 8, computation of d_tilde and the risk of being locked in the loss-
> based mode due to non-linear warping:  fully agree that this is a limitation of
> the algorithm we should acknowledge in the draft.  In the meanwhile would
> like to work with Julius to look into specific example failure cases, and
> hopefully derive further insights on how to better tune the algorithm
> parameters.  Will update both our ns3-rmcat implementation and NADA draft
> after that investigation.
> 
> * Section 5.1.2 : Estimation of p_mark : as I understand your comments point
> to the fact that different ECN marking behaviors may occur and some (e.g.,
> L4S) may not interact well with NADA.  It is true that we have not tested
> NADA extensively against different variants of ECN markings --- the one's
> we've tried before are RED (marking instead of dropping).  One way to help
> clarify that will be to point to our Appendix A.2 as the specific ECN marking
> behavior we expect, in Sec. 5.1.2.  Do you think that will address your
> concern?
> 
> And, thanks to Julius for sharing with us issues you've observed in your
> implementation efforts.  We've set up a separate email thread to meet up
> remotely and go over that in greater technical details.
> 
> Question back to the Chairs:  our plan now is to update both our draft (to
> address both sets of Ingmar's comments) and corresponding ns3 open-
> source implementation, within a month.  Procedure-wise, are we expecting
> the updated version (-05) to go through WGLC again at that point?  Just trying
> to understand what to expect next.
> 
> Thanks,
> Xiaoqing
> 
> 
> 
> > As for Julius's tests, we thank Julius for the effort put into it, and would like
> to see how his results compare to our latest ones (see Slides presented by
> Xiaoqing in April's RMCAT interim).
> >
> 
> I haven't done too much testing, but as far as I can tell there is an issue when
> the bottleneck bandwidth is smaller than RMAX:
> If you have a competition scenario between NADA and a loss based flow, it
> seems that nada gets stuck in the competitive mode because itself creates
> queueing delay that is larger than QTH.
> 
> I actually made a little mistake when I said the problem has gotten worse
> compared to the last version of the draft. The issue actually has been
> introduced in version 03 with the introduction of TEXPLOSS. Before, the draft
> was very unclear and stated that one should warp the congestion signal IFF
> packet loss is present, but without stating a time span.
> 
> Therefore the implementation just used the knowledge it had from the
> recent observation window LOGWIN (500 ms). That worked really well
> because it helped the algorithm to return to the normal mode of operation
> more easily, because after that timespan it would stop warping regardless
> whether there is lots of queueing delay present or not.
> 
> I don't know if this is an issue with my implementation or if there is an actual
> issue with the algorithm, but this is what I have observed.
> 
> I'll contact Xiaoqing and Sergio directly so we can work together on this issue
> and post our findings to this mailing list directly.
> 
> Regards,
> 
> Julius
> 
>

---

From: Ingemar Johansson S <[email-masked]>
Date: Tue, 25 Jul 2017 07:22:51 +0000

Hi
An errata...

Referring to "classic" AQMs like CoDel
"It is AFAIK possible to accurately indicate the degree of congestion" should read
"It is AFAIK not possible to accurately indicate the degree of congestion"

/Ingemar

> From: Ingemar Johansson S
> Sent: den 20 juli 2017 21:55
> To: 'Xiaoqing Zhu (xiaoqzhu)' <[email-masked]>; Flohr, Julius
> <[email-masked]>; Sergio Mena de la Cruz (semena)
> <[email-masked]>
> Cc: Anna Brunstrom <[email-masked]>; [email-masked]; draft-ietf-
> [email-masked]
> Subject: RE: [rmcat] Still looking for reviews of draft-ietf-rmcat-nada
> 
> Hi Xiaoqing + others
> 
> To your ECN question... This is tricky, if I understand things right you tie NADA
> to a specific ECN marking algorithm. For instance CoDel will display a
> completely different ECN marking behavior. The snag is that with classic ECN
> marking like this you can only indicate the existence of congestion. It is AFAIK
> possible to accurately indicate the degree of congestion.
> > To: Flohr, Julius <[email-masked]>; Sergio Mena de la Cruz
> > (semena) <[email-masked]>
> > * Page 8, computation of d_tilde and the risk of being locked in the
> > loss- based mode due to non-linear warping:  fully agree that this is
> > a limitation of the algorithm we should acknowledge in the draft.  In
> > the meanwhile would like to work with Julius to look into specific
> > example failure cases, and hopefully derive further insights on how to
> > better tune the algorithm parameters.  Will update both our ns3-rmcat
> > implementation and NADA draft after that investigation.
> >
> > * Section 5.1.2 : Estimation of p_mark : as I understand your comments
> > point to the fact that different ECN marking behaviors may occur and
> > some (e.g.,
> > L4S) may not interact well with NADA.  It is true that we have not
> > tested NADA extensively against different variants of ECN markings ---
> > the one's we've tried before are RED (marking instead of dropping).
> > One way to help clarify that will be to point to our Appendix A.2 as
> > the specific ECN marking behavior we expect, in Sec. 5.1.2.  Do you
> > think that will address your concern?
> >
> > And, thanks to Julius for sharing with us issues you've observed in
> > your implementation efforts.  We've set up a separate email thread to
> > meet up remotely and go over that in greater technical details.
> >
> > Question back to the Chairs:  our plan now is to update both our draft
> > (to address both sets of Ingmar's comments) and corresponding ns3
> > open- source implementation, within a month.  Procedure-wise, are we
> > expecting the updated version (-05) to go through WGLC again at that
> > point?  Just trying to understand what to expect next.
> >
> > Thanks,
> > Xiaoqing
> >
> >
> >
> > Cc: Anna Brunstrom; Ingemar Johansson S; [email-masked];
> > draft-ietf-rmcat- [email-masked]
> > to see how his results compare to our latest ones (see Slides
> > I haven't done too much testing, but as far as I can tell there is an
> > issue when the bottleneck bandwidth is smaller than RMAX:
> > If you have a competition scenario between NADA and a loss based flow,
> > it seems that nada gets stuck in the competitive mode because itself
> > creates queueing delay that is larger than QTH.
> >
> > I actually made a little mistake when I said the problem has gotten
> > worse compared to the last version of the draft. The issue actually
> > has been introduced in version 03 with the introduction of TEXPLOSS.
> > Before, the draft was very unclear and stated that one should warp the
> > congestion signal IFF packet loss is present, but without stating a time span.
> >
> > because it helped the algorithm to return to the normal mode of
> > operation more easily, because after that timespan it would stop
> > warping regardless whether there is lots of queueing delay present or not.
> >
> > I don't know if this is an issue with my implementation or if there is
> > an actual issue with the algorithm, but this is what I have observed.
> >
> > I'll contact Xiaoqing and Sergio directly so we can work together on
> > this issue and post our findings to this mailing list directly.
> >
> > Regards,
> >
> > Julius
> >
> >

---

From: Anna Brunstrom <[email-masked]>
Date: Wed, 26 Jul 2017 03:05:49 +0200

Hi Ingemar,

I think this is actually even more related to the feedback than the 
marking, the "classic" feedback provided by TCP does not tell you how 
many marked packets you have received. But as the RTCP feedback gives 
you per packet information I think there will be no difference from when 
you have an AQM with loss in this case?

L4S will give you a different marking, but I think that should be left 
out of scope at the moment and treating lost and marked packets the same 
in accordance with RFC 6679 is fine.

BR,
Anna (as an individual)

---

From: Anna Brunstrom <[email-masked]>
Date: Wed, 26 Jul 2017 03:20:01 +0200

Hi Xiaoqing,

Reply on the chair question inline.

> Question back to the Chairs:  our plan now is to update both our draft (to address both sets of Ingmar's comments) and corresponding ns3 open-source implementation, within a month.  Procedure-wise,…

If the update consists of clarifying the text in relation to the WGLC 
comments, there is no need for a second WGLC. We will just confirm that 
the updated draft addresses the comments. If it turns out that you make 
substantial updates to the algorithm based on your discussions with 
Julius then we may need to think about if a second WGLC is needed.

BR,
Anna