Subject: [aqm] WGLC for draft-ietf-aqm-ecn-benefits

From: Wesley Eddy <[email-masked]>
Date: Fri, 24 Apr 2015 00:37:45 -0400

To keep moving forward with the set of documents, we'd like to start
a Working Group Last Call on:
http://datatracker.ietf.org/doc/draft-ietf-aqm-ecn-benefits/

Please send any comments you have on this document to the list
by May 8th.  Other than those raised in the thread Mirja has just
opened with her review, I'm not aware of any other open issues.

Thank you to everyone that's been following up on the reviews
pledged at the last meeting!

-- 
Wes Eddy
MTI Systems

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 24 Apr 2015 11:25:22 +0200 (CEST)

> opened with her review, I'm not aware of any other open issues.

I have re-read this document as if I never read it before. Writing my 
thoughts as I read it.

The title says "benefits" but the abstract states that the document will 
talk about both benefits and potential problems. I therefore think the 
title does not reflect the abstract.

1. Introduction: I don't understand the "(potential)", doesn't transport 
always treat loss and CE marking as indications of congestion? Actually, 
the way the introduction reads today one might interpret that TCP and SCTP 
are "Internet transports" but UDP is not, because it's merely "transport".

Can we have a reference to AQM in the second paragraph? What is the 
difference between a router and a middle box?

I don't like the first sentence in the 3rd paragraph, shouldn't it say 
that the device "might mark instead of drop", and not say "as the queue 
builds"? Because if we have a cisco-style policer, there is no queue but 
instead a token-bucket rate limiter. I don't like "as the network queue 
builds" anyhow, if it should stay, it should be rephrased to "interface 
queue size increases" or something like that. What if there are several 
queues? I'd like the last part to not be included at all. Also, is the 
document really saying that ECN should be done before incuring loss for 
non-ECN traffic? The current wording can be interpreted as such.

Fourth parahraph. I have no idea what "incipient" means. I looked it up. 
Can we use the word "developing" or some other synonym?

2.2. Here we have incipient again, and isn't drop or CE marking an 
indicator of actual congestion, not just developing/incipient congestion?

How do we stop older bleaching-boxes that do not understand ECN bits and 
still bleach them, from just forwarding the packet even if CE=1? I 
understand the must requirement and it makes sense, but I don't see how 
this can be achieved with older equipment?

3.1. I would like to see text after 0.02 that says something like (1 
packet in 50) or something like that, just for clarification, or that the 
packet drop rate is 2%. It's not obvious that "the packet drop ratio is 
0.02" means these things.

My summary: I would like to see some text on the downsides of having ECN 
transport, ECN enabled routers and having middle box bleaching ECN bits, 
thus removing the congestion signal. This downside, and the operational 
problem that some old equipment just simply drop packets with ECN bits set 
should be summarized the same way as the benfits. These are the only 
drawbacks of enabling ECN that I can see, and potentially text describing 
how to detect these problems could be added/extended (it's somewhat hinted 
at currently).

Generally, I support the document, it's useful, provides good insight, and 
is valuable to the community. Thanks for writing it!

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Gorry Fairhurst <[email-masked]>
Date: Tue, 28 Apr 2015 13:37:41 +0100

It is great to see some detailed comments from you looking afresh at 
this. Please see below for initial replies.

GF/MW: We looked at both approaches, and revised to include "drawbacks" as
well, but then the list discussion seemed to favour focusing on the
"benefits", since deployment issues could change with time. I suggest we
remove the sentence about "problems" from the abstract.
 >1. Introduction: I don't understand the "(potential)", doesn't transport
always treat loss and CE marking as indications of congestion?
Actually, the way the introduction reads today one might interpret that
TCP and SCTP are "Internet transports" but UDP is not, because it's
merely "transport".
GF/MW: We need to add REF to AQM recommendations [RFC2309.bis].
GF/MW:  We'd agree, and be happy to remove "(potential)" from section 1
para 1.
 >difference between a router and a middle box?
GF/MW: This echoes what is said in the AQM Recommendations draft: "The
mechanisms described in this document may be implemented in
network devices on the path between end-points that include routers,
switches, and other network middleboxes." - The difference may be in the
eyes of the beholder - all are network devices that can deploy AQM.
instead a token-bucket rate limiter. I don't like "as the network queue
builds" anyhow, if it should stay, it should be rephrased to "interface
queue size increases" or something like that.
GF/MW: I agree policiers and other methods can track congestion state and
set a CE-mark. I'm OK with some change to "queue builds", but do not yet
understand the difference between "interface queue size increases", except
that would seem to limit only to an interface. It is OK with us to remove
"as the queue builds".
 >
 >What if there are several queues? I'd like the last part to not be
included at all.
 >
 >I suggest we add this sentence after the next para also, to balance this:
 >"An endpoint that receives a congestion mark
 > causes a congestion control reaction at the sender to reduce
 > the maximum rate permitted by the endpoint."
 >
 >Also, is the  document really saying that ECN should be done before
incurring loss for non-ECN traffic? The current wording can be
interpreted as such.
GF/MW: It enables marking before the point where a drop-tail queue would
have dropped. The ID intentionally does not describe how to implement or
to evaluate if this is done "fairly". The suggested sentence seems to be
unnecessary because it echoes what is in RFC3168, to some degree, but also
uses phrasing that can be misleading, by saying: "maximum rate permitted".
 >
 >Fourth pararaph. I have no idea what "incipient" means. I looked it up.
GF/MW: OK this came from the list discussion. This term has already been
used in early discussion of RED marking (e.g.,
ftp://ftp.ee.lbl.gov/papers/early.pdf) It was used in RFC 2481, and is
used several times in RFC 3168. I suggest we add some explanatory words
(e.g., "where congestion is beginning to be experienced")
 >indicator of actual congestion, not just developing/incipient congestion?
GF/MW: See above, one advantage of talking about "incipient" congestion is
we avoid talking about "mild" or "initial" or some other term that could
perhaps be misconstrued. It is is congestion, but reported when first
discovered.
 >this can be achieved with older equipment?
GF/MW: We don't see more options than:
* Declare them as not complying with current use
* Go and fix those that can be fixed, and live with the others until in
the future they are finally replaced.
- I'm not sure what extra there is to say about this, apart from to
verify the operation across the path (as in section 4.1).
GF/MW: The phrase:
"A network device should therefore not remark an ECT(0) or ECT(1) mark to
zero [RFC2309.bis]."
- This deployment issue is consistent with the intention of RFC 2309.bis,
but is not specifically called-out in RFC2309.bis, and this reference
needs to be removed.
 >packet in 50) or something like that, just for clarification, or that
the  packet drop rate is 2%. It's not obvious that "the packet drop
ratio is 0.02" means these things.
GF/MW: We think this change should be made.
 >My summary: I would like to see some text on the downsides of having ECN
transport, ECN enabled routers and having middle box bleaching ECN
bits, thus removing the congestion signal. This downside, and the
operational problem that some old equipment just simply drop packets
with ECN bits set should be summarized the same way as the benfits.
These are the only drawbacks of enabling ECN that I can see, and
potentially text describing how to detect these problems could be
added/extended (it's somewhat hinted at currently).
GF/MW: We could certainly add text, how about adding something like:
After:
A network device should therefore not remark an ECT(0) or ECT(1) mark to 
zero
Add:
  This can result in packets that were originally marked as
  ECN-capable being dropped by ECN-capable network devices further
  along the path. This eliminates the advantage of enabling ECN.
And after:
   Such a packet has already received ECN
   treatment in the network, and remarking it would then hide the
   congestion signal from the endpoints.
Add:
   The endpoint will then be unaware of the signalled congestion, and
   will only trigger a congestion response later if the congested
   queue drops a packet. Note that ECN-enabled
   networks devices need to drop excessive traffic [Section
   4.2.1 of RFC2309.bis] even when this is marked as ECN-capable.
   This eliminates the advantage of enabling ECN, and could slow
   down the response to congestion compared to using AQM, but is not
   expected to be worse than drop-tail performance.
 >
 >Generally, I support the document, it's useful, provides good insight,
and is valuable to the community. Thanks for writing it!

We plan another revision of this document based on the received comments.

Please let us know if we have understood correctly.

Gorry & Michael

---

From: John Leslie <[email-masked]>
Date: Tue, 28 Apr 2015 09:34:51 -0400

Gorry Fairhurst <[email-masked]> wrote:
> To: Mikael Abrahamsson <[email-masked]>, "[email-masked]" <[email-masked]>
> 
>> My summary: I would like to see some text on the downsides of having
>> ECN transport, ECN enabled routers and having middle box bleaching
>> ECN bits, thus removing the congestion signal. This downside, and the

   This text is a bit confusing -- I had to read it three times to
get (I think) your point. Perhaps:
" 
" This would negate the advantages of ECN for congestion which may
" occur later in the path, in that an ECN-capable network device would
" not be able to forward the packet with congestion signaled by "CE".

   Putting the effect first seems easier to read...

   Also, the forwarding node later in the path might have signaled
congestion via "CE" at an earlier threshhold than it would drop
packets.

>   expected to be worse than drop-tail performance.

   I'm still failing (after five readings) to extract a clear meaning. :^(

   There are several points:

- clearing a "CE" marking destroys the congestion signal. Barring some
  extraordinary programming, this would make an ECN-enabled flow
  unresponsive.

- with an unresponsive flow, the congested node _might_ reach a level
  of congestion requiring tail-drop. But it also might not. :^(

- _if_ a receiver added a way to observe this abuse (possibly by
  having the sender set an occasional "CE" mark before sending the
  packet), the responsiveness would still be degraded. Besides,
  there's really no chance that _all_ flows would ever be programmed
  to do this.

   Myself, I see no reason to add text to cover all of these: I think
text covering the first point is sufficient.

--
John Leslie <[email-masked]>

---

From: Mikael Abrahamsson <[email-masked]>
Date: Tue, 28 Apr 2015 15:49:42 +0200 (CEST)

Hi,

it's quite hard to read the below email, considering how it's quoted.

I think I need to see the new document to be able to tell if the changes 
works or not.

Also, I actually still don't understand the concept of "developing 
congestion". For me "congestion" means there is a sustained, constant, 
non-trivial buffer fill. So "incipient congestion" doesn't compute at all. 
I understand this term has been used before, but that doesn't help me 
understand the reasoning behind the wording. For me sending a congestion 
signal, means saying there is congestion. It's not saying there is 
developing congestion (whatever that might be).

I would greatly appreciate if someone could point me to text explaining 
this so I can better understand the reasoning behind this wording.

For instance from the linked early.pdf:

"The goal of the gateway is to detect incipient congestion that has 
persisted for a “long time” (several roundtrip times)."

So we have "developing congestion" that has "persisted for a long time". I 
don't understand this wording at all. If it has persisted for a long time, 
then it's not "developing" anymore, or is there some hidden meaning to 
"incipient" that I don't understand?

On Tue, 28 Apr 2015, Gorry Fairhurst wrote:

> It is great to see some detailed comments from you looking afresh at this. 
> Please see below for initial replies.
>
> A network device should therefore not remark an ECT(0) or ECT(1) mark to zero
> Add:

---

From: Vishal Misra <[email-masked]>
Date: Tue, 28 Apr 2015 10:00:23 -0400

One looks at the derivative of the queue length. If that derivative continues in the positive direction, you can proactively send a congestion signal and ask the flows to slow down before the queue size reaches “congestion levels”. It is the theory behind almost all control systems, and was the idea behind the design of the PI controller.

RED was reactive, it waited for the average queue length to rise above a threshold and that introduced too much of a delay. PI is pro-active, it predicts onset of congestion and responds quickly.

-Vishal
--
http://www.cs.columbia.edu/~misra/

> On Apr 28, 2015, at 9:49 AM, Mikael Abrahamsson <[email-masked]> wrote:
> 
>  So "incipient congestion" doesn't compute at all. I understand this term has been used before, but that doesn't help me understand the reasoning behind the wording. For me sending a congestion signal, means saying there is congestion. It's not saying there is developing congestion (whatever that might be).

---

From: Gorry Fairhurst <[email-masked]>
Date: Tue, 28 Apr 2015 15:33:51 +0100

> congestion via "CE" at an earlier threshhold than it would drop
I agree and so we will improve the words.

However, I'm not keen on delving into what happens if (and when) 
thresholds differ (or the AQM algorithm is different).

>     I'm still failing (after five readings) to extract a clear meaning. :^(
We will look at simpler words.

>    extraordinary programming, this would make an ECN-enabled flow
It would be less responsive than an ECN-enabled flow should be and could 
congest a queue - perhaps negating all benefits - but unresponsive? - I 
hope not, if it really contributes to congestion the queue should 
finally drop and the flow adapt before congestion collapse.

>    of congestion requiring tail-drop. But it also might not. :^(
Could be so.

>    there's really no chance that _all_ flows would ever be programmed
Yes, there are ways to be smart.

> _______________________________________________
> aqm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/aqm
>

Gorry & Michael

---

From: John Leslie <[email-masked]>
Date: Tue, 28 Apr 2015 10:44:23 -0400

> developing congestion (whatever that might be).

   I agree that "developing congestion" doesn't convey the same meaning
to all readers.

   When I suggested we use "incipient congestion", I was thinking of
the queueing-theory definition.

> this so I can better understand the reasoning behind this wording.

   I recommend my slides from IETF-77:

https://www.ietf.org/proceedings/77/slides/iccrg-7.pdf

starting with slide 11; and the Bauer_Clark_Lehr_2009 paper it refers
to (Section 3 of that paper).

   Briefly, the Queuing Theory definition is the most useful for
control-loop purposes:
"
" congestion is said to occur if the arrival rate into a system exceeds
" the service rate of the system at a point in time.

   This is what I meant "incipient" congestion to mean.

   (NB it _does_ make sense to talk of "incipient congestion" persisting
for a longer period of time, believe it or not!)

---

From: Mikael Abrahamsson <[email-masked]>
Date: Tue, 28 Apr 2015 21:50:06 +0200 (CEST)

> to (Section 3 of that paper).

This link is no longer valid, but I found the paper.

>   This is what I meant "incipient" congestion to mean.

Ok, I think I understand the reasoning behind it. Now you and the other 
authors need to decide if you want these documents to be read and 
understood by people like me for instance, who works at service providers 
or other networking professions who are not heavy inty queueing theory but 
instead are just trying to get their gear to do the right thing. I have 
studied english from grade 4 to grade 11, and I was best in my class in 
english. I had never even heard the word "incipient" before 2 days ago, 
and even now I am not sure I understand the full meaning trying to be 
conveyed.

Are people like me the target of these documents? If then, I recommend you 
try to simplify the terminology in these documents, striking "incipient" 
would be a first step, or at least try to explain them in the document, or 
point to a document that does.

If I for instance read RFC2884 is has 3 mentions of "incipient congestion" 
but 95 mentions of "congestion". Am I to understand that these 3 mentions 
of "incipient" means that the congestion that is "incipient" is different 
from the mentions of just "congestion" without "incipient" before it? 
Because that subtle difference is completely lost to me and if I were to 
guess, completely just deleting the 3 mentions of "incipient" from RFC2884 
wouldn't change the document at all. Am I wrong here?

For instance here:

"Jacobson [14] proposed gateways to monitor the average queue size to 
detect incipient congestion, and to randomly drop packets when congestion 
is detected."

I just don't understand the difference between the first mention of 
"incipient congestion" and then the second mention where "incipient" is 
not used. What am I missing here?

> for a longer period of time, believe it or not!)

If a queue has a buffer fill of between 40 and 60 ms worth of packets over 
60 seconds, is this link congested or is it experiencing a prolonged 
period of "incipient congestion"? Is the "incipience" (I don't even know 
if that is the correct term) just when the buffer fill time is increasing 
(for instance from 45 to 55 ms) and not when it's decreasing from 55 to 45 
ms, or is it just "incipient congestion" for the entire duration of the 
buffer having a constant non-trivial amount of packets in it?

---

From: John Leslie <[email-masked]>
Date: Tue, 28 Apr 2015 16:42:37 -0400

Mikael Abrahamsson <[email-masked]> wrote:
>>...
> instead are just trying to get their gear to do the right thing.

   Speaking for myself, I would like the document to be easy for network
operators to read.

   But I am not an author, and I certainly can't speak for the authors.

> I have studied english from grade 4 to grade 11, and I was best in my
> class in english.

   Alas, that doesn't mean as much as it used to...

> I had never even heard the word "incipient" before 2 days ago, 

   Q.E.D. ;^)

> and even now I am not sure I understand the full meaning trying to be 

   Alas, you're probably right. :^(

   The temptation becomes overwhelming, at some point in the RFC-generation
process, to use terms that "sound right" without defining them well.

   I have a history of annoying folks by asking for good definitions at
and after this point in the process. But I would like to see a definition
of "incipient congestion" -- perhaps something like:
"
" incipient congestion: the detection of congestion when it is just
"   starting, perhaps by noticing that the arrival rate exceeds the
"   forwarding rate.

   I don't think this is inconsistent with previous usage, but YMMV...

> If I for instance read RFC2884

   "Performance Evaluation of ECN..."

> it has 3 mentions of "incipient congestion" but 95 mentions of
> "congestion". Am I to understand that these 3 mentions of "incipient"
> means that the congestion that is "incipient" is different from the
> mentions of just "congestion" without "incipient" before it? 

   RFC 2884 is water over the hill: we cannot change it.

   My opinion is that it uses "incipient congestion" to indicate that
Random Early Detection signals oncoming congestion before it reaches
the point of tail-drop. "Congestion" used alone seems to be a generic
term covering a quite-wide range of meanings.

   Note, please, that RFC 2884 is informational -- which means that
only the authors are responsible for defining their terms, and that,
unless quite specifically cited, its definitions do not affect the
meaning of the terms it uses when they appear in other documents.

   (That is to say, don't bother looking there for definitions that
we should have included in _this_ document.)

> wouldn't change the document at all. Am I wrong here?

   Yes, IMHO, but YMMV. It meant (IMHO) to distinguish congestion signals
from RED from tail-drop which went before it.

> detect incipient congestion, and to randomly drop packets when congestion 

   I don't know the context of what you're quoting here, so I can't
respond.

> period of "incipient congestion"?

   Perhaps yes; perhaps no. Particular AQM algorithms may define this
either way.

> Is the "incipience" (I don't even know if that is the correct term)

   Yes, it's a word, meaning what I think you intend.

> just when the buffer fill time is increasing (for instance from 45 to
> 55 ms) and not when it's decreasing from 55 to 45 ms,

   That would be my preference: when arrival rate exceeds forwarding
rate, it's a bad idea to probe for additional capacity; but when the
forwarding rate exceeds the arrival rate, there probably is unused
capacity which it's reasonable to probe for.

   (Note, of course, that I do _not_ claim this is consistent with
current TCP congestion-control algorithms, which are optimized for
the lack of information which we must live with.)

> or is it just "incipient congestion" for the entire duration of the 

   Alas, YMMV. The RED algorithm seems to assume that there is some
"incipient congestion" whenever some fraction of the buffer is full.
I doubt we can fix that in this document, but I'd sure like to try!

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Wed, 29 Apr 2015 03:23:47 +0000

I similarly support the document. One comment, for Mikael.

> On Apr 24, 2015, at 2:25 AM, Mikael Abrahamsson <[email-masked]> wrote:
> 
> I have re-read this document as if I never read it before. Writing my thoughts as I read it.
> 
> The title says "benefits" but the abstract states that the document will talk about both benefits and potential problems. I therefore think the title does not reflect the abstract.
> 
> 1. Introduction: I don't understand the "(potential)", doesn't transport always treat loss and CE marking as indications of congestion? Actually, the way the introduction reads today one might interpret that TCP and SCTP are "Internet transports" but UDP is not, because it's merely "transport".

Delay-based transports may or may not. For example, if Caltech FAST is in use, the window is changed according to
                       least measured RTT
	cwnd’ = cwnd * ------------------ + alpha
                           mean RTT

And the obvious thing to do on receipt of CE might well be to simply arrest upward growth on the window, allowing the natural process to subsequently reduce it. That would mean, in the above, solving for mean RTT when cwnd’ == cwnd (what RTT value would have cwnd not change in the above equation?), and setting the mean RTT variable to

                                                         cwnd
    mean RTT := min ( Mean RTT, least measured RTT * ------------ )
                                                     cwnd - alpha