Subject: [aqm] I-D Action: draft-ietf-aqm-eval-guidelines-01.txt

From: [email-masked]
Date: Thu, 05 Mar 2015 04:19:23 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories.
 This draft is a work item of the Active Queue Management and Packet Scheduling Working Group of the IETF.

        Title           : AQM Characterization Guidelines
        Authors         : Nicolas Kuhn
                          Preethi Natarajan
                          Naeem Khademi
                          David Ros
	Filename        : draft-ietf-aqm-eval-guidelines-01.txt
	Pages           : 28
	Date            : 2015-03-05

Abstract:
   Unmanaged large buffers in today's networks have given rise to a slew
   of performance issues.  These performance issues can be addressed by
   some form of Active Queue Management (AQM) mechanism, optionally in
   combination with a packet scheduling scheme such as fair queuing.
   The IETF Active Queue Management and Packet Scheduling working group
   was formed to standardize AQM schemes that are robust, easily
   implementable, and successfully deployable in today's networks.  This
   document describes various criteria for performing precautionary
   characterizations of AQM proposals.  This document also helps in
   ascertaining whether any given AQM proposal should be taken up for
   standardization by the AQM WG.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-aqm-eval-guidelines/

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-ietf-aqm-eval-guidelines-01

A diff from the previous version is available at:
http://www.ietf.org/rfcdiff?url2=draft-ietf-aqm-eval-guidelines-01

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: David Lang <[email-masked]>
Date: Fri, 20 Mar 2015 17:11:50 -0700 (PDT)

re: gaming ECN

apologies for not replying to the right message, I deleted them earlier before 
deciding to respond.

Programmers are going to game ECN to their advantage (or at least to their 
perceived advantage)

There are three ways to mark flows as congested

1. at the same level that you drop packets and then drop the packets at this 
level (why bother)

2. at the level that you drop packets for non-ECN flows, you mark and allow ECN 
flows to continue

3. at a level below the point that you drop packets for all flows, you mark ECN 
flows as congested

If you do #2, then flows with ECN effectively get priority over flows without 
ECN (as you don't actually force them to back off, you are just asking them to)

   Programmers will mark everything with ECN and not back off

If you do #3, then flows with proper ECN yield to flows without ECN, giving 
effective priority to flow without ECN

   Programmers will not use ENC as it puts their traffic at a disadvantage

If everyone uses ENC and backs off properly, then #3 is better as you don't have 
the delays caused by re-sending packet. But in a mixed environment, ECN is going 
to be gamed one way or the other.

David Lang

---

From: John Leslie <[email-masked]>
Date: Fri, 20 Mar 2015 21:23:29 -0400

> Programmers are going to game ECN to their advantage (or at least to their 

   Some programmers will, no doubt...

> There are three ways to mark flows as congested

   (Note any real path will likely have a mixture of these...)

> 1. at the same level that you drop packets and then drop the packets at 
>    this level (why bother)

   Many nodes do this today, simply by ignoring ECN-capable marking.

> 2. at the level that you drop packets for non-ECN flows, you mark

   This _is_ the prescribed way to implement ECN today.

>    and allow ECN flows to continue

   Without deep-packet-inspection and saving _considerable_ state,
no node can "allow" or "disallow" a particular flow to continue.

   Thus essentially no nodes try to "disallow" flows -- they merely
provide signals of congestion.

> 3. at a level below the point that you drop packets for all flows, you mark 
>    ECN flows as congested

   This is what we have been talking about -- claiming it could result
in better signals.

> If you do #2, then flows with ECN effectively get priority over flows 
> without ECN

   It's not "priority". It's an occasional packet which gets through
instead of being dropped.

> (as you don't actually force them to back off, you are just asking
> them to)

   We're not even "asking them" -- we're providing a signal.

   We _can't_ force them to back off.

   A not-ECN-capable flow could achieve the same result with Forward-
Error-Correction tactics: include enough redundancy to regenerate
all lost packets, and also decline to back off the sending rate.

> Programmers will mark everything with ECN and not back off

   Perhaps some will.

   Certainly others will use FEC and not back off.

   So what?

> If you do #3, then flows with proper ECN yield to flows without ECN,

   As ECN is defined today, indeed they will.

> giving effective priority to flows without ECN

   We are not (yet) proposing any particular mechanism for ECN-marking
at an earlier congestion status. If we do sometime propose a mechanism,
this will serve as an incentive to upgrade to it.

> Programmers will not use ECN as it puts their traffic at a disadvantage

   Programmers will use ECN when it offers them an advantage.

   In some cases, reducing packet loss _is_ that advantage; and
backing off the sending rate is perfectly acceptable.

   In the future we may define algorithms that encourage stopping the
exponential increase in sending rate without asking for a radical
reduction in sending rate at the first ECN mark. There are types of
traffic which would find that tradeoff _very_ attractive.

   But that's out of scope for us in this document -- just as any
attempt to force flows to back off is out of scope.

> If everyone uses ENC and backs off properly, then #3 is better as you
> don't have the delays caused by re-sending packet.

   Thank you.

> But in a mixed environment, ECN is going to be gamed one way or the other.

   So what?

   (BTW, I'd ask that same "So what?" for any problems of gaming
not-ECN-capable traffic.)

   I would support explaining the problems you see; but I can't see
abandoning the effort because you see problems.

--
John Leslie <[email-masked]>

---

From: David Lang <[email-masked]>
Date: Sat, 21 Mar 2015 15:47:08 -0700 (PDT)

> instead of being dropped.

is it? or is it that in order to keep the link from being congeted, flows with 
ECN marked (but not honored) will consistantly get more packets through than 
ones wihtout ECN?

If it' just an occasional packet, it's not a big deal, but if the non-ECN flows 
get slowed more because the ECN-marked flows are getting more packet through, 
that's a priority difference, not just an occasional packet.

I wrote this based on the responses of "how could anyone game ECN" where people 
seemed to be thinking that there is no way to gain an advantage from claiming to 
do ECN but then ignoring the congestion signals.

It may be that nobody ever abuses ECN this way, but when introducing something 
new we need to look at ways that it could be abused and what the effects of 
someone trying to abuse it would be.

David Lang

---

From: KK <[email-masked]>
Date: Sat, 21 Mar 2015 16:57:38 -0700

I have been reading the discussion related to ECN and the recent draft put
together by Gorry and Michael.
When we wrote RFC 3168, we constructed the protocol in such a way that it
would not provide an incentive for an ECN capable flow to benefit or for a
flow that purports to be an ECN flow would gain by Œgaming¹ the
environment.
There were several people who questioned us at that time with regard to
our conservative choice to treat flows Œsimilarly': mark flows that are
ECN capable and drop flows that are not ECN capable with the same
criterion (probabilistically or whatever other means people might think of
in the future).
We specified that a packet that was ECN capable wouldn¹t be viewed as
receiving any notion of Œpriority¹. Moreover, we stated that when a buffer
is full, packets would be dropped, irrespective of whether it is ECN
capable or not. 

So, let us look at the two adversarial scenarios:
1) A flow that claims to be ECN capable, gets marked by congested
router(s). The end-system (either end) does not honor the protocol and
continues to send packets at a higher rate (not reducing the window or
whatever in accordance with TCP) than a flow that would react properly to
the ECN marks. It would presumably get more packets through on a short
term basis. 
2) A flow that is not ECN capable has packets dropped by congested
router(s). The end-system (either end) does not honor the protocol and
continues to send packets at a higher rate (not reducing the window or
whatever in accordance with TCP) than a flow that would react properly to
the packet drop as an indication of congestion. It would presumably get
more packets through on a short term basis.

Would there be any difference in the number of packets that are
successfully delivered to the destination in either of these cases? I
would think that there would not be a significant, consistent difference.
Yes, these non-cooperative flows would be able to deliver more packets
than the ones that cooperate, whether it is through marking or dropping, I
would think.
Ultimately, if the non-cooperative behavior persists, buffers overflow and
it would not make a difference whether the flow (claims to be or is) ECN
capable or not.

We thought we were being careful in  ensuring that we address the
sentiment that David expresses: "when introducing something
new we need to look at ways that it could be abused and what the effects
of someone trying to abuse it would be"

Thanks,  
-- 
K. K. Ramakrishnan
Professor
Dept. of Computer Science and Engineering
University of California, Riverside
Rm. 332, Winston Chung Hall
Tel: (951) 827-2480
Web Page: http://www.cs.ucr.edu/~kk/

On 3/21/15, 3:47 PM, "David Lang" <[email-masked]> wrote:

>_______________________________________________
>aqm mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/aqm

---

From: Michael Welzl <[email-masked]>
Date: Sat, 21 Mar 2015 23:04:40 -0500

> On 21. mar. 2015, at 17.47, David Lang <[email-masked]> wrote:
> 
> is it? or is it that in order to keep the link from being congeted, flows with ECN marked (but not honored) will consistantly get more packets through than ones wihtout ECN?
> 
> If it' just an occasional packet, it's not a big deal, but if the non-ECN flows get slowed more because the ECN-marked flows are getting more packet through, that's a priority difference, not just an occasional packet.

Let's consider two cases:

1) an attacker who does not care about congestion control and just wants to flood.

Such a sender can get more packets into the queue with ECN, but sending way too many will eventually create drops at the physical queue limit. In contrast, without ECN, the sender would get occasional drops before it reaches the hard queue limit. How "occasional" these drops are depends on how aggressively the AQM mechanism drops packets - the more aggressive it is, the closer it becomes to operating like the hard queue limit that is there anyway.

This doesn't strike me as a convincing scenario for ECN being truly harmful.

2) an application developer who wants to be better off than everyone else by ECN-enabling packets but ignoring ECN marks.

This application developer cares about network performance, and will therefore want to do some form of congestion control (as illustrated by various applications deliberately doing it: Skype, Adobe's RTMFP, QUIC, ...). Will ECN give this application a persistently better behavior then?

I don't have an answer for the case when the queue is shared with others (but if /  when / how often is not known to the app programmer beforehand). If we consider only flows by this application, and assume TCP-like congestion control, we have example graphs:
https://www.duo.uio.no/handle/10852/37381
see Figure 14.

=> because every ECN-capable packet enters the queue, it influences the dynamics of the AQM mechanism, and what then happens depends on how aggressive the AQM mechanism reacts. Note that these graphs vary quite a bit - they show a rather inconsistent behavior, and definitely do not show that TCP with ECN had persistently higher throughput. Also note that incorrect behavior might increase the delay, and clearly raw throughput is not the main metric of interest to all applications.

So that was ECN vs. no-ECN with TCP. What about applying a different congestion control mechanism to get a benefit?
Well - CUBIC is already more aggressive than standard TCP, with or without ECN, and will "win" against it because it increases faster and backs off by multiplying with 0.7, not 0.5. THIS gives you a higher priority - but you don't need ECN for that.

All in all, the "more throughput by gaming ECN" image seems to be blurry at best.

Cheers,
Michael