Subject: Re: [re-ECN] CONEX Principles & Constraints

From: Leslie Daigle <[email-masked]>
Date: Sun, 01 Nov 2009 18:20:19 -0500

Commenting on just one point:

John Leslie wrote:
 >>> 2. The protocol should be open about the responses it allows to the
 >>> conex information. For example, it should not make assumptions about
 >>> the behaviour of a particular application, and it should allow a
 >>> diversity of potential traffic management practices (of networks and
 >>> end-hosts).
 >
 >    Diversity of traffic management practices is critical: I'm not sure
 > whether the rest of this paragraph helps or tends to confuse.

I think it's worth emphasizing application-agnoticism, though I agree 
the wording above may inadvertently support it (in a "don't look over 
here" kind of way).

How about:

2.  The protocol should be open about the responses it allows to the 
conex information, supporting diversity of practices to manage network 
(not application) flows.

Leslie.

[Original message]
> [email-masked] <[email-masked]> wrote:
>>> [email-masked]
>>>
>>> 1. the purpose is to standardise the conex protocol...
>> I might even go so far as saying "Potential uses for such information
>> are explicitly beyond the scope of this initial
>> {WG/BoF/protocol/whatever}"
> 
>    "the planned WG".
> 
>>> 2. The protocol should be open about the responses it allows to the
>>> conex information. For example, it should not make assumptions about
>>> the behaviour of a particular application, and it should allow a
>>> diversity of potential traffic management practices (of networks and
>>> end-hosts).
> 
>    Diversity of traffic management practices is critical: I'm not sure
> whether the rest of this paragraph helps or tends to confuse.
> 
>>> 3. the conex information needs to be easily visible to all network
>>> elements, and be application-independent (including encrypted apps).
>>> Reason: this ensures widest possible applicability -- it is visible
>>> to both end-systems (to help them control their sending rate) and to
>>> network managers (to help them manage a bottleneck, whether traffic
>>> originates from their own users or from other networks).
>> I would either merge this with the later points about IP or at least
>> re-order the list so they occur in a row.
> 
>    I wouldn't recommend "merging" this with anything else.
> 
>>> 4. the conex information needs to be sufficiently up-to-date, for
>>> example so that it can usefully assist congestion control by end
>>> systems. Reason: stale information is little use. [Question:
>>> instead of "sufficiently up-to-date", would "real-time" or "near
>>> real-time" be better?]
>> How about "Sufficiently up-to-date to enable near-real-time traffic
>> management". 
> 
>    I think Toby captures the essential point: that this needs to be
> used for real-time traffic management. I don't think "up-to-date"
> helps.
> 
>> Incidentally this is nothing to do with congestion control at
>> end-systems - the end-system already sees congestion! So the only
>> link to congestion control is if this in turn allows the network
>> to be more lenient towards end-systems.
> 
>    "Usefully assist congestion control by end systems" isn't really
> the point. While we envision a system where ECN marking is "re-inserted
> into the stream being sent," this is mostly unrelated to what the end
> user application and/or stack does to control congestion.
> 
>    I'm not sure what we can say about how this benefits end users.
> I do imagine some overlap with the Multipath TCP WG, but it's way too
> early for any specifics.
> 
>    Our point really is that ConEx marking can enable network operators
> to distinguish "heavy" traffic from "congestion-causing" traffic, thus
> enabling experimentation with transport algorithms that do a better
> job of filling the pipes without harming other uses.
> 
>>> 5. the conex information needs to be sufficiently granular, to allow
>>> identification of those sending traffic to the bottleneck [Comment:
>>> this is trying to get a point that John made about 'granular enough
>>> for cost allocation' but in a more generic way.
> 
>    I guess I should comment on that.
> 
>    In hindsight, I was already too "generic" when I wrote that phrase.
> I meant allocation within the accounting system of the network operator
> that might perform the upgrade; but I wished to allow for policing
> which either charged for congestion from a particular forwarder or
> dropped / delayed / whatever packets that would "cause" more congestion
> than "normal".
> 
>    The point I wanted to make is that upgrading a bottleneck link needs
> cost justification, and we need to collect information that can justify
> the cost.
> 
>>> "Identification" is a bad word as, for some readers, it might be
>>> imply more than intended
> 
>    Agreed.
> 
>>> - but I can't think of a better one at the moment.
> 
>    Better to leave out anything that even resembles "identification"...
> The point is strictly related to who _forwards_ packets, not who _sends_
> them; and how that is determined is pretty strictly outside the scope
> of ConEx.
> 
>    The point of "granularity" is that we can't afford to try to aggregate
> to "flows" and then de-aggregate to portions of those "flows" that cause
> congestion: we need to operate packet-by-packet, determining a probability
> that the packet in question will "cause" congestion.
> 
>>> I hope that the following bullet about conex information in the IP
>>> header solves how to do it.]
>> The information needs to granular enough to hold the originator of the
>> traffic accountable?
> 
>    I hope not!
> 
>>> 6. the conex information is carried in the IP header. Note: it is in
>>> scope to work out which IP header bits to use (and the implications
>>> for other stuff that may already be using those bits).
> 
>    This is really weak on justification for the use of scarce bits...
> 
>> Better: "This may require defining new codepoints in the IP header or
>> possibly even the re-definition of existing codepoints. We will be
>> careful to consider the impact of our choices both on existing protocols
>> and on deployability.
> 
>    I would try to avoid trespassing on that ground.
> 
>    While there is space in the IPv6 header, we want to experiment in
> IPv4-land. Talking of "defining new codepoints" is likely to get our
> scope limited to IPv6. Better to experiment in IPv4, and perhaps _then_
> define IPv6 codepoints.
> 
>    BTW, I firmly believe the "granularity" issue is essential to justify
> experimenting with IPv4 codepoints: if we don't get buy-in to the need
> for granularity, the whole experiment is endangered.
> 
>>> 7. a protocol will be developed for both IPv4 and IPv6. Reason: both
>>> are needed for full applicability. (There was push-back on an earlier
>>> suggestion of narrowing the scope to IPv4.)
> 
>    I agree, but justification is weak. Bob Briscoe should clarify how
> we'd experiment in IPv6 space: I believe that would be very good;
> however, I suspect we'll eventually find a better way to do congestion
> exposure in IPv6, and we shouldn't limit ourselves to designing the
> final IPv6 solution too early.
> 
>> I might consider phrasing this "although on the face of it an IPv6
>> protocol may be easier, we will also develop solutions for IPv4 as it is
>> still the dominant network-layer protocol.."
> 
>    As I've already said, we need to _experiment_ in IPv4 in order to
> collect sufficient data and observe how congestion exposure information
> is actually used. I would avoid calling the IPv4 stuff "solutions"
> 
>>> 8. TCP options will be used to inform the sender about congestion
>>> received at the receiver. (Note, current proposals require this
>>> information transfer.) Reason: narrow the scope by looking only at
>>> TCP, which is used by the vast majority of the traffic (~85-90%).
> 
>    I thorouhly disagree!
> 
>    Limiting the scope to TCP would be fatal. We need to collect data
> on congestion caused by UDP packets; and any limitation to particular
> transport(s) incents the use of other transports to "end-run" ConEx.
> 
>> Do we want to specify using TCP options at the moment? Option space is
>> limited and already used for lots of other things + MPTCP will be using
>> even more (and there is a very high chance of eventual overlap between
>> conex and MPTCP). I would prefer to leave this as "we will concentrate
>> on TCP as this provides a feedback path..."
> 
>    TCP option "space" is limited in that packets are limited to 44 bytes
> of options per packet. We _would_ get burned by that limit. :^(
> 
>    More importantly, we _really_ don't want to wander into the weeds of
> how to juggle TCP options when we're dealing with the forwarding of IP
> packets. (Yes, I know we do have folks clever enough to pull it off
> 99.999% of the time; but the rest of us will quickly lose track of the
> details, and sooner or later the folks that are clever enough will find
> themselves on the "rough" end of "rough consensus.)
> 
>>> 9. a baseline assumption is that routers are unmodified. Reason:
>>> deployment viability
>> "This MUST work with unmodified routers, even if they don't do ECN...
>> although modifying routers may allow additional refinements...
> 
>    A reasonable way to say it...
> 
>>> 10. a baseline assumption is that conex information is signalled
>>> explicitly (through the IP header) and not implicitly signalled
>>> through packet loss, delay or jitter. Reason: explicit signalling
>>> allows a "better" reaction, for example faster, without impairment
> 
>    Pretty good... but the justification needs to be stronger.
> 
>    First of all, explicit signaling can carry more-than-one bit,
> instead of less-than-one (due to the multitude of other things that
> can lead to packet loss); and can reach the actor who needs to react
> significantly faster.
> 
>    Second, explicit signaling carries consistent information to every
> router along the path, meaning that incentives to slow down a sending
> rate can be applied "close enough" to the sender to be effective; and
> if a packet actually does need to be dropped, it could be dropped
> before it causes other congestion earlier in the forwarding path.
> 
>> In theory implicit signalling already exists - if you observe a TCP
>> source halving its rate you can assume it was (probably) in response to
>> a loss event...
> 
>    Hmm... that's getting "too clever" for this bear-of-small-brain...
> 
>>> 11. but we will also develop a solution where the conex information
>>> is signalled through packet loss. Reason: ECN is not enabled in most
>>> routers today; incremental deployment is critical
> 
>    Am I missing how to "re-insert" that signal?
> 
>> No! - we may develop a solution that informs the network of congestion
>> the sender has learnt about both as a result of explicit notification
>> (ECN) and implicit notification (loss). However we DON'T use loss to
>> signal ConEx information (which is surely impossible anyway?)! 
> 
>    I strongly recommend against any statement along those lines.
> 
>>> 12. (possibly) we will study an incremental deployment step where only
>>> the sender is conex-enabled [Step suggested by Matt]
>> We certainly need to leave this open as a possible step
> 
>    I don't see any reason to mention this. Clearly, a ConEx-aware sender
> can re-insert a signal whether or not the end-receiver is ConEx aware.
> And that signal can enable all the experimenting we need along the
> forwarding path.
> 
>    We need only specify the _meaning_ of the ConEx signal the sender
> re-inserts -- not any particular algorithm for generating it.
> 
>>> 13. (possibly) another incremental deployment step?
> 
>    Way premature, IMHO.
> 
>>> 14. we will describe threats, especially from falsifying or
>>> suppressing the conex information - whether by ISPs or end-hosts.
>>> The protocol needs to allow such threats to be dealt with (in as
>>> simple a manner as possible), but the actual mechanisms to tackle
>>> those threats (using policer boxes, for instance) are out of scope.
>>> Reason: such mechanisms are about uses of conex, rather than exposing
>>> the information.
> 
>    I agree.
> 
>>> 15. anything else?
> 
>    Not at this time.
> 
> --
> John Leslie <[email-masked]>
> _______________________________________________
> re-ECN mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/re-ecn

-- 

-------------------------------------------------------------------
"Reality:
      Yours to discover."
                                 -- ThinkingCat
Leslie Daigle
[email-masked]
-------------------------------------------------------------------

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 02 Nov 2009 09:36:14 +0000

Folks,

>>>>Reason: such mechanisms are about uses of conex, rather than exposing

Suggest we replace the middle sentence with:

The protocol must support at least one simple mechanism that can 
tackle those threats without compromising any other principles and 
constraints, but standardisation of a particular mechanism is out of scope.

[[[Reason: The ability to counter cheating, particularly by single 
packet flows, introduces all the toughest constraints on the 
protocol. Saying the actual mechanisms are out of scope precludes us 
from considering whether even one nominal mechanism will be feasible 
without compromising other constraints. ]]]

>>    Not at this time.

I have a very subtle principle to add:

15. Spontaneously cooperative behaviour between end-systems and 
between networks and end-systems should not be precluded or 
discouraged by any measures designed to prevent cheating or to 
encourage cooperation.

BOb

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: <[email-masked]>
Date: Wed, 4 Nov 2009 16:35:28 -0000

Thanks for your very usefull comments about this. I've attempted a
revised version

[CONEX information = expected rest-of-path congestion]

1.	Application-agnostic - the CONEX protocol should be open about
the responses it allows to the CONEX information, supporting a diversity
of traffic management practices by networks and end-hosts. The proposed
WG will encourage experiments on traffic management, but not standardise
them (in order to reduce its scope).
2.	Visibility of CONEX information - it should be easily visible to
all network elements (even if the data traffic is encrypted), in order
to ensure the widest possible applicability -- it is visible to both
end-systems (to help them control their sending rate) and to network
managers (to help them manage a bottleneck, whether traffic originates
from their own users or from other networks).   
3.	Granularity of CONEX information - it should be sufficiently
granular to allow near real-time, per packet traffic management. So that
any node can see the impact of the packets it is asked to forward. The
CONEX information is inevitably a minimum of 1 RTT out-of-date
4.	CONEX information is in IP header - 2 & 3 imply that the CONEX
information is carried in the IP header. Signalling of congestion
information is thus explicit, rather than implicitly through packet
loss, delay or jitter, which allows a faster and more accurate reaction.
The proposed WG will specify a packet structure (v4 & v6) to encapsulate
CONEX information (header bits, interpretation)
5.	Transport protocol - the receiver needs to inform the sender
what CONEX information to set (TBD: whether the capability of an ECN
receiver is sufficient). 
6.	Threats analysis - the proposed WG will study the threats,
especially from falsifying or suppressing the CONEX information
7.	Deployability analysis - the proposed WG will study technology
to support incremental deployment steps: ECN routers vs non-ECN routers
vs CONEX routers; receivers with CONEX-capability, ECN-capability, or
neither; opportunistic deployment.

3 - John, does this now briefly summarise your granularity point? 

5 - some people have suggested we should restrict initial work to TCP
(to limit WG scope). John has argued that UDP needs to be in scope.
Views?

7 - bob, does 'opportunistic deployment' get your "subtle point 15"?

thanks

phil

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of [email-masked]
Sent: 29 October 2009 18:03
To: [email-masked]
Subject: [re-ECN] CONEX Principles & Constraints

We wanted to start some discussion on "principles" and "constraints" ie
what we want to do in the proposed WG and how would its scope would be
restricted (building especially on emails from John (about "principles")
and Rich (about "requirements"). Eventually this would feed into the
Charter. This is just a first guess, so please bash. Something on these
lines would be in the "Constraints" part of the Agenda.  

1.	the purpose is to standardise the conex protocol - this exposes
congestion information along the forwarding path of the Internet
("rest-of-path congestion"). (This is in addition to the current
capability that exposes the congestion on the "path-so-far".). Note, the
purpose doesn't include studying how the conex information could be
used. Reason: there are lots of potential uses for the information. In
order to reduce the scope of the proposed WG, we concentrate on the
underlying information protocol.
2.	The protocol should be open about the responses it allows to the
conex information. For example, it should not make assumptions about the
behaviour of a particular application, and it should allow a diversity
of potential traffic management practices (of networks and end-hosts).
3.	the conex information needs to be easily visible to all network
elements, and be application-independent (including encrypted apps).
Reason: this ensures widest possible applicability -- it is visible to
both end-systems (to help them control their sending rate) and to
network managers (to help them manage a bottleneck, whether traffic
originates from their own users or from other networks).
4.	the conex information needs to be sufficiently up-to-date, for
example so that it can usefully assist congestion control by end
systems. Reason: stale information is little use. [Question: instead of
"sufficiently up-to-date", would "real-time" or "near real-time" be
better?]
5.	the conex information needs to be sufficiently granular, to
allow identification of those sending traffic to the bottleneck
[Comment: this is trying to get a point that John made about 'granular
enough for cost allocation' but in a more generic way. "Identification"
is a bad word as, for some readers, it might be imply more than intended
- but I can't think of a better one at the moment. I hope that the
following bullet about conex information in the IP header solves how to
do it.]
6.	the conex information is carried in the IP header. Note: it is
in scope to work out which IP header bits to use (and the implications
for other stuff that may already be using those bits). 
7.	a protocol will be developed for both IPv4 and IPv6. Reason:
both are needed for full applicability. (There was push-back on an
earlier suggestion of narrowing the scope to IPv4.)
8.	TCP options will be used to inform the sender about congestion
received at the receiver. (Note, current proposals require this
information transfer.) Reason: narrow the scope by looking only at TCP,
which is used by the vast majority of the traffic (~85-90%). 
9.	a baseline assumption is that routers are unmodified. Reason:
deployment viability
10.	a baseline assumption is that conex information is signalled
explicitly (through the IP header) and not implicitly signalled through
packet loss, delay or jitter. Reason: explicit signalling allows a
"better" reaction, for example faster, without impairment
11.	but we will also develop a solution where the conex information
is signalled through packet loss. Reason: ECN is not enabled in most
routers today; incremental deployment is critical
12.	(possibly) we will study an incremental deployment step where
only the sender is conex-enabled [Step suggested by Matt]
13.	(possibly) another incremental deployment step?
14.	we will describe threats, especially from falsifying or
suppressing the conex information - whether by ISPs or end-hosts. The
protocol needs to allow such threats to be dealt with (in as simple a
manner as possible), but the actual mechanisms to tackle those threats
(using policer boxes, for instance) are out of scope. Reason: such
mechanisms are about uses of conex, rather than exposing the
information.
15.	anything else?

Thanks

phil

---

From: <[email-masked]>
Date: Wed, 4 Nov 2009 17:07:29 -0000

I think this is getting to be a good list... Specific comments by number:

1)    I would actually suggest spelling out that ConEx is not just open about the responses but is truly independent of any response

2)    Congestion information is already generally visible to the end points...

3)    Join the first two sentences...

4)    Perhaps phrase it that the logical result of constraints 2 & 3 is to put ConEx info in the IP header... Also this is not necessarily explicit signalling of all congestion info - just of whole=path info... packet structure à codepoint structure? Set of codepoints? The packet structure will still be IP.

5)    No, the receiver needs to inform the sender of its most recent knowledge of whole-path congestion. It doesn't necessarily follow that this is the same as what the sender will then set (flow start, sudden increases in sending rate, change of path, etc...). However some form of feedback does seem to be required in general

6)    This seems out of place in the list. Perhaps the list needs to be split into constraints and aims as two separate items

7)    This would also be an aim not a constraint.

Regarding the specific question of transport. I would favour limiting to TCP BUT explicitly stating that UDP is in scope as an extension once TCP is sorted. The problem with UDP is the lack of feedback...

Regarding "opportunistic" deployment, I think I might prefer unilateral or spell it out a bit better...

Toby

From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: 04 November 2009 16:35
To: [email-masked]
Subject: [re-ECN] CONEX Principles & Constraints (version 2)

Thanks for your very usefull comments about this. I've attempted a revised version

1.	Application-agnostic - the CONEX protocol should be open about the responses it allows to the CONEX information, supporting a diversity of traffic management practices by networks and end-hosts. The proposed WG will encourage experiments on traffic management, but not standardise them (in order to reduce its scope).
2.	Visibility of CONEX information - it should be easily visible to all network elements (even if the data traffic is encrypted), in order to ensure the widest possible applicability -- it is visible to both end-systems (to help them control their sending rate) and to network managers (to help them manage a bottleneck, whether traffic originates from their own users or from other networks).   
3.	Granularity of CONEX information - it should be sufficiently granular to allow near real-time, per packet traffic management. So that any node can see the impact of the packets it is asked to forward. The CONEX information is inevitably a minimum of 1 RTT out-of-date
4.	CONEX information is in IP header - 2 & 3 imply that the CONEX information is carried in the IP header. Signalling of congestion information is thus explicit, rather than implicitly through packet loss, delay or jitter, which allows a faster and more accurate reaction. The proposed WG will specify a packet structure (v4 & v6) to encapsulate CONEX information (header bits, interpretation)
5.	Transport protocol - the receiver needs to inform the sender what CONEX information to set (TBD: whether the capability of an ECN receiver is sufficient). 
6.	Threats analysis - the proposed WG will study the threats, especially from falsifying or suppressing the CONEX information
7.	Deployability analysis - the proposed WG will study technology to support incremental deployment steps: ECN routers vs non-ECN routers vs CONEX routers; receivers with CONEX-capability, ECN-capability, or neither; opportunistic deployment.

5 - some people have suggested we should restrict initial work to TCP (to limit WG scope). John has argued that UDP needs to be in scope. Views?

thanks

phil

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: 29 October 2009 18:03
To: [email-masked]
Subject: [re-ECN] CONEX Principles & Constraints

We wanted to start some discussion on "principles" and "constraints" ie what we want to do in the proposed WG and how would its scope would be restricted (building especially on emails from John (about "principles") and Rich (about "requirements"). Eventually this would feed into the Charter. This is just a first guess, so please bash. Something on these lines would be in the "Constraints" part of the Agenda.  

1.	the purpose is to standardise the conex protocol - this exposes congestion information along the forwarding path of the Internet ("rest-of-path congestion"). (This is in addition to the current capability that exposes the congestion on the "path-so-far".). Note, the purpose doesn't include studying how the conex information could be used. Reason: there are lots of potential uses for the information. In order to reduce the scope of the proposed WG, we concentrate on the underlying information protocol.
2.	The protocol should be open about the responses it allows to the conex information. For example, it should not make assumptions about the behaviour of a particular application, and it should allow a diversity of potential traffic management practices (of networks and end-hosts).
3.	the conex information needs to be easily visible to all network elements, and be application-independent (including encrypted apps). Reason: this ensures widest possible applicability -- it is visible to both end-systems (to help them control their sending rate) and to network managers (to help them manage a bottleneck, whether traffic originates from their own users or from other networks).
4.	the conex information needs to be sufficiently up-to-date, for example so that it can usefully assist congestion control by end systems. Reason: stale information is little use. [Question: instead of "sufficiently up-to-date", would "real-time" or "near real-time" be better?]
5.	the conex information needs to be sufficiently granular, to allow identification of those sending traffic to the bottleneck [Comment: this is trying to get a point that John made about 'granular enough for cost allocation' but in a more generic way. "Identification" is a bad word as, for some readers, it might be imply more than intended - but I can't think of a better one at the moment. I hope that the following bullet about conex information in the IP header solves how to do it.]
6.	the conex information is carried in the IP header. Note: it is in scope to work out which IP header bits to use (and the implications for other stuff that may already be using those bits). 
7.	a protocol will be developed for both IPv4 and IPv6. Reason: both are needed for full applicability. (There was push-back on an earlier suggestion of narrowing the scope to IPv4.)
8.	TCP options will be used to inform the sender about congestion received at the receiver. (Note, current proposals require this information transfer.) Reason: narrow the scope by looking only at TCP, which is used by the vast majority of the traffic (~85-90%). 
9.	a baseline assumption is that routers are unmodified. Reason: deployment viability
10.	a baseline assumption is that conex information is signalled explicitly (through the IP header) and not implicitly signalled through packet loss, delay or jitter. Reason: explicit signalling allows a "better" reaction, for example faster, without impairment
11.	but we will also develop a solution where the conex information is signalled through packet loss. Reason: ECN is not enabled in most routers today; incremental deployment is critical
12.	(possibly) we will study an incremental deployment step where only the sender is conex-enabled [Step suggested by Matt]
13.	(possibly) another incremental deployment step?
14.	we will describe threats, especially from falsifying or suppressing the conex information - whether by ISPs or end-hosts. The protocol needs to allow such threats to be dealt with (in as simple a manner as possible), but the actual mechanisms to tackle those threats (using policer boxes, for instance) are out of scope. Reason: such mechanisms are about uses of conex, rather than exposing the information.
15.	anything else?

Thanks

phil