Subject: [Tsvwg] [Digest] Fourth ECN code-point

From: Bob Briscoe <[email-masked]>
Date: Thu, 08 Mar 2001 10:55:01 +0000

tsvwg,

Here's the first of 5 digests of the off-line discussions that 
went on between:
1/ two of the authors of standards track "The Addition of Explicit 
Congestion Notification (ECN) to IP" 
(draft-ietf-tsvwg-ecn-{01|02}.txt Ramakrishnan, Floyd, Black) 
http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-ecn-01.txt
and 
2/ the new "An Open ECN Service in the IP layer" 
(draft-ietf-tsvwg-ecn-ip-00.txt Briscoe, Crowcroft).
http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-ecn-ip-00.txt

Also, to repeat Sally's clarification - although the intro to the 
Briscoe/Crowcroft one says we hope it will be absorbed into the 
standards track one, it wasn't... But by agreement, additions on 
diffserv and fragmentation were made to the standards track one 
that don't go as far as we would have liked but are the best we
can hope for at this stage (except I'm still concerned about the 
complexity of the fragmentation solution for implementors - I'll 
now take that up on the list).

Apologies in advance if the digests aren't complete, or aren't 
understandable.

The 5 digests will be:
1. Fourth ECN code-point (this one)
2. ECN & diffserv
3. ECN & fragmentation
4. IP's ECN service to other congestion control protocols
5. Summary

    DB> = David Black
    JC> = Jon Crowcroft
    SF> = Sally Floyd
    BB> = Bob Briscoe

BB>>>>Date: Wed, 21 Feb 2001 17:14:12 +0000
BB>>>>To: [email-masked], [email-masked], [email-masked]
BB>>>>From: Bob Briscoe <[email-masked]>
BB>>>>Subject: Important additions to ECN Proposed Standard?
BB>>>>Cc: Crowcroft Jon
BB>>>>
BB>>>>This screws up your timetable, I'm sure. Having read it, how would you
BB>>>>propose we proceed?

SF>>>At 09:58 21/02/01 -0800, Sally Floyd wrote:
SF>>>Well, so far I have only read it once over lightly, but mostly 
SF>>>I disagree.  Fairly strongly.  First I will tell you what is
SF>>>going on, and later I will read it over more carefully.
SF>>>But I don't agree with most of your proposals, so I am not
SF>>>going to be encouraging you to proceed.
SF>>>We (the authors of the ECN draft) have been given a Last Call
SF>>>requirement from the IESG to review and try to incorporate ECN
SF>>>nonces into a revision of the newest ECN draft.

JC>>To: Sally Floyd <[email-masked]>
JC>>cc: Bob Briscoe <[email-masked]>,
JC>>        "Crowcroft, Jon" <[email-masked]>, [email-masked],
JC>>        [email-masked], [email-masked]
JC>>Subject: Re: Important additions to ECN Proposed Standard?
JC>>Date: Wed, 21 Feb 2001 19:25:14 +0000
JC>>From: Jon Crowcroft <[email-masked]>
JC>>
JC>>quick point
JC>>
JC>>we really really do not want to hold up the baseline ECN for TCP -
JC>>please believe that this is a million miles from our goal so if,
JC>>as you think, anything in what we wrote would have that effect,
JC>>then we have to rethink
JC>>
JC>>what we wanted to was to make sure that since its close to getting
JC>>thru, that we don't close doors to later use of scarce bit's
JC>>functionality for other things such as you yourself mention you
JC>>are working on - the ECN capable UDP 
JC>>
JC>>plus we wanted to put on the table for discussion,
JC>>some ideas on avoiding some more complex definitions for possible
JC>>multicast ecn behaviour
JC>>
JC>>the wording may have been over zealous in the use of standards
JC>>terminology!

BB>>Sally,
BB>>
BB>>Just would like to agree wholeheatedly with Jon. I tried 
BB>>to find a way of defining everything that there are strong 
BB>>views on in a way that allowed all views to continue being 
BB>>supported into the future. 
BB>>
BB>>I really appreciate the (8) years of work you've put into 
BB>>ECN. I believe it's the most important thing around for the 
BB>>future Internet. I'm sorry you've taken my intervention as 
BB>>negative. Additive was the intention.
BB>>
BB>>Anyway, to answer your points, I've digested them all back 
BB>>together again with replies inline...

SF>>>
SF>>>We decided that there were three options:
SF>>>(1) Do nothing
SF>>>(2) Modify the ECN mechanism so that the fourth
BB>>     ^^^^^^^^^^^^^^^^^^^^^^^^
BB>>What do you mean by this? Standardise it? Implement it? Both?

SF>
SF>Yep.  I am modifying the ECN draft to specify that 01 is semantically
SF>identical to 10.
SF>
SF>>>	code point ( <CE, ECT> = <1,0>) behavior
SF>>>	at network nodes and receivers matches
SF>>>	what the nonce mechanism needs, but put
SF>>>	the nonce mechanism (parity calculations,
SF>>>	additional TCP header bit and sender logic)
SF>>>	into a separate (experimental) draft.

BB>>Ah! then we have a potential conflict here. I can't 
BB>>sort this in the time of an e-mail. Will have to weigh 
BB>>these two using the process you suggest below. I'm just 
BB>>concerned if your meaning of "Modify the ECN mechanism" 
BB>>is to standardise 01 to be semantically identical to 
BB>>10. While the nonce idea is still experimental, this 
BB>>would be wrong, surely?
BB>>
BB>>The nonce idea only protects against misbehaving CE 
BB>>*erasure*, of course. I found it very difficult to see 
BB>>a serious motivation for such an attack. Do you have 
BB>>one in mind? Perhaps you are thinking that, if an 
BB>>upstream network is congested, a competing downstream 
BB>>network can drive it further into trouble? Given the 
BB>>malicious network can usually do this by dropping ACKs 
BB>>or NACKs in the other direction, this seems a rather 
BB>>esoteric concern. Neat idea from a theory viewpoint, 
BB>>whatever. But, it seems strange to use a code-point to 
BB>>protect against this. 

SF>I agree that the motivation of protecting against
SF>misbehaving CE *erasure* is not very compelling.  The other motivation
SF>is to protect against receivers failing to report CE to the senders.
SF>I agree that, for TCP, this is again only a mildly compelling
SF>concern.  But several of us were up in Seattle two weeks ago,
SF>talking with people at RealNetworks about ECN and other issues,
SF>and I think that the idea of a mechanism for conformant senders
SF>(e.g., big web servers) to protect against misbehaving receivers
SF>for non-TCP mechanisms down the line is in fact fairly compelling.
SF>But you get to disagree with it when it goes to Last Call,
SF>or to lobby Allison, or whatever.  As long as it doesn't sink
SF>ECN entirely, in which case I would be annoyed.

BB:There's a solution:
BB:
BB:The nonce scheme isn't relevant for catching misbehaving 
BB:receivers in multicast anyway, as you only nack, never ack. 
BB:So the only packets where the nonce has passed through the 
BB:network are never fed back. So a multicast sender might just 
BB:as well send 10 all the time anyway, unless it wants to 
BB:catch a misbehaving network (unlikely) by telling the 
BB:receivers the nonce as a bit field.
BB:
BB:(Using <xy> to mean <ECT,CE>):
BB:And in our scheme, if a multicast router turns <11> into 
BB:<01> on all but one i/f, downstream congestion can turn it 
BB:into <11> again. So <01> behaves like <10>  ('unmarked') to 
BB:the network. All your I-D needs to say is that the *network* 
BB:treats <10> & <01> as semantically equivalent (but a 
BB:multicast router can turn 11 into 01 (experimentally)). The 
BB:hosts can treat <01> as 'potentially marked' in multicast 
BB:and 'unmarked' in unicast. 
BB:
BB:These could be the standard default meanings of <01>, but 
BB:you could allow hosts to agree their own meaning overriding 
BB:this between themselves, as long as it didn't conflict with 
BB:the network's semantics.
BB:
BB:Could EVEN have a scheme to use the nonce in multicasts with 
BB:ack feedback (getting complicated tho, but only for the 
BB:hosts):
BB:- If the sender guaranteed a equal ratio of <01>:<10> over a 
BB:moving window of n packets, where say n/2 randomly selected 
BB:ones are affected by the nonce and padding is inserted into 
BB:the others to balance the nonce packets.
BB:- As before, a multicast router would take an incoming 11 
BB:(nonce already lost), copy it to one i/f and write 01 to all 
BB:other i/fs
BB:- Receiver would detect the difference between number of 01 
BB:& 10 packets over moving window of n packets as the level of 
BB:'potentially marked' traffic.
BB:
BB:Issue:
BB:- The level of potentially marked traffic is fairly slow to 
BB:measure, and noisy given losses too, but may be enough to 
BB:decide to drop a layer (in e.g. RLM) or select the 'slowest' 
BB:receiver (e.g. in pgmcc) both of which have some hysteresis 
BB:anyway.

SF>>>(3) Incorporate all of the nonce text in addition
SF>>>	to making the changes required for (2).
SF>>>
SF>>>Our opinion is that option (3) would be premature, but we think
SF>>>that option (2) is in fact the best use for that extra codepoint,
SF>>>and today I am in the process of modifying the ECN draft to add
SF>>>this definition for the fourth codepoint.  The draft will also
SF>>>include a discussion of the other competing proposals for that
SF>>>codepoint, including yours.  Some of the others are listed below.

BB>>To mimic what you have said to me, this could slow down the
BB>>move to standardisation. I'd prefer the solution I give: 
BB>>ensure everyone treats the field as 4 code-points not 2 bits, 
BB>>and leave <ECT=0,CE=1> as an experimental code-point. 
BB>>(See my IANA section at the end.)

SF>>>Allison Mankin ([email-masked]) is the IESG Area Director handling
SF>>>this, so you should feel free to talk with her.  The modified ECN
SF>>>draft defining the fourth codepoint will receive another Last Call
SF>>>in the TSVWG working group.  So that the IETF/IESG can make
SF>>>an informed decision about whether or not to define the fourth
SF>>>codepoint for the ECN nonce.

BB>>I've just sent the draft to Allison/Scott pending it being 
BB>>uploaded (primarily to get a file-name).

SF>>>---------------------------------------------------------
SF>>>
SF>>>This is what I can think of as the other possible uses for
SF>>>the extra code-point:

SF>>>* Possible use #1:
SF>>>The extra ECN codepoint could indicate ECN-Capable not for the
SF>>>standard semantics of ECN, but for the alternate semantics.  However,
SF>>>this could also be signalled with diff-serv, in two possible ways:
SF>>>
SF>>>  (1) Diffserv method 1:
SF>>>  Use the standard ECN-capable code-point(s), but have a diffserv 
SF>>>  field that means: "use the alternate semantics for ECN".  This has
SF>>>  the drawback that a router that doesn't understand that diffserv
SF>>>  class will use the default ECN semantics.

BB>>Yes, by definition, DSCP is not end-to-end, except for some v. 
BB>>sought after points in the pool.

SF>>>  (2) Diffserv method 2: 
SF>>>  Don't use the standard ECN-capable code-point(s), but use diffserv
SF>>>  to signal ECT as well, for these alternate ECN semantics.  But use
SF>>>  the CE bit for the response.

BB>>Problem getting more real-estate from diffserv again.

SF>>>* Possible use #2:
SF>>>The extra ECN codepoint could be used by the router to indicate
SF>>>"very mild congestion", or something like that, giving an extra
SF>>>congestion signal from the router.  I would be perfectly happy to
SF>>>give this up, particularly as we are developing TCP-friendly
SF>>>congestion control mechanisms that don't halve their sending rate
SF>>>in response to a single drop.   And it seems likely that if something
SF>>>like this is developed, it will need more than an extra codepoint.
SF>>>But this tradeoff should certainly be made clear.

BB>>Surely you only need one code-point. You 'just' alter 
BB>>the semantic of only reacting to one mark per RTT. 
BB>>Instead, you react fractionally to each mark. We've 
BB>>already designed an alternative TCP/ECN protocol to do 
BB>>this (without soft state acks but still detecting ack 
BB>>losses), but I wanted to get the IP layer sorted first 
BB>>(we want to simulate the new TCP before we crow too loud 
BB>>- however it uses the TCP fields differently to the 
BB>>Proposed Standard - but we have to take our chances - we 
BB>>can't do all the research yesterday).

SF>>>* Possible use #3:
SF>>>The extra ECN codepoint could be used to signal corruption instead
SF>>>of congestion:
SF>>>"The data payload has been deleted, but the packet with this header
SF>>>was lost in the network due to corruption and not congestion".
SF>>>Sent by the link-level sender on the corrupted link.  But I could
SF>>>imagine that this one would need more than a single codepoint also.

BB>>Use ICMP for this, surely? This gives the space you 
BB>>need to say what you want, and copies the header.

SF>>>* Possible use #4:
SF>>>Multicast congestion control.

BB>>Is this our idea, or something else?

BB>>>David,
BB>>>
BB>>>At 20:56 21/02/01 -0500, [email-masked] wrote:
DB>>>>-- Section 6
DB>>>>
DB>>>>Section 6 on Multicast forwarding describes work that
DB>>>>is at the stage of "informal discussion in the research
DB>>>>community", and hence is surely premature for standardization.
DB>>>>OTOH, if we regard multicast duplication as being a separate

JC>>>not true - the IETF RMT working group is actively working on protocols
JC>>>in the standardazation track (if it was still research ,the IESG would
JC>>>never have let it move from RMRG  to RMT) - the  use of ECN is not
JC>>>part of that work yet, but a lot of the people there have the skills
JC>>>to use it if its deployed in the IP level, and there have been several
JC>>>people with a lot of experience who have suggested it would help a lot

SF>>>Yep.  And many of those proposals (including the use of ECN with
SF>>>the multicast extensions of equation-based congestion control that
SF>>>I have been working on) would use ECN with the exact same semantics
SF>>>as for unicast.   
SF>>>
SF>>>My own inclination, however, would be not to have a strong push at
SF>>>this time to make it easy for any application to tell IP to set
SF>>>the ECT codepoint in a packet.  I think it is a little premature,
SF>>>since router mechanisms have not yet been deployed to protect the
SF>>>network against misbehaving flows (though I have been working on
SF>>>it - you could see the draft paper on "Controlling High-Bandwidth
SF>>>Flows at the Congested Router" listed on my web page).  Until such
SF>>>router mechanisms are deployed, I am not inclined to be rushing to
SF>>>try to make it easy for unreliable, non-congestion-controlled
SF>>>applications not in the kernel to just blithely set the ECT codepoint
SF>>>in a packet.
SF>>>
SF>>>Instead, I have been talking with Mark Handley and others, who
SF>>>are writing a rough first pass of an internet-draft, to be
SF>>>submitted tomorrow, about a new UDP-ish protocol for unreliable
SF>>>unicast transfer that incorporates ECN and end-to-end congestion
SF>>>control.  Taking it one step at a time, starting with the deployment
SF>>>of ECN in TCP, and then in a new UDP-ish protocol, and then
SF>>>in more places, makes the most sense to me.  But I agree that the
SF>>>greatest power for ECN in the long run will be not with TCP,
SF>>>but with unreliable unicast, unreliable multicast, and reliable
SF>>>multicast applications.
SF>>>
SF>>>As for the ECN nonce, some of the multicast applications might even
SF>>>find an ECN nonce useful.  For example, the ack-based variants of
SF>>>multicast congestion control could probably use the nonce perfectly
SF>>>effectively, though I must say I haven't thought it through...
SF>>>
SF>>>- Sally

JC>>>so this is good - i think we are all pretty much trying to get to the
JC>>>same page...
JC>>>
JC>>>i guess we need to talk f2f soonest to try and avoid messing up the
JC>>>various timescales of work!

DB>>>>action from the forwarding for which ECN marking is specified,
DB>>>>then both the nonce and the "potentially marked" multicast
DB>>>>usage require the same behavior in the output queue (if
DB>>>>congestion would result in dropping a packet with
DB>>>><ECT,CE> = <0,1>, the packet may be marked as an ECN
DB>>>>notification [i.e., <ECT,CE> = <1,1>] and forwarded instead).
DB>>>>If this is specified, then the unicast vs. multicast
DB>>>>differences at the end systems follow from the fact that
DB>>>>TCP doesn't work over multicast - I don't think we need
DB>>>>to change any wording to accomplish this, although it
DB>>>>wouldn't hurt to add a sentence pointing out that how
DB>>>>end systems treat multicast traffic that uses ECN is a
DB>>>>research topic.  So, I'd propose specifying the router
DB>>>>behavior for <ECT,CE> = <0,1> that's needed for both
DB>>>>the nonce and multicast, and then making it clear that
DB>>>>both multicast duplication and how end systems handle
DB>>>>multicast traffic that uses ECN are research topics and
DB>>>>hence out of the scope of this document.  Note that how
DB>>>>the nonce mechanism works on end systems will be in
DB>>>>a separate document in any case.

BB>>>My latest draft says this now.

JC>>>I disasgree - most of the RMT WG is hedged about with phrases about
JC>>>"TCP friendly" or "TCP equivalent" - if RMT is to make any preogress
JC>>>and you have ECN-capabile IP and TCP deployed, then we MUST start to
JC>>>consider the use of ECN for non TCP protocols SOONEST.

DB>>It looks like a number of us have converged on the solution
DB>>of specifying the behavior of the fourth code point in the
DB>>network in a fashion that works for both the nonce and
DB>>multicast "probably marked" scenario.  We need to be careful
DB>>to say that we are not specifying the behavior of ECN for
DB>>multicast packet duplication.  FWIW, Jon's exposition on
DB>>the state of multicast standardization is actually arguing
DB>>with Bob's text, rather than with me :-).

BB>Hence my reply saying, make sure you make this clear.

________________________________________________________
Notice: This contribution is the personal view of the 
author and does not necessarily reflect the technical nor 
commercial direction of British Telecommunications plc.
________________________________________________________
Bob Briscoe      http://www.labs.bt.com/people/briscorj/

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg