Subject: Re: [tsvwg] confirming adoption of L4S drafts

From: "De Schepper, Koen (Nokia - BE/Antwerp)" <[email-masked]>
Date: Sun, 2 Apr 2017 11:24:27 +0000

Hi Michael,

Thanks for bringing up these points. See inline:

From: tsvwg [mailto:[email-masked]] On Behalf Of Michael Welzl
Sent: vrijdag 31 maart 2017 21:47
To: tsvwg IETF list <[email-masked]>
Subject: Re: [tsvwg] confirming adoption of L4S drafts

Hi all,

Sorry, I would have gone to the mic to say this yesterday, but time was running short, there were lots of others going there, AND I was scribing… hence now this email:

I’m a little worried that we’re adopting a behavior inside the network, without specifying how end systems should work with it.

[KDS] Currently the L4S Identifier draft is specifying how a CC should behave and respond to CE, when it uses the L4S-ECT code point. The idea is to write it rather in requirements than specific mechanisms. Probably also the correct place to write down the TCP-Prague requirements (currently they are a bit lost in an appendix in the DualQ draft). A separate draft can then be made per specific L4S complaint CC to explain its mechanisms. Would that help?

In particular, what makes me feel uneasy is slide 2 from yesterday’s presentation:
https://www.ietf.org/proceedings/98/slides/slides-98-tsvwg-sessb-l4s-drafts-01.pdf
which showed TCP Prague as an ICCRG item. I think this misrepresents the role of an IRTF group: yes ICCRG can publish documents, but these have an IRTF boilerplate, and I don’t think that’s the right way to publish an e2e counterpart to an IETF Experimental RFC. As I explained in tsvarea earlier this week:
https://www.ietf.org/proceedings/98/slides/slides-98-tsvarea-sessb-discussion-on-congestion-control-work-in-the-ietf-the-role-of-iccrg-michael-welzl-00.pdf
… the more common role of ICCRG is to pre-evaluate proposals before they’re even ready for the IETF.

[KDS] Yes, sorry for the confusion. For TCP-Prague, the ICCRG reference was mainly to show where currently the work is presented (which I think is the right wg for this, not?). As I said in the tsvwg presentation, seen the recent iccrg and tsvwg discussions it is not clear to me if a CC draft will be needed. As far as I understood, if accepted by ICCRG the process allows a draft in TCPM.

Technically, these are not so minor issues. DCTCP, as it stands, doesn’t work on the Internet; there are Internet-specific problems that would first need to be addressed. Two cases were discussed yesterday:
1) how will this work when there are two bottlenecks in a row?
2) what if TCP Prague traffic gets into a “classic” ECN queue?
… and these are likely not the only ones. E.g., the second problem was identified long ago, at the L4S BoF - see slide 3:
https://www.ietf.org/proceedings/96/slides/slides-96-l4s-6.pdf
where it says "Fall-back to classic TCP on classic ECN bottleneck”. Worryingly, the “starting point” column says “none”, and so far it doesn’t seem that we have an answer to this question.
=> what will this do to the ongoing effort of Apple to enable (classic) ECN?

[KDS] Indeed, DCTCP can be used with DualQ in fair-share mode with classic TCP, but in a controlled environment only. This could be in a data center, or in a service provider’s network, where the congestion points are known and support L4S. Extra L4S identification (diffserv codepoints/IP ranges) could be used if needed. As, I think Gorry stated, the current DCTCP draft restricts its use to controlled environments only:
   DCTCP as described in this draft
   is applicable to deployments in controlled environments like
   datacenters but it MUST NOT be deployed over the public Internet
   without additional measures
To my understanding this allows the above described controlled service provider environment too.

Are your other concerns all covered by the TCP-Prague requirements? Should we add more?

Regards,
Koen.

Cheers,
Michael

On Mar 31, 2017, at 9:08 AM, Wesley Eddy <[email-masked]<mailto:[email-masked]>> wrote:

Hello, this is a note to confirm that due to the positve response at the last couple IETF meetings and on the mailing list, and currently no negative responses, we (TSVWG co-chairs) think there's consensus to adopt the set of 3 L4S drafts.

https://tools.ietf.org/html/draft-briscoe-tsvwg-l4s-arch
https://tools.ietf.org/html/draft-briscoe-tsvwg-aqm-dualq-coupled
https://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-l4s-id

We've asked the authors to wait a week, just to confirm this with the mailing list, before submitting the documents as draft-ietf-tsvwg-...

Please don't hesitate to start reviewing and discussing the open items Koen highlighted yesterday.

---

From: Michael Welzl <[email-masked]>
Date: Mon, 3 Apr 2017 11:08:32 +0200

Hi,

In line:

> On 02 Apr 2017, at 13:24, De Schepper, Koen (Nokia - BE/Antwerp) <[email-masked]> wrote:
> 
> Hi Michael,
>  

I don't know? It's certainly a good starting point but I'm a little concerned that we'll end up with RFCs to describe an inner network behavior (and reserve a codepoint for it!), but maybe never with an end system mechanism *for the Internet* that uses it (because there are some issues to be addressed to which I've so far not even seen hints of a solution).

> [KDS] Yes, sorry for the confusion. For TCP-Prague, the ICCRG reference was mainly to show where currently the work is presented (which I think is the right wg for this, not?). As I said in the tsvwg presentation, seen the recent iccrg and tsvwg discussions it is not clear to me if a CC draft will be needed.

I heard you say this but I don't understand your point. Why should a CC draft not be needed?

> As far as I understood, if accepted by ICCRG the process allows a draft in TCPM.

I confirm that ICCRG is the right place to present such work, for now.
"if accepted by ICCRG the process allows a draft in TCPM" doesn't sound right. It's not like we (ICCRG) accept anything and that then automatically has a meaning in TCPM. We just review things until they are ready for the IETF. So given that this is one piece of a larger puzzle, I think we should first see a dedication from TCPM or TSVWG that they'd accept a CC draft following the requirements that you lay out (the envisioned draft about CC requirements that you mention above). Then I think that would be a better basis. But that's just my personal opinion.

I only wanted to make it clear that the ICCRG doesn't have any direct influence and presenting something there doesn't automatically mean anything in the IETF.

> To my understanding this allows the above described controlled service provider environment too.

Ah... understood. The "controlled service provider environment" scenario makes sense to me. To be clear, this would be about situations where both the sender and receiver are also within the same controlled ISP network. 

Yet: the intention of L4S is not to be limited to such scenarios only, I suppose?  If the plan is still to try this more openly across the Internet, then I think my concerns are unchanged...

>  Are your other concerns all covered by the TCP-Prague requirements? Should we add more?

Actually I think my concern is covered well by requirement #4.2 here: 
https://tools.ietf.org/html/draft-briscoe-tsvwg-l4s-arch-02#appendix-A

However I disagree with the discussion:

***
   Discussion: It is not clear at this point if it is possible to design
   a mechanism that always detect the aforementioned cases.  One
   possibility is to base the detection on an increase on top of a
   minimum RTT, but it is not yet clear which value should trigger this.
   Having a delay based fall back response on L4S may as well be
   beneficial for preserving low latency without legacy network nodes.
   Even if it possible to design such a mechanism, it may well be that
   it would encompass additional complexity that implementers may
   consider unnecessary.  The need for this mechanism depends on the
   extent of classic ECN deployment.
***

I can't understand how having such a mechanism would be unnecessary: that would be assume that EVERYONE who would enable ECN would ONLY enable it using L4S.
(there's no old code out there that treats ECT(1) and ECT(0) equal, and nobody would just turn it on - even today that's probably already wrong).

I also can't understand why it might not be possible: I'd say it's a strong requirement. What might happen is that you get false positives - so be it. Then you'll just have to be conservative.
BTW I guess there's more input you could use - possibly the pattern of returning ECN-marks can also be a hint as to the use of an L4S vs. classic ECN queue.

Cheers,
Michael

---

From: Wesley Eddy <[email-masked]>
Date: Fri, 7 Apr 2017 11:43:40 -0400

Hi Michael, as co-chair shepherding these documents in TSVWG, I want to 
make sure if we're okay to proceed.

You're obviously right that the corresponding scalable end-host behavior 
is work that needs to be done, and I agree that where that work is done 
in terms of WG, RG, etc. will need to be figured out, but I see those 
more as action items for follow-up in parallel, rather than roadblocks 
to proceeding with adopting work on the in-network behavior.  I think 
the ADs will make sure this works out (and if it's not working out in 
time, they could kill off the L4S work at some point).  I think your 
concern about defining the in-network behavior and codepoint without an 
end-host algorithm is something we should track as that work proceeds in 
parallel, and evaluate when we think L4S is mature enough for a last call.

Certainly nobody would want to just go ahead and publish RFCs here, if 
it turns out to that there isn't promise of mature end-host algorithm 
work to go with it!  I think WG participants, ADs, chairs, etc, can 
evaluate this as the work proceeds, but it shouldn't gate adoption.

In my personal opinion, the ability to support DCTCP in private/managed 
networks is probably enough motivation alone, and that something for 
Internet end-hosts will obviously make it even better, but is just an 
expansion of the use cases rather than absolutely required to see value 
out of L4S.

Would you agree that we can assume end-host algorithm research will keep 
proceeding (somewhere to be defined) in parallel, and that TSVWG should 
be doing these Experimental+Informational in-network pieces in the meantime?

---

From: "Black, David" <[email-masked]>
Date: Sat, 8 Apr 2017 01:11:29 +0000

A few comments as a WG chair ...

The concern about what transport protocol to use with L4S was raised in discussion in Chicago.

> "if accepted by ICCRG the process allows a draft in TCPM" doesn't sound
> right. It's not like we (ICCRG) accept anything and that then automatically has
> a meaning in TCPM. We just review things until they are ready for the IETF.

Process point acknowledged, although I had read Koen's use of "accepted" loosely as ICCRG generally having positive feedback on the work.  I suspect everyone's in agreement here, as while ICCRG does not have a formal IETF process role, if ICCRG has negative views on a congestion-related draft, then that draft is unlikely to make much progress in any IETF WG in the Transport Area.

> Ah... understood. The "controlled service provider environment" scenario
> makes sense to me. To be clear, this would be about situations where both
> the sender and receiver are also within the same controlled ISP network.

Yes, definitely, and also in related networks under single administrative control.  This "controlled service provider environment" scenario is anticipated by the new UDP Guidelines RFC (RFC 8085), and worked examples of applicability to that scenario can be found in MPLS/UDP (RFC 7510) and GRE/UDP (RFC 8086).

Thanks, --David

> -----Original Message-----
> Sent: Monday, April 3, 2017 5:09 AM
> To: De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper@nokia-
> bell-labs.com>
> Cc: tsvwg IETF list <[email-masked]>
> > On 02 Apr 2017, at 13:24, De Schepper, Koen (Nokia - BE/Antwerp)
> <[email-masked]> wrote:
> >
> > Hi Michael,
> >
> > Sorry, I would have gone to the mic to say this yesterday, but time was
> running short, there were lots of others going there, AND I was scribing…
> hence now this email:
> >
> > I’m a little worried that we’re adopting a behavior inside the network,
> without specifying how end systems should work with it.
> >
> > [KDS] Currently the L4S Identifier draft is specifying how a CC should
> behave and respond to CE, when it uses the L4S-ECT code point. The idea is
> to write it rather in requirements than specific mechanisms. Probably also
> the correct place to write down the TCP-Prague requirements (currently they
> are a bit lost in an appendix in the DualQ draft). A separate draft can then be
> made per specific L4S complaint CC to explain its mechanisms. Would that
> help?
> 
> I don't know? It's certainly a good starting point but I'm a little concerned that
> we'll end up with RFCs to describe an inner network behavior (and reserve a
> codepoint for it!), but maybe never with an end system mechanism *for the
> Internet* that uses it (because there are some issues to be addressed to
> which I've so far not even seen hints of a solution).
> 
> 
> > which showed TCP Prague as an ICCRG item. I think this misrepresents the
> role of an IRTF group: yes ICCRG can publish documents, but these have an
> IRTF boilerplate, and I don’t think that’s the right way to publish an e2e
> counterpart to an IETF Experimental RFC. As I explained in tsvarea earlier this
> week:
> > https://www.ietf.org/proceedings/98/slides/slides-98-tsvarea-sessb-
> discussion-on-congestion-control-work-in-the-ietf-the-role-of-iccrg-michael-
> welzl-00.pdf
> > [KDS] Yes, sorry for the confusion. For TCP-Prague, the ICCRG reference
> was mainly to show where currently the work is presented (which I think is
> the right wg for this, not?). As I said in the tsvwg presentation, seen the
> recent iccrg and tsvwg discussions it is not clear to me if a CC draft will be
> needed.
> 
> "if accepted by ICCRG the process allows a draft in TCPM" doesn't sound
> right. It's not like we (ICCRG) accept anything and that then automatically has
> a meaning in TCPM. We just review things until they are ready for the IETF.
> So given that this is one piece of a larger puzzle, I think we should first see a
> dedication from TCPM or TSVWG that they'd accept a CC draft following the
> requirements that you lay out (the envisioned draft about CC requirements
> that you mention above). Then I think that would be a better basis. But that's
> just my personal opinion.
> 
> I only wanted to make it clear that the ICCRG doesn't have any direct
> influence and presenting something there doesn't automatically mean
> anything in the IETF.
> 
> 
> >  Technically, these are not so minor issues. DCTCP, as it stands, doesn’t
> work on the Internet; there are Internet-specific problems that would first
> need to be addressed. Two cases were discussed yesterday:
> > … and these are likely not the only ones. E.g., the second problem was
> identified long ago, at the L4S BoF - see slide 3:
> > where it says "Fall-back to classic TCP on classic ECN bottleneck”.
> Worryingly, the “starting point” column says “none”, and so far it doesn’t
> seem that we have an answer to this question.
> > [KDS] Indeed, DCTCP can be used with DualQ in fair-share mode with classic
> TCP, but in a controlled environment only. This could be in a data center, or in
> a service provider’s network, where the congestion points are known and
> support L4S. Extra L4S identification (diffserv codepoints/IP ranges) could be
> used if needed. As, I think Gorry stated, the current DCTCP draft restricts its
> use to controlled environments only:
> Ah... understood. The "controlled service provider environment" scenario
> makes sense to me. To be clear, this would be about situations where both
> the sender and receiver are also within the same controlled ISP network.
> 
> Yet: the intention of L4S is not to be limited to such scenarios only, I
> suppose?  If the plan is still to try this more openly across the Internet, then I
> think my concerns are unchanged...
> 
> 
> I can't understand how having such a mechanism would be unnecessary: that
> would be assume that EVERYONE who would enable ECN would ONLY enable
> it using L4S.
> (there's no old code out there that treats ECT(1) and ECT(0) equal, and
> nobody would just turn it on - even today that's probably already wrong).
> 
> I also can't understand why it might not be possible: I'd say it's a strong
> requirement. What might happen is that you get false positives - so be it.
> Then you'll just have to be conservative.
> BTW I guess there's more input you could use - possibly the pattern of
> returning ECN-marks can also be a hint as to the use of an L4S vs. classic ECN
> queue.
> 
> Cheers,
> Michael

---

From: Michael Welzl <[email-masked]>
Date: Sat, 8 Apr 2017 11:00:56 +0200

Hi,

Yes to everything here.

My crystal ball says that the work on the necessary heuristics might never really happen, which is what made me worry: then, what’s the use of L4S altogether?
However, that doesn’t matter much in the “controlled service provider” environment, and this convinces me; sorry, I hadn’t really thought this scenario through. My bad.

So just to restate, I do agree that it would be good to adopt this work in TSVWG.

Cheers,
Michael

> On Apr 7, 2017, at 5:43 PM, Wesley Eddy <[email-masked]> wrote:
> 
> Hi Michael, as co-chair shepherding these documents in TSVWG, I want to make sure if we're okay to proceed.
> 
> You're obviously right that the corresponding scalable end-host behavior is work that needs to be done, and I agree that where that work is done in terms of WG, RG, etc. will need to be figured out, but I see those more as action items for follow-up in parallel, rather than roadblocks to proceeding with adopting work on the in-network behavior.  I think the ADs will make sure this works out (and if it's not working out in time, they could kill off the L4S work at some point).  I think your concern about defining the in-network behavior and codepoint without an end-host algorithm is something we should track as that work proceeds in parallel, and evaluate when we think L4S is mature enough for a last call.
> 
> Certainly nobody would want to just go ahead and publish RFCs here, if it turns out to that there isn't promise of mature end-host algorithm work to go with it!  I think WG participants, ADs, chairs, etc, can evaluate this as the work proceeds, but it shouldn't gate adoption.
> 
> In my personal opinion, the ability to support DCTCP in private/managed networks is probably enough motivation alone, and that something for Internet end-hosts will obviously make it even better, but is just an expansion of the use cases rather than absolutely required to see value out of L4S.
> 
> Would you agree that we can assume end-host algorithm research will keep proceeding (somewhere to be defined) in parallel, and that TSVWG should be doing these Experimental+Informational in-network pieces in the meantime?
> 
> 
> 
>

---

From: Wesley Eddy <[email-masked]>
Date: Sun, 9 Apr 2017 16:31:41 -0400

Ok, thanks for everyone's feedback.  Since it's been a week, and there 
are no objections, I think we've fully confirmed the adoption call, and 
that the L4S editors should submit the next versions as 
draft-ietf-tsvwg-$X, and I"ve marked them as adopted in the datatracker.