Subject: FW: Last Call: <draft-farrell-ft-03.txt> (A Fast-Track way to RFC with Running Code) to Experimental RFC

From: "Adrian Farrel" <[email-masked]>
Date: Fri, 11 Jan 2013 16:21:49 -0000

Hi Alexa,

Please be aware of this document that has just entered a four-week IETF last
call. The document describes a proposed IETF process experiment under the rules
of RFC 3933.

The proposed experiment calls on the IETF Secretariat to take specific actions
under certain circumstances in corner cases of the experiment. Could you please
have someone in the Secretariat look at the draft and comment on the
practicalities of the actions. Note that, at this stage, no changes to the tools
are proposed so any actions would require manual intervention (if the experiment
were successful and resulted in permanent changes to IETF process we might make
changes to the tools at some future time).

Thanks,
Adrian

> -----Original Message-----
> From: [email-masked] [mailto:ietf-announce-
> [email-masked]] On Behalf Of The IESG
> Sent: 11 January 2013 15:15
> To: IETF-Announce
> Subject: Last Call: <draft-farrell-ft-03.txt> (A Fast-Track way to RFC with
Running
> Code) to Experimental RFC
> 
> 
> The IESG has received a request from an individual submitter to consider
> the following document:
> - 'A Fast-Track way to RFC with Running Code'
>   <draft-farrell-ft-03.txt> as Experimental RFC
> 
> The IESG plans to make a decision in the next few weeks, and solicits
> final comments on this action. Please send substantive comments to the
> [email-masked] mailing lists by 2013-02-08. Exceptionally, comments may be
> sent to [email-masked] instead. In either case, please retain the
> beginning of the Subject line to allow automated sorting.
> 
> Abstract
> 
>    This memo describes an optional, fast-track way to progress a working
>    group document to IESG review.  It is provided as a process
>    experiment as defined in RFC 3933 for use when working group chairs
>    believe that there is running code that implements a working group
>    Internet-Draft.  The motivation is to have the IETF process
>    explicitly consider running code, consistent with the IETF's overall
>    philosophy of running code and rough consensus.
> 
>    In this process all of working group last call, IETF last call, and
>    Area Director review are carried out in the same two week period.
>    Only comments that meet IESG Discuss criteria need to be addressed
>    during this stage, and authors are required to make any changes
>    within two weeks.
> 
>    This experiment will run for one year.
> 
> 
> The file can be obtained via
> http://datatracker.ietf.org/doc/draft-farrell-ft/
> 
> IESG discussion can be tracked via
> http://datatracker.ietf.org/doc/draft-farrell-ft/ballot/
> 
> No IPR declarations have been submitted directly on this I-D.

---

From: Stephan Wenger <[email-masked]>
Date: Fri, 11 Jan 2013 21:02:09 +0000

Hi,

Sorry for replying to this "advise to secretariat" thread and not to the
ietf-announce thread--I'm not subscribed to ietf-announce.
I have three comments, and regret that I have not followed all of the
discussions regarding this draft before, so please advise if those
comments have already been raised and/or resolved.

First, I'm glad that the direct preferences of open source implementations
over implementations compliant with other business models are mostly gone.
 Still, there is one reference that worries me, and that is the reference
to GPLv3 code as an "extreme" in section 2.1.  Yes, the GPL (and similar
copyleft licenses) is an extreme, at least in terms of open source
licensing models.  However, it is not an extreme of openness or
accessibility of the source code for review by WG chair, AD, and
community.  I would hope that we are all aware that many (most?)
commercial software developers, by company policy or common sense, avoid
looking at GPL-ed code, out of fear of contamination of their own closed
source code.  GPL-ed code  is, therefore, inaccessible for verification by
a large part of the IETF community, and does not serve as a good example
for "openness", which is how I interpret the spectrum laid out in section
2.1.  A better example would be source code that is almost universally
accessible.  The extreme here would be source code in the public domain.
Somewhat less convincing but perhaps a bit more realistically, source code
under a BSD-style license like the one the IETF Trust is using.

Second, the draft suggest that the existence of an implementation of the
specification should serve as a shortcut towards RFC, presumably because
such an implementation speak favorably to the implementability of the
specification.  That, however, is not universally true.  Specifically, if
implementer and spec writer are the same person (or part of the same
team), it is quite likely that the spec takes certain assumptions made by
the implementers for granted, and does not document it.  That would be a
bad spec, accessible implementation or not.  The solution to this issue is
IMO not, as the draft appears to be to suggest, to put burden on WG chairs
and ADs to ensure that the spec and the implementation match.  Both WG
chairs and ADs have enough to do, and the incentive for rubber-stamping is
quite high.  A more sensible solution may be to require that the
implementer is unaffiliated with the spec author; in other words, the
implementation is derived from the spec + IETF discussion context.

A third comment would be that, if you interpret the draft strictly, it is
highly unlikely that the experiment would ever be exercised, as the
implementation needs to "match" the draft to be advanced, and that would
mean that the implementation has to follow in lockstep with the draft
development up until the final version.  With respect to the core subject
matter of a draft, that may be fine.  However, many of the final edits in
a draft come as input from IETF wide community review; things like
security, congestion control, and the like.  Those are often trivial to
write down, but can have a major implementation impact.  To cure this, it
would IMO be acceptable if the implementation would be required to match
the latest or a reasonably young, i.e. a few months old version of the
draft.

Please consider this.
Thanks,
Stephan

---

From: "Eggert, Lars" <[email-masked]>
Date: Sat, 12 Jan 2013 09:35:26 +0000

Full agreement with Stephan.

Lars

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 12 Jan 2013 18:32:59 +0000

Hiya,

> over implementations compliant with other business models are mostly gone.

FWIW, I'm not at all glad about that personally but the text we have
probably does represent a consensus. Note, there are others like me
who'd prefer to favour open-source here, as well as some, presumably
including you, that would prefer something like the opposite.

> source code.  

<rant>

I am aware that lawyers and others do encourage ignorance in such
ways. Its an incredibly stupid way to behave IMO even if understandable
in a horrendously short-sighted kind of way.

Thankfully I don't work in a place that has to deal with such
anti-scientific idiocy. (Being a university we have our own
idiocies:-) And there are even many commercial companies that
have yet to adopt such silliness. So there's hope for us all
yet.

</rant>

> GPL-ed code  is, therefore, inaccessible for verification by

I'm not at all sure what concrete suggestion you're making, other
than disparaging GPL. If your suggestion is s/GPLv3/BSD license/
then that could be done but would make no difference at all to
this draft, so I plan to leave this as-is.

> specification.  That, however, is not universally true.  

Very few things in the IETF are universally true;-)

> Specifically, if

That's a fair point but, a) we don't consider affiliations like that
in the IETF, at least not in a written-rules kind of way, and b) there's
nothing in principle wrong with the editor writing code. While it
is better if the code is written independent of the editor and even
better if someone who hasn't been involved in the WG has done the
coding, that'd be too high a burden to require IMO, especially given
that this is an experiment.

I'd have no problem adding text that encouraged some form of
independence though, if you'd like to provide some.

> the latest or a reasonably young, i.e. a few months old version of the

Another fair point but again one where its not really possible to
describe a rule that'd be better and more precise than "match."
I don't think a rule mentioning earlier or older drafts would be
useful for this.

I'll see if I can come up with something better than "match" but
if you have text to suggest, that might help.

Cheers,
S.

---

From: Stephan Wenger <[email-masked]>
Date: Mon, 14 Jan 2013 07:50:37 +0000

Hi,
Please see inline.
Stephan

>On 01/11/2013 09:02 PM, Stephan Wenger wrote:
[...]
>in a horrendously short-sighted kind of way.

I understand that this is a rant.  And, I'm not ranting back, even if
tempted.  Well, maybe a little.  That said, I would like to understand the
"ignorance" part.  

It is factually true that there is a risk of contamination, and a risk of
being required to publish formerly closed source code under a very onerous
license (patent rights, whatnot) when a developer exposes himself/herself
to GPLed code.  I'm not including citation showing this at this point
(call it time and space constraints, and off-topic--but it's easy enough
to do), but the logic is approximately: copyleft licenses require to grant
back derivative work in source code form, and inclusion of GPLed code
fragments into your source code can be viewed as derivative work.
Publishing source code can (and in most cases, is) not a strategy closed
source companies follow.

If a Free Software developer, probably with the help of folks like the
SFLC, sue your company over alleged creation of derivative work from GPLed
code without granting that source code back, you need to defend yourself,
or publish the code under the GPL with all its onerous conditions (again,
patent license and whatnot).  Whether fair use, de minimis, or similar
theories apply for a specific case depends on the amount of included GPLed
code, legislation, the quality of your lawyer and the one on the other
side, and so on.  In most cases of accidental contamination, it is IMO
likely that you get of the hook easily enough, and even more likely that
no one is suing you.  Still, even the smallest legal action is costly.

For this reason, in most closed source companies I'm aware off, GPLed code
is off-limits for developers and/or for all employees, perhaps minus those
folks in the IT department which are not involved in product development
but deal with production servers and such.  It's common business sense.

It's OK if you don't like that business model.  It's ok if you like "Free
Software" and its business model.  Form a religion about the so-called
"Free Software" model, for all I care.  (Oh wait, there is already one :-)
 However, I do not believe that your draft, and especially the section in
question, is the right place to do so.

[...]
>this draft, so I plan to leave this as-is.

Yes, I'm arguing in favor of replacing "GPL" with "BSD", or, even better,
with "public domain".  To rephrase the reason for it: ideally, in order to
verify "matching" between software and specification, the software would
need to be accessible to as many people (in practice: IETF participants)
as possible.  One extreme is correctly identified: closed source code is,
in almost all cases, accessible only to a very small group of people.  The
other extreme is "public domain" source code, where no reasonable lawyer
or business guy has a reason to tell their developers to not touch it.
BSD licensed code comes close, and is more common.

The GPLed code is not at one of the extremes, but middle ground.  I would
wager a guess that perhaps less than half of the IETF participants are
allowed to check GPLed code for something as business-uncritical as
fast-tracking a draft.  Therefore, GPLed code should not be an example for
universally accessible and verifiable code.

Clearer?  Please reconsider your position.

>in the IETF, at least not in a written-rules kind of way,

"Affiliation" was not meant as "same employer". Clearly, same team in the
same company is a problem.  Co-developer in the same (small) open source
project is a problem.  US-based Office of CTO full-time standardizer (of
which the IETF has a fair share, we may like it or not) and code-grunts in
China are IMO not a problem, even if they work for the same mega-company.
There may be a better word than "affiliated" to describe this
relationship.  That word escapes me, though.  (Standard excuse: not an
English native speaker.)

>and b) there's
>nothing in principle wrong with the editor writing code.

For the purpose of short-cutting an approval mechanism (and only in this
context), I disagree, for the reasons already mentioned.  Outside of that
context, I'm fully with you, and I wish there would be more of those
multi-purpose types.

>While it

Ack.

>independence though, if you'd like to provide some.

How about:

"If the source code has been developed independently of the authoring of
the draft (and ideally by non WG participants), it is likely that the
implementation and the draft match, and that pitfalls unaware developers
may find have been found and dealt with.  If, on the other hand, draft
author(s) and implementation developer(s) overlap, then it is sensible to
scrutinize the draft more closely, both with respect to its match with the
implementation and for assumptions that author/developer may have taken
for granted which warrant documentation in the draft."
 
>if you have text to suggest, that might help.

Trying:

"Match means that all, or substantially all, protocol mechanisms of the
draft are implemented, that no other code points are implemented that
would reasonably fall into the scope of the draft in question, that all
documented state machines are implemented and no other state machines, and
so forth.  The over-the-wire behavior of the implementation and of the
draft should substantially match, including more subtle points such as
timing relationship of messages, .  Minor divergences in details stemming
from unaligned development cycles of draft and implementation are
acceptable."

>>> call. The document describes a proposed IETF process experiment under
>>>the
>>> rules
>>> of RFC 3933.
>>>
>>> under certain circumstances in corner cases of the experiment. Could
>>>you
>>> please

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Mon, 14 Jan 2013 01:08:28 -0800

On 01/11/2013 01:02 PM, Stephan Wenger wrote:
> ietf-announce thread--I'm not subscribed to ietf-announce. I have three
> comments, and regret that I have not followed all of the discussions
> regarding this draft before, so please advise if those comments have
> already been raised and/or resolved.
> 
> 
> Still, there is one reference that worries me, and that is the reference to
> GPLv3 code as an "extreme" in section 2.1.  Yes, the GPL (and similar 
> accessibility of the source code for review by WG chair, AD, and community.
> I would hope that we are all aware that many (most?) commercial software
> developers, by company policy or common sense, avoid looking at GPL-ed
> code, out of fear of contamination of their own closed source code.  GPL-ed
> code  is, therefore, inaccessible for verification by a large part of the
> IETF community,

Commercial software developers can do whatever they want, how idiotic it is
(see Oracle vs Google), that does not make them an example to follow.

I think that you underestimate the IETF community, who certainly know how to
see through all the FUD about the GPL.  Sure it may be a bad idea to literally
copy 300 lines of GPL code in your code, but that does not apply to what we
are talking about, which is reading code.

> and does not serve as a good example for "openness", which is how I
> interpret the spectrum laid out in section 2.1.  A better example would be
> source code that is almost universally accessible.  The extreme here would
> be source code in the public domain. Somewhat less convincing but perhaps a
> bit more realistically, source code under a BSD-style license like the one
> the IETF Trust is using.

- -- 
Marc Petit-Huguenin
Email: [email-masked]
Blog: http://blog.marc.petit-huguenin.org
Profile: http://www.linkedin.com/in/petithug

---

From: "Eggert, Lars" <[email-masked]>
Date: Mon, 14 Jan 2013 09:10:52 +0000

Hi,

> are talking about, which is reading code.

I have worked for employers before where reading GPL code was considered highly problematic.

Lars

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Mon, 14 Jan 2013 01:33:45 -0800

On 01/14/2013 01:10 AM, Eggert, Lars wrote:
>> I think that you underestimate the IETF community, who certainly know how
>> to see through all the FUD about the GPL.  Sure it may be a bad idea to
>> literally copy 300 lines of GPL code in your code, but that does not
>> apply to what we are talking about, which is reading code.
> 

Sure, if the IETF is part of your job, and your employer prevents you to read
GPL code (or read Spinoza, or sing under the shower), then do not do it for
the IETF.  Now if your employer does not regulate what you do on your own
time, then there is no problem here.

If someone said to writers to never read any book that is still copyrighted
because they might be contaminated and that may somehow induce them to
plagiarize without knowing it, how do you think they would react?  There is no
difference with software - excepted that there a lot more people that would
like us to think that software is different.

- -- 
Marc Petit-Huguenin

---

From: Stephan Wenger <[email-masked]>
Date: Mon, 14 Jan 2013 09:43:28 +0000

Inline.
S.

On 1.14.2013 10:33 , "Marc Petit-Huguenin" <[email-masked]> wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA256
>
>plagiarize without knowing it, how do you think they would react?

Many would laugh.  Some politicians in Germany, who were recently found
that they plagiarized in the PhD theses, would probably notŠ  Their risk
is low, though.

>There is no

This is incorrect.  If you are found to infringe on someone's copyright in
a book you put together, you have a copyright problem (only).  If you are
found to infringe on certain open source licenses (including the GPL
variants, but by no means limited to them), and the rightholder forces you
to take a license, you also have to provide a patent license under terms
you may not like.  Even if you don't take a license, it is quite possible
that you have (by copying GPLed code) provided in implied patent license.

Please don't oversimplify.

Thanks,
Stephan

>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.4.12 (GNU/Linux)
>
>iQIcBAEBCAAGBQJQ89D3AAoJECnERZXWan7EticQAIXvbyfRD/LX9YT+/SiW21Wp
>NBuVji+aAPyGaKvwpQ+G1pqrp74hUk2ylvmqOGi9JWc0/E3H/dhyBbksvEHrbyY8
>1+4fGKtOAUkAW97IaBaWRQT9jbo7w9TytRVz8o3+Gh0b4qTmTHzBh76zXjS5v2RM
>qRHFcCnSitXMUV4Y+PjtP8ZjpTd/J4Ahwsw9xa07WeNQIrfQ3PHYX8tC2xrrAuHm
>1on3qI5FuumDjatWwNMhAb+/Sm0XLOiRJifjQxENOfOPW4JtkEcqi9Us3+7fNFMe
>Z06qtrCC2oflODJw+7xZwguIJe8V+Eu8ea3wNnVaa3tqqBw+twZT/vhBSehrCgrU
>hTSvj9zphutbRK/1xvEkMwggYhUPrwH7GTNKrANYYu8xb9O2MD2OB2OdrwtX4Aax
>Z11lBsY7GC0Vum3X/aa+3UWu2cyiBDcDJym+pOvzOYPGJ7aNpEAdlRIYVdE7KV1t
>1n7SrK960uX2WE9VgFdWptfm9QXGDjW417PDxlunNUzlY6Xb/C2uaQuuR6gZG7Vi
>P3I+Yiz2LvioE3ZwTPDSboLPgbw29g3oEkCBjyfP14KjP2F8MTUfI3S5vTl0ZrAh
>RaqkdT6sfREVxFFszyeAhDDAcXGkzvkZd1wgHwfw1HcKLlTUH6QNPVT5OGsA3DLQ
>7CYGpSyyjONrEgZqxBXD
>=/ZJe
>-----END PGP SIGNATURE-----
>

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 14 Jan 2013 09:46:42 +0000

Hiya,

>> <rant>
...
> tempted.  
...

Yes, its tempting, but I'm going to resist since its irrelevant IMO.

...

> Clearer?  Please reconsider your position.

Yes, that's clearer. We're talking about two different continuums
(or continua:-), so either would work, and neither is important to
this draft. I'll make the change if there's enough support for it,
but as I guess you can see, this is an area where we won't get folks
to entirely agree. In this case, we don't need to, since the text
in question is just explanatory and has no significant impact on
the experiment at all.

(BTW, I'll just note in passing that this kind of contentious
irrelevancy is exactly the kind of thing that an author/editor
running the experiment could freely ignore as not meeting the
discuss criteria. That'd be nice, but doesn't apply this time
unfortunately;-)

> For the purpose of short-cutting an approval mechanism (and only in this

s/short-cutting/speeding up/

> for granted which warrant documentation in the draft."

That'd be no harm to add. I don't know that it improves the document
enough to bother though. I'll think about it, but let's see if anyone
else cares.

> from unaligned development cycles of draft and implementation are

I like most of that, thanks. For now, I've added this to 2.1, let me
know if it works:

   We do not give a precise definition for "match" here but the intent
   is that all, or substantially all, protocol mechanisms of the draft
   are implemented, that the over-the-wire behavior of the
   implementation and of the draft should substantially match, including
   more subtle points such as timing relationship of messages, etc.
   Minor divergences in details stemming from unaligned development
   cycles of draft and implementation are acceptable.

Cheers,
S.

PS: There's a working draft at [1] in case that helps.

[1] http://down.dsg.cs.tcd.ie/misc/draft-farrell-ft-04.txt

---

From: Stephan Wenger <[email-masked]>
Date: Mon, 14 Jan 2013 10:03:07 +0000

Perhaps my final comment on this.  Also cutting the thread down to
something readable.  Inline.
S.

On 1.14.2013 10:46 , "Stephen Farrell" <[email-masked]> wrote:

[...]
>in question is just explanatory and has no significant impact on

Definitely my final comment on this topic:  I agree that the change has no
impact on the operation of the iETF under the draft.  IMO it simply fixes
a flaw of logic.  

[...]

>>>[Š]

Thanks.

[...]
>   cycles of draft and implementation are acceptable.

Works for me.

>>>>> Please be aware of this document that has just entered a four-week
>>>>>IETF
>>>>> last
>>>>> The proposed experiment calls on the IETF Secretariat to take
>>>>>specific
>>>>> actions
>>>>> practicalities of the actions. Note that, at this stage, no changes
>>>>>to
>>>>> the tools
>>>>>> Subject: Last Call: <draft-farrell-ft-03.txt> (A Fast-Track way to
>>>>>>RFC
>>>>>> with
>>>>> Running
>>>>>> [email-masked] mailing lists by 2013-02-08. Exceptionally, comments
>>>>>>may
>>>>>> be

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Mon, 14 Jan 2013 02:31:40 -0800

On 01/14/2013 01:43 AM, Stephan Wenger wrote:
> Inline. S.
>>>>> I think that you underestimate the IETF community, who certainly
>>>>> know how to see through all the FUD about the GPL.  Sure it may be
>>>>> a bad idea to literally copy 300 lines of GPL code in your code,
>>>>> but that does not apply to what we are talking about, which is
>>>>> reading code.
>>>> 
> Sure, if the IETF is part of your job, and your employer prevents you to 
> read GPL code (or read Spinoza, or sing under the shower), then do not do
> it for the IETF.  Now if your employer does not regulate what you do on
> your own time, then there is no problem here.
> 
> If someone said to writers to never read any book that is still 
> copyrighted because they might be contaminated and that may somehow induce
> them to plagiarize without knowing it, how do you think they would react?
> 

And?  Plagiarizing is extremely wrong.  But preventing people to read book is
not how any sane person would try to solve the plagiarism problem.

And yet that is what is tried - with some success as I discover - with software.

> 
> There is no difference with software - excepted that there a lot more
> people that would like us to think that software is different.
> 
>> This is incorrect.  If you are found to infringe on someone's copyright
>> in a book you put together, you have a copyright problem (only).  If you
>> are found to infringe on certain open source licenses (including the GPL 
>> variants, but by no means limited to them), and the rightholder forces
>> you to take a license, you also have to provide a patent license under
>> terms you may not like.  Even if you don't take a license, it is quite
>> possible that you have (by copying GPLed code) provided in implied patent
>> license.

Do you have a case to share where that happened with a GPL license?

- -- 
Marc Petit-Huguenin

---

From: Stephan Wenger <[email-masked]>
Date: Mon, 14 Jan 2013 11:45:35 +0000

Hi,
Please see inline.
Stephan

>not how any sane person would try to solve the plagiarism problem.

I agree.  

>Do you have a case to share where that happened with a GPL license?

It has been pointed out to me that the formulation "take a license" above
was misleading, and I agree.  What I meant is that the rightholder forces
you to re-license your code under GPL (what is sometimes called the
"viral" effect of the GPL).

As for your question, I vaguely recall an opinion which upheld the
requirement to grant a patent license for GPLv2(?) code (GPLv2 is not
pointing out the requirement to license patents as clearly as GPLv3 does).
 But I would have to dig,and have limited motivation to do so right now.
As for the implied patent license, I'm not aware of case law.

Stephan

>iQIcBAEBCAAGBQJQ896LAAoJECnERZXWan7EKHoQALpP81quorS9MiMD0D5bFfkS
>4B3sJS8VCCZJxasMNuZY1b0foJl1u/n1fQTuyle3/TD0Ld2yz3BXWk8fav/dlaBz
>5ZaGKRG3QzoDJG3afDmDPmP1bQ6u0riJ6U1V2QuzfiuIjp9h2Vzf/3dYSiqTGITB
>vBCImbpUkgKP7U7/SsfsPEIeBEdJLuaZ+DmPa0pQ8wLPjKEGJbvfNnMuNcBYIHkf
>b2uyEn1kcJbrkAsnWJkVWCGJukiHzbSDSRB7s9TWbLl4GXykBHZ9PvzNCARTrJD3
>WZ2n9/2roewaAgKP9I8xaGuTTepFnkciUgwGCFSKUNqb+j/oPtFnl0ng2OIFAIX2
>WBKQQMTYzu/tY6Rudwdnu6qBMN6Wag/wUG1QxDIIR/daA4ikmdZxJibZEeZ2FQ4f
>uf4QAgMfHBSPjQIQ6Zgm0ch36LxifewPyOlU/LXui0MTXB6QClKYFdqLNc9Mzpdf
>pB3K7KJtBEBTQshZ6Ad97f4S0PXupXcv8yJCrCJHYnh7BKL0GXcf3lnqmLh1TwFR
>18rtMHiKdcwcHzQSGOt0/ny4tJcsaNp3rtWN1XZVfrLPnh3THM6vXNCO0WkTwTXL
>w6uplJLhny/47RXRBEIPRHB+XDxKiDZ+JbVsWVfpfhlLCe2dMHMZpE7ELzEK9WuG
>nzBiUyNwdJge+C0nzedQ
>=Wkhd

---

From: Hector Santos <[email-masked]>
Date: Mon, 14 Jan 2013 12:05:13 -0500

I have two concerns and comments:

- How will success or failure be measured?  Number of appeal increases 
or lesser amount?  I have a concern that once this door is open, there 
will be increase appeals and also apathy of outcomes.  There should be 
a statement of what sort of problems or issues are possible and 
envisioned and the plans to deal with them or not.

- The drafts indicates that fast tracking probably applies better 
towards BIS work. I cited concerns that this ideas should only be used 
for new work, not BIS work where there is already established 
implementators and working code.  I prefer to work in "No Surprises" 
environment where it doesn't require me to pay attention to all 
protocol change work to make sure nothing fell thru the crack or some 
group of vendors with their questionable "code" implementation 
different from the norm being mandated in new BIS work.

I only see this RFC being used to negate voiced concerns in the name 
of fast tracking, not win people over if they already have a problem 
with the work, especially if its BIS WORK.  But then again, maybe that 
is the intent.

I believe the experiment should only be applied first to new protocol 
work, not BIS work and if so, iff there are absolutely no question of 
possible conflict of interest, rush to judgment and no new code change 
impacts.  Existing protocol implementers should not be put into a 
position, and worst within a shorter time period, to argue or just 
review what BIS changes made entail and force them into appeal actions.

Perhaps the experiment can be done in two phases. I believe there is 
good in fast tracking new ideas and methods, especially if simple and 
one that most people seem to agree with with little concerns. Many 
vendors have much to contribute in this area.  So the first phase can 
explore only with fast tracking new concepts. If this works out well 
using some measuring tool, then BIS work can be better explored for 
fast tracking by gaining some insights from the first phase.

--
HLS

-- 
HLS

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 14 Jan 2013 18:12:47 +0000

Hi Hector,

On 01/14/2013 05:05 PM, Hector Santos wrote:
> will be increase appeals and also apathy of outcomes.  There should be a
> statement of what sort of problems or issues are possible and envisioned
> and the plans to deal with them or not.

RFC 3933 describes that I guess. Basically, if the IESG consider
it a success then they could make the RFC into a BCP or more likely
find a victim to write an I-D for that. That'd go through the
usual process.

I don't think this'll have any impact on appeals (hope not anyway),
and for me the main metric of success will be whether or not it
gets used, and if used, if the people involved in those drafts
think its worth making permanent or not.

> towards BIS work. 

Not really. Its says that -bis RFCs might be suitable.

> I cited concerns that this ideas should only be used
> environment where it doesn't require me to pay attention to all protocol
> change work to make sure nothing fell thru the crack or some group of
> vendors with their questionable "code" implementation different from the
> norm being mandated in new BIS work.

I don't get the concern.

> I only see this RFC being used to negate voiced concerns in the name of
> fast tracking, not win people over if they already have a problem with
> the work, especially if its BIS WORK.  But then again, maybe that is the
> intent.
> 
> using some measuring tool, then BIS work can be better explored for fast
> tracking by gaining some insights from the first phase.

Ah, you're worried some WG will shoot out a -bis draft this way
and that existing implementers will miss that because of the
shorter timescale and no longer be "compliant" with the latest RFC?
Is that it?

To be honest, I can't really see that this'll have any impact
there. If the existing coder who'd be caught out isn't aware that
a WG is doing stuff then this will make no difference. If they
are aware that the WG is doing a -bis, then they can comment as
always before and during IETF LC. I really can't see this
experiment causing any real surprise in cases like that.

S.

>> IETF last
>> the rules
>> you please
>>> RFC with
>>> may be

---

From: Joe Touch <[email-masked]>
Date: Tue, 22 Jan 2013 08:39:20 -0800

Hi, all,

On 1/11/2013 8:21 AM, Adrian Farrel wrote:
> under certain circumstances in corner cases of the experiment. C

This is a silly idea.

First, running code should already be considered as part of the context 
of review.

Second, running code is not correlated to correctness, appropriateness, 
or safety. See Linux for numerous examples.

Third, running code doesn't mean the doc is sufficient that multiple 
parties can generate interoperable instances. It's merely the sound of 
one hand clapping ;-)

Finally, NOTHING should circumvent the multi-tiered review process. That 
process helps reduce the burden on the community at large via the 
presumption that smaller groups with more context have already reviewed 
proposals before they get to the broader community.

This is a bad idea even as an experiment.

Joe

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 22 Jan 2013 17:00:44 +0000

Hi Joe,

>> under certain circumstances in corner cases of the experiment. C

So you're in two minds about it eh:-)

> parties can generate interoperable instances. It's merely the sound of

Your second and third and points seem opposed to your first.
The latter ones imply that running code is useless, the first
one says its not.

I don't believe any of us have any quantitative basis on which to
base assertions that this will improve or dis-improve our processes
or output, or be neutral. (Hence proposing it as an experiment.)

> proposals before they get to the broader community.

I disagree with the shouted "NOTHING" - if there are non-silly
ways in which we figure we can improve our processes then we
ought be open to trying 'em out. You may or may not be right
that this is silly, but merely asserting that it is doesn't
make it so.

Being stuck with current processes or only ever adding more
review tiers would IMO be sillier than this proposal. But
that seems to be where we're mostly at.

> This is a bad idea even as an experiment.

Sorry, I don't get the "bad" aspect - rhetoric aside, in what
way do you see running this experiment doing harm?

S.

---

From: Joe Touch <[email-masked]>
Date: Tue, 22 Jan 2013 09:14:10 -0800

...
> The latter ones imply that running code is useless, the first

I never said "useless"; I explained several ways in which it alone is 
correlated to any of the issues relevant to speeding up the review process.

Multiple interoperable implementations helps ensure a doc sufficiently 
describes a protocol - nothing less, but also NOTHING MORE.

> or output, or be neutral. (Hence proposing it as an experiment.)

It takes more than an "unknown" to make an experiment. There has to be 
an hypothesis. Near as I can tell, yours is "running code means it's OK 
to run concurrent review at multiple levels".

Please explain why you think that is true. I gave multiple reasons why 
it is not.

> that seems to be where we're mostly at.

OK, so let's try an experiment where authors with the first name 
"Stephen" pay everyone $1,000 to review their docs. It certainly hasn't 
been tried, so - by your metric - it's worth considering?

Some things are simply not.

> way do you see running this experiment doing harm?

It puts more work on the community at large to review an idea that could 
have been either rejected or significantly improved in a smaller 
community before wasting the larger communities time.

This document is a prime example of such.

Joe

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 22 Jan 2013 17:39:50 +0000

On 01/22/2013 05:14 PM, Joe Touch wrote:
> I never said "useless"; 

That's how I interpreted one hand clapping.

> I explained several ways in which it alone is

I can't parse the above. But sure, there are many things
that running code does not do, and for which running code
is insufficient. Some of those are noted in the draft.

> describes a protocol - nothing less, but also NOTHING MORE.

Multiple interoperable implementations are not required for
PS or experimental RFCs. Perhaps you've misread something.
Or perhaps there is some other process-experiment that you'd
prefer we try that's not this one? If so, writing a draft
would be the way to go.

> Please explain why you think that is true. I gave multiple reasons why

The above is an oversimplification of what the draft proposes
but see below.

> Some things are simply not.

Rhetoric to the fore then.

Feel free to cut the text above this line if responding;-)

> community before wasting the larger communities time.

The proposal actually doesn't formally change what's required
of a WG before a publication request. So I think you're just
incorrect there.

You would perhaps have been correct had you argued that the
proposal might speed things up and hence give the community
less time for review. But seeing if such a speed-up happens
and whether that has other good or bad impacts under these
conditions is an explicit goal of the experiment.

I'm guessing we just disagree about the likelihood that
going faster under these conditions might improve or
dis-improve things. But that's ok. If we get to run the
experiment we might find out. If the draft doesn't become
an RFC, then we won't.

S.

---

From: John Leslie <[email-masked]>
Date: Tue, 22 Jan 2013 14:04:13 -0500

Joe Touch <[email-masked]> wrote:
>> The proposed experiment calls on the IETF Secretariat to take specific 
>> actions under certain circumstances in corner cases of the experiment.

   Specifically:
] 
] 8. If at any point in the fast-track process the responsbile AD does 
]    not act in a timely fashion then the IESG Secretary should ensure 
]    that the draft enters IESG evaluation and is scheduled for the 
]    the soonest IESG telechat that is more than one week after the 
]    end of the two-week post-IETF LC period.  That is, if the 
]    responsible AD does not act, then the default action applies 
]    which is that the draft enters IESG evaluation and is placed on 
]    the earliest IESG telechat. 

   I honestly don't know who or what the "IESG Secretary" might be:
several of us have assumed this is a typo, and "Secretariat" was
intended.

   (There is one mention of "Secretariat" in the draft, but it speaks
of what WG Chairs must do, not what the Secretariat should do.)

> This is a silly idea.

   Assuming that Joe refers to the item quoted above, I find myself
forced to agree with Joe (which is scary ;^)...

   (There is, BTW, an email address <[email-masked]>, which
feeds into an automated ticket-generation system. I have been asked
not to respond to it because it tends to get overloaded.)

   If we actually wanted the Secretariat to take the actions listed
above, they would need an automated tool to "discover" the inaction
of the AD involved. From the paragraph above, I frankly have no idea
how to program such a tool.

   "Enters IESG evaluation" is a simple state variable, mostly harmless.
"Scheduled" for an IESG telechat is a little vague. Usually this means
creating a "ballot" where each IESG member can enter a "position," but
there are other ways to get on the agenda.

   Who gets to decide whether this paragraph calls upon the Secretariat
to create that ballot?

   What time-limit applies between adding the item to the agenda and
the actual start of the telechat? I have seen items appear _after_
the "FINAL Agenda" is posted the afternoon before (and have done my
best to discourage such a practice).

> First, running code should already be considered as part of the context 

   Considered how?

> or safety. See Linux for numerous examples.

   Exactly! Evaluating the correlation of running code to the printed
standard-to-be is a long-winded process.

> parties can generate interoperable instances. It's merely the sound of 

   Here I disagree with Joe (familiar territory!) -- the existence of
one open-source implementation decidely helps other implementors to
produce interoperable instances.

   (Alas, at the same time it tends to hide ambiguities of the spec.)

> Finally, NOTHING should circumvent the multi-tiered review process.
> That process helps reduce the burden on the community at large via
> the presumption that smaller groups with more context have already
> reviewed proposals before they get to the broader community.

   This "multi-tiered review process" is somewhat mythical. To the
extent it actually exists, I believe it benefits us; but accomplishing
it in limited time is iffy at best.

> This is a bad idea even as an experiment.

   I'd like to call folks' attention to one paragraph in the Introduction:
] 
] In summary, this experiment collapses three stages of the publication 
] process to run concurrently: working group last call, AD review, and 
] IETF last call.  Additionally, the experiment will only require 
] issues raised during these three stages to be addressed if they meet 
] the IESG's Discuss criteria.  The former is not formally a process 
] change, although it is a change to the normal conventions.  The 
] latter is a process change and requires this experiment to be run 
] under the auspices of [RFC3933].

   All the other verbiage aside, _this_ is the point of the experiment:
to remove any implied requirement to respond to LastCall comments that
don't meet (current?) IESG Discuss criteria. (There is, fortunately,
a normative reference to these, but being a web-page, one wonders if
it will be entirely stable...)

   It is unreasonable to expect every commenter to read that web-page;
and I can testify that IESG members sometimes disagree on how it should
be interpreted. In practice, I believe this change will mean that the
effort now spent responding to comments from respected IETFers will
tend to be replaced by justifying why no response is needed. :^(

   To me, it seems unfortunate we should even be "experimenting" with
paying less attention to comments from the more-experienced IETFers.
The existence of "running code" doesn't make evaluation simpler. :^(

   I wonder if we might do better to encourage the "Experimental"
status for potential standards _with_ running code, and distribute
the evaluation during the period of experiment (rather than try to
squeeze "enough" evaluation into IETF-wide LastCall).

====
   Quoting from RFC 2026:
] 
] A Proposed Standard specification is generally stable, has resolved
] known design choices, is believed to be well-understood, has received
] significant community review, and appears to enjoy enough community
] interest to be considered valuable...

   Will this still be true under this proposed experiment?

] A Proposed Standard should have no known technical omissions with
] respect to the requirements placed upon it. However, the IESG may
] waive this requirement in order to allow a specification to advance
] to the Proposed Standard state when it is considered to be useful and
] necessary (and timely) even with known technical omissions.

   I would feel less nervous about this experiment were it to include
a specific IESG action to waive the "no known technical omissions"
clause.

--
John Leslie <[email-masked]>

---

From: Thomas Narten <[email-masked]>
Date: Tue, 22 Jan 2013 16:31:20 -0500

FWIW, I share Joe's concerns. And Stephen's responses don't really
change my mind.

This document seems to have a bit of missing the forest for the
trees. In the overall scheme of things, I don't believe the draft will
materially help, and is at best a distraction from dealing with
meaningful problems.

The crux of the issue is that any attempt at "fast tracking" is
fundamentally about short-circuiting some aspect of our review
processes. We should only do that very carefully. In almost all cases,
individual judgement is needed as to when it is appropriate to do
that. ADs/WG chairs already have some flexibility here. e.g., a WG can
skip WG LC if they think its not needed.  And nothing stops an AD from
going to WGLC before doing a careful review. But I'd rather that be
done because the case at hand justifies such an optimization, not
because there is some "running code" checkmark that allows a special
case to be fast tracked..

This document risks substituting process for judgement. I'd rather see
more of the latter and less of the former.

Some specifics from the doc:

>    In summary this experiment collapses three stages of the publication
>    process to run concurrently: working group last call, AD review, and
>    IETF last call.  Additionally, the experiment will only require

IMO, it is almost never appropriate to collapse WG LC and IETF LC. In
cases where it is appropriate to do so, there presumably isn't a need
for a WG LC to start with.  And an AD has always been able to start an
IETF LC without doing a detailed review. The point, is judgement is
needed, and all such cases are best handled on a case-by-case
basis. 

Surely, we don't need a document to justify doing this in those cases
where it makes sense...

>    IETF last call.  Additionally, the experiment will only require
>    issues raised during these three stages to be addressed if they meet
>    the IESG's Discuss criteria.  The former is not formally a
>    process

and later:

>    4.  Only comments that would be "DISCUSS-worthy" according to the
>        IESG Discuss Criteria [DCRIT] need be handled during last call.
>        Other comments can be handled or not, at the authors/editors
>        discretion.

The above can easily become a way to dismiss legitimate review
comments. No doubt, when a AD or reviewer suggests "this needs
fixing", the proponents will hold up this document and say "you
shouldn't do this, per the RFC -- you're violating the spirit of this
document, only really really critical stuff needs to get addressed..."
No thanks. That is not what the IETF is about.

If it is really urgent to get a document done, it is far better to
take steps to make sure the editors are engaged and responsive. That
is more likely the real issue!!!

>    It is understood that the savings in time for the end-to-end delivery
>    of a proposed standard or experimental RFC may be modest, however,
>    the modifications to normal IETF process will also serve as an
>    indication of the importance that the IETF places in running code.

Running code is valuable. Always has been, always will be. But we need
to resist the temptation of making "running code" more equal than
other criteria or putting it on a pedestal as some sort of holy
grail. Running code by itself is just a sound bite. The importance of
running code is what it tells us about a protocol specification. Just
the mere fact that there is running code doesn't mean there is
anything particularly enlightening to learn from an
implementation. For example, "running code" of a router function in
software doesn't necessarily say anything as to whether the code can
be implemented efficiently using ASICs, which may be the real
requirement.

> 
>    5.  The responsible AD for the WG concerned makes the decision as to
>        whether changes are required as a result of review comments, and
>        determines whether or not those have been completed.  If
>        significant change or extended discussion is required, or if
>        changes are not complete within two weeks after the end of fast-
>        track last call, then the draft is returned to the WG by the
>        responsible AD and the document is withdrawn from the experiment.

The two week figure is too stringent. The above is too prescriptive to
be practical. For starters, how will the IESG telechat align with the
ending of LC? Or what if an IESG member issues a defer (or will those
be disallowed?).

Section 4 has a bit too much detail and process in it. I'm sure it has
both too much and too little actually. I.e., it probably misses some
corner cases, and then what do you do given how prescriptive the rest
of the section is?

But overall, I see this document at best as a no-op. However, I fear
that it can be used to short-circuit our review processes in a way
that doesn't help the IETF and that won't really speed things up
enough to justify the downsides.

Thomas

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 22 Jan 2013 22:11:40 +0000

Hiya,

> community before wasting the larger communities time.

Actually it occurs to me that there might be something
here of note (while continuing to disagree with you about
the overall idea)...

If we do the experiment and a draft changes significantly
during last-call based on new comments from someone active
in the relevant WG then that would imply that either that
draft wasn't suited for fast-tracking or perhaps that
the experiment indicates a problem with fast-tracking.

I'd be surprised if that happens at all to be honest,
unless lots of WGs choose to try the experiment, but it
could.

I don't think there's really a high probability that a
draft from a WG will be "rejected" - almost all docs
that get to the start of WGLC tend to arrive at the IESG
as far as I can see, though I'm sure there'll be some WG
that operates differently.

S.

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 22 Jan 2013 22:35:49 +0000

Hi Thomas,

> FWIW, I share Joe's concerns. And Stephen's responses don't really

Ah well. I'm willing to keep trying:-)

> materially help, and is at best a distraction from dealing with

Do you mean the experiment or if this became a permanent
part of the process? Your comments read as if you're concerned
mostly about the latter, and you make no mention of the
former.

As it happens, I agree that this is not going to produce
a huge improvement. But its not intended to, and modest
improvements (as opposed to modest proposals:-) should also
be ok. If not then we're in a worse process-rut than even I
think.

> processes. 

I'd prefer s/short-circuiting/speeding-up/

> We should only do that very carefully. 

For example as an experiment? I think we are being careful.

> In almost all cases,

Disagree. The WG chairs exercise judgement to kick off the
process. The IESG exercise judgement as usual with no changes.

> needed, and all such cases are best handled on a case-by-case

What's wrong with the judgement of a set of WG chairs? AFAIK, ADs
are no wiser in general and this one is certainly dumber than a
lot of WG chairs.

> comments. 

Equally, gratuitous nit-picking can be used to try kill something
with a thousand cuts. Neither are at all common that I can see,
during IETF LC.

I think incorporating the discuss criteria should be an interesting
part of the experiment. (I've found them really helpful in reviewing
documents these last couple of years.)

> No doubt, when a AD or reviewer suggests "this needs

I don't see how you get to "No doubt..." I do entirely doubt that'll
happen for most all editors.

> is more likely the real issue!!!

Urgency isn't a part of this. Going faster doesn't imply urgency.

> grail. 

I agree. And this draft doesn't do that. Or are you saying it does?

> Running code by itself is just a sound bite. The importance of

Sure. The proposal says that this won't be suitable for all drafts.
In response to other comments we had the idea of building up text
about that as part of a wiki.

> The two week figure is too stringent. 

Possibly. But I think its a useful quid-pro-quo - if the authors
want fast track review but aren't willing to do fast edits then I
think they ought not get fast-track review.

I think the timers in general are something that an experiment
may show to be ok or not. And if not, then changing 'em would
be easy if we wanted to continue this after the experiment.

> The above is too prescriptive to
> ending of LC? 

That's covered in the draft. Could be it needs to be better
stated. (Basically, next telechat that's more than one week
after the authors have done their changes, if they got those
done within the two weeks after the end of LC.)

> Or what if an IESG member issues a defer (or will those
> be disallowed?).

No change there. IESG evaluation is entirely unaffected by this.
The draft says that.

> corner cases, and then what do you do given how prescriptive the rest

That's fair. Its hard to get the right balance there. I think
some of that text could be better on a wiki, as discussed in
response to some earlier comments.

> enough to justify the downsides.

I think the worst case is nothing happens but that if this
does get tried, we'll learn something about our processes and
(of course:-) I reckon we ought do the experiment.

Cheers,
S.

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 22 Jan 2013 22:46:05 +0000

Hi John,

Bits and pieces below...

> of what WG Chairs must do, not what the Secretariat should do.)

Fair enough. I think the terms effectively mean the same set of
folks though.

> of the AD involved. 

Perhaps ultimately, were this to be a part of our permanent process.

> From the paragraph above, I frankly have no idea

"AD didn't hit button to move forward or backward"? Absent a tool,
my guess is a WG chair would complain and some other AD would tell
'em to talk to their AD or ask the secretariat to push the button.
This is very much a corner case.

> best to discourage such a practice).

I'm not seeing the vagueness there sorry.

> it will be entirely stable...)

Stable enough for an experiment I think. That might be something that
would need attention should we want to adopt this permanently after
the experiment I agree.

> tend to be replaced by justifying why no response is needed. :^(

I don't think that'll happen much. Maybe a little though.

> The existence of "running code" doesn't make evaluation simpler. :^(

Yet, we have found these criteria valuable in IESG review, right?
I think they could be equally valuable here too.

>    Will this still be true under this proposed experiment?

I think it'll be no less true than today for documents as they
arrive at the start of IESG review.

> a specific IESG action to waive the "no known technical omissions"

But IESG evaluation is unchanged.

Cheers,
S.

---

From: Thomas Narten <[email-masked]>
Date: Wed, 23 Jan 2013 09:02:25 -0500

I do not really have time or desire to enter an extended discussion on
this document. It's pretty clear to me we just disagree. But I did
want to be on record as not supporting this document so that silence
wouldn't be taken as agreement or support.

A few specific followups below.

> mostly about the latter, and you make no mention of the

Mostly the latter. But given I don't see this experiment being
successful, I think the community has better ways of spending its
limited resources.

> be ok. If not then we're in a worse process-rut than even I

If there is an improvement, I expect it will be too small to be worth
bothering with. And given the community's limited resources (a not
insubstantial amount of collective cycles has already gone into this
document), it seems better to use those resources where there is some
hope of making a real difference.

> I'd prefer s/short-circuiting/speeding-up/

Sorry, I think short-circuiting is the right term. Saying you are just
speeding things up is just spin.

The reality is that it takes time to do quality standards/technical
work. And, it takes iteration. The only kind of iteration that makes
sense is sequential. Get some review, revise, then review the new
version. When one tries to overlap sequential reviews, the result is
almost always a poorer quality document.  It is important that the
revisions get reviewed by a fresh set of eyes. If you short-circuit
such iteration, you inevitably get poorer quality documents. Also,
it's really unfair to reviewers to have all of them stumble across the
same issues.

> process. The IESG exercise judgement as usual with no changes.

This document would use a different kind of judgement. One that is
more formulaic. The kind of judgement I'd rather see is the WG/WG
chairs/ADs getting together and making the case for a particular
document being so important/urgent that it needs to be
accelerated. And then have them work out a plan to actually get
reviews in a more timely manner, etc. E.g., get committments from folk
to review and revise quickly. Have a realistic workplan that folk
commit to. If its not possible to get folk to commit to being
responsive, doesn't that implicitely say something about the overall
urgency of the document?  But make that case on a number of factors
(importance of having the RFC in place, etc.) rather than just on the
fact that there was some sort of running code associated with it.

> documents these last couple of years.)

Discuss criteria are an IESG-level construct, developed to find
balance between "nit picking" by the IESG very late in the process
vs. correcting real problems. That same criteria is not appropriate to
apply to a document that hasn't gone through WG LC, which is at a much
earlier stage of the process. 

> happen for most all editors.

This community is really good at citing process to justify whatever
action (or inaction) they would like to see. Like it or not, this
could become another such tool.

Thomas

---

From: John C Klensin <[email-masked]>
Date: Thu, 24 Jan 2013 13:34:20 -0500

--On Tuesday, January 22, 2013 16:31 -0500 Thomas Narten
<[email-masked]> wrote:

> FWIW, I share Joe's concerns. And Stephen's responses don't
> really change my mind.
> 
> trees. In the overall scheme of things, I don't believe the
> draft will materially help, and is at best a distraction from
> dealing with meaningful problems.
> 
> processes. We should only do that very carefully. In almost
> all cases, individual judgement is needed as to when it is
> appropriate to do that. ADs/WG chairs already have some
> flexibility here. e.g., a WG can skip WG LC if they think its
>...

Hi.
First of all, I am completely in agreement with Thomas and his
analysis.  Anything  that can reasonably and appropriately be
done by this sort of effort can be done by discretion without
adoption of a formal procedure and adoption of a formal
procedure creates additional and unnecessary risks. 

As co-author of the process experiment model, I also object to
its use when it is not demonstrably necessary, for example to
give extra status and force IETF-wide use where the actions
suggested can be carried out as a matter of normal discretion
(see Section 5 of the document).  In this particular case, a
better experiment would be to use a process like this for some
otherwise-relevant drafts and in some areas only and compare the
results.  The "force IETF-wide use" part doesn't apply as long
as its invocation is a matter of WG Chair and AD discretion and
belief that the right conditions are met, so, other than
consuming a rather large amount of community time since it was
first proposed almost two months ago, I just don't see what it
accomplishes other than adding to the collection of process
documents and constraining the review process, including about
issues that running code doesn't help to inform.  The process
experiment mechanism is, IMO, an entirely inappropriate way to
simply make a statement that the IETF really, really, does
believe in running code and interoperability... especially since
one would hope that such a statement would not expire after a
year.

Another thing this draft accomplishes is to shift the
responsibility for determining whether a document is "good
enough" from an IESG interpretation of community consensus to
the decision of a single AD as to whether or not changes are
required (Section 3, item 5).  While it would probably improve
speed of processing, it is not, in general, a desirable change,
especially if one believes in the first part of the Clark quote.

It is probably just a nit in the grand scheme of things but, if
the responsible AD doesn't have time to do a sufficient review
to be confident in voting "yes" (Section 3, item 7) how can she
possibly sign off on the belief than no changes are required
(Section 3, item 5)?  It seems to me that situation creates a
situation in which an AD can initiate this process, take the
word of the WG that everything is in order, and then not only
avoid responsibility by abstaining but use the provisions of
Section 3 item 8 to let the document go through (as long as some
AD is willing to say "yes") without taking real responsibility.
Especially because we don't have firm rules against a WG Chair
and responsible AD coming from the same organization, I would
hope that those who are concerned about IETF CoI and antitrust
policies would take a careful look at that scenario because it
would seem to avoid many of the protections that are inherent in
wide community review and individual responsibility.  Other than
that, it is nothing more than a specific scenario consistent
with Thomas's concern that this procedure could be used to
short-circuit effective review.

I'm not going to quote extensively from Thomas's other arguments
or waste bits trying to summarize them, but let me add a few
observations to Thomas's about the document and how it is being
handled.

> If it is really urgent to get a document done, it is far
> better to take steps to make sure the editors are engaged and
> responsive. That is more likely the real issue!!!

And that no AD is going to screw around nit-picking or delaying
things in other ways.  Based on experience, I have absolutely no
confidence that a prohibition against anything that isn't
important (such as the one in this draft) will stop that
behavior if, e.g., someone thinks the document is
incomprehensible or ignores important issues.  (And, as Thomas
points out, it shouldn't).
 
> Running code is valuable. Always has been, always will be. But
> we need to resist the temptation of making "running code" more
> equal than other criteria or putting it on a pedestal as some
> sort of holy grail. Running code by itself is just a sound
> bite. The importance of running code is what it tells us about
> a protocol specification. Just the mere fact that there is
>...

Moreover, Dave Clark and sound bites notwithstanding,
interoperability is lots more important to us than running code,
partially because, while a lot of things might be learned by a
good-faith implementation of a specification by someone not
involved in its development, an implementation by an author or
other active participant in the development of a document may
actually tell us very little without third-party and
interoperability testing ... including not telling us whether it
is actually "running", much less conforming, code.  The latter
is a _huge_ can of worms if it is going to be a procedural
gating factor in the way the document proposes.

In that light, the document says "The spirit [...] is that early
implementations and ongoing development and testing throughout
the protocol work can significantly improve the quality of a
protocol and of its specification."  I agree with that and hope
everyone else does too.  But, other than general encouragement
and observations that such testing might be appropriate (e.g.,
in Section 4 item 13) this proposal makes no provision for
enabling or encouraging "ongoing development and testing
throughout the protocol work".  Instead, it concentrates only on
an assertion that an implementation exists, that it is
conformant enough to be informative, and that it "runs".  I
would have more sympathy with this proposal if its gating
condition was serious interoperability testing among independent
implementations, but that could still be done on a discretionary
basis (as 2026 points out in Section 4.1.1).

> But overall, I see this document at best as a no-op. However,
> I fear that it can be used to short-circuit our review
> processes in a way that doesn't help the IETF and that won't
> really speed things up enough to justify the downsides.

Yes.

In addition, this document moves a number of informal IESG
procedures -- including the DISCUSS criteria and even the
Shepherd report requirements (now Informational in RFC 4858 plus
a number of less formal "statements" and "templates" that have
sometimes changed rapidly) -- into the status of formal
procedural requirements despite the details of those procedures
never having gotten community consensus about those
specifications.  Under normal circumstances, the relationship
between this document and those specifications would require
additional actions to prevent this document from going into
permanent publication hold.   [DCRIT] is called out as a
normative reference without the discussion and alerts normally
required; the shepherding procedures and templates are not
called out at all.

I'm also feeling a need to say something that people seem to be
missing or avoiding.  In my experience in the IETF, we almost
never allow a individual submission document that is obviously
very controversial to turn into a extended debate on the IETF
list with the author trying to refute every comment.   Sooner or
later (usually sooner), the sponsoring AD says "hey, not ready
for the IETF List, much less Last Call" and pushes it off into
either a separate mailing list (to emerge only when there is
real consensus) or asks the author to take steps to propose and
organize a WG.

So, Adrian, noting the ratio between discussion of this draft on
the IETF list in the last few weeks and discussions of
everything else, how long does professional courtesy to another
IESG member (presumably in combination with your thinking this
is a good idea) encourage you to allow the unintentional DoS
attack on the IETF list to continue? 

regards,
   john

---

From: t.p. <[email-masked]>
Date: Fri, 25 Jan 2013 10:02:27 +0000

----- Original Message -----
From: "Thomas Narten" <[email-masked]>
To: "Joe Touch" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Tuesday, January 22, 2013 9:31 PM
> materially help, and is at best a distraction from dealing with

I agree.  This proposal asks the ADs and WG Chairs to do more work and I
perceive lack of resouces in that area to be the single biggest reason
for I-Ds to fail to progress as fast as they might.  Putting more load
on the critical path can only make things worse, for the work as a
whole.  Some favoured I-Ds will progress faster but overall, RFCs will
take longer to appear.

In this vein, I have commented before how progress in the IETF tends to
take place in three bursts, in the course of each year, and that if only
we had a way of bringing the three pauses to life, then progress would
be much improved.  Quality would almost certainly also improve, since
data would not get lost or forgotten in the pauses.  This month, most WG
mailing lists seem to be in a mega-pause (I keep checking the archives
to see if I have been accidentally unsubscribed!): that is the real
problem.

Tom Petch

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 25 Jan 2013 10:30:37 +0000

> fundamentally about short-circuiting some aspect of our review

Speaking as a Gen-ART reviewer, I am indeed worried by this aspect.
I feel I would have to spend much longer reviewing a draft if I
knew it had not been through WGLC. The fact that a spec is
implementable does not mean it is correct, which is exactly why
we want review by subject matter experts.

I would expect other late reviewers, including ADs, to react the
same way.

I welcome anything that encourages early implementations;
interop challenges are a good way to do that. But in the end,
is the IETF process the right vehicle for this?

Can we start a new tradition of interop challenges?

    Brian

---

From: Loa Andersson <[email-masked]>
Date: Fri, 25 Jan 2013 11:54:15 +0100

Folks,

I'm very much on the same page as Tom, the normal problem we have
is not too much review, it is that we don't have enough.

Running code is valuable, but does not normally replace review.

/Loa

-- 

Loa Andersson                        email: [email-masked]
MPLS Expert                                 [email-masked]
Huawei Technologies (consult)        phone: +46 739 81 21 64

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 25 Jan 2013 13:07:00 +0000

Responses to some points below but I'd really like to ask
people to consider a few things here:

- what's proposed is an experiment, it'd likely get tried out
  a few times and won't consume any huge resource anywhere
- its optional, WG chairs that want to try it could, those
  that don't can just ignore the whole thing
- argument that there are other or better process issues to
  address is misdirected
- on a meta-level, if every process proposal, even small
  experiments like this, generates huge concerns then we're
  not in a good place and it seems to me that we are in that
  bad place (that's not an argument to do this experiment,
  but just an observation)

> Running code is valuable, but does not normally replace review.

This proposal doesn't "replace review." Its an optional way
to get some of that done in parallel with a few other rule
tweaks.

On 01/25/2013 10:30 AM, Brian E Carpenter wrote:
> knew it had not been through WGLC.

Its an experiment that would be used for a few drafts, your
concern might be justified if this were proposed as a
permanent change (it's not) that applied to many drafts (it
won't ever IMO, but certainly not during the experiment).

On 01/25/2013 10:02 AM, t.p. wrote:
> This proposal asks the ADs and WG Chairs to do more work

It does not. WG chairs *choose* to take this route if they
want to. The timers and checking the implementation could
increase the workload for the responsible AD at a time not of
her choosing and its the timing that's more pertinent but not
so bad since ADs are already clearly gullible enough when
it comes to additional workload:-)

On 01/24/2013 06:34 PM, John C Klensin wrote:
> its use when it is not demonstrably necessary,

That's a really weird objection. I just re-read 3933 and it
says nothing whatsoever like that, in fact it says 'This
specification permits and encourages the IESG to adopt and
institute "process experiments"...' and the whole tenor of
the RFC is that stuff ought be tried out.

John also said:

> this document moves a number of informal IESG
> procedural requirements

That's just wrong. It includes those as part of an experiment
that is optional to use which I don't think qualifies as
"formal process requirements" in any useful sense.

And lastly...

> In my experience in the IETF, we almost
> list with the author trying to refute every comment.

"Trying to refute every comment" is (IMO wildly) inaccurate.
This was proposed a couple of months ago. Some people liked
the idea of trying it out, others didn't. I'm following up
on that as described in 3933. Every proposal to do with
process seems this controversial unfortunately, even teeny
little ones like this. I'd love if it were otherwise.

And if Adrian or the IESG decide we should not go ahead and
try this particular experiment, that's just fine and the
world will not end. (And nor will it end if this experiment
is run.)

What's less fine, but I suspect is also the case, is that
any proposal for process change would get the same kinds
of negative reception from a similarly sized set of folks
and with similarly good, bad and irrelevant arguments
against whatever change or experiment is proposed.

Regards,
S.

---

From: Eliot Lear <[email-masked]>
Date: Fri, 25 Jan 2013 14:36:55 +0100

On 1/22/13 10:31 PM, Thomas Narten wrote:
> a WG can
> skip WG LC if they think its not needed.

???

When was the last time that happened?  Did it require a consensus call
to determine?

Eliot

---

From: John C Klensin <[email-masked]>
Date: Fri, 25 Jan 2013 10:27:10 -0500

--On Friday, January 25, 2013 14:36 +0100 Eliot Lear
<[email-masked]> wrote:

> When was the last time that happened?  Did it require a
> consensus call to determine?

Chair discretion.  It is seems clear that the WG is in agreement
on all relevant issues and has adequately reviewed the document,
a WG LC is just another way to waste time.  Keep in mind that,
whether there is a WG LC or not, WG members are permitted to
comment in IETF LC.  Although it may be bad taste for a WG
member to disagree during IETF LC about issues already settled,
we don't forbid that and, because of a lot of edge cases and the
vague possibility of abuse, it would be a terrible idea to
forbid it.

Other than added bureaucracy, a WG LC serves only the following
purposes:

	* If the WG participants haven't adequately reviewed the
	document(s), a WG LC can provide a deadline and forcing
	function for more reviews.
	
	* Surprises during IETF LC and long debates on the IETF
	list waste the community's time and distract from other
	discussions.  If a WG LC might contribute to identifying
	and understanding the issues before the document reaches
	IETF LC and possibly even resolving them in the WG, that
	is almost certainly A Good Thing.   A WG Chair who can't
	make an guess about whether such issues are likely to
	arise (or issues that were thought settled really were
	not) probably needs to retire (and should certainly err 
   in the direction of doing a WG LC).

In the context of draft-farrell-ft, the above makes the idea of
WG LC in parallel with IETF LC either irrelevant or bad news.
If the WG Chair (or AD) concludes that a WG LC is needed, then
the procedure should not be invoked.  If a WG LC is not needed,
then organizing things so that one is conducted in parallel with
the IETF LC risks having two discussions going on in parallel
with comments made to the WG list not being exposed to the
community (remember the days when IETF LC comments were
routinely sent to the IESG and not to the IETF list?).   And, if
the WG does conduct a WG LC, the document can be read as
forbidding use of the FT procedure to condense and set timelines
on the rest of the process (something I'd suggest fixing if we
are going to go ahead with this).

All of this points out one of my main concerns.  Almost as a
side-effect, the proposal formalizes a number of informal
procedures and mechanisms work pretty well most of the time but,
because they are informal, can be used flexibly without a big
fuss.  If we are going to formalize them, we really should be
asking questions about consensus, exception cases, and so on...
and risk another several steps in the direction of trying to
substitute procedures for judgment, responsibility, and
accountability.

    john

---

From: Eliot Lear <[email-masked]>
Date: Fri, 25 Jan 2013 16:31:06 +0100

John,

On 1/25/13 4:27 PM, John C Klensin wrote:
> Chair discretion [... and five of paragraphs of text]

None of which answered my above questions.  When was the last time
chairs USED that discretion?!

Eliot

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 25 Jan 2013 15:34:05 +0000

> are going to go ahead with this).

That's a fair point and I'd be open to incorporating
a change to make that better if we are going ahead
with the experiment. I'll add a pointer to this mail
to the working copy [1] in a bit.

> substitute procedures for judgment, responsibility, and

I disagree that doing an experiment formalises these things.
I do agree that more thought would be needed if the
experiment turned up something that we did want to make
permanent.

S.

---

From: John C Klensin <[email-masked]>
Date: Fri, 25 Jan 2013 10:45:47 -0500

--On Friday, January 25, 2013 16:31 +0100 Eliot Lear
<[email-masked]> wrote:

> None of which answered my above questions.  When was the last
> time chairs USED that discretion?!

I didn't answer because I don't know.  It isn't something I've
tried to track and my first-hand experience is limited because
I've tried to avoid WG Chair roles in favor of advising for the
last decade or so.   The times when I didn't do a WG LC are both
long ago and probably before much of the community assumed they
were required.  We did short ones in EAI, not because they were
required but because we were trying to encourage additional
internal reviews and to verify that any issues that were missed
were identified.  But that WG was quite difficult in terms of
getting explicit reviews and discussion so WG LC was used as a
tool to address known problems.

   john

---

From: John C Klensin <[email-masked]>
Date: Fri, 25 Jan 2013 11:37:22 -0500

--On Friday, January 25, 2013 15:34 +0000 Stephen Farrell
<[email-masked]> wrote:

>> procedures and mechanisms work pretty well most of the time
>> but, because they are informal, can be used flexibly without
>> a big fuss.  If we are going to formalize them, we really
>> should be asking questions about consensus, exception cases,
>> and so on... and risk another several steps in the direction
>> of trying to substitute procedures for judgment,
>> responsibility, and accountability.
> 

If I correctly understand the above, it lies at the root of the
problem I was trying to describe.  This is really an experiment
if the effect of deciding we didn't want to make it permanent
was that we were at status quo ante, i.e., as if the experiment
had never been initiated.  If the experimental procedure doesn't
turn out to be useful but we then need to rethink things that
have changed, then we are in rather different territory.   In
other language, if the worst possible consequence of this
proposed experiment would be a harmless waste of time, then a
number of us who are now expressing reservations or opposition
would be saying "ok, try it and let us know how it works out".

To the degree to which you can modify the document to be clear
than the experiment has no permanent procedural side-effects
whether it is judged to be a success or not, the specification
becomes more attractive.

As an example, Barry has initiated an experiment in a different
shepherd write-up regime that moves away from the "yes, I
checked this and yes, I checked that too" format and toward
narrative about notable points.   Because the shepherd write-up
format and requirements have never been part of a formal
procedure, that particular experiment could be initiated without
a big fuss (and, in particular, without a 3933 procedure). I
don't know if the requirements on shepherd write-ups in item 11
of Section 4 is compatible with Barry's template, would require
modifications to it, or would require use of some other template
if one intended to use the Fast Track procedure.  Nor do I know
whether having this provision would frustrate other experiments
with shepherd templates during the course of the experiment.

Similarly, this experiment is heavily dependent on retention of
the IESG's current voting categories and DISCUSS criteria.  Does
it prevent changing those categories and criteria during the
course of the experiment?  Does it give them a status they don't
now have (I think it does, YMMD) and, if it does so, does their
status revert at the end of the experiment?

I think you could add language to the document that would
clarify these things and either eliminate those concerns or
bring them more clearly into focus.

All of that said, I'm about to go back to lurking but want to
make a prediction about a likely outcome of this experiment
(assuming it is approved) that the document doesn't mention.  A
document is moved into this procedure and fast-tracked.  Someone
concludes, as Thomas, Brian, and others have suggested, that
some aspect of the procedure has frustrated adequate review and
that there are bad associated consequences.  They appeal on the
grounds that the things that this procedure shortcuts are
insufficient to ensure fairness given the things that the claim
that there is a running implementation doesn't demonstrate
(e.g., as others have suggested, while "cannot be implemented"
would certainly indicate there is something wrong with a spec,
"can be implemented" doesn't show that the spec is correct or
does what is wanted).   The moment that appeal is filed, the
document in question moves from "fast track" to the slowest
track of all.

    john

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 25 Jan 2013 17:25:53 +0000

Hiya,

On 01/25/2013 04:37 PM, John C Klensin wrote:
> had never been initiated.  

That is definitely my intention. I'm happy to make it
clear in the draft, though I guess I'd assumed it was
inherent in the use of 3933. Anyway, yes, I'll add text
saying that explicitly. For now, I've put in a note
in the working copy. [1]

> ... The moment that appeal is filed, the

I think I recall someone sometime hoping there'd be more
appeals:-)

Yes, if this generated more appeals then that'd be a bad
outcome. Would it be more likely? I don't know to be honest
but I'd hope not, though perhaps you're right that it sort
of paints a target on a draft. I think this is similar to
points made about picking suitable drafts/implementations
and would be a good thing to say in a wiki rather than
put into the draft. I've added a reminder note to do that
also in [1].

Thanks,
S.

---

From: t.p. <[email-masked]>
Date: Fri, 25 Jan 2013 17:59:16 +0000

----- Original Message -----
From: "Eliot Lear" <[email-masked]>
To: "John C Klensin" <[email-masked]>
Cc: "Thomas Narten" <[email-masked]>; <[email-masked]>;
<[email-masked]>
Sent: Friday, January 25, 2013 3:31 PM
> chairs USED that discretion?!

I believe that it was used for
draft-ietf-tls-renegotiation
in December 2009.

Tom Petch

---

From: [email-masked] (Martin Rex)
Date: Fri, 25 Jan 2013 22:36:24 +0100 (CET)

Eliot Lear wrote:
> >>> a WG can skip WG LC if they think its not needed.
> >>
> >> When was the last time that happened?
> >> Did it require a consensus call to determine?
>

I don't know about the last time it happened, but I know about
one time in Nov-2009 in the TLS WG (now rfc5746).

The WG chair *did* a WG consensus call before AD & WG chair short-cutted
to IETF LC.  IMHO, at the very minimum, a WG consensus call with a specific
question or choice is necessary to gauge consensus in a WG.

WG Consensus call (20-Nov-2009):
  http://www.ietf.org/mail-archive/web/tls/current/msg04571.html

AD taking over (27-Nov-2009):
  http://www.ietf.org/mail-archive/web/tls/current/msg04928.html

IETF LC (30-Nov-2009):
  http://www.ietf.org/mail-archive/web/tls/current/msg04962.html

While I was OK with skipping the WGLC, I really didn't like some
of the AD's shortcut, in particular with respect to standardizing
two methods for signaling, and his refusal to perform a consensus
call for mandating the simpler version of the signaling, which
would have significantly facilitated the creation of fixes for
the installed base.

  http://www.ietf.org/mail-archive/web/tls/current/msg05287.html

There is a REAL issue with rushing a document pointing to implementations,
when it turns out pretty quickly that what has been implemented is really
just half of a solution.  I'm actually afraid that rushing things on
the process side is going to worsen problems.

What I was slighly missing in that process was (a) leadership, in
recognizing that mandating one signaling scheme will simplify the
spec, implementations and interop effort and (b) true consensus calls
on issues that are obviously contentious.  Even running a
consensus call during / in parallel to an IETF LC is better than
blaming time-constraints and the AD infering his "desired" outcome
from prior discussions of *MUCH* broader topics. 

Process-wise, we need to be extra careful about collision of interest,
each time when short-cutting processes.

-Martin

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 25 Jan 2013 22:47:18 +0000

Hi Martin,

> one time in Nov-2009 in the TLS WG (now rfc5746).

I recall that and agree with the sequence of events you
describe, but I'm not sure that that situation is
relevant when considering this draft, for two reasons:

- First, that was the IETF in security-incident-handling
mode, and that's just different from normal process for
us, whether fast-tracked or not. Its in the nature of things
that the vast majority of security incidents don't
directly affect IETF protocols so as to require a
backwards incompatible change. So I think that was a
highly unusual case. (And let's hope things stay that
way.)

- Second, there was significant controversy within the
WG before the last calls, (with many hundreds of mails;-)
so a set of WG chairs that chose to try a fast-track
experiment in such circumstances would be crazy basically.
(Remember, we're only talking about an experiment here.)

As for Eliot's question, I don't recall any case when
a WG skipped WGLC. Even if its not part of 2026, right
now it's a de-facto but mandatory part of the process
as far as I can see. I'd be interested if there are cases
where WGLC was skipped, esp. if its been regularly done.

S.

---

From: [email-masked] (Martin Rex)
Date: Sat, 26 Jan 2013 03:20:32 +0100 (CET)

> relevant when considering this draft, for two reasons:

Uh-oh!

> us, whether fast-tracked or not.

If I hadn't stumbled upon the TLS renegotiation issue while
discussing the proposal for TLS channel bindings, then
the "solution" would have been developed and shipped
under NDA and entirely with involvement of the IETF TLS WG
(just some selected TLS WG members), and likely there would
have been a request for rubber stamping the (half-baked) solution
*AFTER* deploymend of the fixes by major vendors.

  http://www.ietf.org/mail-archive/web/tls/current/msg03928.html
  http://www.ietf.org/mail-archive/web/tls/current/msg03942.html

Myself, I'm worried by the idea of shortcutting review and marginalization
of comments that do not meet the discuss criteria.

> (Remember, we're only talking about an experiment here.)

Admittedly, I deserve part of the blame for the heated discussion.

But if occasional heated discussions turn out to be a serious problem
for creating or improving technical solutions, then we may need more
procedural safeguards (such as independent leadership), not less.

The true essence of most of the 2009 TLS WG discussion was actually
fairly small and simple:

  http://www.ietf.org/mail-archive/web/tls/current/msg05365.html

-Martin

---

From: t.p. <[email-masked]>
Date: Sun, 27 Jan 2013 11:19:23 +0000

----- Original Message -----
From: "Stephen Farrell" <[email-masked]>
To: <[email-masked]>
Cc: "John C Klensin" <[email-masked]>; "Thomas Narten"
<[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Friday, January 25, 2013 10:47 PM
> relevant when considering this draft, for two reasons:

Sounds a bit like 'don't confuse me with the facts, my mind is made
up:-)'

The point that Thomas made and John endorsed is that when we want to
speed things up, our current procedures allow us to do just that.  We do
not need a formal process (more complications, more work).  And as John
pointed out, having two independent Last Call discussions, on two
different lists, on communities that may have little overlap, is not the
way, IMO, to establish a clear consensus.

Tom Petch

---

From: Stephen Farrell <[email-masked]>
Date: Sun, 27 Jan 2013 11:50:54 +0000

> Sounds a bit like 'don't confuse me with the facts, my mind is made

Absolutely not. Handling the TLS renegotiation bug was just
not the kind of thing that's relevant for this experiment.
It'd have been entirely dumb, stupid and crazy to try use any
kind of process experiment to handle that situation.

*After* this experiment is run, and if it turns up something
that we might want to make part of the normal formal process
then perhaps we might want to think about whether security
incident handling could be done that way or not. (I'd think
maybe not.) That'd happen in a year or so at the end of the
experiment if the experiment seems to have gone well. But,
that's not relevant now.

This IETF LC is about whether or not to run a process experiment
for a limited time that's optional to use at the discretion
of WG chairs.

This IETF LC is not about whether this or any proposal ought
be part of our formal, long-term process.

> way, IMO, to establish a clear consensus.

John's point about the two discussions in parallel was well
made and already acknowledged as such.

S.

---

From: Eliot Lear <[email-masked]>
Date: Sun, 27 Jan 2013 20:18:21 +0100

On 1/27/13 12:19 PM, t.p. wrote:
> speed things up, our current procedures allow us to do just that. We
> do not need a formal process (more complications, more work). And as
> John pointed out, having two independent Last Call discussions, on two
> different lists, on communities that may have little overlap, is not
> the way, IMO, to establish a clear consensus.

If we don't need a formal process, then I'd like it clarified in WG
training that this procedure already exists.  Furthermore, I don't buy
the notion about confusion.  In the cases where this is likely to be
done, either an emergency exists (which was the case with TLS, IMHO), or
the draft is likely not to elicit much in the way of objection.  This
DOES happen.  But that having been said, if both a WG chair and an AD
make a bad call, there are numerous opportunities to correct it in this
process, the NOMCOM process, appeal, etc.

Eliot

---

From: Joe Touch <[email-masked]>
Date: Sun, 27 Jan 2013 20:27:17 -0800

About the idea of an "experiment":

>    a few times and won't consume any huge resource anywhere

If this is an experiment, then you presumably answers to the following 
questions:

	1- what is your an hypothesis?
	2- what you intend to measure?
	3- what is your 'control' against which to compare the results?
	4- what is your objective metric for success/failure?

I've heard only one hypothesis - that this reduces time to publication. 
I disagree that this is a useful hypothesis to test for the following 
reasons:

	- time to publication isn't a goal of the IETF
		IMO, any doc that isn't useful in 5 years ought
		to not be published here; we don't need to
		document every sneeze

	- thorough review ought to be a requirement
		and this 'experiment' potentially compromises that
		by reducing the overall time of review

	- community resources ought to be considered
		and this 'experiment' burns group resources
		due to having a broad group concurrently review
		a doc that could have been reviewed by smaller
		groups first

Given the limited cycles this community has to review docs, I cannot see 
a benefit to this experiment that is worth the cost.

Having this entire community burn cycles on this document speaks for 
itself. It should have been vetted in a smaller, more invested community 
first.

Calling something an 'experiment' doesn't make it worthwhile to test.

Joe

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 28 Jan 2013 11:12:35 +0000

> About the idea of an "experiment":

Right. The context being its an RFC 3933 IETF process
experiment.

>     4- what is your objective metric for success/failure?

Well, its not a scientific experiment, and nor does it need to
be. Quoting 3933:

-   " A statement of the problem expected to be resolved is
      desirable but not required (the intent is to keep the firm
      requirements for such an experiment as lightweight as possible).
      Similarly, specific experimental or evaluative criteria, although
      highly desirable, are not required -- for some of the process
      changes we anticipate, having the IESG reach a conclusion at the
      end of the sunset period that the community generally believes
      things to be better (or worse) will be both adequate and
      sufficient. "

My take is that even though 3933 says "desirable" a couple of
times there, it's probably best to not go there, at least in this
case, but for now probably more generally. The reason I think
that is that I reckon the IETF has got itself into a log-jam
that almost prevents us from modifying our processes in any way
and every additional word provides additional barriers to doing
anything, since there'll always be someone for whom those added
words raise what seems to them to be a red flag.

I do agree that's far from ideal, but so is our current
process-change log-jam. (BTW, I already said this a few months
ago in response to a similar comment on I think -00 or -01.)

Separately, I don't believe this proposed experiment does have
any objective criteria on which we could get consensus and that
would be practical to measure - I think the outcome is more
likely to be subjective, (e.g. "seemed to help because of X")
and that the follow-up, if any, would require further
consideration - my guess is that if the experiment were some
form of success then we'd need to start over on documenting
the thing(s) that might be worth including in an update to
BCP9, but then that'd be based on some experience with
running the experiment rather than on your or my speculation
as to what might or might not work well or badly.

>         to not be published here; we don't need to

IMO reduced time to publication is definitely *a* goal.
I've heard lots of IETF participants complain about how
long things take, lots of times. Perhaps you're in the
rough on that. (I also note that this experiment doesn't
actually aim for any major reduction in time to publication
as it says in the draft.)

The draft itself also discusses reasons why running code
might also lead to better quality specifications.

>         by reducing the overall time of review

I think the liklihood of that doing damage during the
experiment is very small. In addition, I might be wrong,
but the thoroughness of review doesn't correlate that
well with the duration of review would be my guess. While
secdir reviews are pretty variable, they are almost all
done in a tight timeframe and a good number of them are
very good. Same for gen-art and apps-dir reviews. So
we do have evidence that good review can be done in a
short timeframe. Equally, I've seen quite a few drafts
that have reached the IESG after years that still
have quite a lot that needs to be fixed and that a
thorough review ought have caught.

>         a doc that could have been reviewed by smaller

Yes, this experiment should bring a few drafts to IETF
review earlier. The WG chairs are still supposed to
only send a publication request when they're satisfied
the draft represents the rough consensus of the WG.
However, I'm looking at changing some of the wording
on this in response to John's point about having two
parallel discussions. I'll post to the list when I've
had a chance to write that up in the working version.

> a benefit to this experiment that is worth the cost.

Fair enough. We disagree.

> itself. It should have been vetted in a smaller, more invested community

I'm following the 3933 process.

I don't know of any smaller but open venue for discussing rfc
3933 experiments. Perhaps there ought be such, but given how
rarely 3933 experiments are proposed, that'd be a very quiet
list.

In any case I would confidently predict that any process change
or experiment will generate this much traffic during IETF
last call so I'm not sure there's a saving;-) If someone manages
to get any kind of process change approved without a major flare
up on this list then I'll happily buy them a beer to celebrate.

If you'd rather we make 3933 historic or replace it with
something else, that's a different topic.

> Calling something an 'experiment' doesn't make it worthwhile to test.

I believe this draft does meet all the requirements of
3933 and is an experiment in that sense. I agree that not all
such things would be worthwhile to test, but a number of
people have agreed that this was worthwhile to test. Whether
or not that means we ought go forward with this is a call
for Adrian and the IESG at the end of IETF LC.

S.

---

From: Joe Touch <[email-masked]>
Date: Mon, 28 Jan 2013 07:23:18 -0800

...
> words raise what seems to them to be a red flag.

Lightweight != vacuous.

Perhaps the lack of the discussion of these issues is part of the reason 
so many previous proposals have not been tried.

That isn't a reason to ignore those issues here. But a bigger concern is 

> as it says in the draft.)

There are plenty of places where existing process is in a logjam. My 
experience over the past 15 years is most of the delay happens during 
the following:

	- changes in the IETF process where ideas like this throw
	a wrench into the process, and create confusion

		I had a few informational docs wait over a year
		in the queue because a process change was put
		into effect before the necessary boilerplate
		was resolved.

	--> this has a *simple* solution. Grandfather everything that
	has been submitted to a given queue to changes in that queue's
	process.

	- IESG review, during which issues are often raised that
	were addressed during WG review that are then re-hashed,
	even though they are not relevant to the area of their
	appointment

Overlapping community review has no impact on either of these.

> might also lead to better quality specifications.

No disagreement there, but "better quality" doesn't mean the doc 
wouldn't still need - and substantively benefit from - serial review in 
increasingly larger communities.

> I think the likelihood of that doing damage during the
> experiment is very small.

Damage =
	- publishing ideas not sufficiently vetted
	that later need to be updated/errata'd

	- wasting the community's time by having a
	large group review an issue that could have been
	addressed and corrected within a smaller community

Do you seriously think these concerns should outweigh a few weeks of 
reduced time to publication?

> In addition, I might be wrong,
> well with the duration of review would be my guess.

I've heard many on this list - including myself - point out specific 
reasons why this should not be believed.

If still believe every process can be accelerated, I encourage you to 
review Brooks "the Mythical Man Month".

> rarely 3933 experiments are proposed, that'd be a very quiet

This is a perfect issue for a bar BOF, FWIW.

Joe

---

From: Spencer Dawkins <[email-masked]>
Date: Tue, 29 Jan 2013 15:34:46 -0600

Just as a follow-up here ...

I was John's co-author on RFC 3933 (http://tools.ietf.org/html/rfc3933). 
When we were working on the draft, the problem I thought we were 
solving, was that the IESG needs to update the IETF's BCP processes from 
time to time, but (1) it was like 32 simultaneous root canals to 
actually get community consensus to modify the IETF's process BCPs 
including untried changes that might be improvements, so (2) the IESG 
was using informal IESG statements developed with much less community 
involvement than was expected for process BCP changes, in order to get 
anything done at all.

I (and, I think, John as well) saw RFC 3933 process experiments as a 
middle path between lightweight IESG decisions and full process BCP 
revisions. That's what I thought Section 2 of RFC 3933 was trying to say.

I had assumed that if the IESG clearly had discretion to do something 
under the current process BCPs, and thought it was the right thing to 
do, they would just do it, rather than set up a process experiment.

For what that's worth.

Spencer

> (see Section 5 of the document).

---

From: t.p. <[email-masked]>
Date: Thu, 31 Jan 2013 12:04:19 +0000

Time to publication of an RFC can be reduced by cutting out the pauses,
which could achieved by highlighting when they occur.

This can be done by a tool which, for every active Working Group, runs
monthly and, for every draft adopted by the Working Group, records
whether or not there has been a change and posts this list as an e-mail
to the list of the Working Group.  Where a new version has been
submitted to Tools, then this new version is listed with the date of
submission.  Where the status has changed, as recorded in the tracker
(AD Review, IETF LC, Publication Approved etc), then the new status is
listed with the date of change.  Where nothing has changed, then this is
listed with the date of the last change.  Given the number of I-Ds that
some Working Groups are working on, the best sequence is alphabetic of
I-D name.

Such a list would shine a light on what is moving and what is not.  This
in turn would encourage, motivate, spur those on whom the draft is
waiting to progress it (and anyone else who could contribute).

This works.  I know it does, because something of this ilk happens on a
WG I subscribe to; I imagine the chair does it manually, or perhaps they
screen scrape, but I see it happening.

My first thought was that this would be at the option of the AD or WG
Chairs but on second thought, I think this a mistake.  Often the light
will shine on the AD or Chair but that it healthy - it is not a
criticism of them, rather an invitation for everyone to see what they
could do to move things on.

Tom Petch

----- Original Message -----
From: "t.p." <[email-masked]>
To: "Stephen Farrell" <[email-masked]>
Cc: "John C Klensin" <[email-masked]>; "Thomas Narten"
<[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Sunday, January 27, 2013 11:19 AM
Subject: Re: FW: Last Call: <draft-farrell-ft-03.txt> (A Fast-Track way
toRFCwith Running Code) to Experimental RFC

> speed things up, our current procedures allow us to do just that.  We
do
the

---

From: John C Klensin <[email-masked]>
Date: Thu, 31 Jan 2013 09:44:10 -0500

--On Thursday, January 31, 2013 12:04 +0000 "t.p."
<[email-masked]> wrote:

> Time to publication of an RFC can be reduced by cutting out
> the pauses, which could achieved by highlighting when they
> occur.
> 
> This can be done by a tool which, for every active Working
> Group, runs monthly and, for every draft adopted by the
> Working Group, records whether or not there has been a change
> and posts this list as an e-mail to the list of the Working
> Group.  Where a new version has been submitted to Tools, then
> this new version is listed with the date of submission.  Where
> the status has changed, as recorded in the tracker (AD Review,
> IETF LC, Publication Approved etc), then the new status is
> listed with the date of change.  Where nothing has changed,
> then this is listed with the date of the last change.
>...

Tom,

This is interesting but could also introduce a pathology in
which drafts are generated too frequently to encourage (or even
permit) healthy discussion.  If one were going to do this, also
collecting some statistics on how much (or whether) a given
draft was being discussed on a WG mailing list might be very
important.  New drafts and indications of motion like status
changes may be good clues but what really counts for measures of
progress and consensus is whether real discussion is going on. 

  john

---

From: SM <[email-masked]>
Date: Thu, 31 Jan 2013 08:51:41 -0800

Hi Tom,
>which could achieved by highlighting when they occur.

[snip]

>waiting to progress it (and anyone else who could contribute).

There was an interesting quote from #overlyhonestmethods.  I cannot 
locate it right now as the web site is whaling out.

There is a Daily Dose web page (see http://tools.ietf.org/dailydose/ 
).  The web page provides a snapshot of what's being "published" in 
the IETF.  However, it does not provide a view of what is moving and 
what is not moving.  I don't have a view of what an area or what a 
working group is up to.

Sometimes it come as a surprise when working group participants hear 
that their working group is being shut down.  It is possible to tell 
whether a working group might be shut down.  It is possible to 
identify some of the issues which can delay publication.  The data 
for what you suggested is available from the datatracker.  The data 
to identify WG activity is also available.  I don't know how to 
assess progress though.

Regards,
-sm