Subject: Re: Last Call: <draft-farrell-ft-03.txt> (A Fast-Track way to RFC with Running Code) to Experimental RFC

From: SM <[email-masked]>
Date: Fri, 11 Jan 2013 11:33:23 -0800

At 07:14 11-01-2013, The IESG wrote:
>The IESG has received a request from an individual submitter to consider
>the following document:
>- 'A Fast-Track way to RFC with Running Code'
>   <draft-farrell-ft-03.txt> as Experimental RFC
>
>The IESG plans to make a decision in the next few weeks, and solicits
>final comments on this action. Please send substantive comments to the
>[email-masked] mailing lists by 2013-02-08. Exceptionally, comments may be

In Section 1:

   "Additionally, the experiment will only require issues raised
    during these three stages to be addressed if they meet the
    IESG's Discuss criteria."

Does this mean that a document does not have to represent consensus?

   "In contrast, a "-bis" RFC that aims for Proposed Standard or
    Experimental is likely to be a fine candidate."

I read what the document proposes as lowering the barrier of entry 
for first publication.  My preference is to say nothing about "-bis" 
RFCs (see quoted text).  Some of the "-bis" drafts I have come across 
(note that this is not related to a draft being discussed currently) 
are not well-written even though there is running code.  The running 
code might be due to author intervention instead of a blind test of 
the specification.

In Section 2:

   "Implementations developed during the production of an Internet-draft
    can indicate that a working group has had the opportunity to get
    early confirmation of its engineering choices."

Agreed.

In Section 3:

   "Any Working Group Last Call (WGLC) or Area Director (AD) review
   (which are routinely done, though not part of the formal [BCP9]
   process) will run in parallel with the two-week IETF Last Call
   (IETF-LC) period."

I am not suggesting changing the two weeks.  It can cause logistical 
problems.  I'll take the risk of trying this experiment.

   "Only comments that would be "DISCUSS-worthy" according to the
    IESG Discuss Criteria [DCRIT] need be handled during last call.
    Other comments can be handled or not, at the authors/editors
    discretion."

See my previous comment about this criteria.

In Section 4:

   "The fast-track process can be used for "bis" RFCs and might well
    be quite suitable for those."

I suggest removing this.

   "If the timers (IETF LC or the two-weeks after IETF LC for fixing
    things) co-incide with a major holiday period or IETF meeting
    then the responsible AD can add a week or two as appropriate."

I suggest not using the Fast-Track if it is less than two weeks 
before or after an IETF meeting.  Some people only do IETF work 
during that period and that results in a spike.  I don't think that 
it is a good idea for this experiment to encourage work during the 
meeting phase instead of the mailing list.

In Section 5:

   "An AD who wishes to do her review in parallel with Last Call is
    always free to make that choice."

"his" or a gender neutral term.

   "This memo itself has no impact on appeal processes.  However, in
    considering an appeal that relates to a document that has been
    fast-track processed, the relevant judge (WG chair, AD, IESG or
    IAB as appropriate) should consider the requirements posited here."

The WG Chair is not a judge in such cases as it would be his/her 
decision being appealed.

Some people are discouraged from bringing work into the IETF because 
of the long debates (which I likely contributed to).  Big companies 
have an incentive to do work within the IETF.  There is a perception 
in open source circles than there isn't much to gain in having a 
specification published as a RFC.

The young, free and wild will not listen to the fogies.  The better 
approach, in my opinion, is not to act as a gatekeeper or encourage a 
wall-garden attitude.

Regards,
-sm

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 11 Jan 2013 20:36:28 +0000

Hiya,

> Does this mean that a document does not have to represent consensus?

You mean rough consensus of the IETF I guess? Good question.

First, WG rough consensus is formally unaffected. As is IESG
review. And if IETF LC comments are received that do meet the
IESG discuss criteria then those should be handled as now.

So I guess we're left with cases where there's a lack of
rough consensus during IETF LC but where the meat of the
disagreement is something that doesn't qualify for an IESG
discuss ballot.

I'd say that'd be quite likely to allow the responsible AD
to say that there had been so much debate during IETF LC that
this experiment ought not be used.

So, I don't think this experiment has any major effect
there really but maybe has a subtle one.

I'll be interested in seeing if this happens if we do the
experiment.

> I read what the document proposes as lowering the barrier of entry for
> first publication.  My preference is to say nothing about "-bis" RFCs
> (see quoted text).  Some of the "-bis" drafts I have come across (note
> that this is not related to a draft being discussed currently) are not
> well-written even though there is running code.  The running code might
> be due to author intervention instead of a blind test of the specification.

I don't get what you mean. Can you explain? (I get that you don't want
to mention -bis cases, but I don't get why.)

> I suggest not using the Fast-Track if it is less than two weeks before
> or after an IETF meeting.  Some people only do IETF work during that
> period and that results in a spike.  I don't think that it is a good
> idea for this experiment to encourage work during the meeting phase
> instead of the mailing list.

Not a bad idea. Something like "fast-track must be started at least
one month (longer?) before an IETF meeting starts" ?

I guess the only problem I'd have with that is that for an experiment
that'd run for one year, that takes out about 4 months (3 meetings
and the year-end holidays) which is a lot.

If we extended the experiment to 18 months duration I'd have no
problem with something like that though. Or with leaving it as-is.
Let's see if we get any other opinions.

> "his" or a gender neutral term.

I'll leave that to the rfc editor. He'll fix it:-)

> The WG Chair is not a judge in such cases as it would be his/her

Can be. If a WG participant says "the decision you made on that
draft is broken: I appeal" then they first send that to the WG
chair.

> Some people are discouraged from bringing work into the IETF because of
> the long debates (which I likely contributed to).  Big companies have an
> incentive to do work within the IETF.  There is a perception in open
> source circles than there isn't much to gain in having a specification
> published as a RFC.
> 

I don't see any text change being suggested there. Correct me if
I'm wrong.

Cheers,
S.

---

From: Abdussalam Baryun <[email-masked]>
Date: Fri, 11 Jan 2013 22:25:38 +0100

Hi SM,

I totally agree with your comments and suggestions, the draft SHOULD
mention the important clarifications and the answers to SM's
questions. This is an important draft and SHUOLD be clear about such
important details in sections, why it ignores them without refering to
informative procedure items to link things for understanding. How do
authors write these drafts are they done with following procedure,

AB

++++++++
In Section 1:

be due to author intervention instead of a blind test of the
specification.
In Section 2:

Agreed.

In Section 3:

In Section 4:

I suggest removing this.

Some people are discouraged from bringing work into the IETF because
of the long debates (which I likely contributed to). Big companies
have an incentive to do work within the IETF. There is a perception in
open source circles than there isn't much to gain in having a
specification published as a RFC.

-sm

---

From: SM <[email-masked]>
Date: Fri, 11 Jan 2013 13:34:29 -0800

Hi Stephen,
>You mean rough consensus of the IETF I guess? Good question.

No, I mean consensus as that's also part what is gauged during a Last Call.

>IESG discuss criteria then those should be handled as now.

I am not concerned about the WG rough consensus angle (for the 
experiment) as it is a subset of a Last Call.

>disagreement is something that doesn't qualify for an IESG

Hmm, you mentioned "rough consensus" on an IETF Last Call.  The IETF 
Last Call is about "consensus".  I may have misunderstood the 
experiment as I did not read it as "rough consensus and running 
code".  To say it differently, there isn't any change to the IETF 
Last Call; this is more about fast-tracking the WGLC and the IETF Last Call.

>this experiment ought not be used.

Ok.

>I'll be interested in seeing if this happens if we do the

Let's see how the experiment works out.

>to mention -bis cases, but I don't get why.)

What the experiment can do is make it easier to move from unpublished 
specification to RFC.  The "-bis" draft is from a previous RFC.  If 
you try to cover it the experiment can turn into a significant 
process change.  I prefer to leave that unspecified and see how the 
experiment works out instead of telling people to use it for "-bis" 
drafts.  In a "-bis" document is in good shape it should not be a 
significant problem to get it through the process.

>one month (longer?) before an IETF meeting starts" ?

   The fast-track process cannot be initiated within two weeks of an
   IETF meeting.

You lose five weeks, two before, one for the meeting, and two after.

>and the year-end holidays) which is a lot.

Ok, make it one week then (see my previous comment).

>problem with something like that though. Or with leaving it as-is.

It can be left out as an implementation detail if it is easier to 
keep the experiment to 12 months.

>draft is broken: I appeal" then they first send that to the WG

Ok.

>I don't see any text change being suggested there. Correct me if

You're not wrong.

Regards,
-sm

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 12 Jan 2013 19:44:26 +0000

Hiya,

So I think the actions arising are:

- consider whether to have a "not before IETF meetings" restriction
  and make this an 18 month experiment
- maybe remove the text about -bis RFCs. (I slightly prefer it as-is
  fwiw, but let's see if we get more input)

Let me know if that's wrong.

Cheers,
S.

On 01/11/2013 09:34 PM, SM wrote:
> Last Call is about "consensus".  I may have misunderstood the experiment
> as I did not read it as "rough consensus and running code".  To say it
> differently, there isn't any change to the IETF Last Call; this is more
> about fast-tracking the WGLC and the IETF Last Call.
> 
> specification to RFC.  The "-bis" draft is from a previous RFC.  If you
> try to cover it the experiment can turn into a significant process
> change.  I prefer to leave that unspecified and see how the experiment
> works out instead of telling people to use it for "-bis" drafts.  In a
> "-bis" document is in good shape it should not be a significant problem
> to get it through the process.
> 
> It can be left out as an implementation detail if it is easier to keep
> the experiment to 12 months.
>

---

From: Bernard Aboba <[email-masked]>
Date: Sat, 12 Jan 2013 14:47:51 -0800

+1

[IAB Chair hat off].

> Date: Fri, 11 Jan 2013 22:25:38 +0100
> Subject: Re: Last Call: <draft-farrell-ft-03.txt> (A Fast-Track way to RFC	with Running Code) to Experimental RFC
> From: [email-masked]
> To: [email-masked]
> CC: [email-masked]; [email-masked]
> 
> Hi SM,
>

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 14 Jan 2013 09:49:33 +0000

Hi Bernard,

I'm sorry, I have no idea what it is that you agree with.
Can you elaborate?

Thanks,
S.

---

From: John Leslie <[email-masked]>
Date: Mon, 14 Jan 2013 08:05:41 -0500

The IESG <[email-masked]> wrote:
> final comments on this action.

   I'm pretty darn uncomfortable _ever_ picking a fight with any
sitting AD, But I feel obligated to say this seems like a terrible
idea to me.

   As a background, I'm a long-time believer in "rough consensus" for
Proposed Standard and "running code" for advancement along the
standards track. I do not believe the two mix well.

   A standard needs to be as simple as possible (but no simpler);
running code needs to be complex.

   Stephen Farrell wants to "speed up" our process by adding an
incentive for "having" running code. Specifically:
] 
] This memo describes a way to fast-track some final stage reviews for
] a working group draft when the working group management and area
] directors believe that the existence of one or more implementations
] serve as a practical review of the engineering choices made by the
] working group.

   Actually, now that I read that carefully, it doesn't even need to be
"running" code. :^(

] This experiment needs an implementation that matches the specification
] contained in the draft.

   (Verifying that has proven to be surprisingly hard. One guesses
that the "match" may not be rigidly enforced...)

   Understand, IMHO there is much to be gained by testing the
interoperability of independent implementations of a specification;
but that is a long-winded process, hardly anything which could be
done during the "fast-track" Stephen is proposing.

] It is up to the WG chairs and responsible AD to satisfy themselves
] on this point within the normal bounds of trust

   "Normal bounds of trust" isn't particularly compatible with our
paradigm of transparency of process...

   I don't even think it means the same thing to the rather limited
set of people named here.

] and without any implication about the licensing related to an
] open- or closed-source implementation.

   Any mention of licensing seems out of place here.

] Note that the existence of an implementation is not sufficient to
] ensure that the draft will automatically pass working group or IETF
] last call, nor that it will receive IESG approval.

   Nor should it be, of course...

   Getting to the devil-in-the-details:
] 
] 1. Any Working Group Last Call (WGLC) or Area Director (AD) review
]    ... will run in parallel with the two-week IETF Last Call

] 2. The IETF last call announcement message must say that the draft
]    in question is following the fast-track process...

] 3. The draft itself should note (e.g. in the introduction) that it
]    has been fast-track processed and reference this RFC.

] 4. Only comments that would be "DISCUSS-worthy" according to the
]    IESG Discuss Criteria [DCRIT] need be handled during last call.
]    Other comments can be handled or not, at the authors/editors
]    discretion.

   This is actually normal for IETF-wide LastCall (though it leads
to raised tempers when seemingly valid comments are ignored).

   It is not normal for WG LastCall; nor is it even compatible with
"rough consensus" -- by which we mean that any new questions raised
have been considered to the satisfaction of a substantial majority
of the participants.

] 5. The responsible AD for the WG concerned makes the decision as to
]    whether changes are required as a result of review comments, and
]    determines whether or not those have been completed.  If
]    significant change or extended discussion is required, or if
]    changes are not complete within two weeks after the end of fast-
]    track last call, then the draft is returned to the WG by the
]    responsible AD and the document is withdrawn from the experiment.

   This is not an easy read. :^(

   First, it centralizes the decision in one person (whereas normal
WGLC process has two or more WG Chairs judging consensus of the group).

   Second, it centralizes the decision in one person whether changes
in the document properly address any WG or IETF-wide comments. This
kind-of matches current practice in the IESG for Area Director
comments; but does not match the theory -- and we could on very short
notice regress from the level of trust which allows this to happen
now.

   Third, the two-week drop-dead time limit is unworkable. Not just
IMHO; in practice it hardly ever happens that everything resolves
itself that quickly. AD DISCUSS comments tend to bunch up in the
hours before the telechat, and frequently need to be clarified
_after_ the telechat.

   Fourth, it's not really clear what happens after "the document is
withdrawn from the experiment." It would appear at first blush that
it must now start a separate WGLC, followed by a new IETF LastCall.
I would never consent to this process for a document _I_ cared about.

] 6. As soon as the responsible AD has confirmed that the authors/
]    editors have made any changes required as a result of fast-track
]    last-call, then the document should enter IESG review and be
]    placed on the next IESG telechat agenda that is more than one
]    week in the future.

   Oh... Stephen wasn't talking about the IESG process at all!

   I'm even more confused now.

   I guess this means that one person (the responsible AD) takes a
look at any and all comments during the "combined" LastCall and
responsible-AD-review and makes a unilateral decision of what
changes the authors/editors should make (on short notice); then
things open for IESG-member DISCUSS comments.

   This would seem to preclude the current practice of putting
drafts on the IESG agenda _before_ the end of IETF LastCall. I'm
only getting more confused as to how this speeds up our process.

] 7. Given the reduced time associated with fast-track processing, the
]    responsible AD may not have time to carry out sufficient review
]    prior to IESG evaluation to be confident in balloting "Yes" for
]    the document.  Draft progression during and after IESG review is
]    otherwise unaffected, so a "Yes" ballot is needed from some AD.

   I _do_ understand that part, thought it's a bit obscure to those
who haven't been IESG members: no document gets approved by the
IESG without at least one "Yes" ballot-position; and Stephen wants
to be able to decline to ballot "Yes" on a document he (as
responsible-AD) has almost unilaterally re-written. (This would
be a really-bad idea, IMHO.)

] 8. If at any point in the fast-track process the responsbile AD does
]    not act in a timely fashion then the IESG Secretary should ensure
]    that the draft enters IESG evaluation and is scheduled for the
]    the soonest IESG telechat that is more than one week after the
]    end of the two-week post-IETF LC period.  That is, if the
]    responsible AD does not act, then the default action applies
]    which is that the draft enters IESG evaluation and is placed on
]    the earliest IESG telechat.

   (This is too complicated for me to try to explain in this email.)

====

   There's a lot more I could review, but I won't in this post.

   Were I an AD responsible to ballot on this, I'd ballot "abstain".
"Abstain" means "There's no point trying to satisfy me," and is
the way an IESG member essentially votes "No."

   I don't believe "running code" is even particularly helpful in
run-of-the-mill Proposed Standards. I've seen it _seriously_ harm
the documents in cases where the "running code" has shipped and
there is reticence to change it: no matter how many weaknesses are
pointed out, you can't get WG consensus to change the spec.

   In cases where "running code" has proven helpful, it's my
honest opinion that the spec has become way too complicated (and
thus the "running code" has proven which parts are unworkable).

   I don't believe it's possible to "fast-track" "runnng code";
nor do I believe it would be desirable were it possible. Code
needs to deal with complex cases that, for the most part, don't
need to be standardized in the first place. But code which
doesn't deal with these is useless in the marketplace.

   IMHO, we designed a separate "Proposed Standard" step to get
a specification published quickly without delving into the many
questions that arise when writing actual code. There are plenty
of Standards-Development Organizations that start from "code".
We don't need to become one of them.

--
John Leslie <[email-masked]>

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 14 Jan 2013 13:24:19 +0000

Hi John,

> sitting AD, But I feel obligated to say this seems like a terrible

Just a quick response to say that there's no fight required.
Its just fine that you think its a terrible idea. And you may
be right, or maybe not, I guess we might find out if the
experiment is done. Or if enough others hate it well enough,
we won't find out because the experiment won't start. That's
all ok.

And given that its a process-thing, I started it fully expecting
to see more or less all flavours of agreement and disagreement:-)

So I'm not at all put out to see someone just hates this.

I'll respond in detail later on when I get a chance.

Cheers,
S.

---

From: Barry Leiba <[email-masked]>
Date: Mon, 14 Jan 2013 10:54:49 -0500

> Does this mean that a document does not have to represent consensus?

This bothered me too: by limiting what last call comments we address,
aren't we limiting the consensus process?  Last call (which includes
directorate and review team reviews) often results in many comments
that will make the document better, but that wouldn't be likely
candidates for DISCUSS positons in IESG Evaluation.  Isn't it a bad
idea to say, outright, "Oh, well, you don't have to pay attention to
those."?  GenART reviews, in particular, are designed to be very
detailed and to call out not just substantive issues, but editorial
ones.  Aren't we showing disrespect to reviewers who spend a lot of
time on a document if we then say, "Yeah, thanks, but we're going to
ignore those."?

But here's the thing: Judgment.

I noted in another conversation recently that we aren't just automata
who mechanically handle process.  And what we're talking about here
isn't a situation where the responsible AD will look at each comment
as if putting it on a scale, and will say, "Doesn't meet DISCUSS
criteria; toss it."

What we have is a case where working group chairs have brought the
document forward ans said, "The working group has consensus on this
document.  Maybe the last of the 't's are not crossed and there's
still an 'i' or two un-dotted, but there's broad consensus in the WG
and running code that shows it's solid, and we're ready for a
fast-track last call."  After that, the community and the IESG get to
comment.  ADs can only block progress on a document (essentially going
against WG consensus) with DISCUSS positions, and those positions need
to meet the DISCUSS criteria.  This is, the document is saying, the
case also with last call comments: in order to go against the WG's
consensus, there needs to be some strength to the argument.

In fact, this document isn't saying anything different to what can
already happen: if the document's authors, shepherd, and responsible
AD all say, "That batch of last call comments don't really matter,"
they can leave those comments un-addressed.  And if another AD
disagrees with that assessment, she can put a DISCUSS position on,
saying that important last call comments were not addressed, the
document does not appear to have rough consensus, etc.

Also, in fact, most document authors are pleased to get comments that
improve the document, and will *not* ignore comments just to save a
few days in the process.  ADs regularly put many non-blocking comments
on documents, ones that do not meet DISCUSS criteria, and document
authors are usually happy to consider them, to discuss them, and to
make changes to the document to address them when that's the right
thing to do.

I'll also note that the quote above from the document doesn't say or
imply that non-DISCUSS-worthy last call comments will never be
addressed -- only that they are not required to be addressed before
the document goes to the IESG.  If the goal is to run things in
parallel, it would be entirely reasonable for people to say, "This
batch of comments are things we can work on while the IESG is
evaluating the document."  Again, judgment.

I'm really not worried about that aspect of this proposal.  Whatever
faults I think this has, I do not think it will result in Proposed
Standard documents with any less consensus than we have now.

Barry

---

From: Olafur Gudmundsson <[email-masked]>
Date: Mon, 14 Jan 2013 11:39:22 -0500

On 11/01/2013 10:14, The IESG wrote:
>
> The IESG has received a request from an individual submitter to
> consider the following document: - 'A Fast-Track way to RFC with
> Running Code' <draft-farrell-ft-03.txt> as Experimental RFC
>
> final comments on this action. Please send substantive comments to
> the [email-masked] mailing lists by 2013-02-08. Exceptionally,
> comments may be sent to [email-masked] instead. In either case, please
> retain the beginning of the Subject line to allow automated sorting.
>

I have experience in process like this, as my WG DNSEXT has required
multiple implementations and inter-op testing before advancing before
advancing documents that make significant changes to the DNS protocol.
Having done this I'm confident that the resulting specifications and
code was much better.

I support this experiment but offer the following comments.

Comment #1: The important part of running code is to assess
clarity of the specification, thus implementation by editors of the
document should not count as one-of-two required implementations
Implementations by editors co-workers are ok IFF the the editors keep
track of communications that lead to changes in code or draft.

Comment #2: It is important that participants all realize that point of
the exercise is not to point figurers at bugs. Rather the goal is to
improve the specifications and make ALL the implementations as compliant
and bug free as possible.

Comment #3 (Section 4 point #6)
Test cases used for interoperability are critical. These test
cases MUST be public. Evaluations of test cases generated by the
implementors and/or other working group participants are critical as
that is a great indicator of the quality and thoroughness of the tests.
IMHO public test cases render the point of open vs. closed source
irrelevant.

Comment #4: The IETF-LC and WGLC statements SHOULD contain references to
the testing performed and the implementations that participated.

	Olafur

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 14 Jan 2013 17:39:37 +0000

Hi again John,

Substantive response this time...

> standards track. I do not believe the two mix well.

I don't get why they don't mix well to be honest. And I don't
get it from your mail (so far). But I think you're maybe
interpreting the draft differently from how its intended so
let's see if we can figure it out.

I'd also point out that this experiment is optional. And
the intent is that if the experiment were a success that
the fast track process would always be optional - to be
used or not as WG chairs choose, with the co-operation of
authors/editors and ADs, pretty much any of whom could
basically force the draft to be processed in the normal
manner.

Some of your concerns read as if the goal were that all PS
RFCs would be processed this way. That's not the goal and
that's stated in the draft.

>    Actually, now that I read that carefully, it doesn't even need to be

Well I reckon its pretty clear when you read the whole thing,
but text tweaks are welcome.

> done during the "fast-track" Stephen is proposing.

It may be a long process. I think that'll depend on the
subject matter. Could well be that this isn't suitable for
things where sensibly checking the implementation matches
the draft is too complex. And maybe that's a good thing
(if it encourages us to write simpler specifications for
example).

> set of people named here.

Yeah you may be right. Can't recall who suggested that phrase
(maybe it was me, but it doesn't sound like me:-) but it could
be improved on I guess.

We do also trust WG chairs and ADs for lots of stuff already, so
would it be better if it said, e.g.:

   This experiment needs an implementation that matches the
   specification contained in the draft.  It is up to the WG chairs and
   responsible AD to satisfy themselves on this point.

   There is no implication here as to the licensing of the
   implementation whether open- or closed-source.  ...

>    Any mention of licensing seems out of place here.

But it only says that licensing can be anything. I do think
that's worth saying but maybe could be better split out as
above.

> have been considered to the satisfaction of a substantial majority

Well WGLC isn't part of 2026, and others have argued that that means
that there's no need for this to even be a process-experiment. That
seems to be at odds with your position.

My take on this is that yes, this is a proposal for an experiment
that changes what we do now with WGLC, and its one I think we ought
try. I think we just disagree on whether that's a good thing to
try or not.

> WGLC process has two or more WG Chairs judging consensus of the group).

I disagree. All the WG chairs must already want to use the fast-track
experiment so they presumably are already ok with the draft at the
start of IETF LC. So the only thing the responsible AD is judging is
whether the IETF LC comments are such that going ahead to IESG review
is ok, which what happens now.

> notice regress from the level of trust which allows this to happen

I don't get what you mean there about the theory.

> itself that quickly. 

You might be right. But I think if some WG chairs and authors/editors
ask for fast-track then they also need to be willing to work quickly
to handle IETF LC comments, and they ought only do this where they
expect it to work and are willing to try make it happen. So I'd
argue to keep this timer. (The duration of the timer I agree might
be wrong.)

> AD DISCUSS comments tend to bunch up in the

I don't know what you mean about AD DISCUSSes - IESG review isn't
changed in that respect and the draft does not require authors to
handle those any differently from today.

> I would never consent to this process for a document _I_ cared about.

You are correct, the intent is that the draft goes back to the WG
and is then processed as normal.

>    I'm even more confused now.

I'm confused that you're confused!

> things open for IESG-member DISCUSS comments.

Right. That's the idea. But note the WG chairs kick off the
process.

> only getting more confused as to how this speeds up our process.

It may be that the set of drafts for which this is useful and the
set that are reasonable to put on a telechat agenda before the end
of IETF LC are disjoint. But its an interesting, though I'd say
minor, point that might come up during the experiment if we try it.

> be a really-bad idea, IMHO.)

No, that's not the intent, nor I think would it be what'd happen;
"almost unilaterally re-written" would not be appropriate I entirely
agree, but I don't see any AD doing that after IETF LC now, and I
can't see it happening with this experiment.

Today, the convention is that the responsible AD ballots yes to
kick off the process of IESG review. This is just saying that
that AD might start IESG review with a different position (I
would guess a no-objection, though there are exceptions to
everything;-) because that AD hasn't had time to fully evaluate
the draft. That AD might e.g. have asked someone else to check
the implementation (allowed by this draft) but might want to
do a fuller review during IESG evaluation before presumably
balloting yes. Requiring the yes ballot up front is just a bit
too demanding with the fast track timers.

>    (This is too complicated for me to try to explain in this email.)

Maybe you'll come back to it?

> run-of-the-mill Proposed Standards. 

Right. I guess that's where we disagree.

> I've seen it _seriously_ harm

I agree that happens. I don't think this'll change how often.
But if it made that happen more often then yes, we should probably
declare the experiment over and not make this part of the
formal process. I could argue that this might reduce that
problem, on the basis that the draft and code are more likely
to be the same, but I've no empirical basis on which to base
that argument. (Same as you wouldn't if you argued that this
experiment will make it happen more often.)

> thus the "running code" has proven which parts are unworkable).

Hmm. I've had somewhat the opposite experience, so I guess
YMMV. I've not gone to dig up examples though but in DKIM I
think we ended up with a simpler spec. due to some folks
doing good work with running code before and during the WG's
processing of the drafts.

> doesn't deal with these is useless in the marketplace.

That may be right, but we're not talking about fast-tracking the
code.

> questions that arise when writing actual code. 

That's fair. I think this experiment partly reflects one way
in which the standards process theory and IETF practice have
diverged over the years. The bar for PS is mostly now much
higher than was the original intent. I think that in some
cases, this experiment might lower that bar a little.

> There are plenty

We also have issues with RFCs for which there is no code and
never will be;-)

But more to the point, I think that in a lot of cases where
the IETF has done a good job, there has been running code
before the WG even started, so I disagree that starting from
code is bad. Rubber stamping stuff whether based on code
or not would be bad, but isn't part of this experiment.

Cheers,
S.

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 14 Jan 2013 18:02:09 +0000

Hi Olafur,

> I support this experiment but offer the following comments.

Phew, so I'm not entirely crazy, good to know:-)

> track of communications that lead to changes in code or draft.

First this draft only requires one implementation and not two as
your WG did. Second, I don't know what you mean by "keep track
of communications" - can you explain? (Or send a pointer to
one you used in the WG?)

That's also sort of like the point Stefan W. raised. And he
suggested:

         "If the source code has been developed
         independently of the authoring of the draft (and ideally by non
         WG participants), it is likely that the implementation and the
         draft match, and that pitfalls unaware developers may find have
         been found and dealt with.  If, on the other hand, draft
         author(s) and implementation developer(s) overlap, then it is
         sensible to scrutinize the draft more closely, both with
         respect to its match with the implementation and for
         assumptions that author/developer may have taken for granted
         which warrant documentation in the draft."

Are you saying something like that would help?

Personally, I think there are thousands of nuggets of advice
that we could possibly add, but I'm not convinced that we
should - I think we'd be better off trying to keep this draft
shorter and simpler and then if the experiment succeeds we can
incorporate those things that experience has shown are worth
including.

> and bug free as possible.

Agreed. Not sure what text would change though.

Same comment about "nuggets" as above:-)

> IMHO public test cases render the point of open vs. closed source

I think that's arguable, but a reasonable argument. Again though,
I don't think we're at the point where a MUST ought go into this
draft and that'd be better done when the experiment's done.

Or do you have a suggestion for text? (I'm not at all sure how
I'd write that up now.)

> the testing performed and the implementations that participated.

I could buy that as a SHOULD or "ought." I've noted that
in the working version as a change to maybe make. [1]

Cheers,
S.

---

From: John Leslie <[email-masked]>
Date: Mon, 14 Jan 2013 13:52:39 -0500

Stephen did well to respond quickly!

   I will respond to most of his comments in private email, rather than
increase the noise-level on the <ietf> list. But a couple of points
deserve a better community understanding...

Stephen Farrell <[email-masked]> wrote:
>...
> that there's no need for this to even be a process-experiment...

   I think the Narrative Minutes show the IESG members questioned why
Stephen didn't simply bring things to IETF LastCall without a WG
LastCall. I invite readers to peruse the Narrative Minutes yourself
and see if it helps your understanding of what Stephen wanted beyond
that.

   But clearly the IESG has (not-yet-declared) consensus that Stephen
could do that without needing an experiment.

> try.

   Clearly our process documents leave a lot of room for something
other than a formal WG LastCall -- but whatever is done is under the
management of the normal WG Chairs. I don't really understand how
Stephen wants to change that; but the his draft reads as if he
wants to take over responsibility for any last WG comments from the
WG Chairs. (It's _hard_ to believe that's what he wants -- the AD
job seems quite large enough without taking on such responsibilities.)

>... 

   This text would better explain what Stephen was trying to say
in item 7. I'm not convinced it needs to be said at all -- this
is about internal IESG process; and most IETF participants neither
understand internal IESG process nor have any desire to learn it.

> I agree that happens. I don't think this'll change how often.

   At first blush, I would expect WG Chairs to try this experiment
when they happen to already have implementations (and not try for
it when they don't). The reticence to change shipping code is
both very human and good generic-management: I don't expect this
reticence to go away.

   I _do_ think this experiment would make the problem worse. Any
change to shipping code involves some level of management decision;
and squeezing such management decision into two weeks ensures it
won't happen absent _customer_ complaints.

   Am I being unfair to talk about squeezing that decision into
two weeks? Possibly I am... But this whole experiment _is_ about
squeezing the timeframe; and in my experience squeezing timeframes
tends to crowd out careful consideration. YMMV...

> processing of the drafts.

   Yes: DKIM clearly had become too complicated. Stephen and I
agree there!

> cases, this experiment might lower that bar a little.

   I'm always fascinated at the things folks think will "lower
the bar" for Proposed Standard. ;^)

>... 
> before the WG even started...

   This perhaps explains where Stephen is coming from. Such
cases do exist; and it is arguable that the process Stephen has
describe in this draft are well-suited to such cases (though I
would still have to quibble about some details).

   I remain unconvinced, however, that fast-tracking the cases
that start from running code is a good use of WG efforts. Such
cases seem better suited to Independent Submissions from a
design team (which would _greatly_ speed the process). When
a WG is formed, IMHO, they should be encouraged to look at the
problem from some quite different angles, in search of a way
to "Simplify!".

---

From: [email-masked] (Martin Rex)
Date: Mon, 14 Jan 2013 21:32:52 +0100 (CET)

> standards track. I do not believe the two mix well.

I don't have the resource to participate the discussion, but
these statements capture my opinion pretty well.

I also believe that the existence of "running code" would be a terrible
excuse for not resolving technical objections. 

While implementing a spec can help finding inconsitencies and ambiguities
in a spec, infering the reverse when the implementation is *NOT*
created as a by-product of a formal correctness proofing tool for the
spec does not hold.

Weird implementations that are incompliant with a spec, or that ignore
rather than recognize and report defects in a spec are extremely common.

And if the sequence of events is, that an implementation is created first,
and then producing a spec describing the alleged implementation behaviour,
is going to result in lots of surprises about the frequent lack of
correctness of both, the spec and the implementation it is based upon.

-Martin

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 14 Jan 2013 21:34:39 +0000

Hiya,

>    Stephen did well to respond quickly!

Responsive with dodgy ideas - sounds like me:-)

> could do that without needing an experiment.

Sorry John, I'm lost. There was no WG here and hence no WGLC.
The IESG hasn't as far as I know reached any consensus for
or against this approach, whether declared or not.

The IESG chatted about this on an informal telechat just after it
was previously discussed on this list, and some ADs had concerns
about the experiment. Others seemed less concerned. Some changes
to the draft resulted from that discussion, but my guess is that
the existing concerns are not fully resolved.

On last week's telechat Adrian as sponsoring AD asked if anyone
felt this was implausible enough that an IETF LC ought not be issued
(as RFC 3933 says) and there was no objection to that. So we're
now at IETF LC for this draft. Again, there's no implication that
the IESG like or will approve this experiment.

> management of the normal WG Chairs. 

As would be the case during this experiment. Its the WG chairs
that initiate the fast-track process and that's very clear in
the draft.

> I don't really understand how

Indeed that'd be hard to believe. Again, the WG chairs initiate
this, so I'm confused how you think I could think that. I think
the experiment would put some additional time-pressure on ADs
but doesn't really add to the absolute amount of text review work
for each draft except in terms of dealing with/checking the
implementation, which is new.

> understand internal IESG process nor have any desire to learn it.

Fair point. OTOH, I think its a real aspect of the experiment so
ought be mentioned even if that's not very useful for most readers.

> both very human and good generic-management: I don't expect this

And that'd be fine. This experiment is not intended for all drafts.

> won't happen absent _customer_ complaints.

Reasonable point. But you've conflated shipping code and running
code and also code changes with product release cycles. All of
those can be independent things so I don't think your conclusion
holds.

> tends to crowd out careful consideration. YMMV...

Not unfair, no. Just incorrect:-)

>    Yes: DKIM clearly had become too complicated. Stephen and I

Heh. I think DKIM is pretty simple in itself actually. Mail
of course isn't, nor is spam.

> the bar" for Proposed Standard. ;^)

Yep. Its a process-proposal, and thus involves tilting and
windmills:-)

> would still have to quibble about some details).

Right.

> problem from some quite different angles, in search of a way

Maybe. But that's a different proposal, perhaps a better one,
but a different one, and this is the one on the table right
now. (I make no claim that this experiment is the best
possible or anything like that.)

Cheers,
S.

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 14 Jan 2013 22:14:43 +0000

Hi Martin,

> excuse for not resolving technical objections. 

I agree.

> in a spec, 

Still agree.

> infering the reverse when the implementation is *NOT*

Formal correctness proofs aren't really an IETF thing, so I'm lost.
(I.e. I've no idea what you mean that's relevant to this draft.)

> rather than recognize and report defects in a spec are extremely common.

Still lost.

> correctness of both, the spec and the implementation it is based upon.

You're saying that this experiment will lead to RFCs that are
actually reverse engineered code being produced by IETF WGs?

Really? It seems to me you're forgetting the entire process of
chartering a WG and doing all the other work that precedes IETF
LC.

S.

---

From: Olafur Gudmundsson <[email-masked]>
Date: Mon, 14 Jan 2013 17:29:08 -0500

Hi Stephen,

> of communications" - can you explain? (Or send a pointer to

If you only require one then it OUGHT to be by someone other than the 
editors to count, otherwise it is just a proof of concept and we have 
limited confidence  that spec and code match up.

Well this can be done in number of ways,
- Issues tracker
- Postings to WG mailing list
- Notes in Appendix,

In our case we used the second one.

> Are you saying something like that would help?
I agree with Stefan W.  but would make it stronger that implementation by
editors does not count except for interoperability testing.

> incorporate those things that experience has shown are worth
I agree, thus I suppressed lots of comments/suggestions

> Same comment about "nuggets" as above:-)
I was hoping we could work this into paragraph #3 of introduction section.
I will think about text changes and send you off-line.

> Or do you have a suggestion for text? (I'm not at all sure how
I will take a stab at re-write second paragraph of section 2.1 to 
reflect this but that might take a day or two.

> in the working version as a change to maybe make. [1]

good.

     Olafur

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 14 Jan 2013 22:38:26 +0000

Hi Olafur,

Thanks for the offer of text. Looking forward to getting that.
Just on this one though...

> editors does not count except for interoperability testing.

Stephan W. also suggested that at first.

My concern is that this is too onerous during a 12 month experiment.
If we constrain it too much, then nobody will try it.

It could be that such constraints may be reasonable after a successful
experiment but I'm really wary of killing this before it starts
with too many pieces of well-meant advice or too many reasonable
sounding rules.

I'd have no problem adding an appendix with such ideas or putting
'em on the wiki page for the experiment though if that'd work.

Cheers,
S.

---

From: [email-masked] (Martin Rex)
Date: Tue, 15 Jan 2013 03:10:31 +0100 (CET)

> problem from some quite different angles, in search of a way

I also believe that being more honest about something that is
more a design team effort than the result of more traditional
WG work should probably carry through that notion of an
Independent Submission, and, when being fast-tracked, that it
is essentially a request for rubber-stamping.

-Martin

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 15 Jan 2013 09:42:03 +0000

Martin,

> is essentially a request for rubber-stamping.

Oh come on. That seems like a fairly wild exaggeration. Please
explain how this experiment as documented in this draft could
lead to some new way to get things "rubber stamped."  If it did
have that effect then I'd be against it, but that just seems
like nonsense to be honest.

S.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 15 Jan 2013 10:55:35 +0000

On balance I think this experiment is safe to carry out, and therefore
probably should be carried out. There are a few comments below.

However, I would urge the IESG to update the page at
http://www.ietf.org/iesg/process-experiment.html, including current
status of the experiments mentioned, and the history of concluded
experiments, which I'm pretty sure was there a few years ago.
At least, the history of the RFC 4693 experiment isn't there.

1. It seems clear that the explicit mention of GPLv3 should be removed.
It's contentious for a number of reasons. The phrase "e.g., under a Free
Software or Open Source license" seems necessary and sufficient.

2. There's a slight inconsistency between the mention of interoperablity
in a few places and the proposal to use *one* implementation as a
criterion. It doesn't seem unreasonable to use one implementation
as a criterion for PS, since a criterion for IS under RFC 6410 is
multiple interoperable implementations. But we should not imply that
one successful implementation implies anything at all about
interoperability. I think this needs to be stated quite explicitly:

Note that the existence of one implementation does not in any way
demonstrate the interoperability required for advancement on the
standards track [RFC6410].

3. However, I remember very bad experience some years ago with the
6NET project attempting to deploy Mobile IPv6 based on implementations
of several different versions of the main MIPv6 draft. They simply
could not interoperate. There is a risk that fast-tracking a draft could
actually be damaging in such a situation. I think we would need a clear
consensus that the draft is stable as well as viable. I suggest adding
an item in section 4 something like this:

The WG chairs and responsible AD must be satisfied that the draft is
in a stable state and that significant technical changes are unlikely
to be proposed in the near future.

Regards
   Brian

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 15 Jan 2013 11:32:35 +0000

Hi Brian,

> At least, the history of the RFC 4693 experiment isn't there.

Fair point. I've noted that that ought happen in the working
copy, [1] just so's we remember to look at it again later
during/after this LC.

> Software or Open Source license" seems necessary and sufficient.

Ok, I reluctantly fold on that since I got other off-list comments
to the same effect. My working copy now says:

   There is no requirement here as to the licensing of the
   implementation whether open- or closed-source.  At one end of a
   spectrum the source code of the implementation could be available
   under a Free Software or Open Source License.  Publishing the code
   source under such a license permits the public at large to read,
   compile and run it and removes all ambiguity about the existence of
   the implementation.  At the other end of the spectrum, a public
   statement by an employee of a known vendor, or the publication of an
   interoperability report from multiple implementers, can give
   sufficient confidence.  In all cases the WG chairs and AD do need to
   be able to say why they consider the implementation is appropriate
   for fast-track processing.

> standards track [RFC6410].

Seems ok to me. Added to working copy. Changed the reference
to BCP9 which includes 6410.

> to be proposed in the near future.

That would be reasonable to add, yes. But, there are lots of
other similar things that could be reasonable to add, that
cumulatively might result in the experiment not happening (e.g.
things from Olafur and Stephan yesterday). While this one is
less of a deal maybe, I'm still reluctant to add it because
if we add everyone's favourite nugget of wisdom, then we'll
likely end up with something unworkable.

Would it be reasonable to either add a section on "stuff that
came up in IETF LC that the IESG ought consider after the
experiment" or else to put this kind of thing in the wiki
for the experiment rather than state it as a rule or guidance
in the draft?

Cheers,
S.

[1] http://down.dsg.cs.tcd.ie/misc/draft-farrell-ft-04.txt

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 15 Jan 2013 11:49:53 +0000

See below...

> for the experiment rather than state it as a rule or guidance

I hate it when we end up legislating for common sense, so I
agree that for the experiment, this point could be put in the
wiki. Perhaps we could just add "Participants must exercise
common sense at all times" to the Note Well.

   Brian

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 15 Jan 2013 13:15:00 +0000

Hiya,

On 01/15/2013 11:49 AM, Brian E Carpenter wrote:
> wiki. 

Great. I'm accumulating stuff like that in the "changes"
section (9.1) of the working version [1] for now, so when
we get towards the end of LC, I'll look at adding things
to the wiki.

> Perhaps we could just add "Participants must exercise

But if that worked, we'd have much less fun;-)

S.

---

From: [email-masked]
Date: Tue, 15 Jan 2013 14:31:24 -0800 (PST)

> these statements capture my opinion pretty well.

I'm reluctantly going to have to join John and Martin, with one very important
difference: I think there is huge value in implementing specifications during
the process leading up to proposed standard. (As many people know, I do this
myself quite often for drafts I'm interested in.) I would rather phrase it as
having demonstrable interoperability for advancement, and ideally having
implementations done much earlier.

More specifically, where I part ways with this draft is in believing that very
early implementation helps find interoperability issues. I have not found that
to be the case. What it helps find are issues affecting implementability. In
the applications area at least, it's surprisingly easy to specify things that
look good on paper but turn out to suck when you try to code them. (My own sins
in this area are well known - RFC 2231 - and one of the reasons for that it is
one of the few RFCs I've written without implementing it first.)

Now, it's quite true that implementability problems can lead to
interoperability problems, like when different people take different shortcuts
in coding an unnecessarily problematic specification. But other outcomes are
possible, including fragility, unnecessary restrictions, and most especially
scalability problems.

Another problem with focusing on interoperability specifically as opposed to
overall implementability is that it makes it hard to argue that one or two
implementations provide that much benefit. In my experience having just one
implementation, even one done by a draft author, is surprisingly helpful in
cleaning up drafts.

I guess what I would like to see is for the draft to talk a little more about
finding implementation issues in general and a lot less about finding
interoperability issues specifically. I also think the draft goes a bit far in
the "carrots" it provides, but that may have more to do with my own experiences
in the applications area, where important comments have a way of only showing
up at the last second and where therefore the abbreviated process might be a
little dangerous to use.

Finally, I'm going to apologize for the tardiness of this comment, which really
should have been made sooner. I'm also going to apologize in advance for
probably not being able to fully participate in this discussion due to severe
time constraints.

				Ned

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 15 Jan 2013 23:14:14 +0000

Hi Ned,

at the end...

> probably not being able to fully participate in this discussion due to severe

Actually I think you make a couple of great points that ought be
mentioned in the draft about implementability. (No chance you'd
have time to craft a paragraph? If not, I'll try pinch text from
above:-) Now that you point it out like that, I'm irritated at
myself for not having included it already! (It rings bells for me.)

Cheers,
S.

---

From: [email-masked]
Date: Tue, 15 Jan 2013 19:40:37 -0800 (PST)

> myself for not having included it already! (It rings bells for me.)

OK, I think the place for a new paragraph is just before the last
paragraph of section 2. How about something along the lines of:

   A complete and correct specification is not in and of itself a guarantee of
   high quality implementations. What may seem like minor details can
   increase implementation difficulty substantially, leading to implementations
   that are fragile, contain unnecessary restrictions, or do not scale well.
   Implementation experience has the potential to catch these problems before
   the specification is finalized and becomes difficult to change.

You might also want to change the final paragraph of the section a bit in
light of the addition; I'll leave that to you to work out.

Hope this helps.

				Ned

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 16 Jan 2013 21:23:29 +0000

Hi Ned,

> light of the addition; I'll leave that to you to work out.

Did that, working copy at [1]. Lemme know if there're any changes
that are needed.

I think that's a really good addition, thanks.

S.

---

From: [email-masked]
Date: Thu, 17 Jan 2013 09:47:20 -0800 (PST)

> Did that, working copy at [1]. Lemme know if there're any changes

It looks good to me. 

I also note that there's a reference to interoperability in the fourth
paragraph of section 1. Perhaps changing

   For example, a framework draft will not be a good candidate because
   implementations of such documents are not, of themselves,
   interoperable.

to something like

   For example, a framework draft will not be a good candidate because
   implementations of such documents are incomplete and therefore do
   not demonstrate either implementability or interoperability of an
   entire protocol.

would be in order.

				Ned

---

From: "Cullen Jennings (fluffy)" <[email-masked]>
Date: Tue, 29 Jan 2013 05:06:12 +0000

My read of this draft is that it eliminates the need for rough consensus at both the WG and IETF level. Basically the WG chair can just decide and even if the WG disagrees with the chair. If the WG does not have consensus in WGLC that they they do want to publish the draft, it still gets published. I realize from the email list discussions this may not be what the author of the draft intended but it is how I read this draft.  

Because of this, I am against approving this and also believe it would need to be BCP not experimental as it changes the fundamental process to approve PS drafts. 

The rest of this draft, the part about overlapping, is already allowed by the process today as the draft points out. The WGLC is not required at all, the AD processing and  IETF LC can overlap. However, I think that the AD should do their processing before they LC the draft because that means they check it is ready before wasting the IESG and everyone else's time revving a document. AD processing can often be done in a few hours or less if the draft is ready. Generally, WGLC avoids late surprises which take more time in the long run but all of this is a general guideline and there are cases where it make sense to overlap all this and put it through. Thus I think it is good that the current process allows this to all be overlapped at the chair and AD discretion. 

I encourage the AD & Chairs to overlap where they think it will 1) is appropriate 2) will speed things up and 3) the speed up actually helps the internet or some groups of users in a meaningful way. I'm certainly not against some chairs, ADs, etc trying to put a draft throughout quickly that they think is ready (running code or not) but I don't see the need for this change to the process. 

I also have a question for the each IESG member that I think is very relevant - Do all of you agree to only put discussed that meet the "Discuss Criteria" this draft refers too? I really hope you do. If you don't that raises even more issues for how this draft changes the process. 

Cullen

On Jan 11, 2013, at 8:14 AM, The IESG <[email-masked]> wrote:

> sent to [email-masked] instead. In either case, please retain the
> beginning of the Subject line to allow automated sorting.
> 
> Abstract
> 
>   This memo describes an optional, fast-track way to progress a working
>   group document to IESG review.  It is provided as a process
>   experiment as defined in RFC 3933 for use when working group chairs
>   believe that there is running code that implements a working group
>   Internet-Draft.  The motivation is to have the IETF process
>   explicitly consider running code, consistent with the IETF's overall
>   philosophy of running code and rough consensus.
> 
>   In this process all of working group last call, IETF last call, and
>   Area Director review are carried out in the same two week period.
>   Only comments that meet IESG Discuss criteria need to be addressed
>   during this stage, and authors are required to make any changes
>   within two weeks.
> 
>   This experiment will run for one year.
> 
> 
> The file can be obtained via
> http://datatracker.ietf.org/doc/draft-farrell-ft/
> 
> IESG discussion can be tracked via
> http://datatracker.ietf.org/doc/draft-farrell-ft/ballot/
> 
> No IPR declarations have been submitted directly on this I-D.

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 29 Jan 2013 09:59:45 +0000

Responses to Cullen below, but this is getting to the point
where unless someone else who likes the idea wants to join
the discussion, I'm going to conclude that we're collectively
either unwilling or unable to consider 3933 experiments and
regard this one as dead, which maybe means 3933 is dead-ish
too, I dunno. (And before someone asks: no, I don't conclude
that its just a problem with this particular experiment, and
yes, I might be wrong there:-)

> My read of this draft is that it eliminates the need for rough consensus at both the WG and IETF level. Basically the WG chair can just decide and even if the WG disagrees with the chair. If the WG…

Right, that is not the intent. I also don't think this
experiment would have that effect, but neither of us
can prove either thing. I'll note yet again that this
experiment would only affect a few drafts where WG chairs
have chosen to try it out, and only for a year, so I
fully disagree that this "eliminates the need for rough
consensus" at all and consider that as a fairly whopping
overstatement that assumes that all drafts must follow
this fast-track process and assumes that the worst thing
happens as a result.

> Because of this, I am against approving this and also believe it would need to be BCP not experimental as it changes the fundamental process to approve PS drafts. 

If you've read 3933 then I don't get how you can say
that - 3933 calls for this draft to be last called for
experimental.

> The rest of this draft, the part about overlapping, is already allowed by the process today as the draft points out. The WGLC is not required at all, the AD processing and  IETF LC can overlap. How…

That seems to contradict your first paragraph where you
seem to say that we do need a WGLC so I'm not getting
how those can go together.

> I encourage the AD & Chairs to overlap where they think it will 1) is appropriate 2) will speed things up and 3) the speed up actually helps the internet or some groups of users in a meaningful way…

Again, its not a process change. Its meant as a process
experiment. That means that after a year, things go back
to where they were, and we evaluate whether or not there's
anything that ought become a process change.

> I also have a question for the each IESG member that I think is very relevant - Do all of you agree to only put discussed that meet the "Discuss Criteria" this draft refers too? I really hope you d…

There's an IESG statement [1] that says that and that
does get brought up when one AD thinks another's
DISCUSS doesn't meet the criteria. That doesn't
happen on every call, but maybe every month or two,
and the DISCUSSing ADs do afaics take that seriously.
Authors and chairs ought to feel free to say if
they think the DISCUSS doesn't fit [1] IMO, (not
many do, mostly former ADs funnily enough;-) so
I don't think other IESG members need to directly
respond here.

Cheers,
S.

[1] https://www.ietf.org/iesg/statement/discuss-criteria.html

---

From: Thomas Narten <[email-masked]>
Date: Tue, 29 Jan 2013 09:34:58 -0500

> yes, I might be wrong there:-)

I object to this (and have come close to saying it a few times
before).

One of the things that further undermines potential support for
something (and further raises questions about intentions) is when
people raise legitmate and substantive issues with a specific
proposal, and the responses start moving away from responding to the
substance to arguments  like "Gee, can't we even do experiments anymore",
etc., etc.

This document is not getting pushback because we "collectively" are
"unwilling or unable to consider experiments". Rather, from the list
traffic, some folk have real issues with the content. Can you please
accept that and stick to the substance?

For me, as I've said before, this just isn't an experiment worth
spending time on. Too little bang for the buck, and I see serious
potential downsides. Can you please just accept that some people have
that view?  And please don't turn around with the "how will we know if
we don't try?" line of argument. There is little need to try something
as an experiment if the likely outcome seems clear.

Thomas

---

From: Dave Crocker <[email-masked]>
Date: Tue, 29 Jan 2013 08:13:37 -0800

On 1/29/2013 6:34 AM, Thomas Narten wrote:
>>  I'm going to conclude that we're collectively

+1

This just-terminated proposal was marked by a rather obscure and 
strikingly weak value proposition, and no apparent concern for costs of 
implementation and use.

d/

-- 
  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net

---

From: SM <[email-masked]>
Date: Tue, 29 Jan 2013 09:49:20 -0800

Hi Stephen,
>yes, I might be wrong there:-)

In 2004 it was written that the IETF has designed process changes 
over the last ten years in one of two ways: announcement by the IESG, 
sometimes based on informal agreements with limited community 
involvement and awareness, and formal use of the same mechanism used 
for protocol specification.

My first reaction was that IETF participants are unwilling or unable 
to consider 3933 experiments.  My second reaction was: what if 
draft-farrell-ft was an IESG statement?  Would the same outcome be 
reached?  The comments posted to this mailing list were 
addressed.  The concerns were left open.

Regards,
-sm

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 29 Jan 2013 18:32:13 +0000

Hiya,

On 01/29/2013 05:49 PM, SM wrote:
> In 2004 it was written that the IETF has designed process changes over
> the last ten years in one of two ways: announcement by the IESG,
> involvement and awareness, and formal use of the same mechanism used for
> protocol specification.
> 
> My first reaction was that IETF participants are unwilling or unable to
> consider 3933 experiments.  My second reaction was: what if
> reached?  The comments posted to this mailing list were addressed.  The
> concerns were left open.

I don't think that draft would ever have been suitable for
an IESG statement (maybe some bits might, but not the whole
thing) so speculating on that is probably not useful.

I also don't think it was by any means a perfect proposal,
but I do think it was good enough to try out or would have
been with the text changes discussed during LC.

It was proposed as an experiment, but I think was interpreted
as a proposed process change, at least as I read a lot (but
by no means all) of the comments. And I think it was that
kind of concern that was left open.

I'm sure that misinterpretation was partly my fault and that
some or all of the folks commenting disagree with me that their
comments were based on a misinterpretation of what was proposed.
I was also surprised by the extent to which comments focused
on near worst possible outcomes.

But in any case, that's life - I don't think the draft had
anything like rough consensus however you look at it, so
Adrian's call was correct and on we go to the next thing.
I just hope the next thing isn't more process discussion
involving me:-)

S.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 29 Jan 2013 11:11:05 -0800

On 1/29/2013 10:32 AM, Stephen Farrell wrote:
> as a proposed process change,

wow.

If you don't think it was a process change, exactly what do you think 
would have been experimented on?

The label "experiment" is a qualifier for the substantive reference, not 
a replacement.  In this case, the substance was with IETF process.

You sought an experiment /with/ the process.

The statements of intended benefit were:

    Abstract:

    "have the IETF process
    explicitly consider running code, consistent with the IETF's overall
    philosophy of running code..."

and

    Introduction:

    "speed up the
    progression of certain working group documents in the latter stages
    of the process."

which is then countered and added to, by:

    "the savings in time for the end-to-end delivery
    of a proposed standard or experimental RFC may be modest, however,
    the modifications to normal IETF process will also serve as an
    indication of the importance that the IETF places in running code."

The Abstract's justification text doesn't actually describe a 
"motivation" as it claims, but rather simply describes a behavioral 
change.  It says what will be done, not why it is worth doing.

That the Introduction discounts its own, first-stated purpose, by 
finally casting the Abstract's reference in terms of "importance" adds 
to some confusion about the nature and purpose of the experiment.

In other words, you essentially characterized the experiment as:

     "Let's do something that will have little direct benefit but will 
symbolize the importance of something the IETF has already valued for.. 
ever."

This didn't make for a very compelling foundation.

And thereby makes this exercise particularly ill-suited for 
generalizations about the IETF's ability to conduct experiments.

d/

---

From: Barry Leiba <[email-masked]>
Date: Tue, 29 Jan 2013 21:32:19 -0500

> IETF participants are unwilling or unable to
> consider 3933 experiments.  My second reaction was: what if draft-farrell-ft
> was an IESG statement?  Would the same outcome be reached?

When Stephen proposed his draft to the IESG, I counter-proposed an
IESG Statement that would essentially say that because implementation
experience is important, the IESG will consider it in its evaluation.
More text around that, but that's the short version.

The IESG did not consider that approach at the time, preferring to
proceed with the 3933-experiment proposal.

Barry (App AD)