Subject: Re: I-D Action: draft-crocker-rfc2418bis-wgguidelines-01.txt

From: Brian E Carpenter <[email-masked]>
Date: Thu, 29 Oct 2015 11:50:32 +1300

Hi,

I've been meaning to review this draft for some time, but I have just
realised why I can't do so effectively.

It's a proposed update of one our two most basic process documents, where
we currently have more than 120 instances of currently running code. But
I can't actually tell whether the draft reflects the original RFC, plus
its updates and errata, plus learnings from the running code.

I hate to say this, but I think the draft needs to go back to the drawing
board. I think it needs to proceed in small steps (a good example is
the way rfc2460bis is proceeding in 6man).

1. Publish a -00 draft which is simply the text of 2418 as-is but in current
format.

2. Publish a -01 draft with the errata fixed.

3. Publish a -02 draft with the RFC 3934 update patched in.

4. Publish a -03 draft with the RFC 7475 update patched in.

5. Publish a -04 draft with the RFC 7221 material patched in.

(If there are other RFCs that should really have been updates to 2418,
repeat the process.)

At that point we'd have a draft that represents agreed consensus up
to date.

Then we could discuss what needs adding or changing based on operational
experience.

Trying to jump straight to the end point like the current draft seems
highly unlikely to converge.

     Brian

---

From: Dave Crocker <[email-masked]>
Date: Wed, 28 Oct 2015 19:53:08 -0700

On 10/28/2015 3:50 PM, Brian E Carpenter wrote:
> But

Brian,

Fortunately, the considerable effort of a protracted, painful sequence
you prescribe isn't necessary, because all of the different types of
evaluation you seek aren't really all that useful.  This is not a
scientific process that needs fine-grained quality control.

It's merely an attempt to describe how to do some things.

So a simpler and -- I'll suggest -- more productive approach is to just
read the thing and react to it in its entirety.

The document is meant to be useful now.  Today.  Based on the rules and
expectations of the IETF.  Today.

That should make it possible to read the text against that simple
framework and offer comments on that basis.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Scott Bradner <[email-masked]>
Date: Thu, 29 Oct 2015 00:11:24 -0400

imo - any document that updates an existing document (especially technical standards and process documents)
needs to include a section that says what the differences are between the old version and the new version so
that readers can find out what they need to do differently than they have been doing

I do not see such a section in this document

Scott

---

From: Scott Bradner <[email-masked]>
Date: Thu, 29 Oct 2015 00:20:50 -0400

while I’m at it -

RFC 2418 was developed in a general area working group (poisson) 

it would seem to me that a document that proscribes how the fundamental work unit of the IETF is
to function is of sufficient importance to be developed in the same way - i.e. if the IETF feels that
an update to the existing document is need it should charter a working group with such a revision
as its goal

this is purely a process point - it is unrelated to the quality of the ID - it is my opinion that the creation or revision
of the basic IETF process documents needs to be done with significant deliberation

Scott

---

From: Dave Crocker <[email-masked]>
Date: Wed, 28 Oct 2015 21:35:48 -0700

> I do not see such a section in this document

Scott,

Section 1.1 contains a general statement about the nature of changes made.

But more importantly I believe your assertion of the specifics that are
needed is pretty much never met for any of the IETF document revisions
I've seen over the years.

d/

---

From: Scott Bradner <[email-masked]>
Date: Thu, 29 Oct 2015 00:42:42 -0400

you mean like appendix A of RFC 7622?

a grep of "Differences from RFC” in an RFC repository comes up with 71 cases
Changes from RFC comes up with 538

now some of these might be double counted in the ToC and in the file but such sections are
not all that uncommon

Scott

> On Oct 29, 2015, at 12:35 AM, Dave Crocker <[email-masked]> wrote:
>

---

From: Dave Crocker <[email-masked]>
Date: Wed, 28 Oct 2015 21:45:02 -0700

> RFC 2418 was developed in a general area working group (poisson) 

As a relatively constrained increment over the original document that
Erik Huizer and I developed on our own, yes.

And forgive me, but I do not see a section in RFC 2418 that satisfies
the documentation requirement about changes, that you expressed in your
other note.

> an update to the existing document is need it should charter a working group with such a revision

A necessary premise for your line of suggestion is that there are
essential deficiencies in the substance of the current draft.  What are
they?

Better:  Do you see a groundswell of community interest to form a
working group and take the considerable time and expend the considerable
work that would be needed for such an effort?

> this is purely a process point - 

Indeed it is.

it is unrelated to the quality of the ID - it is my opinion that the
creation or revision
> of the basic IETF process documents needs to be done with significant deliberation

Scott, What the heck do you think the current posting and request for
comments is meant to be for?

If the community has enough basic concerns about the /substance/ of the
document and enough energy to pursue them, then the community knows how
to decide on a resolution path.

But asserting the need for a particular path in the absence of that
foundational community assessment will cause the cart to quite mangled
by the hooves of the horse as it gets run over.

d/

---

From: Scott Bradner <[email-masked]>
Date: Thu, 29 Oct 2015 00:45:21 -0400

I also do not think section 1.1 would inform any reader of any changes in process that this ID proposes
if there are no changes in process than it would be fine to say that otherwise it would seem to help 
the working group participants to know what they would need to do differently

Scott

---

From: Scott Bradner <[email-masked]>
Date: Thu, 29 Oct 2015 00:47:40 -0400

> the documentation requirement about changes, that you expressed in your

yup - that was before it became clear to me (and many others) that such sections made life easier for the readers

Scott

---

From: John C Klensin <[email-masked]>
Date: Thu, 29 Oct 2015 00:55:58 -0400

--On Wednesday, October 28, 2015 21:45 -0700 Dave Crocker
<[email-masked]> wrote:

> Scott, What the heck do you think the current posting and
> request for comments is meant to be for?
> 
> If the community has enough basic concerns about the
> /substance/ of the document and enough energy to pursue them,
> then the community knows how to decide on a resolution path.

Dave,

I won't attempt to speak for the community, but when a proposal
is made to replace a fundamental document, there is grounds for
concern about accidental (or even "submarine") changes.  To that
end, and _especially_ if the sort of review we normally
associate with WGs is not anticipated, the kind of detailed
summary of the changes that are intended that I think Scott and
Brian are asking for seems appropriate and even necessary.  Such
a list would permit readers/reviewers to consider whether the
intentional changed are what is wanted and to check for
unintended ones.  Those checks often require different skill
sets and might be performed by different members of the
community.   In addition, if unintended changes were discovered,
that would provide justification for more intense scrutiny in
either WG or some other form.  

If procedures were being created for the first time, the
situation would be different, as it was when you are Erik did
the initial drafts that lead to 2418.  But it seems to me that
"what, if anything, does this mean we need to do differently" is
a relevant, and key, question for any revision, especially one
separated from the original by multiple updates and many years.

    john

---

From: Dave Crocker <[email-masked]>
Date: Thu, 29 Oct 2015 08:36:33 -0700

Scott, Brian, John, et al,

> I also do not think section 1.1 would inform any reader of any changes in process that this ID proposes

Since it doesn't seek to do that, it should not be surprising that it
doesn't accomplish it.

Here are a few simple points:

     1.  The draft dramatically reorganizes the text.  The draft has
extensive wordsmithing.  And, yes, the draft probably has bits of
procedure that differ from what is in RFC 2418.

     2.  Changes that extensive do not permit the sort of bookkeeping
you are focusing on.  Or rather, such bookkeeping is extremely expensive
to develop, while -- I believe -- being is entirely useless for any
reader who isn't a bookkeeper.  Broadly speaking, the actual practice of
process management in the IETF is not nearly precise enough or
consistent enough or faithful enough to the precise details of RFC 2418
to make such fine-grained historical auditing anything close to useful.

     3.  The actual IETF working group practices today often differ from
what is in RFC2418.  No one seems bothered by that.  So rather than
focus on auditing for changes, the goal for the draft is to accurately
reflect what is done today and/or what the community seems to believe
/should/ be done today.  Anyone wanting to audit the details of the
document needs to focus on current reality, not historical reference.
For those who feel otherwise, they are free to do the considerable work
of creating a changes audit.

     4.  The draft was first released publicly 7 months ago.  A mailing
list was set up for discussion.  36 people are subscribed, though it's
notable who some of the folk /not/ on the list are...  On that list,
Brian offered some detailed comments and they were factored into
revisions of the draft.  But other than Brian's comments back in March,
/no one has offered comments on the substance of the draft/.  Not on the
wgguide mailing list and not on the ietf list.  Seven months.

The draft was produced because Ralph and I felt that the community
needed a revised handbook of working group operations.  Revised both in
terms of making the details better match current practice (and/or
current /desired/ practice) and in terms of organization to facilitate
actual use as a reference document.

The organization that I put together for the original version of the RFC
(maintained in the current RFC version) has an expository style typical
of specifications.  The organization Ralph and I have developed is meant
to be more helpful for multiple kinds of readers and multiple kinds of
consultation.

The substance of open processes has to do with making things available,
allowing public discussion, and making changes responsive to input.
We've been doing that.  A working group format is not a mystical process
for open collaboration.  It's a tool.  Other tools work well too.  In
particular, working groups only perform well when there is serious
community motivation to participate.  After 7 months and multiple draft
revisions, there is no evidence that the community wants to put in the
very considerable overhead effort of having a working group and making
it succeed.

So I hope that the energy you folk are currently putting into arguing
about process formalities can instead be channeled into the substance of
the draft.

Section 1.3 has a significant list of significant open issues for the
draft.  I invite folk to put some effort into suggesting content that
will permit closing those open issues.

d/

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 30 Oct 2015 08:21:08 +1300

Dave,

> So I hope that the energy you folk are currently putting into arguing about process formalities can instead be channeled into
> the substance of the draft.

I don't care about the formalities. I care about the fact that I can't
understand in detail from the draft what it intends to change in the way
WGs operate and, incidentally, what text in RFC 2418 is wrong in practice.

Speaking for myself, I could never accede to a consensus on this draft
without that understanding. You want me to "react to it in its entirety."
My reaction is that I don't understand what problems it fixes.

Now I have a rotten track record in this area. About ten years ago I
wanted to trigger an update of RFC 2026, and decided that the way to do
it was to write a careful analysis of its defects. I won't bore everybody
with the details, but basically I failed. I think the reason I failed
was because my analysis was just Too Much Information. Subsequent reformers
have done better by fixing one thing at a time, but RFC 2026 is even less
a description of what we actually do now than it was 10 years ago.

For anybody who thinks this is all very simple, please review
http://www.ietf.org/about/process-docs.html

Oh, and to be clear: thanks for working on this. It does matter.

Regards
   Brian

---

From: Dave Crocker <[email-masked]>
Date: Thu, 29 Oct 2015 14:00:04 -0700

On 10/29/2015 12:21 PM, Brian E Carpenter wrote:
> I care about the fact that I can't

Brian,

You are focused on ways the draft is different from the RFC, while I am
instead focused on the draft's accurately reflecting current (and/or
community-desired) practices.

In that regard, the draft does not seek to "change" wg practices at all,
but rather to document what the community considers best practices.

You can't evaluate that by looking at RFC2418; you need to look at what
is done today or what people generally seem to agree /should/ be done today.

That is, the way to read the document is in terms of the present, not
the past.

d/