Subject: Re: Why ask for IETF Consensus on a WG document?

From: RJ Atkinson <[email-masked]>
Date: Fri, 24 Jun 2011 11:21:06 -0400

Earlier, Paul Hoffman wrote, in part:
> ...the IESG just approved publication of X, even with
> what appears to be a lack of consensus in the comments
> on the ietf@ mailing list.
> 
> (some other text elided here.)
> 
> For a document such as this, why even ask for IETF consensus 
> if the IETF consensus doesn't matter?
>  
> (remaining text elided here.) 

Next, as a reference, here is a boilerplate snippet from
a recent IETF Last Call:
> The IESG plans to make a decision in the next few weeks, and solicits
> final comments on this action. Please send substantive comments to the
> ietf at ietf.org mailing lists by 2011-07-21. Exceptionally, comments
> may be sent to iesg at ietf.org instead. In either case, please retain
> the beginning of the Subject line to allow automated sorting.

The "IETF Consensus" matters, and I believe the IESG 
did fully and properly consider that matter in the case
that appears to have driven your note.

There appears to be a confusion underlying your note.

Paul's assertion that public postings to the IETF Discussion 
list are the only metric for "IETF Consensus" is NOT correct 
-- and never has been in the past.  

"IETF Consensus" includes ALL of the inputs that the IESG
receives about a document or issue that is put to IETF
Last Call.   The Last Call announcement specifically says,
for example, that comments may be sent privately to the
IESG.

Consensus inputs are NOT limited to posted public comments,
BUT ALSO include formal liaisons, communications with various 
WG Chairs (not limited to the sponsoring WG), private email 
to the IESG, private email to an AD, and more.

Note also that this is THE SAME as how WG Chairs are supposed
to operate.  WG Chairs are supposed to consider ALL inputs,
NOT ONLY public postings to the WG list, BUT ALSO private
inputs from WG participants and other applicable folks,
and whatever other inputs arrive after a WG Last Call is
announced.

I hear that Paul H is unhappy with this particular outcome,
but there is no evidence of any kind of process problem
or process mistake here.

Yours,

Ran

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 24 Jun 2011 08:50:27 -0700

> -- and never has been in the past.

Come on, Ran. I never said "only metric", did I? This whole thread is about asking the IESG how they determine IETF consensus, with this being an example.

The IETF Discussion list and the the IESG balloting history are the only observable metrics we have, thus my question on this thread.

> for example, that comments may be sent privately to the

Completely correct.

> to the IESG, private email to an AD, and more.

Completely correct. You also left out the shepherd report, which also contains valuable information about the level of consensus in the WG.

And you have now listed so many variables, it begs the question: how does the IESG actually make the call when the observable comments on ietf@ make consensus seem questionable?

> and whatever other inputs arrive after a WG Last Call is

We probably disagree here: I don't think that WG chairs should make consensus decisions based on private inputs that cannot be validated. Doing so is begging for WG chair over-reaching.

> I hear that Paul H is unhappy with this particular outcome,

...which means that you did not read the messages I sent. I explicitly said I didn't care about this one, which is why I didn't say anything during the Last Call.

> but there is no evidence of any kind of process problem

Nor did I say that there was. I asked a question that this particular Draft helps illuminate, and I did so after talking to ADs who thought there were indeed interesting process problems. YMMV.

--Paul Hoffman

---

From: Keith Moore <[email-masked]>
Date: Fri, 24 Jun 2011 11:54:02 -0400

> Consensus inputs are NOT limited to posted public comments,

I agree with this much:  IESG, at least, should be able to consider private comments about document and standards actions, as input to its determinations. 

But one of the important attributes of consensus, one of the things that makes it so powerful, is that ideally, it's visible to everyone.  Take the example where a bunch of people in a room are asked a question and asked to raise hands to indicate yea or nay.   If only one or two people express a particular opinion, they can each see that for themselves, and that the "rough consensus" is clearly against them.   Likewise, the other participants will be able to see that the rough consensus is on their side of things. 

On a mailing list, it's a bit harder.  But still, when the traffic is visible to all list participants, it isn't too hard to know whether your opinion is widely supported, or whether you're part of a small minority.

In my experience, private input to IESG is more valuable in helping the IESG deal with technical evaluation or other concerns, than with consensus determination.   At least when I was on IESG, the volume of private Last Call responses was fairly small and I don't recall a case when I thought it would affect a determination of community wide consensus.    But private input was occasionally quite valuable in other ways.

While I do think that IESG should be able to consider private input in consensus determination, from a process standpoint it's problematic.  If the IESG decision doesn't reflect the visible consensus or lack thereof, this quite properly raises a question as to whether IESG has properly evaluated consensus.  So in the event that there were a substantial amount of private input, IESG would do well to find a way to make it clear to the public that is conclusions were based on that input, in what way, and for what reasons.

In this particular case I don't see any reason to believe that IESG has received a substantial amount of private input.  e.g. I didn't see any mention of such input in the IESG writeup or the IESG members' comments.

Keith

---

From: RJ Atkinson <[email-masked]>
Date: Fri, 24 Jun 2011 12:18:09 -0400

On 24  Jun 2011, at 11:50 , Paul Hoffman wrote:
> And you have now listed so many variables, it begs the question:

Paul,

I disagree that it begs that question.  

The IETF processes have always been open to ALL inputs from whatever 
source, in all parts of its processes.  You write that text above as if 
listening to all inputs is news, and it just isn't.

> how does the IESG actually make the call when the observable
> comments on ietf@ make consensus seem questionable?

The make their best effort determination, just as they always do.
That is how things have ALWAYS worked.  It works well.

Consensus in the IETF has NEVER been a numbers game, 
counting merely the public postings.  The IETF doesn't vote.

Just counting the numbers of public postings would be voting,
and this organisation has made a quite explicit decision 
NOT to vote.  

I'm told IEEE 802 votes, and I'm told they are happy with the results.
That's great for them, if true, but it isn't sensible for IETF.

To repeat some of what Don Eastlake wrote (I read his note 
just after sending my earlier note):
> But a narrow rule that the IESG is required to put on blinders and 
> only consider the IETF discussion list IETF Last Call email, 
> ignoring previous discussions on other relevant IETF mailing lists 
> and ignoring WG, area, and IETF plenary meeting discussions they
> have attended, is just arbitrary nonsense.

I totally agree with Don's quoted text just above.  

There is not now, has never been, and could never be a crisp rule 
defining a formula or algorithm on how the IESG determines consensus 
(or on how a WG Chair determines consensus).

I know of a large number of cases where one or more people sent private 
inputs to WG Chair(s) or to IESG folks on some Last Call item -- simply 
to avoid IETF-related controversy.  

Cheers,

Ran Atkinson

---

From: Martin Rex <[email-masked]>
Date: Fri, 24 Jun 2011 18:32:30 +0200 (MEST)

RJ Atkinson wrote:
> 
> The "IETF Consensus" matters,

I believe it should, but I currently completely fail to see
that it did matter to the IESG.

>
> and I believe the IESG did fully and properly consider that matter
> in the case that appears to have driven your note.

I would be curious about the IETF Last Call writeup from the responsible
Area Director about the issue that were raised during IETF Last Call
and their resolution, rfc-4858 Section 3.2 (2.i) => (2.b)-(2.h)

  http://tools.ietf.org/html/rfc4858#page-10

I don't see that IETF LC Writeup in the Datatracker

  https://datatracker.ietf.org/doc/draft-ietf-v6ops-6to4-to-historic/writeup/

And I don't see the significant objection specifically on the
procedural issue that moving to historic is an abuse of the
IETF process in the comments of other ADs here:

  https://datatracker.ietf.org/doc/draft-ietf-v6ops-6to4-to-historic/ballot/

So I agree to Pauls original request for clarification:
Has the IESG given itself entirely new procedures for protocol
actions, and when are those going to be published  (maybe using
the same bypass of the traditional IETF consensus
process that the above document seems to have used).

> -- and never has been in the past.  

If you read rfc4858 section 3.2 above, there was supposed to be a collection,
determination and resolution process for issues raised in an IETF LC in
just the same fashion as for the WG LC, and an IETF writeup of
significant issues raised during IETF LC and their resolution
as a prerequisition for an IESG decision.

If the new IESG approach to faster publication of RFC is to
simply ignore issues raised during IETF LC and rely primarily
on WG Last Call, and to make this change to the IETF procedures
bypassing public discussion and IETF consensus, then it would
be helpful for the resource management of IETF participants
when determining where to spend their valuable time: on WG
discussions or IETF LCs.

> Last Call.

IETF Consensus precludes ignoring procedural issues and
in-scope technical issues.  Otherwise it is IESG consensus
at best.

Why was there no serious consideration to downgrade 6to4 to
experimental instead of moving it to historic?
Are there any serious procedural issues against "experimental"?

>
> The Last Call announcement specifically says,

The irritation is about consensus-precluding input that has been
openly discussed in a quite elaborate fashion and a complete
lack of writeup and resolution of the IETF LC issues on the part
of the responsible AD.

...which bears resemblance with what this words of rfc2418
specifically cautions against:

   http://tools.ietf.org/html/rfc2418#section-2.3

   Proposed working groups often comprise technically competent
   participants who are not familiar with the history of Internet
   architecture or IETF processes.  This can, unfortunately, lead to
   good working group consensus about a bad design.

The job of the IESG is _not_ to defend a WGs interests against the interests
of the IETF at large, but the other way round, to ensure that the
interests of the IETF at large are sufficiently addressed by the output 
of the WG (or the author of an independent submission) before a document
is published as RFC.

> and whatever other inputs arrive after a WG Last Call is

Yes, but in an open standardizations body, the WG Chairs need to
document "openly" whatever input (content, not necessary contributor)
that they based their decision on.

A WG Chair or AD writing "I've received important input from important
lobbies, but I am unwilling or unable to disclose any content,
so I declare consensus to be that certain way." would not be appropriate
for the original open standardization process of the IETF.

Neither the absence of a write-up or deliberate ignorance of
raised procedural and in-scope technical issues.  Also contentious
matters of taste should be documented in the write-up.

-Martin

---

From: Keith Moore <[email-masked]>
Date: Fri, 24 Jun 2011 12:37:16 -0400

> and this organisation has made a quite explicit decision 

It's true in many ways that IETF does not vote.   Last Call inputs are not limited to strictly yea or nay; and there are no well-defined criteria for when an action has sufficient support (or lack thereof) from participants.

At the same time, when evaluating consensus, it's often useful to do a count of the number of approvals vs. objections.   Anytime there is a significant number of objections, "rough consensus" is in doubt. 

Sometimes when weeding through a long exchange of emails, it's hard to get a sense of just what the balance is without doing an actual count.  Actually counting names helps remove some kinds of bias, e.g. the tendency to count opinions with which one agrees, more than those with which one disagrees.     

But even counting is an approximate process.   Not all responses are entirely yea or nay, and sometimes peoples' opinions shift as the conversation progresses.  (which, IMO, is a good sign.)

Keith