Subject: Re: "Discuss" criteria

From: "Robert Sayre" <[email-masked]>
Date: Mon, 1 Jan 2007 06:17:34 -0800

On 12/29/06, Sam Hartman <[email-masked]> wrote:
>
> If there are people on the IESG who you don't trust with the ability
> to say "Hey, I think we have a problem here.  Let's take the time to
> find out," then please work to remove them.

I don't think that is an accurate description of a DISCUSS. In
particular, it skirts the issue Dave mentioned:

Dave Crocker wrote:
> There is often a failure to distinguish between new and peculiar problems
> created by a particular specification, versus general problems that already exist.
>
> A classic example of this is citing basic DNS problems, for specifications that
> are merely consumers of the DNS and, hence, are not creating any new problems.

A WG can agree with the AD that there is a problem, but disagree that
it needs to be solved in their document. Too often, the compromise
ends up being the insertion of text that satisfies the AD's concerns,
but disenfranchises the WG. The WG either ignores the text in
practice, or the document author couches the text in so many
qualifiers that it becomes easy to explain any implementation in terms
that make it seem conformant. The result is inaccurate or misleading
documentation of Internet technology.

-- 

Robert Sayre

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 02 Jan 2007 00:21:37 +0100

John Leslie wrote:
>    This is venturing into dangerous territory. The best expertise on
> the technical issues involved _should_ be in the WG that produced the
> document. Expecting to find _better_ expertise within the IESG seems
> less than rational...
>   
Expecting the best available expertise on charset issues and string 
matching in the Kerberos WG is, in my opinion, less than rational.

But the Kerberos WG still has to define protocol operations where 
strings are compared.
Sometimes a WG needs help with issues outside their core purview, and 
sometimes they won't discover that until their documents hit the IESG. 
That's life.

(apologies to the Kerberos folks, who, I think, are now very much 
educated on those specific issues, for using them as an example....)

                    Harald

---

From: John C Klensin <[email-masked]>
Date: Mon, 01 Jan 2007 19:49:59 -0500

--On Tuesday, 02 January, 2007 00:21 +0100 Harald Alvestrand
<[email-masked]> wrote:

> Expecting the best available expertise on charset issues and
> string matching in the Kerberos WG is, in my opinion, less
> than rational.
> 
> But the Kerberos WG still has to define protocol operations
> where strings are compared.
> Sometimes a WG needs help with issues outside their core
> purview, and sometimes they won't discover that until their
> documents hit the IESG. That's life.

Harald, we certainly agree that, regardless of what we do, some
of these will ultimately hit the IESG during final review (as
well as agreeing that the Kerberos WG is a poor example).
However, I think your comment could be construed as a little too
accepting of the situation.  From my point of view, any time
that one of these situations is not discovered before Last Call
is an indication of a failure in the system, a failure that the
relevant ADs should be examining carefully in the hope of
preventing future failures of the same or similar nature.

    john

---

From: Keith Moore <[email-masked]>
Date: Mon, 01 Jan 2007 20:25:00 -0500

-------- Original Message --------

> to follow up on Dave's note
> 
>> * The IETF as a whole does not have consensus on the technical
>>   approach or document. There are cases where individual working 
>>   groups or areas have forged rough consensus around a technical 
>>   approach which does not garner IETF consensus. An AD may DISCUSS 
>>   a document where she or he believes this to be the case. While 
>>   the Area Director should describe the technical area where
>>   consensus is flawed, the focus of the DISCUSS and its resolution 
>>   should be on how to forge a cross-IETF consensus.
> 
> what actual evidence must an AD present to indicate that the
> assertion of non-consensus is anywhere but in the one AD's mind?

Like WG chairs, ADs are expected to use their best judgment in assessing 
WG or IETF consensus, respectively.

Ultimately a declaration of consensus has to be a judgment call on 
someone's part.  No amount of procedural rules can get around that.  The 
best we can do to ensure fairness is make the process transparent and 
provide for appeal.

Keith

---

From: Lisa Dusseault <[email-masked]>
Date: Mon, 1 Jan 2007 19:44:48 -0800

On Jan 1, 2007, at 4:49 PM, John C Klensin wrote:
> ... regardless of what we do, some

Indeed, although surprise issues are not completely avoidable, we  
should look on each one as an opportunity to ask "how could we have  
caught this one earlier" (just like bugs in software found after  
checkin).

So... how do we do better generally?  One possible approach:  I'm  
hoping that creating outside-expert review process for early review  
will help with this a bit.  It might not help much but it's not a  
very expensive experiment to run.  We've already got a few groups of  
experts (directorates, area review teams) who've volunteered to do  
early review.  Last week I posted to the WG chairs list asking who  
wanted to volunteer their documents to go through this a little more  
formally, with a bit of tracking of what reviews are requested and  
received.

Lisa

---

From: Sam Hartman <[email-masked]>
Date: Tue, 02 Jan 2007 10:00:05 -0500

>>>>> "John" == John Leslie <[email-masked]> writes:

    John> Sam Hartman <[email-masked]> wrote:
    >> Scott O Bradner <[email-masked]> writes:
    >> 
    >>>> * The IETF as a whole does not have consensus on the
    >>>> technical approach or document. There are cases where
    >>>> individual working groups or areas have forged rough
    >>>> consensus around a technical approach which does not garner
    >>>> IETF consensus. An AD may DISCUSS a document where she or he
    >>>> believes this to be the case. While the Area Director should
    >>>> describe the technical area where consensus is flawed, the
    >>>> focus of the DISCUSS and its resolution should be on how to
    >>>> forge a cross-IETF consensus.
    >> None.  But the AD must be willing to propose a procedure that
    >> the rest of the IESG can go along with to determine whether
    >> there is in fact a lack of consensus or wether the AD is wrong.
    >> This style of discuss is much more of a "Hold on here, let's
    >> work together to check consensus," than a "I'm blocking this
    >> document for ever."

    John>    This is venturing into dangerous territory. The best
    John> expertise on the technical issues involved _should_ be in
    John> the WG that produced the document. Expecting to find
    John> _better_ expertise within the IESG seems less than
    John> rational...

For this type of discuss, the IESG is not making a technical judgment,
but rather a judgment of consensus and process.  And expecting the
IESG to understand our process and be able to execute it is hopefully
a rational judgment.

---

From: John Leslie <[email-masked]>
Date: Tue, 2 Jan 2007 14:38:50 -0500

Sam Hartman <[email-masked]> wrote:
>>>>> technical approach or document...
>>> 
>> This is venturing into dangerous territory. The best
>> expertise on the technical issues involved _should_ be in
>> the WG that produced the document. Expecting to find
>> _better_ expertise within the IESG seems less than
>> rational...
> 

   (I dislike being drawn into what threatens to become a flame-war.)

   Let me be clear: we're not talking about the example Sam gave, in
which there was neither a WG nor technical content. I'm only talking
about possible cases where there is a WG and there is technical
content.

   We must avoid setting the bar too high when a WG is ready to publish
a document. Indeed, the IESG is charged to judge consensus; but that
has traditionally been a fairly low bar: no obvious technical problems
and no undue danger of harm to the Internet being raised during the
IETF-wide Last-Call. The vast majority of IETF participants "consent"
to its publication by their silence, based upon a belief that others
are doing the necessary perusals.

   I submit that _any_ technical problems and _any_ danger of harm to
the Internet should be open to rebuttal by the WG, or to modifications
to the document by the WG to reduce the problem to an acceptable level.
(Such modifications may, of course, justify a new IETF-wide Last-Call.)

   Trying to judge the extent of "IETF-as-a-whole" consensus beyond
the presence or absence of problems noted during Last-Call is akin to
crystal-ball gazing, and should be strongly discouraged. It's bad
enough that we expect the IESG to judge the seriousness of problems
which _are_ raised during Last-Call. To hold them responsible for
problems which _might_have_been_raised_ if everyone was actually
reading every document is downright silly.

   We are _decades_ past the time when everybody _could_ read all the
documents proposed for publication -- and even when we could, we
declined to try.

   It's high time we gave up any pretense that the "IETF-as-a-whole"
should come to "consensus" about the technical details of RFCs before
they're published.

--
John Leslie <[email-masked]>

---

From: "Kyse Faril" <[email-masked]>
Date: Tue, 02 Jan 2007 16:04:01 -0700

Excuse me for interruting at this juncture; I am reading news 'Search for 
the next Google,' and as
I have a somewhat antiquaited (by now!) proposal to the INDUSTRY for a 
SEARCH ENGINE that would be weighted in favor of the FRENCH-LANGUAGE, which 
would be called "OIEAU" ('Wow!')

Just thought I would ANNOUNCE this PROPOSAL again in the NEW YEAR...for 
LEGAL REASONS...

OIEAU Copyright 2007 Casey Farrell

ALL RIGHTS RESERVED

>From: Harald Alvestrand <[email-masked]>
>To: John Leslie <[email-masked]>
>CC: Sam Hartman <[email-masked]>, [email-masked]
>Subject: Re: "Discuss" criteria
>Date: Tue, 02 Jan 2007 00:21:37 +0100
>
>John Leslie wrote:
>But the Kerberos WG still has to define protocol operations where strings 
>are compared.
>(apologies to the Kerberos folks, who, I think, are now very much educated 
>on those specific issues, for using them as an example....)
>
>                    Harald
>
>

_________________________________________________________________
Type your favorite song.  Get a customized station.  Try MSN Radio powered 
by Pandora. http://radio.msn.com/?icid=T002MSN03A07001

---

From: Keith Moore <[email-masked]>
Date: Tue, 02 Jan 2007 18:41:37 -0500

> should come to "consensus" about the technical details of RFCs before

strongly disagree.  I have seen way too many working groups working in 
isolation, often deliberately so, and without due regard for the harm 
that their work will cause.

---

From: Sam Hartman <[email-masked]>
Date: Tue, 02 Jan 2007 19:05:17 -0500

>>>>> "Keith" == Keith Moore <[email-masked]> writes:

    >> It's high time we gave up any pretense that the
    >> "IETF-as-a-whole" should come to "consensus" about the
    >> technical details of RFCs before they're published.

    Keith> strongly disagree.  I have seen way too many working groups
    Keith> working in isolation, often deliberately so, and without
    Keith> due regard for the harm that their work will cause.

I think I agree with both of you.

For most matters silence of the majority plus consensus among those
involved is sufficient.

However the IETF as a hole is an excellent resource for taking a look
in situations where more eyes are required.  An example of this is
when there is a minority who is unhappy with the consensus or when
there are specific issues we want greater review on.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 02 Jan 2007 20:42:03 -0500

I wonder if we as a community are asking the wrong question in this discussion.

It seems to me that there is a very large value in having a clear 
document that the IESG says documents what they will do.

I personally am much more interested in having a document that serves 
that function, even if I don't like the criteria listed.
If we can get to the point where we have that, then I think it makes 
sense to have a discussion about what changes the community would 
like to see in the IESG procedures.

Is draft-iesg-discuss-criteria-02.txt a statement from the IESG of 
what it is doing?
If not, what is it?
If it is, what I would ask first is that the IESG commit to updating 
this whenever they change the procedures.  (I don't care if it is 
kept as a perpetual I-D, with a pointer so people know it has 
meaning, or a web page, or an ION, or what.)

Then, once we know that, we can have a useful debate on what changes 
we would like to see in what the IESG does.

Yours,
Joel M. Halpern

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 03 Jan 2007 14:55:11 +0100

> Is draft-iesg-discuss-criteria-02.txt a statement from the IESG of what 
> it is doing?

I think it is slightly more accurate to say it's a statement of
what we aspire to do.

> this whenever they change the procedures.  

On a best effort basis - like every other IETF document, we rely on
the edit-token holder having the time to do so.

     Brian

---

From: Dave Crocker <[email-masked]>
Date: Wed, 03 Jan 2007 23:23:52 +0000

> there are specific issues we want greater review on.

1. There are important differences between "getting greater review" and "getting 
IETF-wide rough consensus".  For example, one is an analysis process, while the 
other is a decision-process.

2. What are some examples of a demonstrated IETF-wide consensus against work 
submitted to the IESG?  What improvements resulted?

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Jan 2007 20:04:47 -0500

>>>>> "Dave" == Dave Crocker <[email-masked]> writes:

    Dave> 2. What are some examples of a demonstrated IETF-wide
    Dave> consensus against work submitted to the IESG?  What
    Dave> improvements resulted?

I don't think there was an IETF consensus behind RFC 4285.  I'm not
particularly sure there was a WG consensus behind the protocol,
although there was a WG consensus that we wanted it published.  We
never evaluated whether I was right that there was an IETF consensus;
instead we improved the document by clearly explaining in an IESG note
what the open questions were and what the disagreement was between the
security area and the MIP6 working group.

I think the only time we've ever seriously taken steps on a discuss of
the form "there's no consensus here," is David's discuss on
draft-carpenter-rescind-3683.  We haven't actually managed to improve
the document there, but failing to publish something with inadequate
community support is an improvement of itself.

There have been other discusses that were initially phrased as lack of
consensus discusses or that included such phrasing.  However the IESG
has (typically before or during the telechat) come up with an
alternate way of looking at these discusses.  As you point out lack of
consensus discusses are difficult to deal with and we'd rather have
easier (or no) blocking comments to deal with.  So really the issue
has not come up often enough to have significant data.

---

From: John Leslie <[email-masked]>
Date: Thu, 4 Jan 2007 08:32:28 -0500

I should be ashamed of myself -- letting myself get ensnared in a
flamewar with Keith...

   First, let's restore some context. We're talking about

http://www.ietf.org/u/ietfchair/discuss-criteria.html

specifically section 3.1; and I was taking exception to the last bullet
in 3.1:
] 
] The IETF as a whole does not have consensus on the technical approach
] or document.

Keith Moore <[email-masked]> wrote:
> [John Leslie wrote]

> that their work will cause.

   Keith states a problem -- which although many of us might state
differently, most of us agree is a problem; and then makes a giant leap
to the conclusion that our defense against this must be to depend upon
some AD to call for IETF-as-a-whole consensus after the IETF Last-Call
has failed to turn up any specific issues.

   I contend that Keith's cure looks worse than the disease. Very often,
by the time we reach that stage, the  WG has effectively disbanded, and
if they haven't, this kind of open-ended, "Somebody somewhere might not
like this" statement should push them over the edge.

   The proper cure for the disease Keith names has been agreed upon for
years now: early cross-area expert review. Alas, we don't seem to be
getting there.

   I continue to suggest that this last DISCUSS criteria is wrong.
Instead we should have something like:
" 
" The potential for harm has not been adequately considered. An expert
" review by [fill in the blank] Area is needed.

(which, BTW, I cannot find in the list in section 3.1).

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 04 Jan 2007 15:12:07 +0100

> has failed to turn up any specific issues.

I don't see where you get that from. I can think of two cases where
we might get such an assertion from an AD:

1. The IETF Last Call did generate dissent.

2. The IETF Last Call generated indifference *and* the proposal,
whatever it is, clearly has very wide impact. At that point it
seems perfectly legitimate to question whether people really
know what that have agreed to by their silence. Now, a better way
to handle that is for the concerned AD to send explicit mail
to the community during the last call - but if that hasn't
happened, I don't see anything illegitimate in a DISCUSS asking
for additional community review.

> by the time we reach that stage, the  WG has effectively disbanded, 

A working group that has effectively disbanded before its drafts
have been approved? What kind of a WG is that?

> and

Indeed, that would be a "disconsolate DISCUSS" and not what we aim for.
> years now: early cross-area expert review. 

I fully agree.

> Alas, we don't seem to be
> getting there.

Well, we keep trying, e.g.
http://www1.tools.ietf.org/group/iesg/trac/wiki/EarlyExternalReview

But the "we" in question is everybody here; as somebody once said:
This Means You.

> (which, BTW, I cannot find in the list in section 3.1).

It certainly happens, although a more common complaint these days
is more like "the issues raised in the XXX review have not been
addressed" (where XXX is sometimes, but not always, security).
I would see such a DISCUSS as being part of a flawed/clarity
DISCUSS rather than standing alone.

However, since the IETF works by consensus, I don't see how we can
take an assertion of non-consensus off the list.

     Brian

---

From: Keith Moore <[email-masked]>
Date: Thu, 04 Jan 2007 10:16:44 -0500

-------- Original Message --------

> has failed to turn up any specific issues.

I don't know where you get the idea that I've made such a leap.  Each AD 
is able to evaluate for him or herself whether there is consensus within 
the IETF as a whole.  The way this is normally done is to look at Last 
Call responses (or the lack thereof), though an AD may also choose to 
poll individuals or other working groups or mailing lists to see whether 
such a consensus appears to exist.

> years now: early cross-area expert review. Alas, we don't seem to be

I certainly agree that early cross-area review is badly needed, though 
I'm not convinced that limiting such review to a single or a few 
hand-picked experts is sufficient.

Keith

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 11 Jan 2007 23:20:34 -0500

> matching in the Kerberos WG is, in my opinion, less than rational.

As chair of the WG in question, I have to agree.  Though we do have a 
couple of individuals who seem to have developed more expertise in that 
area than is healthy. :-)

> educated on those specific issues, for using them as an example....)

None needed; I think you illustrated the point perfectly.

-- Jeff

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 11 Jan 2007 23:32:42 -0500

> 1. The IETF Last Call did generate dissent.

I'd expect this to be the common case.  That is, if an AD asserts there is 
lack of consensus, I would normally expect that to be because there is a 
demonstrated lack of rough consensus that has not been resolved.

> for additional community review.

Neither do I, but I would not this to be described not as a lack of 
consensus, but as a case where broad _support_ seems necessary but is not 
present, or one in which the AD is concerned that the document has not 
received sufficient review in a particular area.

> take an assertion of non-consensus off the list.

Agree.