Subject: Re: Last Call: <draft-ietf-pkix-est-07.txt> (Enrollment over Secure Transport) to Proposed Standard

From: Russ Housley <[email-masked]>
Date: Mon, 10 Jun 2013 14:45:06 -0400

I have read the document, I a support publication on the standards track.

Russ

On Jun 10, 2013, at 10:45 AM, The IESG wrote:

> 
> The IESG has received a request from the Public-Key Infrastructure
> (X.509) WG (pkix) to consider the following document:
> - 'Enrollment over Secure Transport'
>  <draft-ietf-pkix-est-07.txt> as Proposed Standard
> 
> The IESG plans to make a decision in the next few weeks, and solicits
> final comments on this action. Please send substantive comments to the
> [email-masked] mailing lists by 2013-06-24. Exceptionally, comments may be
> sent to [email-masked] instead. In either case, please retain the
> beginning of the Subject line to allow automated sorting.
> 
> Abstract
> 
> 
>   This document profiles certificate enrollment for clients using
>   Certificate Management over CMS (CMC) messages over a secure
>   transport.  This profile, called Enrollment over Secure Transport
>   (EST), describes a simple yet functional certificate management
>   protocol targeting Public Key Infrastructure (PKI) clients that need
>   to acquire client certificates and associated Certification Authority
>   (CA) certificate(s).  It also supports client-generated public/
>   private key pairs as well as key pairs generated by the CA.
> 
> 
> 
> 
> The file can be obtained via
> http://datatracker.ietf.org/doc/draft-ietf-pkix-est/
> 
> IESG discussion can be tracked via
> http://datatracker.ietf.org/doc/draft-ietf-pkix-est/ballot/
> 
> 
> No IPR declarations have been submitted directly on this I-D.
> 
>

---

From: Pete Resnick <[email-masked]>
Date: Mon, 10 Jun 2013 15:37:25 -0500

Russ, our IAB chair and former IETF chair, just sent a message to the 
IETF list regarding a Last Call on draft-ietf-pkix-est. Here is the 
entire contents of his message, save quoting the whole Last Call request:

> I have read the document, I a support publication on the standards track.

A month ago, we had another very senior member of the community post 
just such a message (in that case directly to the IESG) in response to a 
different Last Call. I took that senior member of the community to task 
for it. But apparently Russ either disagrees with my complaint or didn't 
notice that discussion on the IESG list, so I think it's worth airing 
here in public:

A statement such as the above is almost entirely useless to me as an 
IESG member trying to determine consensus. It is content-free.

We don't vote in the IETF, so a statement of support without a reason is 
meaningless. We should not be encouraging folks to send such things, and 
having the IAB chair do so is encouraging bad behavior. Had I not known 
Russ and his particular expertise, I would have no reason to take it 
into consideration *at all*. We should not have to determine the 
reputation of the poster to determine the weight of the message. Even 
given my background knowledge of who Russ is, I cannot tell from that 
message which one of the following Russ is saying:

- This document precisely describes a protocol of which I have been an 
implementer, and I was able to independently develop an interoperable 
implementation from the document.
- This document is about a technology with which I have familiarity and 
I have reviewed the technical details. It's fine.
- I've seen objection X to the document and I think the objection is 
incorrect for such-and-so reasons.
- My company has a vested interest in this technology becoming a 
standard, and even though I know nothing about it, I support it becoming 
a standards track document.
- My Aunt Gertrude is the document editor and she said that she needs 
statements of support, so here I am.
- I have a running wager on when we're going to reach RFC 7000 and I 
want to increase my odds of winning.

I take it I am supposed to presume from my friendship and knowledge of 
Russ that one of the first three is true and that the last three are 
not. (Well, maybe the last one might be true.) But if instead of from 
"Russ Housely", the message was from "Foo Bar", I would have absolutely 
no way to distinguish among the above.

I think we should stop with these one-line statements of support. They 
don't add anything to the consensus call. I'm disappointed that Russ 
contributed to this pattern.

Other opinions?

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: "Bradner, Scott" <[email-masked]>
Date: Mon, 10 Jun 2013 21:52:10 +0000

better than saying "I have not read the document but I support publication"

I do not see all that much help in having someone list reasons they support publication unless
there is some particularly wonderful feature or the prose is particularly clear

the reverse is not the case, I think there is real value in someone saying in detail why the do not support
publication of a document

Scott

On Jun 10, 2013, at 4:37 PM, Pete Resnick <[email-masked]> wrote:

> Russ, our IAB chair and former IETF chair, just sent a message to the IETF list regarding a Last Call on draft-ietf-pkix-est. Here is the entire contents of his message, save quoting the whole Last Call request:
> 
> A month ago, we had another very senior member of the community post just such a message (in that case directly to the IESG) in response to a different Last Call. I took that senior member of the community to task for it. But apparently Russ either disagrees with my complaint or didn't notice that discussion on the IESG list, so I think it's worth airing here in public:
> 
> A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free.
> 
> We don't vote in the IETF, so a statement of support without a reason is meaningless. We should not be encouraging folks to send such things, and having the IAB chair do so is encouraging bad behavior. Had I not known Russ and his particular expertise, I would have no reason to take it into consideration *at all*. We should not have to determine the reputation of the poster to determine the weight of the message. Even given my background knowledge of who Russ is, I cannot tell from that message which one of the following Russ is saying:
> 
> - This document precisely describes a protocol of which I have been an implementer, and I was able to independently develop an interoperable implementation from the document.
> - This document is about a technology with which I have familiarity and I have reviewed the technical details. It's fine.
> - I've seen objection X to the document and I think the objection is incorrect for such-and-so reasons.
> - My company has a vested interest in this technology becoming a standard, and even though I know nothing about it, I support it becoming a standards track document.
> - My Aunt Gertrude is the document editor and she said that she needs statements of support, so here I am.
> - I have a running wager on when we're going to reach RFC 7000 and I want to increase my odds of winning.
> 
> I take it I am supposed to presume from my friendship and knowledge of Russ that one of the first three is true and that the last three are not. (Well, maybe the last one might be true.) But if instead of from "Russ Housely", the message was from "Foo Bar", I would have absolutely no way to distinguish among the above.
> 
> I think we should stop with these one-line statements of support. They don't add anything to the consensus call. I'm disappointed that Russ contributed to this pattern.
> 
> Other opinions?
> 
> pr
> 
> --

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 10 Jun 2013 15:54:55 -0600

On 6/10/13 2:37 PM, Pete Resnick wrote:

> I think we should stop with these one-line statements of support.

+1 ;-)

/psa

---

From: Dave Crocker <[email-masked]>
Date: Mon, 10 Jun 2013 16:59:19 -0500

On 6/10/2013 4:52 PM, Bradner, Scott wrote:
> etter than saying "I have not read the document but I support publication"
>

Content free statements of support turn the public query into a pure 
popularity contest and/or a game of personal authority.  Is your support 
more important than Pete's non-support?  Absent substance in the 
statements, we've no idea how to evaluate any disparity.

Statements that include substance about the nature and technical 
adequacies are a form of showing one's work on an exam: they demonstrate 
the substance that is the basis the support, not just the fact of it.

d/
-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Ted Lemon <[email-masked]>
Date: Mon, 10 Jun 2013 22:06:15 +0000

> there is some particularly wonderful feature or the prose is particularly clear

I don't really see any point in expressing support for a document during IETF last call unless it is as a rejoinder to some valid expression of non-support.   A valid expression of non-support would point out some technical point that either was not considered by the working group, or was considered, but the working group wasn't competent to evaluate the question.   Or it would point out some process failure that occurred in the working group.

Even if you review the document during IETF last call, unless you find a problem with it there's not much point in saying anything.

---

From: David Morris <[email-masked]>
Date: Mon, 10 Jun 2013 15:19:50 -0700 (PDT)

On Mon, 10 Jun 2013, Ted Lemon wrote:

> I don't really see any point in expressing support for a document during
> IETF last call unless it is as a rejoinder to some valid expression of
> non-support.  A valid expression of non-support would point out some
> technical point that either was not considered by the working group, or
> was considered, but the working group wasn't competent to evaluate the
> question.  Or it would point out some process failure that occurred in
> the working group.
> 
> Even if you review the document during IETF last call, unless you find a 
> problem with it there's not much point in saying anything.

I don't think there is another way to indicate you've reviewed a draft and
found no issues. Surely rough concensus must include confidence that
that silence means more than ignorance and I'm not aware of any mechanism
to evaluate the level of review exercised during last call.

---

From: Ted Lemon <[email-masked]>
Date: Mon, 10 Jun 2013 22:39:03 +0000

> to evaluate the level of review exercised during last call.

The expectation is that the working group gave the document sufficient review; the point of the IETF last call then is to backstop that review, not augment it.   A document that receives no comment in IETF last call is assumed to have consensus.

---

From: SM <[email-masked]>
Date: Mon, 10 Jun 2013 16:21:49 -0700

Hi Pete,
At 13:37 10-06-2013, Pete Resnick wrote:
>just such a message (in that case directly to the IESG) in response 
>to a different Last Call. I took that senior member of the community 
>to task for it. But apparently Russ either disagrees with my 
>complaint or didn't notice that discussion on the IESG list, so I 
>think it's worth airing here in public:

I don't expect IAB or IESG members to be infallible, i.e. they are 
individuals after all.

>IESG member trying to determine consensus. It is content-free.

Yes.

>We don't vote in the IETF, so a statement of support without a 
>reason is meaningless. We should not be encouraging folks to send 
>such things, and having the IAB chair do so is encouraging bad 
>behavior. Had I not known Russ and his particular expertise, I would 
>have no reason to take it into consideration *at all*. We should not 
>have to determine the reputation of the poster to determine the 
>weight of the message. Even given my background knowledge of who 
>Russ is, I cannot tell from that message which one of the following 
>Russ is saying:

The comment was from an individual.  The issue is that if you do a 
blind review of the messages you don't know who sent the message and 
the only weight you could give is to the content of the message.

>They don't add anything to the consensus call. I'm disappointed that 
>Russ contributed to this pattern.

I agree that one-line statements are not of much use.  It's more 
tedious to write a statement to support a proposal than an objection 
to it.  Non-silent Last Calls usually draw objections.  It's going to 
be difficult to balance that if one-line statements of support (or 
objections) are not considered in a determination of consensus.

Regards,
-sm

---

From: Ted Lemon <[email-masked]>
Date: Mon, 10 Jun 2013 23:46:29 +0000

On Jun 10, 2013, at 7:21 PM, SM <[email-masked]> wrote:
> I agree that one-line statements are not of much use.  It's more tedious to write a statement to support a proposal than an objection to it.  Non-silent Last Calls usually draw objections.  It's going to be difficult to balance that if one-line statements of support (or objections) are not considered in a determination of consensus.

Determining consensus in an IETF last call is a bit more complicated than that.   It's not a working group last call.   If someone objects to publication during IETF last call, and their objection has already been discussed and addressed in the working group, the objection in IETF last call doesn't break that consensus.

---

From: Mark Andrews <[email-masked]>
Date: Tue, 11 Jun 2013 10:31:24 +1000

In message <[email-masked]>, T
ed Lemon writes:
> > I agree that one-line statements are not of much use.  It's more tedious =
> to write a statement to support a proposal than an objection to it.  Non-si=
> lent Last Calls usually draw objections.  It's going to be difficult to bal=
> ance that if one-line statements of support (or objections) are not conside=
> red in a determination of consensus.
> 
> Determining consensus in an IETF last call is a bit more complicated than t=
> hat.   It's not a working group last call.   If someone objects to publicat=
> ion during IETF last call, and their objection has already been discussed a=
> nd addressed in the working group, the objection in IETF last call doesn't =
> break that consensus.

Which breaks some of the reasons why we do IETF last calls.  WGs do get too
focused on a problem and do fail to do a balance response to problems.

Mark
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 11 Jun 2013 01:36:30 +0100

Hi Pete,

> IESG member trying to determine consensus. It is content-free.

In fact, you do know Russ. If you did not, then the above would be
far closer to correct. But in reality you do know a lot more than
you claim below. With overwhelming probability, your choice #2
applies and I'm very surprised you don't also think that. If it
made a significant difference and I wasn't sure I'd ask Russ to
clarify. For me, I read Russ' mail and I concluded he meant #2.
And I'm confident in that conclusion.

So I'm really bemused when you say that you don't know how to
interpret Russ' mail.

Declaring that mail problematic also seems quite purist to me.
And insisting on purity seems to me worse than being slightly
but harmlessly ambiguous.

At the same time I agree we do not want a procession of +1 mails.
But then we won't get that for this draft. And if we did get that
for any draft, some IETF participants (probably incl. you and I)
would notice that and query it or object. So perhaps you're also
being a tad trigger-happy on jumping on this message.

Lastly, I think evaluating IETF LC messages only in terms
of how they help the IESG evaluate consensus or not is wrong.
Those messages are for and from the IETF community. So if
e.g. some renowned NFS person who's hardly known to the IESG
were to have sent an equivalent message, that might have been
quite good input that Martin or Spencer could have translated for
you. And that's just fine. And it would be fine if Russ' mail
had been directed not at the IESG but at some other part of the
community.

So bottom line, I think you're wrong, Russ' mail was not
content-free.

S.

PS: Yes, this is a not-very-good academic accusing an employee
of an industrial behemoth of excess purity. Go figure:-)

---

From: Ted Lemon <[email-masked]>
Date: Tue, 11 Jun 2013 01:31:20 +0000

> focused on a problem and do fail to do a balance response to problems.

If enough IETF last call people agree that the working group made a mistake, that could change the consensus evaluation, but there would have to be a substantive argument made.   Revisiting the arguments that happened during WGLC in IETF last call with the same participants can't possibly change the consensus.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 11 Jun 2013 16:51:28 +0900

I'm seeing two things here.

One is that you need some context of *why* something is supported, as per your examples. 

The other is that you need a level of detail that's more than "one line."

However, I'd note that *all* of those examples are (in my MUA) one line each.

So, can you clarify? I.e., if Russ had included one of those lines, would that have been enough?

I'd support asking for that level of detail. OTOH I'm not for making people "show their work" to a greater level of detail; the overhead of participating in an effort is high, and the most relevant people are often exhausted by this point in the process. We shouldn't pile more work onto them (he says, conscious of the work piled onto ADs as well).

Regards,

--
Mark Nottingham   http://www.mnot.net/

---

From: SM <[email-masked]>
Date: Tue, 11 Jun 2013 01:30:03 -0700

>[email-masked] mailing lists by 2013-06-24. Exceptionally, comments may be

There weren't any comments during the WGLC of 
draft-ietf-pkix-est-06.  The AD review of draft-ietf-pkix-est-06 was 
posted to the mailing list and the only comments after that was "this 
version address my concerns".

I read the document.  It is about the use of an obsolete Proposed 
Standard or later versions of that specification.  The comments from 
three individuals who happen to be Area Directors creates a 
conundrum; should I give more weight to them or to a content-free 
comment?  I do not support the publication of this document as a 
Proposed Standard as it is doubtful that it has the consensus of the 
working group.

Regards,
-sm

---

From: Randy Bush <[email-masked]>
Date: Tue, 11 Jun 2013 10:51:53 +0200

so now i am expected to do a write-up of why i show simple support of a
document i have read?  may i use carbon paper for the triplicate, or
will a copier suffice?  surely we can find a way to waste more time and
effort.

randy

---

From: Ted Lemon <[email-masked]>
Date: Tue, 11 Jun 2013 11:02:28 +0000

> will a copier suffice?  surely we can find a way to waste more time and

If you say "I support publication," in IETF last call without saying why, you are arguably wasting other peoples' time.   If you say "I read the document and I think it's technically solid; furthermore, it addresses a real need," then you've spent a solid 5 seconds more typing, and we have something to go on.   Keep your carbon paper in the drawer where it belongs.

---

From: <[email-masked]>
Date: Tue, 11 Jun 2013 12:17:34 +0100

We have to know, not that you have read the document, but that you have -understood- it.

Lloyd Wood
http://sat-net.com/L.Wood/

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Randy Bush [[email-masked]]
Sent: 11 June 2013 09:51
To: Mark Nottingham
Cc: Pete Resnick; [email-masked]
Subject: Re: Content-free Last Call comments

randy

---

From: <[email-masked]>
Date: Tue, 11 Jun 2013 12:20:39 +0100

Ad-hominem arguments are not good arguments.

Peer review depends on what the peer says, not who the peer is - something any academic should know.

Lloyd Wood

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Stephen Farrell [[email-masked]]
Sent: 11 June 2013 01:36
To: Pete Resnick
Cc: [email-masked]
Subject: Re: Content-free Last Call comments

Hi Pete,

S.

---

From: Abdussalam Baryun <[email-masked]>
Date: Tue, 11 Jun 2013 13:28:19 +0200

A comment is a comment (important for discussing) which I want to see,
no matter if content-free or not, the origin requester (IETF Last
Call/WGLC) of such comments SHOULD specify which type of comment they
want if necessary. As long as it is a comment-on-discuss-lists any can
ask questions to the commentor to know the comment-reason if
necessary. IMO, we don't want no-comment, ignorance, or no answers if
requested, which will mean no discussions.

AB

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 11 Jun 2013 13:33:24 +0200

> We have to know, not that you have read the document, but that you have -understood- it.

Process experiment:
end all Internet-Drafts with a multiple-choice test.

Grüße, Carsten

---

From: Dave Cridland <[email-masked]>
Date: Tue, 11 Jun 2013 12:36:49 +0100

On Mon, Jun 10, 2013 at 9:37 PM, Pete Resnick <[email-masked]>wrote:

> A statement such as the above is almost entirely useless to me as an IESG
> member trying to determine consensus. It is content-free.
>

I think this is, in part, due to the question asked.

The IETF's Last Call announcement presumes much on the part of those
reading it. You're aiming to solicit something that's not asked for.

Compare and contrast with the XSF's Last Call announcements, in particular
the questionnaire at the end. Note that in this thread, almost all
respondents are actually filling it in, and further note that at least some
of those are experienced IETF participants, suggesting that even the jaded
IETF folk might join in.

http://jabber.996255.n3.nabble.com/LAST-CALL-XEP-0308-Last-Message-Correction-td14079.html

I'd suggest that putting together a set of five questions you're hoping to
have answered would be sensible and useful.

Perhaps:

1) Do you believe this document is needed?

2) Is the document ready for publication as-is?

3) Are you intending to, or have you already, implemented and/or deployed
this specification?

4) Does the document adequately explain the risks involved in implementing
and/or deploying this specification?

5) Is the document sufficiently clear to allow unambiguous understanding of
how to implement and/or deploy the specification?

Dave.

---

From: Måns Nilsson <[email-masked]>
Date: Tue, 11 Jun 2013 13:52:46 +0200

Subject: Re: Content-free Last Call comments Date: Mon, Jun 10, 2013 at 11:46:29PM +0000 Quoting Ted Lemon ([email-masked]):
 
> Determining consensus in an IETF last call is a bit more complicated
> than that.   It's not a working group last call.   If someone objects to
> publication during IETF last call, and their objection has already been
> discussed and addressed in the working group, the objection in IETF last
> call doesn't break that consensus.
 
So, if wg discussion has been ordered mute by the wg chairs because
some wg participants believe the group-think consensus is good enough,
can those objections again be raised in IETF LC or are they set in stone?

-- 
Måns Nilsson     primary/secondary/besserwisser/machina
MN-1334-RIPE                             +46 705 989668
NATHAN ... your PARENTS were in a CARCRASH!!  They're VOIDED -- They
COLLAPSED They had no CHAINSAWS ... They had no MONEY MACHINES ... They
did PILLS in SKIMPY GRASS SKIRTS ... Nathan, I EMULATED them ... but
they were OFF-KEY ...

---

From: Måns Nilsson <[email-masked]>
Date: Tue, 11 Jun 2013 13:55:56 +0200

> can those objections again be raised in IETF LC or are they set in stone?

s/they/the WG decisions/

My apologies. 
-- 
Måns Nilsson     primary/secondary/besserwisser/machina
MN-1334-RIPE                             +46 705 989668
Once, there was NO fun ... This was before MENU planning, FASHION
statements or NAUTILUS equipment ... Then, in 1985 ... FUN was
completely encoded in this tiny MICROCHIP ... It contain 14,768 vaguely
amusing SIT-COM pilots!!  We had to wait FOUR BILLION years but we
finally got JERRY LEWIS, MTV and a large selection of creme-filled
snack cakes!

---

From: Dave Crocker <[email-masked]>
Date: Tue, 11 Jun 2013 06:58:21 -0500

On 6/11/2013 6:36 AM, Dave Cridland wrote:
> Compare and contrast with the XSF's Last Call announcements, in

Re-formulating the LC text sounds like an excellent idea, to call for 
more substantive comments.

> I'd suggest that putting together a set of five questions you're hoping
> to have answered would be sensible and useful.
>
> Perhaps:

If we want the statements of support to be meaningful, they need to have 
the creator of the statement do some real work -- more than mechanically 
checking boxes -- demonstrating the 'understanding' that Lloyd suggests. 
  Multiple guess questions don't demonstrate understanding; worse, they 
are too easily plagiarized as part of a campaign.

One of the unfortunate realities in the current IETF is periodically 
seeing patterns of support that have more to do with politicking for a 
draft than for commenting on a critical review of it.  There is no 
perfect protection against this, but asking each statement of support to 
demonstrate the commenter's own understanding will help.

We also sometimes have drafts that have had little working group 
activity.  This is independent of the quality of the work, but it means 
that there's little sense of community need or interest.  It's not 
supposed to happen, but it's become more common in the current IETF. 
Again, there's no perfect protection against that, but seeing public 
activity during IETF LC that demonstrates enough community interest to 
do the minimal work of offering a capsule commentary on the draft will help.

d/

---

From: Ted Lemon <[email-masked]>
Date: Tue, 11 Jun 2013 12:12:02 +0000

> can those objections again be raised in IETF LC or are they set in stone?

You can always challenge the WG chair's finding, but you don't need to hassle the whole IETF to do that—just talk to the responsible AD for the working group, or to the IESG as a whole if the responsible AD fails you.   That's what the appeals process is supposed to be for.

You can of course raise the point on the IETF mailing list, and that is likely to result in the responsible AD considering the question of whether the WG chairs made the right call—I certainly would do so if someone raised such a point on a document for which I was responsible AD.   But if the responsible AD decides that the chair made the right call, the objection you raise in IETF last call doesn't count against the working group consensus.

---

From: Dave Cridland <[email-masked]>
Date: Tue, 11 Jun 2013 13:25:11 +0100

On Tue, Jun 11, 2013 at 12:58 PM, Dave Crocker <[email-masked]> wrote:

>  Multiple guess questions don't demonstrate understanding; worse, they are
> too easily plagiarized as part of a campaign.
>
>
We want understanding, of course, but I think requiring Russ to demonstrate
that by writing a paragraph or six on the finer points of the proposal
would be daft.

> draft than for commenting on a critical review of it.  There is no perfect
> protection against this, but asking each statement of support to
If the politicking is from multiple organizations who all want to implement
and deploy, then I'm all in favour...

If there's only one implementer willing to say as much, then even quite a
slew of deployment wannabees would have me concerned for the viability of
the protocol.

I'd note that the XSF's questions are only concerned with implementation
rather than deployment - maybe that helps, I'm not sure either way.

We also sometimes have drafts that have had little working group activity.
>  This is independent of the quality of the work, but it means that there's
> little sense of community need or interest.  It's not supposed to happen,
> but it's become more common in the current IETF. Again, there's no perfect
> protection against that, but seeing public activity during IETF LC that
> demonstrates enough community interest to do the minimal work of offering a
> capsule commentary on the draft will help.

Perhaps having a shepherd-style write up included in the last call
announcement? (Or available via a URL there).

Dave.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 11 Jun 2013 05:45:15 -0700

On 6/11/2013 5:25 AM, Dave Cridland wrote:
> We want understanding, of course, but I think requiring Russ to
> demonstrate that by writing a paragraph or six on the finer points of
> the proposal would be daft.

That's the problem with special-case exceptions, such as requiring less 
work by an august personage.  It reduces to a cult of personality and it 
doesn't scale.  For an organizational culture of the type the IETF 
expresses, that doesn't fit.  The opinions of people IETF management 
positions are not supposed to automatically have more weight in 
determining the specifics of our specifications; they are supposed to 
make their case, just like everyone else.

We try to distinguish between comment when wearing a formal IETF hat 
versus without a hat.  So it's not the IETF Chair making the comment, 
it's "merely" a well-known personage.

It's easy to give special rights to such folk, such as not requiring 
them to offer the substance behind their statement, but it actually has 
a pretty insidious effect.  It's gets us used to pro-forma postings; it 
gets us relying on a few folk to sway things; it gets us to count rather 
than think.

> If the politicking is from multiple organizations who all want to
> implement and deploy, then I'm all in favour...

Pete Resnick has been working on a careful formulation of what the IETF 
means when it talks about 'rough consensus'.  My own interpretation of 
what he's developing -- and I want to stress this is me speaking, not me 
speaking for Pete -- is that consensus is a combination of both numbers 
and substance.  The mere fact that "almost everyone" is in favor of 
something can't be enough.  What is also required is that the arguments 
of objectors must have inadequately persuasive substance.  One voice 
with a really solid concern, which withstands independent review, needs 
to be able to upset an overwhelming agreement.

So no, the fact that the politicking is from multiple organizations 
needs to be insufficient.

>     activity.
...
> announcement? (Or available via a URL there).

Perhaps something like that, yeah.

d/

---

From: Dave Cridland <[email-masked]>
Date: Tue, 11 Jun 2013 14:27:00 +0100

On Tue, Jun 11, 2013 at 1:45 PM, Dave Crocker <[email-masked]> wrote:

> positions are not supposed to automatically have more weight in determining
> the specifics of our specifications; they are supposed to make their case,
> just like everyone else.
>
> versus without a hat.  So it's not the IETF Chair making the comment, it's
> "merely" a well-known personage.
>
> It's easy to give special rights to such folk, such as not requiring them
> to offer the substance behind their statement, but it actually has a pretty
> insidious effect.  It's gets us used to pro-forma postings; it gets us
> relying on a few folk to sway things; it gets us to count rather than think.
>
>
Ah, sorry, that's not what I meant - I included Russ's name purely because
he was the original exemplar, not because he's special in any particular
way.

I meant that requiring anyone to demonstrate understand of the draft by
jumping through hoops would, ipso facto, require them to jump through hoops.

> means when it talks about 'rough consensus'.  My own interpretation of what
> he's developing -- and I want to stress this is me speaking, not me
> speaking for Pete -- is that consensus is a combination of both numbers and
> substance.  The mere fact that "almost everyone" is in favor of something
> can't be enough.  What is also required is that the arguments of objectors
> must have inadequately persuasive substance.  One voice with a really solid
> concern, which withstands independent review, needs to be able to upset an
> overwhelming agreement.
>
> So no, the fact that the politicking is from multiple organizations needs
> to be insufficient.
>
>
Again, I think you're misunderstanding me - I meant (somewhat facetiously)
I'm in favour of the politicking, not that I think that it should carry the
day automatically. If there are unanswered objections, that should indeed
count against. More generally, in the case of the XSF's small set of
questions, if people answer the last call with one-word answers to those
and nothing further, this gives that community sufficient information to
gauge whether to advance a proposal along the standards track there - in
other words, given a fairly minimal bar, any engagement meeting that bar is
valuable.

That bar has to be high enough to carry more than the single bit of
information Russ's note carried on its own, though, but it also needs to be
low enough that it won't prove a barrier to response.

I think the XSF's questions are close to the right level (I think the XSF
could, if it wanted, tweak these and improve them after this number of
years); I don't think it would be very hard to find some similarly
reasonable start point for the IETF.

Dave.

---

From: Sean Turner <[email-masked]>
Date: Tue, 11 Jun 2013 09:31:35 -0400

> You can of course raise the point on the IETF mailing list, and that is likely to result in the responsible AD considering the question of whether the WG chairs made the right call—I certainly woul…

Engaging during IETF LC on a point made during WGLC or earlier that 
wasn't adopted is a-okay in my book.  I think of it as raising 
awareness.  I mean maybe the authors and WG chair didn't get it right. 
But, then what Ted said kicks in.

spt

---

From: Randy Bush <[email-masked]>
Date: Tue, 11 Jun 2013 15:41:29 +0200

> more substantive comments.

perhaps we should go to the source of the problem and require a phd
dissertation and defense from draft authors.

how much process chaos can we create?

randy

---

From: Ted Lemon <[email-masked]>
Date: Tue, 11 Jun 2013 13:43:23 +0000

> how much process chaos can we create?

Don't ask questions you don't want answered!   :)

---

From: Dave Cridland <[email-masked]>
Date: Tue, 11 Jun 2013 14:47:48 +0100

> how much process chaos can we create?
I think using a reductio ad absurdum is a little unfair when I've
demonstrated that the proposed reformulation works (whilst being purely
voluntary) in a similar organization with common participants.

In particular, I'm not suggesting any change to process.

Dave.

---

From: <[email-masked]>
Date: Tue, 11 Jun 2013 14:55:44 +0100

How many RFCs describe things that are implemented?
How many RFCs describe things that are deployed?

Lloyd Wood

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Dave Cridland [[email-masked]]
Sent: 11 June 2013 12:36
To: Pete Resnick
Cc: [email-masked] Discussion
Subject: Re: Content-free Last Call comments

On Mon, Jun 10, 2013 at 9:37 PM, Pete Resnick <[email-masked]<mailto:[email-masked]>> wrote:
A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free.

The IETF's Last Call announcement presumes much on the part of those reading it. You're aiming to solicit something that's not asked for.

Compare and contrast with the XSF's Last Call announcements, in particular the questionnaire at the end. Note that in this thread, almost all respondents are actually filling it in, and further note that at least some of those are experienced IETF participants, suggesting that even the jaded IETF folk might join in.

I'd suggest that putting together a set of five questions you're hoping to have answered would be sensible and useful.

Perhaps:

3) Are you intending to, or have you already, implemented and/or deployed this specification?

4) Does the document adequately explain the risks involved in implementing and/or deploying this specification?

5) Is the document sufficiently clear to allow unambiguous understanding of how to implement and/or deploy the specification?

Dave.

---

From: Warren Kumari <[email-masked]>
Date: Tue, 11 Jun 2013 13:21:55 -0400

[ Not sure if this is adding to the Signal or the Noise on the Discuss list, but it *will* help bump up my ranking on the "Weekly posting summary", which I use to justify my participation to my management. That's what it's for, isn't it?!* ] 

> A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free.

I disagree.

> I take it I am supposed to presume from my friendship and knowledge of Russ that one of the first three is true and that the last three are not.
> (Well, maybe the last one might be true.) But if instead of from "Russ Housely", the message was from "Foo Bar", I would have absolutely no way to distinguish among the above.

Actually, yes.

Russ has been participating in the IETF (and specifically in the area where he posted the above email) for a long time -- you know this, because you've also been participating.
In *my opinion* he has shown himself to be diligent and sane. This means that *I* would give his comment and support great weight -- I'd *assume* he has read and understood the document, and is supporting it because #1, 2, 3 and / or 6. If Foo Bar had posted the comment, and in *my* opinion  Foo Bar is a total nutter, I would give his comment less, or possibly negative,  weight. Obviously your opinions of Russ and Foo may be opposite to mine -- you apply your own weighting to each comment -- that's why we pay you the big bucks…

If Foo Bar is new enough to the IETF and cannot reasonably expect everyone on the IESG (or in a WG or wherever) to know and have formed an opinion of him, then it is *Foo Bar's* responsibility to more fully support his comments. 

Do folk who actually *participate* actively and sanely get to assume that they have earned some standing and credibility? Yup. I view this as a feature, not a bug. 

If I go to my doctor and he tells me that I simply have a cold (and not, like I'm convinced, the plague), I should presumably weight his comments higher than those of my crazy next door neighbor (who, apparently, routinely communicates with beings from another dimension), yes? 

We want to reward merit and participation, not make the process so annoying that those who participate get annoyed and wander off.

If anyone *opposes* a draft, I think that it behooves them to explain what the issue is, regardless of who they are. This is similar to at a restaurant -- when the waiter asks if you are enjoying your [steak|tofu] it's fine to say "Yes thanks, great", but if express displeasure you should be ready to explain what you didn't like.

> I think we should stop with these one-line statements of support. They don't add anything to the consensus call. I'm disappointed that Russ contributed to this pattern.

My opinion is that the folk on the IETF / WG chairs / anyone evaluating information uses their opinion on the source of that information as input to it's weight. This is why we have people judging consensus (see http://tools.ietf.org/html/draft-resnick-on-consensus-01 :-P) and not voting / an algorithm.

W

> pr

[*]: :-P
-- 
"He who laughs last, thinks slowest." 
    -- Anonymous

---

From: "John Levine" <[email-masked]>
Date: 11 Jun 2013 17:43:32 -0000

>can those objections again be raised in IETF LC or are they set in stone?

If that were ever to happen, I don't see why not.

In the recent cases I've seen where someone claims this is an issue,
what actually happned is that a person only tangentially involved in
the WG is obsessing about some technical wart and refuses to (or
worse, can't) understand the overall context that led the WG to decide
what it did.

It's hard to see any benefit to rehashing such arguments in front of
the whole IETF.

R's,
John

---

From: Doug Barton <[email-masked]>
Date: Tue, 11 Jun 2013 10:52:01 -0700

> It's hard to see any benefit to rehashing such arguments in front of

The flip side of that argument is that we don't want to assume working 
groups are infallible, or more importantly not subject to the groupthink 
phenomenon. Otherwise what is IETF LC for?

Doug

---

From: Ted Lemon <[email-masked]>
Date: Tue, 11 Jun 2013 18:02:29 +0000

On Jun 11, 2013, at 1:52 PM, Doug Barton <[email-masked]> wrote:
> The flip side of that argument is that we don't want to assume working groups are infallible, or more importantly not subject to the groupthink phenomenon. Otherwise what is IETF LC for?

The IETF last call is for catching things the working group missed, not for rehashing arguments that were beaten to death in the working group.   It is certainly possible for a discussion in the working group to go one way, and then for the same discussion to come up in IETF last call and go the other way, because the experts on the topic were not included in the discussion, or because their advice was inappropriately ignored.   This does happen; unfortunately, when it happens it often doesn't get caught in IETF last call anyway.

---

From: Melinda Shore <[email-masked]>
Date: Tue, 11 Jun 2013 11:09:54 -0800

> phenomenon. Otherwise what is IETF LC for?

Right.  We've had some issues with document quality, and I
can think of several documents that sailed through WG last
call and should not have.

Melinda

---

From: Dave Crocker <[email-masked]>
Date: Tue, 11 Jun 2013 12:13:50 -0700

On 6/11/2013 10:21 AM, Warren Kumari wrote:
> [ Not sure if this is adding to the Signal or the Noise on the
> Discuss list, but it*will*  help bump up my ranking on the "Weekly
> posting summary", which I use to justify my participation to my
> management. That's what it's for, isn't it?!* ]

oh boy. we need to put that on a t-shirt.  it's just perfect.  and 
universal.

d/

---

From: Melinda Shore <[email-masked]>
Date: Tue, 11 Jun 2013 11:14:25 -0800

On 6/11/13 10:02 AM, Ted Lemon wrote:
> The IETF last call is for catching things the working group missed,
> not for rehashing arguments that were beaten to death in the working
> group. 

I am not sure I fully understand why we're having this conversation,
or rather why this aspect of the broader discussion requires attention.
Sometimes working groups make mistakes, and I don't think that in
practice there'd be general objection to having serious problems
identified during IETF last call.  Yes, there have been cases where
cranks who can't let go of an idea that was rejected try to flog it
to death, but that doesn't mean that identifying and dealing with
real problems should be dismissed.

Melinda

---

From: Pete Resnick <[email-masked]>
Date: Tue, 11 Jun 2013 14:45:59 -0500

It's interesting to see that people are interpreting me to mean I want 
more text. I don't. I want less. Save your breath. There is no reason to 
send one line of support, and it only encourages the view that we're 
voting. Details below.

Specifically on Stephen's message:

> So I'm really bemused when you say that you don't know how to

Let's separate out a few issues. There's what the message means itself, 
there's its usefulness and how much weight it should be given, and then 
there's what's problematic about it. I think you've conflated a few things.

As you know, on the earlier message I called out on the IESG list, I 
guessed wrong that the sender was claiming choice #3 (that is, I 
thought, incorrectly, that the sender disagreed with a recent argument 
against publication, but simply didn't explain why). In fact in that 
case, not only wasn't it choice #4, it was also neither choice #1 (an 
implementer claiming implementation) nor choice #2 (an expert on the 
topic), my other two guesses, but rather a generally smart IETFer who 
thought the work seemed like a good idea. So I'm a little disinclined to 
guess on motives. In the case of Russ's message, I did guess it was 
choice #2, but I wasn't really sure.

But let's separate what Russ meant from the weight it should be given. 
The message was that a person (let's presume with some expertise in the 
area) had read it. So what? Didn't this document go through a WG? 
Weren't a good bunch of experts already reading and reviewing this 
document? If this was during the WG discussion and the chairs solicited 
some final check reviews, *maybe* the message could have some use. But 
during IETF Last Call? Even if the message was sent because it was 
trying to say, "I'm an expert on this stuff and it looks technically 
sound", is that going to change anything that the IESG should do about 
the document? I just think at Last Call, these statements of support are 
either harmless-but-useless or they are nefarious.

> Declaring that mail problematic also seems quite purist to me.

I always find it amusing that when Stephen and I disagree, it's almost 
always because he presumes I want purity (and usually a new process), 
which I don't, and it seems to me that he wants no rules-of-thumb at all 
and that everything should be on a case-by-case basis. (I guess that 
means that I'm into purity, where Stephen has no principles. ;-) ) I 
think general principles are a fine thing, and we should try to stick to 
them, but I've got no interest in purity.

The single piece of mail isn't problematic at all. As I said, probably 
harmless but useless. But the overall impression it gives *is* 
problematic. It encourages the view that we are voting, that a simple 
statement of support is important to our process. (What really set my 
hair on end was the "I support publication" bit. That always sounds like 
a vote for a new RFC. From Russ, I have a pretty good idea that he 
didn't mean it that way, but it's a poor formulation and not what we 
should be encouraging.) It also indicates that during Last Call, we want 
to hear that a document is getting reviewed. I would like to presume 
that documents get serious reviews in the WG, and that Last Call time is 
for people who haven't been participating in the WG to do a final check, 
letting us know if there's something *wrong*. If the document wasn't 
getting good review in the WG and needs statements of support at Last 
Call time, something has seriously failed earlier in the process. The 
chair or responsible AD should have been saying early on, "Dear Expert, 
can you please have a read over this document and see if it's sane." 
Last Call is the wrong time for that to happen. This just encourages 
more of the tail-heavy process that we discussed earlier.

> At the same time I agree we do not want a procession of +1 mails.

Right. And having senior members of the community doing so will 
encourage such behavior in the future.

> would notice that and query it or object.

I'd rather have all of us crusty folks model good behavior rather than 
having to complain about bad behavior later, especially when we'll get 
responses like, "But the chair did it." We've already seen a couple of 
comments to my message from non-crusty-folks indicating that any +1 is 
just dandy. Blech.

> had been directed not at the IESG but at some other part of the

So this is the most interesting argument. But I would say that requiring 
this kind of interpretation/translation is problematic as a general rule 
when the messages are going to the IETF list in response to a Last Call. 
If a renowned NFS person is doing a review of an NFS document, I think 
that's great. And I think that's helpful to Martin or Spencer. But if 
all they're going to say is, "I don't have any concerns", it doesn't 
seem to me to be that useful a piece of info on the IETF list at Last 
Call time. Again, at best it encourages the idea that just because Über 
NFS person endorses the document, that weighs in its favor. The 
important thing to hear at Last Call time are objections, not 
endorsements. As a note to the IESG saying, "Just in case you didn't 
think this got review from the right people before, I, Über NFS person, 
hereby say that I reviewed it and have no problem with it", that has 
some content, but it does indicate that we are waiting until too late in 
the process to be thinking about such things.

> So bottom line, I think you're wrong, Russ' mail was not content-free.
>    

Yeah, you haven't convinced me.

> of an industrial behemoth of excess purity. Go figure:-)

I'll note that I hide in a small R&D corner of my industrial behemoth, 
mostly teaching classes to other engineers. You, on the other hand, 
probably still write code. :-)

pr

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 12 Jun 2013 08:05:38 +1200

Pete,

> send one line of support, and it only encourages the view that we're

Just to test what you are saying, let me ask the following. I will
stipulate that a message that can be summarised accurately as "+1" doesn't
serve much purpose. How would you react to one that says something like:
"I've read the draft, and I've considered Joe Blow's objection, but I
still support publication of the draft" (*not" followed by a reasoned
rebuttal of Joe Blow's argument)?

   Brian

---

From: Dave Cridland <[email-masked]>
Date: Tue, 11 Jun 2013 21:24:00 +0100

On Tue, Jun 11, 2013 at 8:45 PM, Pete Resnick <[email-masked]>wrote:

> more text. I don't. I want less. Save your breath.

Well, this thread is surely evidence that you don't always get what you
want.

But more seriously, what are you expecting Russ to do? What did you want
him to write?

If your answer is "Nothing", then how do you read IETF consensus for a
document that gets no response in its Last Call? The XSF's stance is often
"It got nothing in Last Call, it shouldn't advance", which seems reasonable
to me - I don't think defaulting to publication is right - certainly not in
every case.

My suggestion was simply to ask for what you want in the Last Call.

But I imagine you could also just start an endless thread on an already
overloaded list until someone guesses the answer.

Dave.

---

From: Ted Lemon <[email-masked]>
Date: Tue, 11 Jun 2013 20:33:26 +0000

On Jun 11, 2013, at 4:24 PM, Dave Cridland <[email-masked]<mailto:[email-masked]>> wrote:
But more seriously, what are you expecting Russ to do? What did you want him to write?

If your answer is "Nothing", then how do you read IETF consensus for a document that gets no response in its Last Call? The XSF's stance is often "It got nothing in Last Call, it shouldn't advance", which seems reasonable to me - I don't think defaulting to publication is right - certainly not in every case.

I have a modest suggestion for you: read the rest of the message from Pete that you just replied to, in which he answers each of the questions you asked in your reply.

---

From: Pete Resnick <[email-masked]>
Date: Tue, 11 Jun 2013 15:31:03 -0500

On 6/11/13 3:05 PM, Brian E Carpenter wrote:
> Just to test what you are saying, let me ask the following.

Oooo...I love a test.

> How would you react to one that says something like:

I would be rather grumpy with such a message. If there's an outstanding 
(reasonable) objection to a document, I need to know why to consider 
that argument in the rough. I'd have to ask for more detail from the 
sender. If the response I get back is, "I figured it was obvious why Joe 
Blow was full of crap", I'd ask, "Then why did you bother posting?" If 
the sender happens to be an expert (and Joe Blow is not), I'm still not 
going to take it at face value that Joe Blow is wrong. If I did, Joe 
would be well within rights to appeal because his argument got blown off.

So, if you're saying something that is perfectly obvious, no need to say 
it. But if it's not perfectly obvious, I do want more text.

pr

---

From: Doug Barton <[email-masked]>
Date: Tue, 11 Jun 2013 13:50:01 -0700

On 06/11/2013 11:02 AM, Ted Lemon wrote:
>> The flip side of that argument is that we don't want to assume
>> working groups are infallible, or more importantly not subject to
>> the groupthink phenomenon. Otherwise what is IETF LC for?
>

As I understand it cross-disciplinary review is also an important 
function of the IETF LC.

> It is certainly possible for a discussion in the working
> group to go one way, and then for the same discussion to come up in
> IETF last call and go the other way, because the experts on the topic
> were not included in the discussion, or because their advice was
> inappropriately ignored.

Right ... for example the recent issue in regards to the potential 
deprecation of the SPF DNS RRtype.

> This does happen; unfortunately, when it
> happens it often doesn't get caught in IETF last call anyway.

Then the process is faulty. :)

I should also point out that when I mentioned "groupthink" in my message 
I was not doing so in a snarky and/or throwaway manner. The phenomenon 
is very real, and IETF WGs are an ideal breeding ground for it on 
several levels. It's _very_ easy for a group of humans to get into a 
mutual confirmation bias feedback loop, and although we generally do 
better on that point than a lot of groups would in similar 
circumstances, having people from outside the WG review documents 
provides (or should provide) a much-needed sanity check. In many ways 
having people from outside the group provide a well thought out review 
is _more_ valuable than the process of creating the document within the 
WG itself (although obviously we need to give proper credit for that 
creation).

Doug

---

From: Dave Cridland <[email-masked]>
Date: Tue, 11 Jun 2013 22:27:29 +0100

On Tue, Jun 11, 2013 at 9:33 PM, Ted Lemon <[email-masked]> wrote:

>  On Jun 11, 2013, at 4:24 PM, Dave Cridland <[email-masked]> wrote:
>
> I have a modest suggestion for you: read the rest of the message from Pete
> that you just replied to, in which he answers each of the questions you
> asked in your reply.
>
>

If he did, either I am sufficiently stupid I cannot make it out, or he did
so sufficiently obliquely that I can't see - depending on where you want to
place the blame. Either way, I've tried to parse out potential responses
(again), and the closest I can come (again) is that Pete seems to want
nothing from Russ in this instance, and indeed, in any instance from any
person except objections. This is not clear to me, however, and therefore
I'd appreciate some clarification, which is why I asked.

That in turn presumes we are defaulting to publication in all cases, and
that in turn seems problematic to me, because his answers become, in order:

a) Russ, and by extension anyone who supports the document's publication
for whatever reason, is expected to do nothing.

b) Russ, and by extension anyone who supports the document's publication
for whatever reason, should write nothing.

c) IETF-wide consensus is not judged here. IETF-wide apathy is IETF-wide
consensus.

My conclusion from that are:

1) Pete strongly supports publish-by-default.

2) Pete has his RFC 7000 sweepstake money on a much earlier date than I do.

Dave.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 11 Jun 2013 14:33:12 -0700

> As I understand it cross-disciplinary review is also an important

This gets at the reality that the current IETF uses processing phases 
rather more robustly than we used to.  It certainly used to be that IETF 
LC was essentially a last chance for people with objections, but now 
it's a full round of broad reviews.  It is, therefore, a real vetting 
phase by the larger community.  As such, positive comments can be as 
valuable as negative, but of course only if they are backed with substance.

Possibly contrary to Pete's stated preference, I'll suggest that 
statements of pure support -- backed up with substantive commentary -- 
can be useful, given the low level of activity and involvement many 
working groups currently show.

To the extent that the IETF cares whether anyone beyond the authors care 
about the document, this shows that they do.

d/

---

From: Ted Lemon <[email-masked]>
Date: Tue, 11 Jun 2013 21:54:22 +0000

On Jun 11, 2013, at 5:27 PM, Dave Cridland <[email-masked]> wrote:
> That in turn presumes we are defaulting to publication in all cases, and that in turn seems problematic to me, because his answers become, in order:
> a) Russ, and by extension anyone who supports the document's publication for whatever reason, is expected to do nothing.
> b) Russ, and by extension anyone who supports the document's publication for whatever reason, should write nothing.
> c) IETF-wide consensus is not judged here. IETF-wide apathy is IETF-wide consensus.

It is presumed that some degree of consensus to do the work of a working group existed when that working group was chartered; otherwise it would not have been chartered.   When the working group reaches consensus to publish, therefore, it is assumed that the IETF has consensus to publish the document, because the IETF tasked the working group to go off and do its work, and the working group did it.

Therefore, silence during IETF last call is not interpreted as apathy, but rather a lack of objection to the completion of a process that the IETF chose to embark on and that the IETF has brought to completion, through the instrument of the working group that produced the document.

This is in fact how consensus is evaluated during IETF last call.   If you think it should be done differently, write up a document and get IETF consensus on it, and we can change the procedure to whatever you think it should be.   Maybe it would be an improvement.

---

From: Elwyn Davies <[email-masked]>
Date: Tue, 11 Jun 2013 23:02:53 +0100

Pete,

I write gen-art reviews.  A proportion of them at IETF Last Call, give 
or take a bunch of boilerplate, consist of the word 'Ready'*.

How do you distinguish the usefulness (or otherwise) of such a review 
from Russ' one liner?

/Elwyn

* and some of the others say 'Not ready' followed by some or many lines 
of comments.

---

From: Dave Cridland <[email-masked]>
Date: Tue, 11 Jun 2013 23:03:35 +0100

On Tue, Jun 11, 2013 at 10:54 PM, Ted Lemon <[email-masked]> wrote:

> It is presumed that some degree of consensus to do the work of a working
> group existed when that working group was chartered; otherwise it would not
> have been chartered.   When the working group reaches consensus to publish,
> therefore, it is assumed that the IETF has consensus to publish the
> document, because the IETF tasked the working group to go off and do its
> work, and the working group did it.
>
> Therefore, silence during IETF last call is not interpreted as apathy, but
> rather a lack of objection to the completion of a process that the IETF
> chose to embark on and that the IETF has brought to completion, through the
> instrument of the working group that produced the document.
>
> This is in fact how consensus is evaluated during IETF last call.

That's interesting - judging by the messages on this thread, there doesn't
appear to be a strong consensus on this...

>   If you think it should be done differently, write up a document and get
> IETF consensus on it, and we can change the procedure to whatever you think
> it should be.   Maybe it would be an improvement.
>
>
... and how would we judge IETF consensus on a document that doesn't get
done under a charter (which would in turn have been granted consensus
without any IETF comments?)

---

From: Ted Lemon <[email-masked]>
Date: Tue, 11 Jun 2013 22:20:28 +0000

On Jun 11, 2013, at 6:03 PM, Dave Cridland <[email-masked]<mailto:[email-masked]>> wrote:
... and how would we judge IETF consensus on a document that doesn't get done under a charter (which would in turn have been granted consensus without any IETF comments?)

I would expect that you'd start with a mailing list, see if there is interest, come up with a proposal for a BOF, come up with a working group charter, get IESG review on the charter, then get IETF consensus on the charter, and then start working on the document.   That's how it's usually done.

BTW, the fact that a few people think the process ought to work differently does not mean there is consensus for it to work differently.   Also, what there may not be consensus on among the people who have weighed in on the topic is whether positive statements in favor of a document are relevant in IETF last call, but I don't really know how to reduce that to practice, because in reality I think it is rare for a quorum of IETF participants to read a document as a consequence of a last call announcement.   Without that, I don't see how you can have any other last call process than the one we currently have.

---

From: Sean Turner <[email-masked]>
Date: Tue, 11 Jun 2013 19:13:10 -0400

On 6/11/13 4:30 AM, SM wrote:
> There weren't any comments during the WGLC of draft-ietf-pkix-est-06.
> The AD review of draft-ietf-pkix-est-06 was posted to the mailing list
> and the only comments after that was "this version address my concerns".

PKIX is closing down; this is the final draft.  Has the wg lost a bit of 
its furor yes.  Mostly, the way things have worked in the recent past is 
that either people are for something and have some nice conversations 
about this or that improvement and wglc is quiet or people think the 
world is ending by publishing a draft and they object during authoring, 
wglc, and ietf lc.  This one falls in to the earlier bucket.

> Standard or later versions of that specification.

This bit I don't agree with.  PKCS#10 is the way most PKI enrollments 
occur.  RFC 2986, which is where PKCS#10 is documented, is updated by 
RFC 5967 because the draft that contained the media-type registration 
was obsoleted.

This draft is a nod to the fact that PKCS#10 is the way it's done and 
it's one via the web (not email) with TLS as the secure transport (not 
S/MIME/CMS).  The kicker here is that it's actually been implemented and 
not just by the authors.

As for the TLS version, well I'm hoping 1.1 will be overtaken by events 
in short order but that compromise has been worked out and documented in 
a number of drafts.

>  The comments from
> three individuals who happen to be Area Directors creates a conundrum;
> should I give more weight to them or to a content-free comment?  I do
> not support the publication of this document as a Proposed Standard as
> it is doubtful that it has the consensus of the working group.

There has been discussion about this draft on the list albeit not much. 
  Definitely discussions at the meetings where it's been an agenda topic 
at IETF 82, 83, 85, and 86.

spt

---

From: Randy Bush <[email-masked]>
Date: Wed, 12 Jun 2013 03:13:20 +0200

> call and should not have.

there was a doc with which i had a small, but non trivial, issue.  the
author and the wg did not think it worthwhile.  i did not want to argue
endlessly, so i waited and raised it to the iesg at last call, and they
understood it instantly.  the author made the small change to the doc.
i said thank you.

i am sure we can make a constitutional crisis over this if we have too
much time on our hands.

randy

---

From: Dave Cridland <[email-masked]>
Date: Wed, 12 Jun 2013 09:43:43 +0100

On Tue, Jun 11, 2013 at 11:20 PM, Ted Lemon <[email-masked]> wrote:

>  On Jun 11, 2013, at 6:03 PM, Dave Cridland <[email-masked]> wrote:
>
> I would expect that you'd start with a mailing list, see if there is
> interest, come up with a proposal for a BOF, come up with a working group
> charter, get IESG review on the charter, then get IETF consensus on the
> charter, and then start working on the document.   That's how it's usually
> done.
>
>
I think you misread entirely.

But anyway, let's assume we follow those steps.

I suspect the closest we get to getting an idea of IETF consensus is the
interest gauging at the beginning of the process, though interestingly this
is only positive interest - objections to doing the work at all aren't
really relevant here. The IETF consensus on the charter is handled by
Apathy Is Assent rules, so claiming that this consensus call becomes the
default is an interesting argument to make.

 BTW, the fact that a few people think the process ought to work
> differently does not mean there is consensus for it to work differently.
> Also, what there may not be consensus on among the people who have weighed
> in on the topic is whether positive statements in favor of a document are
> relevant in IETF last call, but I don't really know how to reduce that to
> practice, because in reality I think it is rare for a quorum of IETF
> participants to read a document as a consequence of a last call
> announcement.   Without that, I don't see how you can have any other last
> call process than the one we currently have.
>
>
OK, so we don't have voting but we do have a quorum? How wonderful. And
given that the majority of people are silent means they agree (with, mind,
everything - not just that they're not reading), so any number of people
are therefore "few".

But to loop around:

I strongly feel that positive statements have value, as they allow the
community to gauge the level of review and consensus, and I suspect that
human nature means that we get more reviews if people get to brag about it.
I suggest that if more than one bit of data is required, it's simply asked
for. Given that the text of IETF Last Call announcements is not governed by
any process RFC that I'm aware of (feel free to correct), I suggest simply
putting a set of optional questions there. I note this practise has served
the XSF very well. I do not think this needs an endless bikeshed discussion
on what questions; the IESG can pick what it wants to know.

If, on the other hand, only objections are sought, then the text (which
simply asks for "comments") also needs changing. And the GenArt, AppsDir,
and SecDir reviews should only be send when they have objections to
publication, of course.

If you feel that the only way to make either change is to form a working
group and publish an RFC to change something undocumented in the series,
then I think we're stranded in a bureaucratic quagmire with no chance of
escape, but I'll be happy to send "comments", as requested, nonetheless.

---

From: Jari Arkko <[email-masked]>
Date: Wed, 12 Jun 2013 12:28:10 +0300

> dissertation and defense from draft authors.

A couple of years ago I worked with someone who completed his PhD thesis on a topic faster than it took to publish the RFC on the same topic… that was my wake-up call for IETF process needing some work :-) Things have improved since then, though, but probably not enough.

But back to the topic. I, for one, would like to see responses on IETF last calls. It builds my confidence that we know enough about the topic to make an approval decision. Particularly when the input comes from people outside the working group. And I'd like to distinguish "everyone thinks this is fine" from "no one read the document".

And substance of the comments - be it positive or negative - is obviously important. But I find myself strangely between Stephen's opinion and Pete's opinion. Pete is of course right that blind "support" is bad and that we could easily guess wrong what the commenter meant, even when there is substance behind the support but the substance did not get explicitly articulated in the e-mail. But I can also see that there are cases where there's some context. Elwyn's mail illustrates an important example with Gen-ART reviewers. (Thank you by the way Elwyn and others for this great service. It is very much appreciated, and does improve RFC quality.) 

Anyway, we know why these reviews are being done, and what the expectation is. The expectation is that a generalist reads the spec and determines if it is understandable, tries to spot possible errors, and so on. But at the same time, the reviewers are unlikely to be in a position to say, for instance, that they have implemented the spec or plan to deploy the technology. So at the end of the day, if the spec is fine, it will only say "Ready", We will interpret that as an Internet technology generalist outside the working group having read the document and indicating that it has no obvious problems. But there is no other implication - about the reviewer needing the work for himself or herself, for instance.

In Russ' case I took the message to mean that he reviewed it as an expert on the technology. It would probably have helped if he said whether he only reviewed it for correctness or if he was also making a statement about the technology being needed in his opinion. I would have appreciated his opinion on it. That being said, it feels kind of odd to explicitly say things if you have nothing to say about the matter. "I have read the document and it seems fine, but I am not implementing it and I don't know if it addresses a real need in the Internet." We tend to assume that if the commenter is not saying these things, it is because he has no statement about his implementation, for instance.

Jari

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Wed, 12 Jun 2013 11:17:35 +0000

Hi,

I agree with Warren and disagree with Pete on this issue. 

Of course, adding more arguments, being more verbose when expressing support is very useful. However, I consider the brief comments like the one made by Russ useful - at least in determining consensus. I am actually encouraging such comments in the WGs I chair. I would like to add an argument, irrespective of who made the comment (which also counts IMO). Assuming a LC of some sorts (IETFLC, WGLC) gets only two negative comments. Would it not be useful to know that it's (2 (negative) vs. 0 (positive)) or (2 (negative) vs. 10 (positive))? Indeed, we do not count votes in the IETF, but then we also have a problem in interpreting silence, and for this purpose IMO what in this thread is called 'content-free' actually has a lot of content on this respect. 

Regards,

Dan

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Warren Kumari
> Sent: Tuesday, June 11, 2013 8:22 PM
> To: Pete Resnick
> Cc: [email-masked]
> Subject: Re: [IETF] Content-free Last Call comments
> 
> [ Not sure if this is adding to the Signal or the Noise on the Discuss
> list, but it *will* help bump up my ranking on the "Weekly posting
> summary", which I use to justify my participation to my management.
> That's what it's for, isn't it?!* ] On Jun 10, 2013, at 4:37 PM, Pete
> Resnick <[email-masked]> wrote:
> 
> > We don't vote in the IETF, so a statement of support without a reason
> is meaningless. We should not be encouraging folks to send such things,
> and having the IAB chair do so is encouraging bad behavior. Had I not
> known Russ and his particular expertise, I would have no reason to take
> it into consideration *at all*. We should not have to determine the
> > - This document is about a technology with which I have familiarity
> and I have reviewed the technical details. It's fine.
> not.
> > (Well, maybe the last one might be true.) But if instead of from "Russ
> Housely", the message was from "Foo Bar", I would have absolutely no way
> to distinguish among the above.
> 
> Actually, yes.
> 
> Russ has been participating in the IETF (and specifically in the area
> where he posted the above email) for a long time -- you know this,
> because you've also been participating.
> In *my opinion* he has shown himself to be diligent and sane. This means
> that *I* would give his comment and support great weight -- I'd *assume*
> he has read and understood the document, and is supporting it because
> #1, 2, 3 and / or 6. If Foo Bar had posted the comment, and in *my*
> opinion  Foo Bar is a total nutter, I would give his comment less, or
> possibly negative,  weight. Obviously your opinions of Russ and Foo may
> be opposite to mine -- you apply your own weighting to each comment --
> that's why we pay you the big bucks...
> 
> If Foo Bar is new enough to the IETF and cannot reasonably expect
> everyone on the IESG (or in a WG or wherever) to know and have formed an
> opinion of him, then it is *Foo Bar's* responsibility to more fully
> support his comments.
> 
> Do folk who actually *participate* actively and sanely get to assume
> that they have earned some standing and credibility? Yup. I view this as
> a feature, not a bug.
> 
> If I go to my doctor and he tells me that I simply have a cold (and not,
> like I'm convinced, the plague), I should presumably weight his comments
> higher than those of my crazy next door neighbor (who, apparently,
> routinely communicates with beings from another dimension), yes?
> 
> We want to reward merit and participation, not make the process so
> annoying that those who participate get annoyed and wander off.
> 
> If anyone *opposes* a draft, I think that it behooves them to explain
> what the issue is, regardless of who they are. This is similar to at a
> restaurant -- when the waiter asks if you are enjoying your [steak|tofu]
> it's fine to say "Yes thanks, great", but if express displeasure you
> should be ready to explain what you didn't like.
> 
> 
> >
> My opinion is that the folk on the IETF / WG chairs / anyone evaluating
> information uses their opinion on the source of that information as
> input to it's weight. This is why we have people judging consensus (see
> http://tools.ietf.org/html/draft-resnick-on-consensus-01 :-P) and not
> voting / an algorithm.
> 
> W
> 
> >
> > pr
> >
> > --

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 12 Jun 2013 13:38:20 +0200 (CEST)

On Wed, 12 Jun 2013, Jari Arkko wrote:

> But back to the topic. I, for one, would like to see responses on IETF 
> last calls. It builds my confidence that we know enough about the topic 
> to make an approval decision. Particularly when the input comes from 
> people outside the working group. And I'd like to distinguish "everyone 
> thinks this is fine" from "no one read the document".

Should people who supported the document within the WG LC generally avoid 
voicing support in the IETF LC discussion?

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Ted Lemon <[email-masked]>
Date: Wed, 12 Jun 2013 12:12:07 +0000

On Jun 12, 2013, at 4:43 AM, Dave Cridland <[email-masked]<mailto:[email-masked]>> wrote:
I suspect the closest we get to getting an idea of IETF consensus is the interest gauging at the beginning of the process, though interestingly this is only positive interest - objections to doing the work at all aren't really relevant here. The IETF consensus on the charter is handled by Apathy Is Assent rules, so claiming that this consensus call becomes the default is an interesting argument to make.

Working group charters go by much less frequently than new drafts; the burden of checking them is about as low as any comprehensive review burden in the IETF can possibly get.   So if you don't comment on a charter when it goes by, and you don't comment on the work the working group does, you have only yourself to blame.   The IETF doesn't have members, so we can't say "only 10% of IETFers like this idea, so we won't bother with it."   We don't have voting, so we can't issue a ballot and count up the yeas and nays.   If you want to limit the number of RFCs published, you need to voluntarily do the work that is required to make that happen.   If you think there's a problem with the charter, and you raise it on the IETF mailing list, there will be no shortage of discussion.   Trust me on this.

OK, so we don't have voting but we do have a quorum? How wonderful. And given that the majority of people are silent means they agree (with, mind, everything - not just that they're not reading), so any number of people are therefore "few".

Right.   We don't have members, so we can't have a quorum.   That's just not how the IETF operates.   If you prefer to operate in an SDO that operates that way, you can either change the IETF, or work in a different SDO.

I strongly feel that positive statements have value, as they allow the community to gauge the level of review and consensus, and I suspect that human nature means that we get more reviews if people get to brag about it.

If the only reason you are doing a review is so you can brag about it, that seems a bit useless to me.   But in any case, as you say, Pete made his point, you don't agree with it, it's a matter of opinion, so we're bikeshedding.

FWIW, my reason for responding to these questions on [email-masked]<mailto:[email-masked]> is that before I became an AD, I actually didn't _know_ how IETF consensus was determined, and had to do quite a bit of looking around to figure it out when I suddenly needed to know.   So I thought it was worth sharing; if in fact there are a lot of IETF participants who think this is the wrong way to handle last call, you really ought to get together and do a BOF.   That too is how the IETF thinks about things—it is only a bureaucratic quagmire if you make it one.   Should it really be _easy_ to change how the IETF evaluates consensus?

---

From: Eliot Lear <[email-masked]>
Date: Wed, 12 Jun 2013 08:21:59 -0400

On 6/11/13 3:45 PM, Pete Resnick wrote:
> more text. I don't. I want less. Save your breath. There is no reason
> to send one line of support, and it only encourages the view that
> we're voting. Details below.

And a lot of details they were.  I will point out that circumstances
probably matter.  For a working group document that has already received
some scrutiny, you've already addressed many questions, like whether the
work is important, whether it's right for the IETF to do, and whether
the solution is correct.  For AD sponsored, I would expect you'd
actually like a bit more.  Yes, you've answered some of these questions
in principle, but the rest of us haven't had our say.  In these
circumstances, I'd have to say that affirmative support with an eye
toward answering those questions (as well as nits et al) would probably
be useful for me to see as an individual.

Eliot

---

From: [email-masked]
Date: Wed, 12 Jun 2013 07:16:38 -0700 (PDT)

> human nature means that we get more reviews if people get to brag about it.

Agreed 100%.

But also consider the likely effect of calling certain comments "useless".

Discussions like this don't exactly fire me up with enthusiasm to expend 
additional time reviewing and commenting on documents not directly related to
what I do. And I rather doubt I'm alone in this.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Wed, 12 Jun 2013 07:44:50 -0700

On 6/12/2013 2:28 AM, Jari Arkko wrote:
> In Russ' case I took the message to mean that he reviewed it as an
> expert on the technology. It would probably have helped if he said
> whether he only reviewed it for correctness or if he was also making
> a statement about the technology being needed in his opinion. I would
> have appreciated his opinion on it. That being said, it feels kind of
> odd to explicitly say things if you have nothing to say about the
> matter.

The act of posting something means they feel that they do have something 
to say.  That's actually not the issue here.

The issue here is how fully the posting should explain itself.

When we write, we make assumptions about the reader.  Russ talking to 
you, Jari, permits an especially rich set of assumptions to be easy and 
correct.  You know each other.  For informal exchange, that's not only 
acceptable, it's essential.  The alternative is pathologically stilted 
and unworkably inefficient.

We often write things in the IETF -- including formal things -- as if 
they are only for folks we know.

But that's not really an appropriate model for a formal standards 
process.  When contributing to the formal parts of a standards process, 
we need to assume the writer and the reader don't know each other, 
because they usually don't.  The fact that some do is actually 
distracting.

Write for the record; the record does not know the background or intent 
or implications of the writer.  Neither will a random reader, now or 
later.  So make them explicit.

The fact that you, Jari, are still left with a guess about what Russ's 
comment fully meant should make obvious just how important it is for 
folks who post comments to provide context for their comment, where 
'context' means demonstrating an understanding of what is being 
commented on and fully explaining what our comment means.

d/

---

From: Bob Hinden <[email-masked]>
Date: Wed, 12 Jun 2013 08:33:47 -0700

Pete,

> A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free.

Please describe the context of your email.  Are you speaking for the IESG, yourself as an AD, or an individual?  Is this something the IESG has discussed?  Are you (or the IESG) creating rules for the community, that is, "thou should not send one sentence responses to IETF last calls"?  Same question when you "took that senior member of the community to task".  

I read your email that you are trying to control how people respond to IETF last calls and if they don't respond to your liking you can take them to task.  Especially the "Russ either disagrees with my complaint…" text.

This may not have been your intent, but your original email and responses can be read that way.  

Bob

---

From: SM <[email-masked]>
Date: Wed, 12 Jun 2013 09:51:38 -0700

Hi Dave,
At 01:43 12-06-2013, Dave Cridland wrote:
>I strongly feel that positive statements have 
>value, as they allow the community to gauge the 
>level of review and consensus, and I suspect 
>that human nature means that we get more reviews 
>if people get to brag about it. I suggest that 
>if more than one bit of data is required, it's 
>simply asked for. Given that the text of IETF 
>Last Call announcements is not governed by any 
>process RFC that I'm aware of (feel free to 
>correct), I suggest simply putting a set of 
>optional questions there. I note this practise 
>has served the XSF very well. I do not think 
>this needs an endless bikeshed discussion on 
>what questions; the IESG can pick what it wants to know.
>
>If, on the other hand, only objections are 
>sought, then the text (which simply asks for 
>"comments") also needs changing. And the GenArt, 
>AppsDir, and SecDir reviews should only be send 
>when they have objections to publication, of course.
>
>If you feel that the only way to make either 
>change is to form a working group and publish an 
>RFC to change something undocumented in the 
>series, then I think we're stranded in a 
>bureaucratic quagmire with no chance of escape, 
>but I'll be happy to send "comments", as requested, nonetheless.

An interesting point in the above is level of 
review and consensus.  Here's what I know: there 
is going to be apathy, there might be attempts by 
a group to support a draft or even attempts to 
silence critics, there might be someone new to 
all this who might be commenting.  If it was my 
decision to make (and it is not) I would take 
those factors and some other points into 
consideration in making a determination about a document.

As I have read your reviews I have an approximate 
idea of the type of review you would do.  I read 
the draft and I notice some obvious issues; I 
downgrade your statement of support to a tweeter 
comment.  I read the draft and I notice an 
obvious issue; I consider your statement of support as good enough.

I have read the reviews from the IAB Chair.  I 
read the draft and I notice that it is not 
well-written; I downgrade the statement of 
support of the IAB Chair to a tweeter comment.  I 
read the draft and I notice that it is good to 
go.  However, I don't find any comments about it 
except for a statement of support from the IAB 
Chair.  I don't say that there is 
consensus.  Note that this is really a personal 
decision; someone else might say that there is 
consensus.  It's not a problem unless the IESG is 
affected by the Abilene paradox.

The XSF is likely a group of people who can write 
code.  The IETF is a bunch of people who might 
discuss about content-free comments but won't 
comment about the draft. :-)  Drawing up a set of 
optional questions will generate a bottom-up 
discussion and that would be against the values 
which the IETF cherishes. :-)  There isn't 
anything preventing an Area Director or someone 
else from asking optional questions during a Last 
Call.  Optional questions from an IETF 
participant might be ignored if such activity 
would turn a Last Call into the Tribunal del Santo Oficio de la Inquisición.

If you are doing an AppsDir review, for example, and you state:

    The draft is ready for publication as a Proposed Standard.

I presume that you can personally explain the meaning of that sentence.

If you are an individual responding to a Last 
Call you can say anything you wish.  If your 
message is littered with spelling mistakes or it 
does not contain any substantive comment, it 
won't bear much weight.  If your English writing 
skills is not that good but your code is good the 
message will bear more weight.  If your message 
is to show your management that you are 
participating in the IETF the message will not bear much weight.

It's simple enough.  I would send a message if I 
believe that it can affect the decision.  It's up 
to me to know what will influence the content and fate of the draft.

Regards,
-sm

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 12 Jun 2013 19:38:01 +0100

> what I do. And I rather doubt I'm alone in this.
You are not alone. I do occasional reviews of documents not directly 
related to what I do. But I hate paperworks. And if I would be required 
to fill in 10 pages questionnaire about why I think the document is in a 
good shape, I probably will stop doing such reviews altogether.

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 12 Jun 2013 12:44:14 -0600

On 6/12/13 12:38 PM, Alexey Melnikov wrote:
>>> I strongly feel that positive statements have value, as they
>>> allow the community to gauge the level of review and consensus,
>>> and I suspect that human nature means that we get more reviews
>>> if people get to brag about it.
>> Agreed 100%.
>> 
>> Discussions like this don't exactly fire me up with enthusiasm to
>> expend additional time reviewing and commenting on documents not
>> directly related to what I do. And I rather doubt I'm alone in
>> this.
> You are not alone. I do occasional reviews of documents not
> directly related to what I do. But I hate paperworks. And if I
> would be required to fill in 10 pages questionnaire about why I
> think the document is in a good shape, I probably will stop doing
> such reviews altogether.

Much as I love Pete, IMHO this thread started because in good
curmudgeonly style he needed something to grump at. In this instance,
I suggest we just keep doing what we've always been doing.

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: Melinda Shore <[email-masked]>
Date: Wed, 12 Jun 2013 10:44:44 -0800

On 6/12/13 3:17 AM, Romascanu, Dan (Dan) wrote:
> Of course, adding more arguments, being more verbose when expressing
> support is very useful. However, I consider the brief comments like
> the one made by Russ useful - at least in determining consensus. I am
> actually encouraging such comments in the WGs I chair. I would like
> to add an argument, irrespective of who made the comment (which also
> counts IMO). Assuming a LC of some sorts (IETFLC, WGLC) gets only two
> negative comments. Would it not be useful to know that it's (2
> (negative) vs. 0 (positive)) or (2 (negative) vs. 10 (positive))?
> Indeed, we do not count votes in the IETF, but then we also have a
> problem in interpreting silence, and for this purpose IMO what in
> this thread is called 'content-free' actually has a lot of content on
> this respect.

I think Pete is correct, in that the way we do last calls
tends to look like voting, which in turn suggests to participants
that we're voting.  "Are there any objections to <whatever>?"
is, I think, the real question at hand, although I can see where
that could tend to attract cranks and chronic kvetches.  Perhaps
something along the lines of "Does anybody have reasons this
document should not progress towards publication?" might do.

Melinda

---

From: Pete Resnick <[email-masked]>
Date: Wed, 12 Jun 2013 14:12:02 -0500

After reading some of the criticisms, I wonder if folks who think 
they've been disagreeing with me are going to get to the end of this 
message and say, "Oh, if that's all he's on about, who cares?" But *I* 
of course think there is an important issue in here. Anyway, back into 
the breach. David's message seems like a good launching point.

On 6/12/13 3:43 AM, Dave Cridland wrote:

> The IETF consensus on the charter is handled by Apathy Is Assent 
> rules, so claiming that this consensus call becomes the default is an 
> interesting argument to make.

Partially tangential: We in the IETF use the phrase "consensus call" in, 
AFAICT, a unique (and IMO goofy) way. It implies that first we have a 
discussion, and then at the end of the discussion we engage in a short 
act we refer to as a "consensus call", the completion of which *creates* 
the consensus. That isn't a consensus call. That's just an anonymous and 
imprecisely counted vote. I think I've got a new section for my 
consensus document to work on: "Consensus isn't the destination; it's 
the journey." (I should get a job writing text for fortune cookies.)

But back to the topic at hand:

> community to gauge the level of review and consensus...

Reviews are good. We need to do them. They are of value. They should be 
encouraged. People should post the results of reviews. They should brag 
about them as they see fit. No argument there.

Public statements of negative reviews (with specific issues identified) 
are really important. If issues are identified that then go unanswered, 
that breaks consensus.

Public statements (even short ones) of positive reviews have value in 
that we want to know that reviews have taken place. That's useful. Mind 
you, such statements don't help you "gauge consensus": Sure, you *might* 
conclude that one more person is part of the consensus, but of course 
the number of people doesn't matter to the consensus, and the consensus 
hasn't moved at all -- you haven't resolved a new open issue. (There is 
a separate question about what to do when documents haven't gotten 
serious review prior to IETF Last Call and what that means for 
consensus. More on that in a moment.)

So, none of the above things get me torqued. Reviews: Good. Public 
statements of reviews: Good. Issues to be resolved: Good.

Here's what gets me torqued, that I think is useless, and starts people 
thinking that they are voting:

"I support publication of this document."
"I do not support publication of this document."

Even prefacing those with "I reviewed the document" doesn't help. (Of 
course the "I reviewed the document" part itself is potentially useful 
if the person bothers to tell us the result of the review, like "it's 
technically sound" or "it solves a problem that needs solving" or "it's 
technically crappy *and* here are the reasons why".) But the "I do/don't 
support this document" part is what gets my goat. It contains the 
apparent implication that someone is counting noses and that "support" 
matters. It shouldn't. What matters is that we as a community have 
worked this thing out, that it solves a problem that we have identified 
is real, and that nobody in the community has found serious fault in it.

It's not the shortness of the statement that's the problem (though 
sometimes, especially with negative reviews, it is a problem); it's 
treating the call for consensus as a vote that bugs me. It's not useful 
to simply "support" a document.

> If, on the other hand, only objections are sought, then the text 
> (which simply asks for "comments") also needs changing. And the 
> GenArt, AppsDir, and SecDir reviews should only be send when they have 
> objections to publication, of course.

Nope, I'm good with all that. In fact, the GenArt review says exactly 
what it should: It provides "generalist reviews for the General Area 
director (currently the IETF Chair), providing an additional set of eyes 
for documents as they are being considered for publication." It's not 
looking to "gauge consensus". It's looking for a review.

That said, I do believe that objections are *way* more important than 
comments saying, "It's fine". Once something has gone through a WG, the 
presumption is "It's fine." Additional eyes are good, but they're good 
precisely because they might find problems.

That brings us to one last thing which I mentioned above: What if 
documents *aren't* getting serious review prior to IETF Last Call? This 
goes back to our "tail heavy" discussion. The IETF used to be very good 
about getting lots of review (sometimes even cross-area review) *during* 
the consensus building process in the WG. We now have many instances 
where WGs are not really working toward consensus to solve a problem; 
they're doing reviews of a single person's idea of how to solve a 
problem. Even worse, we are seeing many individual submissions that are 
not the product of consensus building at all, and when that 4-week Last 
Call kicks in, we are at best getting reviews, but not building 
consensus around the need for the document. Solving that problem by 
tail-loading more reviews does not address the real issue.

pr

---

From: Pete Resnick <[email-masked]>
Date: Wed, 12 Jun 2013 14:17:46 -0500

On 6/12/13 10:33 AM, Bob Hinden wrote:
> Please describe the context of your email.  Are you speaking for the IESG, yourself as an AD, or an individual?

Oh, crap. And given that I'm usually the one giving people a hard time 
about *this* issue, I feel especially bad about not being clearer.

This is Pete, hatless individual talking. There are (as you've seen) 
IESG people, past and present who disagree with me. Nothing I've said is 
anything we've come to an IESG consensus around. Insofar as I think that 
only saying "I support this document" is a bogus thing to say, I am 
revealing something about how I behave as an AD in the face of such 
statements, and my further explanations that I value reviews and take 
them seriously also indicates my views as an AD, but that doesn't 
indicate some policy that I'm instituting with WGs for which I am 
responsible.

I am terribly sorry about not making this clear up front. My sincere 
apologies. I hope nobody took this as a other than my commentary.

Sheepishly,
pr

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 12 Jun 2013 13:31:44 -0600

On 6/12/13 12:44 PM, Peter Saint-Andre wrote:
>>>> allow the community to gauge the level of review and
>>>> consensus, and I suspect that human nature means that we get
>>>> more reviews if people get to brag about it.
>>> Agreed 100%.
>>> 
>>> Discussions like this don't exactly fire me up with enthusiasm
>>> to expend additional time reviewing and commenting on documents
>>> not directly related to what I do. And I rather doubt I'm alone
>>> in this.
>> think the document is in a good shape, I probably will stop
>> doing such reviews altogether.
> 
> curmudgeonly style he needed something to grump at. In this
> instance, I suggest we just keep doing what we've always been
> doing.

To clarify:

I reject kings, presidents, and voting. But I do believe in the delete
key.

If you think that these "+1" messages are useless, don't factor them
into your thinking about whether we have consensus.

If you think that these "+1" messages are actively harmful (e.g., by
leading to groupthink or poisoning our very understanding of what it
means for us to reach consensus), then proceed as Pete is proceeding.

I think these messages are useless, not harmful. But perhaps I have
more confidence in the inherent skepticism of your average IETF
participant than Pete does...

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: Ted Lemon <[email-masked]>
Date: Wed, 12 Jun 2013 19:42:19 +0000

> participant than Pete does...

FWIW, until I read Pete's document on consensus, I thought that +1 statements were part of the consensus process.   This was not a strongly held opinion—it was just my understanding of how consensus operated, from having watched other working group chairs run their working groups.   I think the point Pete is making is very important, because the consensus process Pete describes is more in keeping with how I think the IETF ought to operate than the process in which +1 counts for something.

(BTW, in case it wasn't obvious, I've been engaging in this discussion with my AD and working group chair experience in the back of my mind, but my AD hat off.)

---

From: Ted Hardie <[email-masked]>
Date: Wed, 12 Jun 2013 13:28:34 -0700

> document should not progress towards publication?" might do.
So, back in the days when we were talking about the move from a 3-stage
process to a 2-stage process, I wrote draft-hardie-advance-mechanics.  That
described our then-extent process  this way:

   In practice, IETF document processing has evolved to a model which
   can be described as "objection based processing".  A document put
   forward by an author team advances from the relevant working group to
   IETF consideration after all objections from within the working group
   have been considered.  The IETF Last Call is, in practice, a way for
   the larger community to object to a document or elements within it.
   IESG document processing is, fundamentally, a process by which a
   sponsor resolves any objections raised by other area directors;
   though the term used is "discuss" and some delegation occurs (to
   document shepherds or review teams), the basic model is clear.
I don't think that true consensus is achieved by the lack of
objection, but the community
seems to have accepted it as "rough consensus".

Could we do better?  Absolutely.  We could, for example,  formalize
the third-party reviews
by other areas such that a demonstration that proposals were
acceptable in a broader Internet
context was fundamental (either at charter time, at working group
draft acceptance time,
at proposed  publication, periodically for advancement, or at all of
the above). That would
get voices known to have analysed it heard, and give us a lot more
confidence that the
proposal wasn't narrow in its examination of the impact of its work.

But this is a volunteer organization, and the practical truth is that
we don't have the volunteer
energy to do that or the funds to pay for it (or trust in anyone we'd
pay, as another issue).
So, bluntly, we should take what we get and use it as we can.  If Pete
can't figure out what
to make of a +1 by a 5 term IESG member and sitting IAB Chair, then
he'll have to evaluate
the document without that input.  He'll have to evaluate it on its
merits anyway, so it's
no big deal.  But  anything that discourages review, however, cursory,
is not really a good
thing.  Perhaps a note to the reviewer like:

"Thanks for taking the time.  It's good to know I can reach out to you
for more data, should
I be on the fence on this one".

(I'm thinking on heavy stock Crane paper, maybe in a cream, nicely
lettered.  Or, you know, as
a jotted email or IM.)

Even sent to a stranger, that would look nice and remind people that
we're not voting.

Just some thoughts on the topic,

Ted

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 13 Jun 2013 08:37:39 +1200

'scuse front posting, but I'm going to outrageously summarise
Pete's point as "I want substance in all Last Call comments", or
alternatively "I will ignore +1 just as I will ignore -1".

That isn't unreasonable, but personally I would interpret "I've
read it and I think it's good work" as substantive, especially
if it comes from a known expert. YMMV

Regards
   Brian

---

From: Pete Resnick <[email-masked]>
Date: Wed, 12 Jun 2013 15:42:39 -0500

> alternatively "I will ignore +1 just as I will ignore -1".

Maybe not outrageous, but certainly wrong because...

> ...personally I would interpret "I've
> read it and I think it's good work" as substantive

I agree with that.

Confused now? Go back and look at my message of an hour and a half ago 
and see if you can tell why I agree. :-)

As I've said to other people, I like straw men. They make fine 
neighbors. They're quiet, and they don't smoke.

But if you're going to summarize (or summarise), best to do so accurately.

pr

---

From: <[email-masked]>
Date: Wed, 12 Jun 2013 22:56:05 +0100

Are the IESG people who disagree with you speaking for the IESG, or for themselves?

I've noticed a tendency for IESG weight to be (inadvertently?) thrown around, lending more weight to comments than would otherwise be given.

Lloyd Wood

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Pete Resnick [[email-masked]]
Sent: 12 June 2013 20:17
To: Bob Hinden
Cc: [email-masked]
Subject: Re: Content-free Last Call comments

Sheepishly,
pr

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 12 Jun 2013 23:10:47 +0100

> Are the IESG people who disagree with you speaking for the IESG, or for themselves?

That's really not clear already? In any case, I was disagreeing
with Pete as an individual since he was wrong regardless of hats.
He and I do that all the time and both find it amusing:-)

> I've noticed a tendency for IESG weight to be (inadvertently?) thrown around, lending more weight to comments than would otherwise be given.

I've not noticed that. Maybe I perceive things differently of
course, but people ought know not to take opinions any more or
less seriously just because someone's on some I* thing at the
moment.

S.

---

From: Pete Resnick <[email-masked]>
Date: Wed, 12 Jun 2013 19:54:20 -0500

> He and I do that all the time and both find it amusing:-)

Indeed, Stephen's wrongness is clearly a personal wrongness and not 
because he is on the IESG. ;-)

Seriously though, Bob and I chatted about this a bit offline, and I see 
how I could be interpreted as saying something IESG-ish, and that was 
absolutely not my intention.

Importantly, I also have no interest in turning anything I've said here 
into some procedure or policy, even if I convince Stephen and others of 
the error of their ways. :-) This conversation is truly about a set of 
principles, that (in my very individual opinion that I hope others will 
share) we should be careful to not fall into the pattern of treating 
Last Calls as simple votes on a topic, that we should not use language 
that leads us (or more importantly newcomers) to believe that we are 
doing so, and that we should be more concerned about why we think a 
document is good or bad rather than the number (or reputation) of people 
who claim that it is. Sometimes we fall into the habit of assuming, "I 
know what Pete is talking when he says he supports a document, so 
everybody else will too." I think that's a bad habit. But this is 
(hopefully) something for the Tao, not something for RFC 2026.

> less seriously just because someone's on some I* thing at the

Much as I would like what Stephen says to be true, I think Lloyd's 
probably right: People give more weight to opinions coming from people 
with dots on their name badges. I try to be careful because of that, but 
I wasn't careful enough this time.

pr

---

From: David Morris <[email-masked]>
Date: Wed, 12 Jun 2013 18:06:35 -0700 (PDT)

On Wed, 12 Jun 2013, Pete Resnick wrote:

> Much as I would like what Stephen says to be true, I think Lloyd's probably
> right: People give more weight to opinions coming from people with dots on
> their name badges. 

One has to be able to see the dots for it to possibly matter. Out here at
the end of the email connection, I have almost no clue who might have a
dot. I give weight to folks I've observed over time to provide thoughful
comments. Many have been ADs, etc. at some point. Unless a message
explicitly says "I'm writing in role as an AD", it doesn't matter. 

The exception would be the WG chairs and ADs associated with WGs I follow.

I'm well aware that any review comment I'd make would need to logically
explain my position because I'm relatively unknown.

Dave Morris

---

From: joel jaeggli <[email-masked]>
Date: Thu, 13 Jun 2013 10:01:48 +0200

> FWIW, until I read Pete's document on consensus, I thought that +1 statements were part of the consensus process.   This was not a strongly held opinion—it was just my understanding of how consensu…
+1 / -1

are conventions that crept in from other standards bodies, they don't 
have any particular place or meaning here, apart from what you can 
literally interpret them as e.g. the equivalent of hand raising in 
agreement or disagreement.

---

From: t.p. <[email-masked]>
Date: Fri, 14 Jun 2013 09:28:07 +0100

----- Original Message -----
From: "Pete Resnick" <[email-masked]>
To: "Stephen Farrell" <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, June 11, 2013 8:45 PM
Subject: Re: Content-free Last Call comments

<tp>

Seems to me that there are more points of view on this thread than there
are members of the IESG, in which case, to help followers of this list,
perhaps they, when instigating a Last Call, should do as some WG Chairs
do when instigating WG Last Call, spell out that expressions of support
from those that have read it are welcome - or not, as the case may be.
Thus the AD might add that any e-mail containing "+1" in the body will
be filtered into a spam trap and discarded without being read.

Tom Petch

</tp>

Let's separate out a few issues. There's what the message means itself,
there's its usefulness and how much weight it should be given, and then
there's what's problematic about it. I think you've conflated a few
things.

pr