Subject: To "lose the argument in the WG"

From: "Pete Resnick" <[email-masked]>
Date: Mon, 13 Feb 2017 22:50:57 -0600

Twice in as many days I've seen folks make the argument that a WG 
participant ought not bring up an issue during IETF Last Call that was 
already discussed in the WG because that participant "lost the argument" 
in the WG. I hate the construction, and I think even making the claim 
harms the process.

In one case, it appears that the chair judged that the WG couldn't come 
to consensus on a definitive statement and therefore concluded that a 
compromise statement was appropriate. It is quite appropriate for WG 
participants to say during Last Call, "We think the chair made an error. 
There remains a technical failing in this document that is not 
appropriately addressed by the compromise language. We tried to convince 
the chair of this, to no avail, and hope that others in the community 
review this particular point and maybe try to make the case clearer than 
we were able to." That's going to require some rehashing, though 
hopefully the chair and/or AD can fairly summarize the arguments and why 
they came to the conclusion they did, which should shorten the 
discussion. But the points need to be brought out, the in-the-rough WG 
participants should be able to identify where the perceived error was 
made, and the community can put in its two cents. You can't say, at this 
point, that anyone "lost the argument", and certainly not that they 
ought to be quiet.

(In that particular case, I've also seen people say, "I give up; I'm not 
discussing it anymore". That's a pain, since the issue hasn't gone away, 
even if the proponent has, so the chair or AD is left to make sure it 
gets dealt with without help. We really need to stop that.)

In the other case, my understanding is that the chair and/or AD judged 
that a particular set of functionality was outside of the scope of the 
document and/or chartered-work in question. The WG participant who felt 
that the functionality should be in-scope and is perfectly within bounds 
to make the case that the functionality is somehow necessary during Last 
Call. The chair and/or AD should summarize why they judged the 
functionality to be out of scope. Others in the community might want to 
take up the argument and explain why it should be added. But again, 
saying that the participant cannot make an argument that was "not won" 
in the WG is unreasonable.

Do note that in both cases, I think it's only fair that the person 
bringing up the issue clearly states what's going on. For instance, in 
the second case the person should really have said, "This was judged 
out-of-scope by the WG/chair, but I think it ought not be out-of-scope 
because of X, Y, and Z". Simply saying, "Please make this change to the 
document" as if it were never discussed before is not helpful to the 
rest of the list during Last Call.

In any event, I think it's vital that the chair and/or AD actively 
manage the Last Call discussion when these sorts of things happen. They 
are the judges of consensus, and they are the ones that are going to 
have to make the call in the end, so having them explain their thinking 
is vital.

End of rant.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: Dave Crocker <[email-masked]>
Date: Mon, 13 Feb 2017 21:07:17 -0800

On 2/13/2017 8:50 PM, Pete Resnick wrote:
> The WG participant who felt that the functionality should be in-scope
> and is perfectly within bounds to make the case that the functionality
> is somehow necessary during Last Call. The chair and/or AD should
> summarize why they judged the functionality to be out of scope. Others
> in the community might want to take up the argument and explain why it
> should be added.

This sounds like reasonable theory, but is actually rather destructive 
practice.

It takes the position that Last Call is acceptable to use as a form of 
appeals process, where folk who have been working on the topic for an 
extended time have to defend their choices to a collection of other folk 
who are new to the topic and are, therefore, making snap judgements.

While, yes, there will be times that the new folk see something new or 
better, that's not the usual occurrence.  The usual occurrence is that 
folk who are experienced with the topic and are tired from the extended 
effort have to rehash their work and defend it to folk who have not done 
their homework.

Either working groups are where the work really does get done or they 
aren't.  The burden of having to worry about and deal with a larger, 
less-involved community being frankly encouraged to second-guess the 
folk who have actual skin in the game, is an example of what makes the 
formality of IETF process onerous.

Last Call should not require a working group to be subject to random 
demands to defend itself.  It should be for independent reviews that see 
something the working group missed.  Missed is different from "we had a 
choice and we made it".

If a fresh reviewer really does do their homework and really does 
present a good case for making a different decision, that's fine.  But 
it also is quite different than supporting the re-hashing exercise that 
occurrs when an existing wg participant expresses dissatisfaction with a 
decision made during normal wg processes.

d/

ps. Pete's other point was about a claim that an issue didn't really get 
settled and needs further review.  That's quite a different case.  Maybe 
it's worthy for LC discussion.  Maybe it isn't.  Dunno.

pps. There's at least one case where I chose to attempt to use LC as a 
kind of appeals process, since I deemed the wg process to have been 
significantly flawed, including the cognizant AD.  Like all good rules, 
there need to be some exceptions thoughtfully permitted, though of 
course my effort in this example of an exception bore no fruit...

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Ted Lemon <[email-masked]>
Date: Tue, 14 Feb 2017 00:19:12 -0500

On Feb 13, 2017, at 11:50 PM, Pete Resnick <[email-masked]> wrote:
> I think it's only fair that the person bringing up the issue clearly states what's going on. For instance, in the second case the person should really have said, "This was judged out-of-scope by the WG/chair, but I think it ought not be out-of-scope because of X, Y, and Z". Simply saying, "Please make this change to the document" as if it were never discussed before is not helpful to the rest of the list during Last Call.

Since I was one of the people who made that point, I should say that "the chair made a mistake" is a different statement than "the working group decided, and I was part of that conversation, and I don't agree, so let's discuss it again."

---

From: "Pete Resnick" <[email-masked]>
Date: Mon, 13 Feb 2017 23:25:00 -0600

On 13 Feb 2017, at 23:07, Dave Crocker wrote:

> extended time have to defend their choices to a collection of other 
> folk who are new to the topic and are, therefore, making snap 
> judgements.
>
> folk who are experienced with the topic and are tired from the 
> extended effort have to rehash their work and defend it to folk who 
> have not done their homework.
>
> demands to defend itself.  It should be for independent reviews that 
> see something the working group missed.  Missed is different from "we 
> had a choice and we made it".
>
> it also is quite different than supporting the re-hashing exercise 
> that occurrs when an existing wg participant expresses dissatisfaction 
> with a decision made during normal wg processes.

Would that we disagreed.

Nowhere did I say that Last Call should be an opportunity to simply 
rehash, without having done homework. Indeed, you will note where I 
said:

>> bringing up the issue clearly states what's going on. For instance, 
>> in the second case the person should really have said, "This was 
>> judged out-of-scope by the WG/chair, but I think it ought not be 
>> out-of-scope because of X, Y, and Z". Simply saying, "Please make 
>> this change to the document" as if it were never discussed before is 
>> not helpful to the rest of the list during Last Call.

For a WG participant to just throw out the same argument made earlier 
without explanation *and* with the expectation that the WG ought be 
required to relitigate the whole discussion is *not* reasonable. The 
case needs to be made that the WG (or the chair in judging) did in fact 
miss something when they came to rough consensus. The chair, not the 
entire WG, should explain why the conclusion was arrived at. (That 
should really be written up in the shepherd report, if we're being 
diligent.) Others can choose to ask questions or push back on either 
position if they think something serious has been missed. But to 
dismiss, out of hand, any such claim with "too late, already decided" is 
also unreasonable.

Homework is required. Pointing out the failure is required. Avoiding 
simply restating a preference is required. I'm just objecting to the 
blanket statement, "If the WG came to consensus, even roughly, the 
discussion is over."

(As I said earlier, in the case of something being declared 
out-of-scope, I'd expect the person bringing up the point to explain why 
being in-scope was essential. To fail to do so is bogus.)

> ps. Pete's other point was about a claim that an issue didn't really 
> get settled and needs further review.  That's quite a different case.  
> Maybe it's worthy for LC discussion.  Maybe it isn't.  Dunno.
>
> significantly flawed, including the cognizant AD.  Like all good 
> rules, there need to be some exceptions thoughtfully permitted, though 
> of course my effort in this example of an exception bore no fruit...

I also agree that this should be fairly unusual. Again, what got to me 
was the automatic rejection of such an argument.

pr

---

From: Mark Andrews <[email-masked]>
Date: Tue, 14 Feb 2017 17:01:56 +1100

In message <[email-masked]>, Dave Crocker wr
> effort have to rehash their work and defend it to folk who have not done 

And workgroups get it wrong.

I gave up trying to convince behave that the DNS64 DNSSEC processing
was insane.  Validating clients send both <DO=1,CD=1> and <DO=1,CD=0>
queries.  You can't expect synthesised AAAA response to be accepted
with either set of flags.  They wanted a flag that said that the
client is validating and that DOES NOT EXIST in DNSSEC.  DO=1 say
a client MAY be validating.  DO=0 indicates that it is NOT validating.
CD is independent of whether the client is validating or not.  Lots
of wishful thinking when that section was written.

I also gave up trying to get 5.9. "Always Set the CD Bit on Queries"
removed from the draft for RFC 6840.  Named totally ignores that
section because it just plain bad advice.  Recursive servers need
to honour CD as originally specified or there are failure modes
that are not recoverable when some of the authorative servers are
broken or the recursive server is under attack unless CD is
controllable.

You need to be able to control validation in the entire chain of
recursive servers for DNSSEC to work.  There are a number of different
failure senarios in DNSSEC.  Some require CD=1 to recover from (if
you initially set CD=0), some require CD=0 to recover from (if you
initially sent CD=1).  In both cases CD needs to passed down the
entire chain for the recovery to work.

Should I have raise these again at IETF last call?

Mark

-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Yoav Nir <[email-masked]>
Date: Tue, 14 Feb 2017 09:28:53 +0200

On 14 Feb 2017, at 8:01, Mark Andrews <[email-masked]> wrote:
> was insane.

<snip />

> Should I have raise these again at IETF last call?

If at IETF LC you said something along the lines of “The specification does this, and I think it’s insane because XXX. It was raised in the WG [1] but was rejected [2],[3].  I still think it’s insane and that the spec should not go forward as it is.”

I think that would be fine, but anyone jumping to agree with you would be properly asked why they didn’t participate in the WG. In any case, it could be useful input for the IESG who are the ultimate judges of IETF consensus.

Yoav

---

From: John C Klensin <[email-masked]>
Date: Tue, 14 Feb 2017 02:43:09 -0500

Pete,

I'm probably going to regret being drawn into this, but I think
you (and Dave) are missing two additional cases, ones that
complicate your analysis considerably even if  they don't change
your key conclusion.

As the Internet and IETF evolve, I see increasing frequency of
very homogenous WGs, ones whose active participants consist
entirely, or almost entirely, of people who represent one
perspective and who are narrowly focused on that WG's topic
area.  I addition to all of the usual possibilities, including
the ones you mentioned, such WGs run multiple risks.  To keep
this from becoming over-long, the one that probably represents
the case we see most often goes like this...

The WG reaches fairly strong consensus on a particular set of
issues, with only one or two people in the minority, taking
positions the WG has decided are not interesting or not
relevant.  In that case, unless we require omniscience as well
as fairness of WG Chairs, there is no error: the WG has reached
rough consensus on a topic and those one or two participants are
in the rough _within_ the WG.   For that case, if IETF Last Call
and the idea of cross-area review are to be more than rituals to
support our mythology, it is absolutely necessary that someone
who participated in the WG be able to come to IETF Last Call and
say "look, the perspective of the WG was too narrow and, as a
consequence, these issues that affect other things didn't get a
reasonable hearing and the IETF as a whole should really
reexamine them independent of the conclusions the WG reached
(and/or the community should reexamine whether the WG was, in
retrospect, properly chartered, constituted, and composed and,
if not, whether its consensus and request for document
publication should be taken seriously,

FWIW, I believe that "strong consensus, but still wrong" was one
of the cases you addressed with RFC 7282.

I think that "too homogeneous a WG and too narrow a perspective"
argument, if plausibly explained and justified, has to be taken
serious even if there are no (other) new arguments or additional
"homework".    I also think it is a particularly difficult type
of problem because, if the consensus recommendation from such a
WG to publish a document is accepted, it reflects badly on the
AD who authorized the Last Call as well as on the WG and its
leadership.  While it doesn't make "I lost the argument in the
WG" is not helpful as a means of ending the conversation, "I
lost the argument in the WG for what I believe were the
following reasons ... and the IETF should review those issues"
may be exactly what we need to hear as part of document review.

That argument is very different from either "the WG CHair made
an error" or "I was on the losing side and I want to keep
whining about it and rehash the argument".

Whether IETF Last Call is the right way to deal what that type
of issue is a separate question.   You and others have heard me
say from time to time that we don't have nearly enough appeals.
I think ADs should be monitoring WGs for sufficient technical
diversity and range of perspectives and that someone
participating in a WG who believes that the profile of
participants in the WG is too narrow to encourage or allow
arguing against WG conclusions on the basis of broader Internet
considerations (whether decisions about WG scope get involved or
not) should be encouraged to discuss that issue with the
relevant AD and, if necessary carry the appeals process further,
rather than quibbling about specific technical issues on Last
Call.  However, success in such an appeal would probably require
shutting down a WG that has an active mailing list and meetings
and is producing results.  Our track record for such shutdowns
is fairly close to non-existent.

That same argument for appeals applies when people who are not
part of the homogeneous majority are treated dismissively or
otherwise discouraged from continuing to participate.  We ought
to be able to deal with those situations as instances of
bullying, but if that has ever worked successfully, I'm not
aware of it -- maybe you know of examples that I don't.

Where I think we agree is that, while there should be a
presumption that a WG that produces a draft and claims consensus
for it has done its work and gotten it right, that presumption
should not be very strong, should be easy to challenge, and that
the position of someone who disagreed with the WG decision
should not be dismissed on that basis.

best,
    john

---

From: Stewart Bryant <[email-masked]>
Date: Tue, 14 Feb 2017 08:40:50 +0000

> should not be dismissed on that basis.

Yes, it is the prospect of a complete re-run of the argument at IETF LC
that keeps WGs honest.

We have to accept the real possibility that fresh eyes on the text
or fresh ears listening to the contra-arguments may consider the WG
to have made an incorrect decision. That is the whole purpose of
IETF LC, and we should do nothing to constrain this important
element of our quality control process.

- Stewart

---

From: Ted Lemon <[email-masked]>
Date: Tue, 14 Feb 2017 10:12:51 -0500

> I think that would be fine, but anyone jumping to agree with you would be properly asked why they didn’t participate in the WG. In any case, it could be useful input for the IESG who are the ultima…

Er, no.  The most important point of last call is to make sure that people who did not have time to participate in the last call in the working group, but who have relevant knowledge, get a chance to weigh in.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 14 Feb 2017 07:32:36 -0800

> Yes, it is the prospect of a complete re-run of the argument at IETF LC

In theory, perhaps.  In practice, no it doesn't.

What it does do is to help make the IETF process onerous, and therefore 
aid in making the IETF a place to avoid when possible.

Having folk spend months (or years) working on a detailed specification 
and then have to go through a detailed defense at the end is 
disrespectful of their effort.  It is also impractical, because it 
essentially requires re-creating the context for making various 
decisions made along the way, months (or years) later, to folk who have 
no skin in the game.

For those rare cases in which there really is a problem that needs 
fixing, forcing a wg to go through this random defense process mostly 
speaks to failures in oversight and review that should be happening 
along the way, not at the end of the process.

And, again, note that it almost never works.  While it creates 
additional work and pain for those who are tired from the lengthy 
process, it rarely (if ever) produces serious benefit.

For the folk who think this is an essential bit of IETF quality control, 
please produce documentation of the times this defense is required and 
compare when it has produced meaningful change and when it hasn't.  Just 
coming up with stray examples of benefits ignores the larger number of 
cases where it incurs costs without benefits.[*]

To the extent that a working group does need course-correction, then 
let's look for ways to do that that are effective and efficient and 
much, much earlier.

d/

[*] Please remember that the trigger for this sub-thread is a 
sour-grapes effort by an existing wg participant to force review of a 
(potentially much) earlier wg decision they didn't like.  A separate 
case is of fresh, non-wg eyes that might see something none of the 
older, more-tired eyes of wg participants caught.  The former case 
really is sour grapes.  The latter case could be naivete or it could be 
insight.  Mostly of the time it is the former.  That ought to mitigate 
the certitude of the fresh eyes, but it usually doesn't.

-- 

   Dave Crocker

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 14 Feb 2017 11:29:34 -0500

I just posted a note on the IETF list in one of the two discussions Pete 
is refering to.

Yes, I watched the discussion in the WG.
It took me a while to realize that the WG had really ended up in the 
state it did.  A state I consider a failure in terms of the document.
So it seems to me that I need to say so during the IETF LC.

Yours,
Joel

On 2/14/17 12:25 AM, Pete Resnick wrote:
>     This sounds like reasonable theory, but is actually rather
>     destructive practice.
>
>     It takes the position that Last Call is acceptable to use as a form
>     of appeals process, where folk who have been working on the topic
>     for an extended time have to defend their choices to a collection of
>     other folk who are new to the topic and are, therefore, making snap
>     judgements.
>
>     While, yes, there will be times that the new folk see something new
>     or better, that's not the usual occurrence. The usual occurrence is
>     that folk who are experienced with the topic and are tired from the
>     Either working groups are where the work really does get done or
>     they aren't. The burden of having to worry about and deal with a
>     larger, less-involved community being frankly encouraged to
>     second-guess the folk who have actual skin in the game, is an
>     example of what makes the formality of IETF process onerous.
>
>     present a good case for making a different decision, that's fine.
>     But it also is quite different than supporting the re-hashing
>     exercise that occurrs when an existing wg participant expresses
>     dissatisfaction with a decision made during normal wg processes.
>
> Would that we disagreed.
>
> rehash, without having done homework. Indeed, you will note where I said:
>
>         Do note that in both cases, I think it's only fair that the
>         person bringing up the issue clearly states what's going on. For
>         instance, in the second case the person should really have said,
>         "This was judged out-of-scope by the WG/chair, but I think it
>         ought not be out-of-scope because of X, Y, and Z". Simply
>         saying, "Please make this change to the document" as if it were
>         never discussed before is not helpful to the rest of the list
>         during Last Call.
>
> without explanation /and/ with the expectation that the WG ought be
> required to relitigate the whole discussion is /not/ reasonable. The
>     pps. There's at least one case where I chose to attempt to use LC as
>     a kind of appeals process, since I deemed the wg process to have
>     been significantly flawed, including the cognizant AD. Like all good
>     rules, there need to be some exceptions thoughtfully permitted,
>     though of course my effort in this example of an exception bore no
>     fruit...
>
> Pete Resnick http://www.qualcomm.com/~presnick/
> <http://www.qualcomm.com/%7Epresnick/>

---

From: Dave Crocker <[email-masked]>
Date: Tue, 14 Feb 2017 09:05:07 -0800

> Er, no.  The most important point of last call is to make sure that people who did not have time to participate in the last call in the working group, but who have relevant knowledge, get a chance…

Unfortunately:

    Last Call Guidance to the Community
    https://www.ietf.org/iesg/statement/last-call-guidance.html

does not provide useful guidance about the purpose and substance of 
doing an IETF Last Call.

The IETF's formal process document on standards:

    The Internet Standards Process -- Revision 3
    https://www.ietf.org/rfc/rfc2026.txt

says nothing about Last Call.

So the formal IETF specification for the role of Last Call is in:

      IETF Working Group Guidelines and Procedures
      https://tools.ietf.org/html/rfc2418

>    This Last Call will announce the intention of
>    the IESG to consider the document, and it will solicit final comments
>    from the IETF within a period of two weeks.  It is important to note
>    that a Last-Call is intended as a brief, final check with the
>    Internet community, to make sure that no important concerns have been
>    missed or misunderstood. The Last-Call should not serve as a more
>    general, in-depth review.

(I should note that that text dates back to the original version of the 
document that Erik Huizer and I wrote, in RFC 1871, in 1994.)

What folks are doing is spontaneously changing the role of this step, 
ignoring the considerable costs and detriments, while relying on a 
theory of benefit that is very, very, very rarely actually demonstrated.

To Ted's point, indulging folk who 'did not have time' to participate 
earlier is frankly abusive of all those who did.

RFC 2416 (and the original, RFC 1871) has explicit text about revisiting 
a topic:

>    It is occasionally appropriate to revisit a topic, to re-evaluate
>    alternatives or to improve the group's understanding of a relevant
>    decision.  However, unnecessary repeated discussions on issues can be
>    avoided if the Chair makes sure that the main arguments in the
>    discussion (and the outcome) are summarized and archived after a
>    discussion has come to conclusion. It is also good practice to note
>    important decisions/consensus reached by email in the minutes of the
>    next 'live' session, and to summarize briefly the decision-making
>    history in the final documents the WG produces.
>
>    To facilitate making forward progress, a Working Group Chair may wish
>    to decide to reject or defer the input from a member, based upon the
>    following criteria:
>
>    Old
>    The input pertains to a topic that already has been resolved and is
>    redundant with information previously available;
>
>    Minor
>    The input is new and pertains to a topic that has already been
>    resolved, but it is felt to be of minor import to the existing
>    decision;
>
>    Timing
>    The input pertains to a topic that the working group has not yet
>    opened for discussion; or
>
>    Scope
>    The input is outside of the scope of the working group charter.

d/

ps. RFC
-- 

   Dave Crocker

---

From: "John Levine" <[email-masked]>
Date: 14 Feb 2017 22:01:32 -0000

>effort have to rehash their work and defend it to folk who have not done 

Sometimes that's true, but sometimes the situation arises to which
John Klensin alluded: a WG does something that crosses topics, and
what they decide makes sense from the viewpoint of topic A but not
from topic B.  I've certainly seen that happen a few times in the
recent past.

R's,
John

---

From: "Pete Resnick" <[email-masked]>
Date: Tue, 14 Feb 2017 15:01:31 -0800

> says nothing about Last Call.

Well, of course, except for defining it in section 14:

    Last-Call - A public comment period used to gage the level of
       consensus about the reasonableness of a proposed standards 
action.

>>    general, in-depth review.

Indeed. If Last Call results in an issue that requires a more general, 
in-depth review, I think the presumption of that text is that it's time 
to send it back to the WG.

> theory of benefit that is very, very, very rarely actually 

I think you've conflated two things: Yes, Last Call should serve as a 
brief final check, not to have a long re-hashed technical discussion. 
But that doesn't mean the outcome of that brief final check is always 
brief and final: When Last Call identifies an issue that was missed or 
mis-evaluated by the WG (perhaps due to JCK's observation about WGs that 
remain homogeneous and insulated), considerable cost and detriment 
should be expected, and is probably appropriate. We have all seen 
efforts that have gone completely off the rails, perhaps due to bad 
management choices, and it is a sign of pathology if that's only noticed 
at Last Call time. Yes, it should probably have been handled during the 
WG process or by interim appeals, but if Last Call is where it's finally 
dealt with, deal with it we must.

Someone described it in private email to me this way: Last Call 
shouldn't be used to argue for a choice where the only consequence is 
"flavor"; it should only be used when the WG has made a choice this is 
"jumping-off-the-cliff". I agree with that in general (allowing that 
people's judgement of how close to the cliff one is might differ), and 
it's the responsibility of everyone to keep their comments to things in 
the latter category.

My main complaint is that people are treating this as a game, with 
points that are "won" or "lost". Telling people that they "lost" or that 
"it's just sour grapes" is obnoxious and has no place in the discussion. 
If someone is abusing the process by bringing up previously 
well-considered topics, of course the person running the Last Call has 
to shut that down. I can even tolerate the occasional random participant 
saying, "Pete, you know this issue was discussed at length, and you 
didn't even bother to fairly say why you think the WG came to the wrong 
conclusion, so this is really just rehashing." Challenging folks to come 
up with why this is not just a "flavor preference" is reasonable: Where 
is the cliff? But just dismissing them with, "Sorry, you were ruled a 
loser" is not how this is supposed to go.

(Speaking for myself: It appears to me that the 2460bis discussion did 
identify a point that is appropriate for the larger community to weigh 
in on. On the other hand, I have yet to be convinced that the major 
point brought up on the SLIM is other than a "flavor preference". In 
neither case should anyone be yelling "sour grapes". That's impugning 
motives and simply ad hominem.)

> earlier is frankly abusive of all those who did.

Yeah, I don't find Ted's point at all convincing either. On the other 
hand, if the WG didn't seek out required expertise and someone does 
notice a showstopper, that's not abusive. The WG screwed up.

(I don't think Dave and I centrally disagree on this.)

pr

---

From: "Pete Resnick" <[email-masked]>
Date: Tue, 14 Feb 2017 15:12:04 -0800

On 13 Feb 2017, at 22:01, Mark Andrews wrote:

> was insane.  [...]
> removed from the draft for RFC 6840.  [...]
>

I would hope that you would have appealed to the chair at the time of 
the chair declaring the document ready for Last Call, and if that failed 
appealed to the AD. And it should have gone to Last Call with a note in 
the shepherd writeup indicating that a participant identified what he 
thought were two essential objections, and the reasoning for why the WG 
still felt that the outcome was reasonable. If that all happened, then 
no, you shouldn't bring it up at Last Call, except perhaps to identify 
the issue to others with expertise in the area who weren't part of the 
discussion and who might be able to better explain why these were 
"WG-running-off-the-cliff" issues and not 
"reasonable-engineering-tradeoffs". But you had better say *that*, not 
simply rehash the issue. Last Call should be about "something went wrong 
in the judgement of consensus", not "I prefer a different outcome".

Now, if Last Call was the first time that you noticed the issue, that 
would suck, but then it is reasonable to bring it up.

pr

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 15 Feb 2017 12:18:37 +1300

On 15/02/2017 06:05, Dave Crocker wrote:
...
> theory of benefit that is very, very, very rarely actually demonstrated.

Two points on this:

1. A claim that a choice made by the WG is not only harmful to the
protocol under review but *also* harmful to the Internet as a whole
would, I hope, always be legitimate under "no important concerns
have been missed" during IETF Last Call. That's certainly the basis
of an issue that I raised recently.

2. As a Gen-ART reviewer I've often seen drafts at IETF LC that
really *need* a general, in-depth review. As a matter of fact, as
a document editor, I just received such a review yesterday (whether
needed or not is not for me to say). So I'd say that that phrase in
RFC2418 takes a very optimistic view of quality control by WGs.

    Brian

---

From: Ted Lemon <[email-masked]>
Date: Tue, 14 Feb 2017 18:18:57 -0500

On Feb 14, 2017, at 6:01 PM, Pete Resnick <[email-masked]> wrote:
>> To Ted's point, indulging folk who 'did not have time' to participate earlier is frankly abusive of all those who did.
> 
> Yeah, I don't find Ted's point at all convincing either. On the other hand, if the WG didn't seek out required expertise and someone does notice a showstopper, that's not abusive. The WG screwed up.

Wow.   You must have a lot more free time than I do, Pete.   My experience is that there are lots of things that get worked on in working groups that I don't even have time to be aware of, and 99% of the time the fact that I wasn't aware of them is not a problem at all.   But 1% of the time it is, and last call is (hopefully) where I become aware of that and say something.   Are you really calling this abusive?

---

From: John C Klensin <[email-masked]>
Date: Tue, 14 Feb 2017 18:42:57 -0500

Pete,

I an in complete agreement with this note, but want to expand on
one point...

>...
>> To Ted's point, indulging folk who 'did not have time' to
>> participate  earlier is frankly abusive of all those who did.
> 
> Yeah, I don't find Ted's point at all convincing either. On
> the other hand, if the WG didn't seek out required expertise
> and someone does notice a showstopper, that's not abusive. The
> WG screwed up.

I think this is key (and another reason I worry about excessive
homogeneity).  It is the WG's responsibility to understand the
context of whatever they are proposing, to seek out additional
expertise if it is needed, and to incorporate relevant work
before requesting IETF Last Call.  While one can go too far
about this (I would not propose a Rule), if a serious issue is
identified during  Last Call, even or especially one that takes
a lot of time and energy to resolve, it should generally be
treated as "shame on the WG for not anticipating and dealing
with the issue" not "shame on the person who finds the problem
for not finding the time and resources to participate in the WG.
This isn't a matter of casting blame, but comments resembling
"you didn't participate in the WG so you have no standing to
raise that objection during Last Call" is as obnoxious and
likely to be harmful to the process as "you raised that issue
and lost, shut up".

best,
    john

---

From: "Pete Resnick" <[email-masked]>
Date: Tue, 14 Feb 2017 15:51:38 -0800

On 14 Feb 2017, at 15:18, Ted Lemon wrote:

>  My experience is that there are lots of things that get worked on in 
> working groups that I don't even have time to be aware of, and 99% of 
> the time the fact that I wasn't aware of them is not a problem at all. 
>   But 1% of the time it is, and last call is (hopefully) where I 
> become aware of that and say something.   Are you really calling this 
> abusive?

Nope. I simply disagreed with you that Last Call is a way to account for 
people who didn’t have time. The WG should have sought the expertise. 
Last Call ought not be relied upon to bring in experts who didn’t 
participate, due to lack of time or otherwise. The process can't depend 
on a single participant's time.

pr

---

From: Ted Lemon <[email-masked]>
Date: Tue, 14 Feb 2017 18:55:29 -0500

On Feb 14, 2017, at 6:51 PM, Pete Resnick <[email-masked]> wrote:
> The WG should have sought the expertise.

This appears to be a rule quite frequently honored in the breach.

---

From: Randall Gellens <[email-masked]>
Date: Tue, 14 Feb 2017 15:56:28 -0800

At 12:18 PM +1300 2/15/17, Brian E Carpenter wrote:

>  really *need* a general, in-depth review.

As a document author, I appreciate the area reviews done by GEN, SEC, 
etc.  However, I think they would be just as useful and perhaps more 
timely if done during WGLC (assuming the WG does a WGLC).

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
    The highlight of the annual Computer Bowl occurred when Bill Gates,
who was a judge, posed the following question to the contestants:
    "What contest, held via Usenet, is dedicated to examples of weird,
obscure, bizarre, and really bad programming?"
    After a moment of silence, Jean-Louis Gassee (ex-honcho at Apple)
hit his buzzer and answered "Windows."
                                          --Recounted by Adam C. Engst

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 14 Feb 2017 19:06:27 -0500

There is a procedure (that I thought was pretty widely known) that 
allows chairs to request early cross-are review when they think it is 
helpful.  I know that several of the review teams support this.

Yours,
Joel

---

From: George Michaelson <[email-masked]>
Date: Wed, 15 Feb 2017 02:51:38 +0200

I'm pretty much not in consensus in DNSOP right now, and I take that
in good part because working consensus is a pretty high value
proposition.

I think if I was 'outside the church' on a matter of technical merit
I'd feel more like standing up and shouting about it. But, mostly,
that WG is talking politics when I chose to disagree (I hate to
mis-characterize it, but thats what I see the ALT and related
discussion as: it has next to no technical merit to me, its a politics
conversation)

People in the WG talk to me outside of the list, and likewise. I'm ok
with that, because in the end, consensus is a high value thing. When
an author says to me "Its AUTH48 and we're in IESG process, if you
make me re-do this it has to go back into the WG to get signed off, so
tell me now: is this real, or just opinion" I take that on board.

So I draw a personal distinction between matters of substance, and
matters of opinion, in how I chose to flag dissent. and I would
encourage others to do the same: try and approach the question from
the perspective of:

  "is this materially relating to bits on the wire, and how protocol
endpoints work"

distinct from

  "is this something about a human process, I just don't agree with"

If you're talking about UTF-8 labels and encoded ASN.1 strings, its
case 1. If you're talking which of IAB and ICANN have primacy
determining what labels exist, you're case 2.

-G

On Wed, Feb 15, 2017 at 2:06 AM, Joel M. Halpern <[email-masked]> wrote:
> There is a procedure (that I thought was pretty widely known) that allows
> chairs to request early cross-are review when they think it is helpful.  I
> know that several of the review teams support this.
>
> Yours,
> Joel
>
>

---

From: John Leslie <[email-masked]>
Date: Tue, 14 Feb 2017 21:50:55 -0500

I would like to weigh in to support Pete's viewpoint.

> to send it back to the WG.

   A lot of the problem is that responsible ADs don't do this often enough.

   (The <ietf> list is a _really_bad_ place to try to resolve such things.
We _always_ find WG participants calling "Foul!" and I'm sure there are
others thinking it...)

> When Last Call identifies an issue that was missed or mis-evaluated
> by the WG (perhaps due to JCK's observation about WGs that remain
> homogeneous and insulated), considerable cost and detriment should
> be expected, and is probably appropriate.

   (It's hard to fully support Pete here, but I do. "Appropriate",
however, doesn't mean "something to die for": it merely means "the
world would be a better place if it worked this way".)

> We have all seen efforts that have gone completely off the rails,
> perhaps due to bad> management choices,

   Pete certainly knows we get what we pay for, when it comes to
management skills...

> and it is a sign of pathology if that's only noticed at Last Call
> time. Yes, it should probably have been handled during the WG process
> or by interim appeals,

   We're scared of appeals, without good reason. (Heck, _I_ was scared
of them five years ago!)

   The IESG, IMHO, is far better able to process appeals than it was
five years ago; and I believe they're essentially ready to formalize
a way to deal with appeals without shooting the messenger.

> but if Last Call is where it's finally dealt with, deal with it we must.

   (Pete _is_ the eternal optimist here: there-aint-no-such-thing-as
"must".)

   But I quite agree, the world would be a better place if we dealt with
it when it comes up in last-call rather than leave it for implementors
to deal with in incompatible ways.
 
> points that are "won" or "lost".

   YES!!!!

> Telling people that they "lost" or that "it's just sour grapes" is
> obnoxious and has no place in the discussion. 

   There's the eternal optimist, again...

   Name-calling _is_ the IETF way, :^( :^(

   We've exhausted yet-another IETF Chair trying to cure people of
discourtesy so obvious that you'd get sued for it in a normal world.
I'm not fool enough to think I'll have any better luck; but I can
at least speak in support of Pete!

> to shut that down.

   Reality check!

   There isn't anybody "running last-call".

   If there were, s/he would have no tools to "shut that down".

> I can even tolerate the occasional random participant saying,
> "Pete, you know this issue was discussed at length, and you didn't
> even bother to fairly say why you think the WG came to the wrong 
> conclusion, so this is really just rehashing."

   (We ought to be able to find a way to ensure someone says that in
private email...)

   But I have a suggestion (which I guess is why I'm bothering to type
this):

   We have pretty good archive tools availabe for every IETF mail-list.

   Occasionally, I see a posting, "Please read the thread starting at

https://mailarchive.ietf.org/arch/msg/ietf/l4Mi7p-KJanEP1FuEmCDclo6REA

before posting any more on this subject."

   This is actually helpful!

   If the person has read the thread and _is_ merely repeating things
already said in that thread, s/he deserves censure, and this will be
obvious to anyone checking that thread.

   If this is new material to the person so advised, it will multiply
greatly the "clue" level going forward.

   If the person can't be bothered to read the cited thread, then a referral
to the sergeant-at-arms is justified.

   And people thinking of joining in will have the opportunity to increase
their clue level _before_ making a fool of themselves.

> notice a showstopper, that's not abusive. The WG screwed up.

   I agree with Pete here, too.

   But I don't agree it's likely the WG Chairs or participants are likely
to see it that way. Somebody will have to help them see this. The obvious
choice is the Responsible AD; but if we limit our choices of ADs to those
who agree this is their job, we might have trouble filling AD slots.

   :^( :^(

--
John Leslie <[email-masked]>

---

From: Yoav Nir <[email-masked]>
Date: Wed, 15 Feb 2017 10:00:35 +0200

In case it isn’t widely known, there is a button on the datatracker page for the draft ( https://datatracker.ietf.org/doc/draft-ietf-something-or-other ) that says “Request review”.

Click that and request from any of the 8 review teams.

Yoav

> On 15 Feb 2017, at 2:06, Joel M. Halpern <[email-masked]> wrote:
> 
> There is a procedure (that I thought was pretty widely known) that allows chairs to request early cross-are review when they think it is helpful.  I know that several of the review teams support this.
> 
> Yours,
> Joel
>

---

From: S Moonesamy <[email-masked]>
Date: Sat, 18 Feb 2017 01:33:40 -0800

Hi John, Dave,
>a way to deal with appeals without shooting the messenger.

There isn't any reason to be scared of appeals if the WG took an 
informed decision.  It might be better to do that instead of going 
through a lengthy discussion.

>at least speak in support of Pete!

Name-calling does not help in a discussion.  At least the IETF Chair 
tried to address uncivil discussions publicly instead of looking the 
other way.  It is obviously an exhausting effort.

>    If there were, s/he would have no tools to "shut that down".

It is possible to take a decision.  That does not shut down the 
discussion; it tells other  people whether it is useful to keep 
discussing about the issue.

>greatly the "clue" level going forward.

Wouldn't it be better to summarize the thread instead of asking a 
participant to go through a long thread?

>participate earlier is frankly abusive of all those who did.

It is a matter of how the comment is framed.  For example, a comment 
along the lines of "I don't know whether the WG considered this 
issue" is not a problem as it should be possible to point to the 
resolution (re. issue tracker, decision posted to the WG mailing list, etc.)

Regards,
S. Moonesamy

---

From: John C Klensin <[email-masked]>
Date: Sat, 18 Feb 2017 10:26:21 -0500

--On Saturday, February 18, 2017 01:33 -0800 S Moonesamy
<[email-masked]> wrote:

> It is a matter of how the comment is framed.  For example, a
> comment along the lines of "I don't know whether the WG
> considered this issue" is not a problem as it should be
> possible to point to the resolution (re. issue tracker,
> decision posted to the WG mailing list, etc.)

It can be something else as well.  We could be more careful
about our vocabulary (but probably won't be), but consider a
scenario in which someone, even someone who is quite expert in a
given topic, looks at a WG charter and likely leadership and
participants and says to herself "This is unlikely to produce
anything harmful or go off into the weeds or is sufficiently
narrowly scoped to not pose a general problem.  I do not have
unlimited time to spend on IETF work, and my knowledge and
skills are needed elsewhere".  That is an "I don't have time"
scenario, but it is a vote of confidence in those actively
participating in the WG (and the WG Chair and AD involved) not
abusive of them.  If it then turns out that the WG, in her
opinion, does go off into the weeds, produces someone harmful to
the Internet or other work, or even produces something outside
the scope of the charter, I think the community needs her to
speak up and that any attempt to dismiss the concerns on the
grounds of "you should have participated more and earlier" are
not only abusive but pose a danger to the IETF.

The only solution to that sort of time conflict is for the IETF
to become much less ambitious about how much work can be done in
parallel, possibly by constraining the number of WGs.  The
latter has been proposed a few times, but has gone nowhere, I
think in part because even its advocates want whatever work they
want to do next to be treated exceptionally.

I also note that a Last Call complaint (or appeal, or suggestion
of intent to appeal) on the grounds that a document on which an
IETF Last Call has been initiated exceeds the scope of the
relevant WG Charter has ever gone anywhere.  It isn't clear to
me how cases like that should be handled because the work has
typically been completed and there is no obvious basis for
sending the document back to the WG for technical corrections.
The community should probably fire ADs who let that sort of
thing happen, but (without guessing at what discussions occur
within the Nomcom) there is little evidence of community
willingness to do that either.

      john

---

From: S Moonesamy <[email-masked]>
Date: Sat, 18 Feb 2017 15:59:06 -0800

Hi John,
>not only abusive but pose a danger to the IETF.

There are two message from Ted and Pete about "abusive" [1][2].  Pete 
made an interesting comment: "The process can't depend on a single 
participant's time."  In the above-mentioned scenario, and assuming 
that it is IETF work which I am supposed to do, I would take the Last 
Call comments into consideration.

>want to do next to be treated exceptionally.

Isn't the issue about the volume of WG discussions which an Area 
Director has to manage instead of the number of WGs?

>willingness to do that either.

It is up to the Working Group Chair and Responsible Area Director to 
assess whether the WG output falls within the WG Charter.

Regards,
S. Moonesamy

1. https://www.ietf.org/mail-archive/web/ietf/current/msg101332.html
2. https://www.ietf.org/mail-archive/web/ietf/current/msg101335.html

---

From: Ted Lemon <[email-masked]>
Date: Sat, 18 Feb 2017 19:41:20 -0500

On Feb 18, 2017, at 6:59 PM, S Moonesamy <[email-masked]> wrote:
> It is up to the Working Group Chair and Responsible Area Director to assess whether the WG output falls within the WG Charter.

This is technically true, but it's worth pointing out that working group charters are approved first by the IETF, and only _then_ by the IESG.   If the IESG decides that the working group charter is much broader than the IETF agreed it is, then that is a process failure. 

In practice the IETF often leaves these issues entirely to the IESG, so in practice this isn't necessarily a problem, but in principle it is, and you can't know whether the IETF said nothing about a charter because we agreed with the scope, or because we didn't look at it; in the former case, the IETF actually does have a position on the scope of the charter—it's just unstated, because it agrees with what was proposed.   Going beyond that scope would definitely be a process failure.

---

From: Ted Lemon <[email-masked]>
Date: Sat, 18 Feb 2017 22:10:02 -0500

> In practice the IETF often leaves these issues entirely to the IESG, so in practice this isn't necessarily a problem, but in principle it is, and you can't know whether the IETF said nothing about…

Joel Halpern pointed out privately that I somewhat misrepresented the process here.   The IETF review process for working group charters does not require that there be IETF consensus to approve the charter.    This is true, butdoesn't actually refute my claim that it is a process failure if the IESG approves a charter and then allows a working group to do work that exceeds the bounds of the charter.   That _is_ a process failure, in the simple literal sense that the process did not produce the right outcome: the IETF community was not given an opportunity to review the charter that would have included the work that was done.   If we think that's okay, why do we have working group charters at all?

The way the process _should_ have gone in this case is that when the IESG noticed that new work needed to be done by a particular working group, they updated the charter to reflect that, and got IETF review of the updated charter.   In this case, you are right that if the IETF community was clearly opposed to the new work, the IESG could still approve the charter.   But this would be extraordinary.

And this is why the IETF community shouldn't treat charter updates as pro forma.   Charter review is an important part of the feedback mechanism that keeps the IETF a consensus-driven organization.   Doing out-of-charter work, or lawyering the charter to say that work that really isn't part of the intent of the charter nevertheless conforms to the letter of the charter, bypasses this important step.

---

From: John C Klensin <[email-masked]>
Date: Sat, 18 Feb 2017 22:56:38 -0500

--On Saturday, February 18, 2017 3:59 PM -0800 S Moonesamy
<[email-masked]> wrote:

>...
>> The only solution to that sort of time conflict is for the
>> IETF to become much less ambitious about how much work can be
>> done in parallel, possibly by constraining the number of WGs.
>> The latter has been proposed a few times, but has gone
>> nowhere, I think in part because even its advocates want
>> whatever work they want to do next to be treated
>> exceptionally.
> 
> Isn't the issue about the volume of WG discussions which an
> Area Director has to manage instead of the number of WGs?

There are a number of dimensions of the problem.  One could
debate their ranking by importance, but I'm not sure the
exercise would be productive.  I would encourage you to find the
earlier I-Ds (there were at least two different models) and
discussions before starting over again.

>> I also note that a Last Call complaint (or appeal, or
>> suggestion of intent to appeal) on the grounds that a
>> document on which an IETF Last Call has been initiated
>> exceeds the scope of the relevant WG Charter has ever gone
>> anywhere.  It isn't clear to me how cases like that should be
>> handled because the work has typically been completed and
>> there is no obvious basis for sending the document back to
>> the WG for technical corrections. The community should
>> probably fire ADs who let that sort of thing happen, but
>> (without guessing at what discussions occur within the
>> Nomcom) there is little evidence of community willingness to
>> do that either.
> 
> It is up to the Working Group Chair and Responsible Area
> Director to assess whether the WG output falls within the WG
> Charter.

Yes, as long as you remember that they are supposed to be
representing the will of the community and accountable to it.
Neither WG Chairs nor ADs have any other source of authority.
The difficulty with your statement above is that it can be read
to imply that there are no constraints one, or appeals from, the
decisions.  That is not true or, more specifically, if one wants
a healthy IETF that produces credible work, it better not be.

best,
   john

---

From: S Moonesamy <[email-masked]>
Date: Sat, 18 Feb 2017 23:07:16 -0800

Hi Ted, John,

[I changed the subject line]

At 07:10 PM 2/18/2017, Ted Lemon wrote:
>Joel Halpern pointed out privately that I somewhat misrepresented 
>the process here.   The IETF review process for working group 
>charters does not require that there be IETF consensus to approve 
>the charter.    This is true, butdoesn't actually refute my claim 
>that it is a process failure if the IESG approves a charter and then 
>allows a working group to do work that exceeds the bounds of the 
>charter.   That _is_ a process failure, in the simple literal sense 
>that the process did not produce the right outcome: the IETF 
>community was not given an opportunity to review the charter that 
>would have included the work that was done.   If we think that's 
>okay, why do we have working group charters at all?

A Working Group charter is approved by the IESG and the IAB [1].  The 
working group charter would be a theoretical exercise if the charter 
which was published diverges significantly from what was actually 
done.  I agree that it is a process failure.

>The way the process _should_ have gone in this case is that when the 
>IESG noticed that new work needed to be done by a particular working 
>group, they updated the charter to reflect that, and got IETF review 
>of the updated charter.   In this case, you are right that if the 
>IETF community was clearly opposed to the new work, the IESG could 
>still approve the charter.   But this would be extraordinary.

It does not seem that difficult to ask for a recharter.  The relevant 
AD(s) could bring the matter to the attention of the IESG instead of 
having to ignore or to deal with a process failure.

>And this is why the IETF community shouldn't treat charter updates 
>as pro forma.   Charter review is an important part of the feedback 
>mechanism that keeps the IETF a consensus-driven 
>organization.   Doing out-of-charter work, or lawyering the charter 
>to say that work that really isn't part of the intent of the charter 
>nevertheless conforms to the letter of the charter, bypasses this 
>important step.

Describing the process issue as "lawyering" conveys the idea that it 
is unreasonable to complain about such issues.  I agree with what you 
wrote about the feedback mechanism.

>exercise would be productive.  I would encourage you to find the

Agreed.

>a healthy IETF that produces credible work, it better not be.

It is possible to appeal the decision of the WG Chair, and if that 
fails, to file an appeal with the Responsible AD.  There is still an 
option to file an appeal if the issue is noticed later in the process.

There are constraints.  At the WG level, the WG Chair is accountable 
to the Area Director.  A competent Area Director would probably ask 
for an explanation if the document exceeds the scope of the WG 
Charter.  I don't know the details of the case [2] to form an opinion 
about the matter.

Regards,
S. Moonesamy

1. The wording which I used is imprecise.
2. https://www.ietf.org/mail-archive/web/ietf/current/msg101489.html

---

From: Ted Lemon <[email-masked]>
Date: Sun, 19 Feb 2017 08:02:16 -0500

On Feb 19, 2017, at 2:07 AM, S Moonesamy <[email-masked]> wrote:
> Describing the process issue as "lawyering" conveys the idea that it is unreasonable to complain about such issues.  I agree with what you wrote about the feedback mechanism.

By lawyering, I mean the practice of trying to fit more in the charter than those who participated in writing that charter intended for it to cover.   This is a very fine line: there are definitely cases where one can legitimately debate what the actual intention expressed in the charter is.   For this reason I think it's best for ADs to err on the side of caution and update the charter if an AD thinks they might be at the edge.

---

From: John C Klensin <[email-masked]>
Date: Sun, 19 Feb 2017 18:49:34 -0500

SM,

--On Saturday, February 18, 2017 11:07 PM -0800 S Moonesamy
<[email-masked]> wrote:

>> The difficulty with your statement above is that it can be
>> read to imply that there are no constraints one, or appeals
>> from, the decisions.  That is not true or, more specifically,
>> if one wants a healthy IETF that produces credible work, it
>> better not be.
> 
> It is possible to appeal the decision of the WG Chair, and if
> that fails, to file an appeal with the Responsible AD.  There
> is still an option to file an appeal if the issue is noticed
> later in the process.

I think we are in danger of having two separate conversations
here and not quite realizing it.  I think Ted and I have been
discussing what happens, or could happen, in practice.  Your
response has been to tell us what the procedures are and allow.
I've confident that I understand those procedures and almost as
confident that Ted does too.  It is probably helpful to remind
us, but let's not get them confused with what happens in reality
(the difference between the procedures and reality may also be a
useful conversation but it is, again, a different one).

The most extreme example of the difference is, of course, that
the written procedures specify a remedy for an AD who has gone
off in the weeds, seriously underformed, or otherwise lost the
confidence of the community in the form of a recall procedure.
That procedure has never been used up to the point that a recall
committee is appointed, much less beyond that point.  One could
infer from that that we've never had a serious problem with an
AD.  I don't think that is true, YMMD.  One could also infer
that informal discussions and interactions, including
encouraging people to resign rather than face the actual recall
procedure, have worked well, making that actual procedure
unnecessary, but the procedures do not call for those actions.

In the charter case, unless someone is following the work of a
WG very closely (for which see the "I don't have time"
discussion that spawned this thread), the first warning that the
WG has produced work that drifted away from the charter involves
a document showing up for IETF Last Call.  That poses a very
difficult problem for the AD for two reasons.  First,
inconsistency between a draft document and charter provisions
can be the result of differing interpretations of the charter or
a discovery that completing the chartered work required some
excursions into directions not anticipated when the charter was
written.  Deviations from the charter are rarely a
black-and-white matter: what is more typical is "some" or "a
bit" and a judgment call as whether or not the issue is actually
significant. In principle, we probably all agree that it would
be best to revise the charter as soon as such situations are
discovered.  In practice, there are lots of reasons why that
often doesn't happen, many of them rather good reasons.   If the
deviations are significant, one of the not-so-good reasons is
the it requires ADs to admit that they let the WG get out of
control.

And, yes, a decision to not put the document on hold while the
charter is revised can be appealed.  But holding work up,
especially work that appears to be finished, is a big deal and
there is, appropriately, a lot of reluctance to do that.  So the
appeals don't happen and, when they do, they often don't go
anywhere.

> There are constraints.  At the WG level, the WG Chair is
> accountable to the Area Director.  A competent Area Director
> would probably ask for an explanation if the document exceeds
> the scope of the WG Charter.

Again, "exceeds" is a judgment call.  Asking is one thing,
getting a useful answer is another.  And telling a WG that
thinks it has finished a document and may be waiting to close
down that they have to start fussing with charter revisions
and/or completely redesign the specification is very hard and
not always a good idea.

best,
   john

---

From: Joe Touch <[email-masked]>
Date: Mon, 20 Feb 2017 08:14:24 -0800

On 2/18/2017 7:26 AM, John C Klensin wrote:
> any attempt to dismiss the concerns on the
+1

It's very easy for large organizations to perform DDOS attacks to push
through ideas that the rest of us have insufficient resources to address.

We need to not prevent that later feedback, even when it's raised early
inside the WG. IMO, an RFC is a result of *community* consensus, and WG
consensus should never supercede that.

Joe