Subject: The plenary and the nomcom-term and review panel proposals

From: John C Klensin <[email-masked]>
Date: Thu, 04 Aug 2005 05:28:49 -0400

Four observations on the plenary discussion of my drafts...

As I said at the end, I had not planned to come to the
microphone at all.  I wanted to listen.  What I heard
included...

(1) To repeat what I did say (since it was apparently hard to
hear) I see, once again, the problem that it has become
very hard to introduce a concept into this community.  If one
tries to do so via a comment on a mailing list, there is never
enough detail for anyone to really evaluate it (or the message
is so long that almost no one reads it).  If the concept is
presented in an I-D, then the document is torn to shreds for
not having enough detail for anyone to evaluate it.  On the
other hand, if details are provided in the initial document,
even as an example to show that a plausible set of details is
possible, the community (often including especially the IESG,
immediately focuses in on the details and picks at them,
ignoring the general concept and the usually-better question of
how to adjust the concept and fill in any blanks.

This behavior has a corollary along the "we cannot handle 
complexity" dimension.  If one writes a single draft that covers 
the several aspects of a problem, it is attacked as "too long, 
too complex, and covering too many issues" and told it should be 
broken up into smaller pieces.  If it is  broken up (the set of 
issues here are now two and I have been advised to produce a 
third that identifies the principles only), then people complain 
that having several drafts at once is too much to deal with.

That way of handling things, especially things that some people 
apparently just do not want to deal with, applies, I think, to 
both process proposals and our technical work.  It is not good 
news if we actually want to get things done.

(2) Several comments, during and after the discussion and most
precisely framed by Spencer Dawkins, that I may have made an
incorrect assumption about transition.   The text more or less
assumes that the review panel membership would be new
and the IESG membership would  be left in place.   Perhaps the
current IESG membership are most skilled in document review
rather than the sort of WG leadership and steering functions
that I had in mind -- the functions I think we had in the
early 1990s.  If that were the case, then we should resolve the
detail of the IESG being larger than the review panel, shift
the current IESG members onto the review panel, and repopulate
the steering/coordination/management entity (probably calling it 
something other than "IESG").

(3) A comment from an IESG member that the notion would probably 
add work, since the document provides for documents rejected  by 
the review panel to cycle back through the IESG. To me, this is 
one of the "sample detail" issues identified above.  If the IESG 
wants to be explicitly in that loop, and the community wants 
that, then "back to the IESG as the submitting body" is the 
right model.  I had anticipated their involvement at that point 
as lightweight, with the AD reviewing the comments and passing 
them on to the WG, but not assuming today's role of negotiator 
with the WG (or between the WG and the review panel).  I think 
that negotiator role, after a document goes out for IETF Last 
Call, is the source of several of our problems and needs to be 
eliminated.

For those who are reading this without having read the document, 
please note that rejection of a document by the review panel is 
intended to be A Big Deal.  The document suggests a process that 
would focus on getting good-quality, pre-reviewed, documents to 
the review panel --with shared responsibility between the WG and 
the IESG's steering/managing function for being sure that 
happens-- with the review panel only organizing a final check.

If the IESG believes that being in the "return" loop would be 
burdensome, then the proposal can easily be adapted to that 
belief.  The change would be to have the review panel return the 
document directly to the WG, with the AD involved only as part 
of WG management.  The IESG would then be involved again only as 
part of routine WG oversight and when the WG concluded that it 
was ready to resubmit the document.  I will make that change for 
-01 unless others object.

As a particularly strong variation, one could have the review 
panel return the document to the WG and then require either a 
"new benchmarks" or recharter activity since "rework a rejected 
document" would not be on the WG's pre-rejection charter.

Again, from my point of view, these are details that can be
sorted out and tuned if the community likes and wants the
general concept.  If the community does not, there is no point
wasting the time to generate and debate those details.

(4) I heard at least one IESG member say something that sounded
suspiciously like:

	"We are really busy and have all of these technical
	documents to review.  If you really want to discuss changes
	and process proposals, we will be happy to defer action on
	all of your technical documents until you come up with a
	process proposal that we like".

I'm sure that was not intended.  However, some reassurance
would be appropriate.  It would also, IMO, be appropriate to
pay careful attention to comments (last night and earlier) that
it is probably time to come up with a process for reviewing and
approving process proposals that does not involve the IESG
other than as community members who can comment on a proposal
along with the rest of us.

Almost as a side-effect of the core changes, the review panel
proposal suggests moving sign-off on process proposals to the
IAB.  Would it be useful to generate a very short I-D that does
that, and that only, as a means of moving things forward?

     john

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Pekka Savola <[email-masked]>
Date: Thu, 4 Aug 2005 14:06:31 +0300 (EEST)

On Thu, 4 Aug 2005, John C Klensin wrote:
> the steering/coordination/management entity (probably calling it something 
> other than "IESG").

Personally, I don't think changing the proposal in this aspect is 
worth the effort.  Changing people around seems more trouble than 
electing new ones.  The existing IESG people, if they felt the review 
panel is more suited for them, could very well apply to the review 
panel, and if selected, the nomcom would pick the IESG replacement. 
In a year or two, everything would be settled in any case.  The people 
will find the roles they're interested in.

(But this wasn't my maint point in replying..)

> (3) A comment from an IESG member that the notion would probably add work, 
> since the document provides for documents rejected  by the review panel to 
> cycle back through the IESG. To me, this is one of the "sample detail" issues 
> identified above.  If the IESG wants to be explicitly in that loop, and the 
> community wants that, then "back to the IESG as the submitting body" is the 
> right model.  I had anticipated their involvement at that point as 
> lightweight, with the AD reviewing the comments and passing them on to the 
> WG, but not assuming today's role of negotiator with the WG (or between the 
> WG and the review panel).  I think that negotiator role, after a document 
> goes out for IETF Last Call, is the source of several of our problems and 
> needs to be eliminated.
>
> For those who are reading this without having read the document, please note 
> that rejection of a document by the review panel is intended to be A Big 
> Deal.  The document suggests a process that would focus on getting 
> good-quality, pre-reviewed, documents to the review panel --with shared 
> responsibility between the WG and the IESG's steering/managing function for 
> being sure that happens-- with the review panel only organizing a final 
> check.

I'm troubled with the assumption that the review panel rejection is A 
Big Deal.  This has unstated assumptions on what kind of people 
you'd expect to be on the review panel and/or what kind of review is 
expected.

As an occasional reviewer myself, a few observations:
  * All the indepth reviews I have done have resulted in issues,
  * If I haven't found a problem or a request for clarification, I
    haven't paid sufficient attention to the review, and
  * IMHO there is nothing more frustrating than finding problems (not
    maybe critical ones) which don't get fixed for one reason or
    another (e.g., because it isn't worth the time to respin the
    document, giving that comment would be a "big deal", ...).

Scientific reviews certainly have "accept", "reject", or "accept with 
modifications".

Maybe forcing "yes, business as usual" or "no, A Big Deal" is 
intentional -- to encouraging the reviewers to perform less in-depth 
reviews (with more responsibility on the chairs and the IESG) so that 
they'd only complain of the most horrid problems, to lessen the load, 
to encourage earlier participation, make sure the majority of the docs 
go through without issues, etc.

As how to address this, a few observations:
  * the review panel's review function has to be roughly equivalent to
    what review the IESG is doing today, so that the IESG can
    "give up" most of its review with good confidence.
  * to achieve this, I think we need "accept", "reject",
    "tentative accept" resolutions where in all cases the panel could
    include feedback on things that were noted.
  * whether or not the review panel (or the reviewers) check the
    revised documents after "tentative accept", whether the checking
    responsibility is with the AD(s), or something else is
    an interesting detail point which probably isn't relevant at this
    point.

Hope this helps in getting a better understanding on what the role of 
the panel could be.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 4 Aug 2005 14:39:35 +0200

Speaking only for myself, and at the slogan level,

> I'm troubled with the assumption that the review panel rejection is 
> A Big Deal.  This has unstated assumptions on what kind of people 

I'm troubled with the fact that at least 70 percent of our documents 
get at least one DISCUSS - which means that at least one AD thought 
the document had a big enough problem that approving the document in 
its current form was not the right thing to so.

I know it eludes most of us, most of the time, that almost every 
standards action we have seen for as long as we have been in the IETF 
has been for Proposed Standards, and (look at 2026 if you think I'm 
kidding) these are SUPPOSED to be specifications at an early stage of 
maturity.

We need to get to the point where we forward specifications that don't 
bounce back most of the time. I haven't said it out loud, but I hope 
that a Standards Review Panel might be able to tackle early and 
cross-area review - with help from the community - which the IESG was 
NOT able to tackle (see ICAR).

So, I'm not asking the Standards Review Panel to approve junk. I'm 
asking the community to support getting to the point where most of our 
documents that reflect WG concensus are not going to bounce back in 
late review. We aren't there yet. We desperately need to be "there".

In My Opinion.

Spencer

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 04 Aug 2005 09:08:13 -0400

I think it would be useful to analyze the nature of current DISCUSS 
comments before drawing conclusions from the 70% figure. They apparently 
range from simple typos ("expand acronyms") to differences of opinion 
("WG chose X, AD prefers Y; both X and Y are at least plausible") to 
adding various disclaimers to fundamental design problems ("broken").

Unless we write like lawyers writing product warranties, we will always 
have differences of opinion as to which disclaimers are important enough 
to call out explicitly and how strongly, regardless of how much 
pre-approval review is done.

Spencer Dawkins wrote:
>> Big Deal.  This has unstated assumptions on what kind of people you'd 
>> expect to be on the review panel and/or what kind of review is expected.
>>
> I'm troubled with the fact that at least 70 percent of our documents get 
> at least one DISCUSS - which means that at least one AD thought the 
> document had a big enough problem that approving the document in its 
> current form was not the right thing to so.
> 
> documents that reflect WG concensus are not going to bounce back in late 
> review. We aren't there yet. We desperately need to be "there".
> 
> In My Opinion.
> 
> Spencer
> 
>

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 4 Aug 2005 15:18:09 +0200

> comments before drawing conclusions from the 70% figure. They 
> apparently range from simple typos ("expand acronyms") to 
> differences of opinion ("WG chose X, AD prefers Y; both X and Y are 
> at least plausible") to adding various disclaimers to fundamental 
> design problems ("broken").

I agree completely with Henning. If I understood Allison in the 
plenary last night, she was saying that most DISCUSSes don't really 
result in much change (Allison, if I misunderstood, please correct 
me).

My point is that each of these DISCUSSes kept a specification from 
being approved for at least one two-week telechat cycle. I believe, in 
the absence of data, that adding delays to a project makes it easier 
to stretch out other delays, so "two weeks" is the minimum amount of 
time one would wait before a specification would be reballoted, but if 
a document editor is on vacation for a week and doesn't provide 
updated text immediately, the actual delay can be longer.

I would like to get out of this situation.

Given that the AUTH48 delay is averaging something like a month (did I 
get this data point right from the plenary last night?), I'm hoping 
that getting crisper and more predictable in the approval cycle will 
also encourage working groups to get crisper and more predictable in 
their part of the process.

Spencer

---

From: Pekka Savola <[email-masked]>
Date: Thu, 4 Aug 2005 16:33:12 +0300 (EEST)

On Thu, 4 Aug 2005, Spencer Dawkins wrote:
> being approved for at least one two-week telechat cycle. I believe, 
> in the absence of data, that adding delays to a project makes it 
> easier to stretch out other delays, so "two weeks" is the minimum 
> amount of time one would wait before a specification would be 
> reballoted, but if a document editor is on vacation for a week and 
> doesn't provide updated text immediately, the actual delay can be 
> longer.

Documents don't need to be reballoted [in IESG] to be approved (after 
being revised). This is up to the nature of the comments/changes and 
the AD's confidence that the discusses have been addressed to the 
satisfaction of commenters, and that new text doesn't have issues.

If time is of the essence, the chair or whoever could certainly take 
up the document from the editor; I've done so personally in a couple 
of cases.

> design problems ("broken").

Exactly.  As I read it, the review panel proposal included "yes" and 
"no".  If you say "yes", the document is shipped as-is.

It is (IMHO) important to be able to fix problems, even if relatively 
minor, even if you would like to say "yes" or would have confidence 
that the problems could be fixed in a day or two.

There are many levels of issues, for example:
  - "there is a significant architectural problem in the doc", NO (or
    big discuss)
  - "there are major issues which require at least significant
    clarification/discussion"
  - there are relatively important issues which need to be fixed but
    are so straightforward that the WG (and the editor in particular)
    can probably do it without further dialogue.
  - there are typos or other things affecting readability, and
    sometimes even the clarity of the document.
  - probably a lot more different classes of issues..

To me, having reviews which wouldn't report any issues would meant 
that with high probability, such reviews wouldn't be sufficiently 
indepth.  Finding issues is one sign of a good review -- if the issues 
found are minor, this is also often a sign of a document in a good 
shape.  Discarding the issues doesn't seem good, though if they are 
minor, the amount of dialogue/energy needed should be minimized.

---

From: John C Klensin <[email-masked]>
Date: Thu, 04 Aug 2005 09:41:05 -0400

Two observations, just my opinion...

--On Thursday, August 04, 2005 15:18 +0200 Spencer Dawkins 
<[email-masked]> wrote:

>> I think it would be useful to analyze the nature of current
>> DISCUSS  comments before drawing conclusions from the 70%
>> figure. They  apparently range from simple typos ("expand
>> acronyms") to  differences of opinion ("WG chose X, AD
>> prefers Y; both X and Y are  at least plausible") to adding
>> various disclaimers to fundamental  design problems
>> ("broken").
>
> I agree completely with Henning. If I understood Allison in
> the plenary last night, she was saying that most DISCUSSes
> don't really result in much change (Allison, if I
> misunderstood, please correct me).

Well, as long as we don't remove the "editor" function from "RFC 
Editor" I hope there is _never_ an IESG DISCUSS on "expand
acronyms".  If there is, we are doing something seriously wrong 
and wasting time, probably a month or more (AD reads document to 
that level, writes up DISCUSS, enters it in tracker, negotiates 
with author, slides to the next telechat...).  The good news is 
that, while I could be wrong, my subjective impression is that 
those happen rarely if at all these days.  The bad news is that 
these DISCUSSes are mostly more substantive, and that brings us 
to...

> My point is that each of these DISCUSSes kept a specification
> from being approved for at least one two-week telechat cycle.
> I believe, in the absence of data, that adding delays to a
> project makes it easier to stretch out other delays, so "two
> weeks" is the minimum amount of time one would wait before a
> specification would be reballoted, but if a document editor is
> on vacation for a week and doesn't provide updated text
> immediately, the actual delay can be longer.
>

There is another element of this, which intersects with NEWTRK 
and Spencer's earlier note.   While we haven't been taking 
advantage of them, there are really strong reasons for a two (at 
least)-stage standards process.  One of them is the ability to 
get those early-stage specifications that Proposed Standards are 
supposed to be out the door.

Conceptually, I think we ought to be adding a new section to 
most Proposed Standards.   That section is exactly what 2026 
indirectly calls for in its prohibition on known technical 
deficiencies, reading that is "we know about them and know how 
to fix them".  Perhaps we should be letting things go out with 
an "Areas of Unresolved Controversy" section in which questions 
and issues that are not absolutely fatal showstoppers, to a 
proposal could just be identified in a Proposed Standard, the 
thing shipped, and those issues resolved before the 
specification could go to Draft (or, in a two-step process, Last 
or whatever it is called).

This is not yet another process change suggestion, just a 
suggestion that we need to, again, think of Proposed Standards 
as what 2026 and the term "proposed" imply, such that the 
response to finding a slightly-important problem is to note it 
and move on, rather than aspiring to have every specification 
perfect, even pre-implementation specifications.

Of course, one could get the same effect by declaring all of 
today's Proposed Standards to be Experimental, taking the "note 
and move on" approach to them, and then having a one-stage 
standards track that requires implementation and deployment 
experience to get to stage 1.

        john

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 4 Aug 2005 16:13:24 +0200

Hi, Pekka,

I rarely if ever argue with you about protocol stuff, because you're 
pretty good at protocols, and our process IS a protocol, but I do see 
"returned to clear DISCUSS" items on the IESG telechat agendas. So, I 
bet you're right, but there is running code that we actually DO end up 
sliding to the next telechat, at least some of the time.

Spencer

>
> Documents don't need to be reballoted [in IESG] to be approved 
> (after being revised). This is up to the nature of the 
> comments/changes and the AD's confidence that the discusses have 
> been addressed to the satisfaction of commenters, and that new text 
> doesn't have issues.
>

In all honesty, there are a lot of things that you do that I wish we 
all did, and this is one of them :-)

Spencer

---

From: Pekka Savola <[email-masked]>
Date: Thu, 4 Aug 2005 17:25:49 +0300 (EEST)

On Thu, 4 Aug 2005, Spencer Dawkins wrote:
> I rarely if ever argue with you about protocol stuff, because you're pretty 
> good at protocols, and our process IS a protocol, but I do see "returned to 
> clear DISCUSS" items on the IESG telechat agendas. So, I bet you're right, 
> but there is running code that we actually DO end up sliding to the next 
> telechat, at least some of the time.

FWIW, I've seen this in basically two scenarios:

  - the "discussing ADs" have proved to be unresponsive, either in 1) 
actually (not) dicussing the comment for one reason or another, or 2) 
verifying that the new version addresses their DISCUSS.  The trick of 
the shepherding AD is to force re-review of the spec by putting it 
back on the agenda.  (This is what I've seen in most cases, and yes, 
it's a problem -- I've also mailed the IESG about it.)

  - the changes have been very extensive (and possibly partially the 
above in verifying whether they address the issues), and re-reviewing 
the document at the IESG has been considered appropriate.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 04 Aug 2005 15:30:22 +0200

(note is long - summary: Review panel SHOULD, in my opinion, be able to 
send back documents to WG without it being a Big Deal. At least once.)

> adding various disclaimers to fundamental design problems ("broken").

I don't know if Bill's database extract contains the actual DISCUSS text, 
but the analysis would at least be interesting.

The category I remember seeing most often was "you have not explained how 
this works" - this might be because it was not written clearly enough, 
because there were states/events that the WG/author had not described, 
because the explanation was in other documents not available to reviewers, 
or because the WG had chosen to ignore that particular issue in the 
document - sometimes because the WG had failed to get consensus on it, and 
preferred to paper over the issue by not mentioning it.

In at least one case I remember, my DISCUSS saying "I don't see how this 
works" resulted in a technical change in the document; the WG effectively 
said "No, we don't see how this can work either - we'd better decide".

I don't remember many recent DISCUSSes based on a fundamental design issue; 
the perception that protocols with fundamental design problems (like 
"broken security") would not pass the IESG might be a reason for the lack 
of such DISCUSSes - putting the quality input where it should be (in the 
WG), but enforced by a strong end-game filter function (in the IESG).
And that's a Good Thing.

But - I don't remember all DISCUSS texts.....

                         Harald

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 4 Aug 2005 17:09:18 +0200

Dear Harald,

I agree, with one edit - s/to WG/to WG early in the process/.

> (note is long - summary: Review panel SHOULD, in my opinion, be able 
> to send back documents to WG without it being a Big Deal. At least 
> once.)

The part where I stroke out about us continuing to think that 
documents coming to a Full Stop in the IESG, or in the SRP, being the 
common path we optimize for is that the Review Panel, as described in 
the current draft, is a LATE review entity, and we don't like late 
surprises (for example, the words "guess again" when the WG thinks 
it's through).

I honestly hope the SRP spends some time spinning up an ICARish 
mechanism for EARLY review, and that the SRP spends its time reading 
reviews from ICARish reviewers that say, "these guys used to be 
idiots, but they've gotten a LOT smarter - please check the 02 and 04 
text in section 6.3 for proof".

And the SRP can spend its time spot-checking drafts that have been 
seriously CARDed (in the SIR sense), and adjourn early to the bars.

And what a wonderful world that would be...

Spencer

---

From: "Steven M. Bellovin" <[email-masked]>
Date: Thu, 04 Aug 2005 12:40:16 -0400

>sliding to the next telechat, at least some of the time.

I think the detailed minutes will clarify this, but there's no one 
answer.  

Often, the DISCUSSing AD will clear a DISCUSS offline, in which case 
things proceed on request of the sponsoring AD.  This is often the case 
for relatively simple issues, especially when dealt with promptly.

If a document takes a while to come back, or if the changes are 
complex, it can be returned to the agenda so that the changes can be, 
well, discussed.  Maybe the objecting AD isn't happy enough, but the 
sponsoring AD thinks they might be persuadable by the other ADs.  
That's been known to happen.  A document with multiple substantive 
DISCUSSes often comes back, because the changes can be interdependent.

Finally -- and this is painful -- sometimes the objecting AD doesn't 
respond to private requests to clear a DISCUSS.  Putting the document 
on the agenda is a forcing function.  This last one shouldn't happen, 
but it does.

		--Steven M. Bellovin, http://www.cs.columbia.edu/~smb

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 09 Aug 2005 15:25:20 +0200

John C Klensin wrote:
...
> please note that rejection of a document by the review panel is intended to be A Big Deal.

I don't think this is a detail, so here goes:

Today, we DISCUSS documents that have (e.g.) a substantive error
in their normative references or an unclear SHOULD that leaves an
implementer in doubt.

Very often, these are resolved in very few days and the document
gets approved before the dust has settled on the DISCUSS.

John, is it your intention that this class of issue would result in
rejection and thus be A Big Deal?

If not, who will detect these small-but-essential issues and who
will get them fixed?

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 09 Aug 2005 15:43:07 +0200

Spencer Dawkins wrote:
>> apparently range from simple typos ("expand acronyms") to differences 
>> of opinion ("WG chose X, AD prefers Y; both X and Y are at least 
>> plausible") to adding various disclaimers to fundamental design 
>> problems ("broken").

Well, please look at draft-iesg-discuss-criteria-00.txt for
a first cut at what is, and isn't, an acceptable DISCUSS. We've got
some pretty good feedback on that draft, so it will be revised.
And I think it would serve as the starting point for a review panel's
criteria.

> 
> 
> I agree completely with Henning. If I understood Allison in the plenary 
> last night, she was saying that most DISCUSSes don't really result in 
> much change (Allison, if I misunderstood, please correct me).

Allison's on vacation. It's very variable. Sometimes the text change
may be null (if the reviewer had simply misunderstood), sometimes it
will be a few words, sometimes it will be a few paragraphs, and
in the worst case it's a rewrite.

> 
> My point is that each of these DISCUSSes kept a specification from being 
> approved for at least one two-week telechat cycle. I believe, in the 
> absence of data, that adding delays to a project makes it easier to 
> stretch out other delays, so "two weeks" is the minimum amount of time 
> one would wait before a specification would be reballoted, but if a 
> document editor is on vacation for a week and doesn't provide updated 
> text immediately, the actual delay can be longer.

It can be a few minutes if resolved by jabber and a short note to the
RFC Editor; it can be months if it needs a rewrite and a new WG
Last Call. It all depends how deep the problem is.

> I would like to get out of this situation.

Some of it is intrinsic, and rearranging the furniture won't
help. But we agree violently that *earlier* review would
help enormously.

> get this data point right from the plenary last night?), I'm hoping that 
> getting crisper and more predictable in the approval cycle will also 
> encourage working groups to get crisper and more predictable in their 
> part of the process.

Look at that feedback loop from the other side: getting WGs crisper and
more predictable will encourage the approval cycle to be crisper and
more predictable.

Pekka and Spencer also wrote:

> 
>>
>> Documents don't need to be reballoted [in IESG] to be approved (after being revised). This is up to the nature of the comments/changes and the AD's confidence that the discusses have been addressed to the satisfaction of commenters, and that new text doesn't have issues.
>>
>> If time is of the essence, the chair or whoever could certainly take up the document from the editor; I've done so personally in a couple of cases.
> 
> 
> In all honesty, there are a lot of things that you do that I wish we all did, and this is one of them :-)

Be careful what you wish for. It's a process violation to make
substantive changes during or after IESG approval without looping
back to the WG. It even attracts appeals. So in such a case,
there is always a judgement call (whether the change is
substantive or editorial) and if you misjudge it, you suffer.

    Brian