Subject: RE: Voting (again)

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Fri, 15 Apr 2005 16:03:02 -0700

> I also believe the nomcom process does provide 
> accountability.  I think that the nomcom interview process 
> was more comprehensive than any job interview process I've 
> gone through.

I think you make a fundamental error here, accountability is determined
by whether we can get rid of someone, not by how they are appointed in
the first place.

The problem that arises with ADs is when they start insisting on silly
ideas that they are personally invested in like the jihad against NAT,
firewalls and anything else that 'breaks' the end-to-end principle. The
IESG rules mean that any AD can vote 'Discuss'.

Federal Judges are appointed by a system that has a large number of
controls, but they are intentionally beyond accountability once they are
appointed.

It is extreemly unlikely that a NOMCON is ever going to change more than
a small number of ADs because they understand that they have no mandate.
One NOMCON made major changes to the IAB but that is rare.

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Keith Moore <[email-masked]>
Date: Sat, 16 Apr 2005 00:22:24 -0400

> a small number of ADs because they understand that they have no mandate.

so in your mind, unless they kill off a lot of ADs, there's no accountability?
what a crock.

---

From: kent crispin <[email-masked]>
Date: Fri, 15 Apr 2005 23:02:17 -0700

> by whether we can get rid of someone, not by how they are appointed in

Oh.  Therefore voting has nothing to do with accountability, since it is a 
mechanism for selecting people in the first place, and therefore the 
premise of this thread is vacuous.

-- 
Kent Crispin 
[email-masked]    p: +1 310 823 9358  f: +1 310 823 8649
[email-masked] SIP: [email-masked]

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 18 Apr 2005 10:27:59 +0200

Phill,

Hallam-Baker, Phillip wrote:
...
> firewalls and anything else that 'breaks' the end-to-end principle.

I think this is immoderate use of language, to say the least.

There is a body of work from the IAB, not the IESG, about the deep
problems caused by NATs and other middleboxes. That doesn't mean that
the IESG has its head in the sand - the IETF has to deal with reality,
even when it's distasteful.

> The

Er, yes, I think it's known as collective responsibility in some
circles. It would happen much less often if WGs conducted their own
cross-area review before drafts were passed on to the IESG.

    Brian

---

From: Ted Faber <[email-masked]>
Date: Mon, 18 Apr 2005 08:07:44 -0700

> controls, but they are intentionally beyond accountability once they are

You know, I hear that a lot, and it's false.  

http://air.fjc.gov/history/topics/topics_ji_bdy.html

Pass it on.

Now back to your regularly scheduled rant.

-- 
Ted Faber
http://www.isi.edu/~faber           PGP: http://www.isi.edu/~faber/pubkeys.asc
Unexpected attachment on this mail? See http://www.isi.edu/~faber/FAQ.html#SIG 
_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Joe Touch <[email-masked]>
Date: Mon, 18 Apr 2005 10:19:58 -0700

Brian E Carpenter wrote:
...
> cross-area review before drafts were passed on to the IESG.

This is a great idea, but ADs can require redundent cross-area reviews
- -- not for re-review, but because they don't always check, ask, or read
the I-D ACKs section.

And no, this isn't a hypothetical.

Joe

---

From: Dave Crocker <[email-masked]>
Date: Mon, 18 Apr 2005 22:11:59 -0700

Brian,

>  Er, yes, I think it's known as collective responsibility in some circles.

When it is used well, yes.

When it is used to reflect the personal preferences of the AD -- no matter the 
history of the working group -- then it is known as something else, and it 
happens with some regularity.

  d/
  ---
  Dave Crocker
  Brandenburg InternetWorking
  +1.408.246.8253
  dcrocker  a t ...
  WE'VE MOVED to:  www.bbiw.net

---

From: Yakov Rekhter <[email-masked]>
Date: Tue, 19 Apr 2005 06:37:58 -0700

Dave,

> happens with some regularity.

Perhaps the IETF traditional motto, "rough consensus and working
code" should be revised to make it clear that the "rough consensus"
goes only up to a certain point, but after that point the IETF
operates solely by a decree from the IESG.

Yakov.

---

From: Eliot Lear <[email-masked]>
Date: Tue, 19 Apr 2005 19:19:38 +0200

Yakov,

> operates solely by a decree from the IESG.

You and I were both in the room when the Ethernet-MIB WG LOUDLY objected 
to Jon Postel having tweaked the Ethernet-MIB to properly align 
definitions with what the IEEE counters were.   The WG chair was rip 
roaring upset.  Here you had rough consensus and running code.  How dare 
others interfere!  Of course it was a broken spec, but nevermind that! 
The WG knew better.

Good thing *someone* was minding the store.

Eliot

---

From: "william(at)elan.net" <[email-masked]>
Date: Tue, 19 Apr 2005 10:36:09 -0700 (PDT)

On Tue, 19 Apr 2005, Eliot Lear wrote:

> You and I were both in the room when the Ethernet-MIB WG LOUDLY objected to 
> Jon Postel having tweaked the Ethernet-MIB to properly align definitions with 
> what the IEEE counters were.   The WG chair was rip roaring upset.  Here you 
> had rough consensus and running code.  How dare others interfere!  Of course 
> it was a broken spec, but nevermind that! The WG knew better.
>

At the same time reverse is not true, i.e. I do not think IESG should be 
allowed to make a decision on document on its own if there is no consensus.

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: Eliot Lear <[email-masked]>
Date: Tue, 19 Apr 2005 19:53:55 +0200

An individual has the ability to write a draft.  The IESG has the 
ability to gauge consensus as to whether that draft should become a 
standard.   So in essence they have the capability today.  The fact that 
it is rarely used is a testament to the good judgement these good people 
display.  In other words, at least this part "ain't broke".

Eliot

william(at)elan.net wrote:
>> You and I were both in the room when the Ethernet-MIB WG LOUDLY 
>> objected to Jon Postel having tweaked the Ethernet-MIB to properly 
>> align definitions with what the IEEE counters were.   The WG chair was 
>> rip roaring upset.  Here you had rough consensus and running code.  
>> How dare others interfere!  Of course it was a broken spec, but 
>> nevermind that! The WG knew better.
>>

---

From: "william(at)elan.net" <[email-masked]>
Date: Tue, 19 Apr 2005 11:02:24 -0700 (PDT)

On Tue, 19 Apr 2005, Eliot Lear wrote:

> An individual has the ability to write a draft.  The IESG has the ability to 
> gauge consensus as to whether that draft should become a standard.

Yes and sometimes they have to take a guess if the number of comments
in regards to the draft is too few. I'm however talking about situation
where there would be enough comments on the it ...

> So in essence they have the capability today.  The fact that it is 
> rarely used is a testament to the good judgement these good people 

I did not say it is. I think its quite clear that decision for document
to become a standard should be based on BOTH consensus AND agreement 
of IESG.

---
William Leibzon
Elan Networks
[email-masked]

---

From: Sam Hartman <[email-masked]>
Date: Sun, 24 Apr 2005 14:36:13 -0400

>>>>> "Dave" == Dave Crocker <[email-masked]> writes:

    Dave> Brian,
    >> Er, yes, I think it's known as collective responsibility in

    Dave> When it is used well, yes.

    Dave> When it is used to reflect the personal preferences of the
    Dave> AD -- no matter the history of the working group -- then it
    Dave> is known as something else, and it happens with some
    Dave> regularity.

Organizations rarely improve by having vague comments about abuse of
power tossed around.  If you are looking to improve the process I
suggest that you raise specific objections to specific actions.  Even
if the community disagrees with you then you will build up a
documented case history which can be used to either prove or refute
your point.

I realize that sometimes your concern is not individual actions but a
concern about a trend or a perception of a trend.  In such cases I've
found that collecting examples together and pointing out that each
example taken individually is fine but that the trend is problematic.

But seriously these vague comments do no one any good.  They don't
allow the people you believe have made a mistake to evaluate their
actions.  They don't help the organization as a whole.  They spread
FUD and create friction without any of the positive benefits of real
introspection or process engineering.

--Sam

---

From: Sam Hartman <[email-masked]>
Date: Tue, 26 Apr 2005 01:22:41 -0400

>>>>> "Dave" == Dave Crocker <[email-masked]> writes:

    >> Organizations rarely improve by having vague comments about
    >> abuse of power tossed around.  If you are looking to improve
    >> the process I suggest that you raise specific objections to
    >> specific actions.

    Dave> Sam,

    Dave> 1.  Apparently you missed the extended, public exchanges
    Dave> about these issues, over the last 3 years.  I am not
    Dave> suggesting anything new or different, merely observing that
    Dave> we have done not one thing to attend to them.

No, I was certainly aware of that discussion.  I followed the
problem-statement list for a while and followed the plenary
discussions.  I seem to have missed the specific complaints thought; I
found a lot of vague claims and a lot of process discussion.  I did
not find claims that specific IESG members had used the discuss power
to advance personal agendas.

I may have missed the specifics.  There really was a lot of traffic on
the problem list.  After I determined that things seemed to be going
in a reasonable direction I did end up spending less energy on it.  I
have only so much time and I chose to spend it on technical work.

    Dave> 2.  The pressures against citing specifics is extreme.  And
    Dave> the IESG/IAB get predictably defensive. Those who take their
    Dave> specific concerns public are roundly punished.  And, yes,
    Dave> that's another vague claim.  However note that you chose to
    Dave> issue a public dismissal about my "vague language" rather
    Dave> than actually pursue the matter through a constructive
    Dave> channel.  (

I considered my suggestion constructive in that I gave specific
recommendations for how I thought you should proceed and honestly
hoped that you would follow those recommendations.  I had (and have)
every desire to seriously consider any specific claims you make or
point me to.

    Dave> No one who has watched the IETF list for any
    Dave> amount of time would seriously suggest that this is a
    Dave> reasonable forum for pursuing such details.)

I'll suggest that it is the best forum we have.  I believe that this
discussion should happen in public and I believe that if there are
specific issues they need to be discussed.

I made the best suggestion I had at the time given that I believe such
discussions need to be public to be effective.

(Note that I'm assuming you believe the problems are bad enough that
they are systemic.  If you just believe someone made a mistake then by
all means start by talking to that person privately.)

    >> I realize that sometimes your concern is not individual actions
    >> but a concern about a trend or a perception of a trend.  In
    >> such cases I've found that collecting examples together and

    >> pointing out that each example taken individually is fine but
    >> that the trend is problematic.

    Dave> We had an entire working group that expressed these
    Dave> concerns.

    Dave> How quickly we forget.

See above.  I just looked at section 2.6 of RFC 3774 and it does not
seem to discuss the sorts of problems that lead to my comment.  If I'm
missing something please point me at it.

--Sam

---

From: Dave Crocker <[email-masked]>
Date: Mon, 25 Apr 2005 23:29:18 -0700

>   I did not find claims that
>  specific IESG members had used the discuss power to advance personal
>  agendas.
>
>  I may have missed the specifics.  

Well, no, you probably did not miss the specifics that you are looking 
forward.  

No one is silly enough to stand up publicly, refer to AD X and declare 
"j'accuse".  That is sometimes done in private communications to the nomcom, 
but those are confidential of course.

The detail that was worth noting was the prevalence of the view.  It's not 
just the stray crazy, like me.

The problem is not that ADs have nasty intentions. It's that the 
otherwise-essential job of quality assurance has been taken to the extreme of 
having ADs block completed work that has been developed in all the proper 
ways.  This includes AD vetos (which are euphemistically called "discuss") 
with non-normative issues or requirements for lists of adjunct -- ie, not 
required -- work.

> >  However note that you chose to issue a public
> > dismissal about my "vague language" rather than actually pursue the
> > matter through a constructive channel. 
....
>  Dave> No one who has watched the IETF list for any amount of time
>  would seriously suggest that this is a reasonable forum for pursuing
>  such details.)

care to point at any equivalent, public discussions that have been productive?

I haven't seen one in the 15 years I've been involved with the IETF.  And from 
what I can tell, they don't happen anywhere else, either.

What open public discussions can be good for is to surface issues, not to 
pursue the details of their solution.

>  (Note that I'm assuming you believe the problems are bad enough that they
>  are systemic.  

Entirely.  It's been an issue since the IESG was put in charge.  It has 
nothing to do with individuals and everything to do with the way we have 
structured (and run) things.

>  See above.  I just looked at section 2.6 of RFC 3774 and it does not seem
>  to discuss the sorts of problems that lead to my comment.  If I'm missing
>  something please point me at it.

as you noted, the group vectored over to process issues.  we like doing that, 
these days.

  d/
  ---
  Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Mon, 25 Apr 2005 19:01:18 -0700

>  Organizations rarely improve by having vague comments about abuse of power
>  tossed around.  If you are looking to improve the process I suggest that
>  you raise specific objections to specific actions.  

Sam,

1.  Apparently you missed the extended, public exchanges about these issues, 
over the last 3 years.  I am not suggesting anything new or different, merely 
observing that we have done not one thing to attend to them.

2.  The pressures against citing specifics is extreme.  And the IESG/IAB get 
predictably defensive. Those who take their specific concerns public are 
roundly punished.  And, yes, that's another vague claim.  However note that 
you chose to issue a public dismissal about my "vague language" rather than 
actually pursue the matter through a constructive channel.  (No one who has 
watched the IETF list for any amount of time would seriously suggest that this 
is a reasonable forum for pursuing such details.)

>  found that collecting examples together and pointing out that each example
>  taken individually is fine but that the trend is problematic.

We had an entire working group that expressed these concerns.  

How quickly we forget.

  d/
  ---
  Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Tue, 26 Apr 2005 09:08:56 -0700

(addendum)

Sam,

>  something please point me at it.

I went back and reviewed that text.  It cites exactly the point I am raising 
here:

  2.6.3.  Procedural Blockages

   The current procedural rules combined with the management and quality
   roles of the ADs can lead to situations where WGs or document authors
   believe that one or two ADs are deliberately blocking the progress of
   a WG document without good reason or public justification.  

That seems pretty clear to me.

You might also want to review the I-D draft-huston-ietf-pact-00, from 2002. 

   I-D:
     <http://www.watersprings.org/pub/id/draft-huston-ietf-pact-00.txt>

   presentation:
     <http://www.potaroo.net/presentations/2002-10-30-ietf-pact.ppt>

It was produced by a group of experienced IETF participants who thought the 
problems were serious enough and long-standing enough to warrant real change.  

Unfortunately, the reaction from some of IETF management could only be called 
paranoid ("What do you REALLY want?")  So, of course, the proposal got 
derailed.

For another perspective on fixing these sorts of issues, take a look at:

 <http://www.watersprings.org/pub/id/draft-bradner-ietf-proc-ideas-00.txt>

I cite it to show that, again, these concerns are not just from one or two 
troublemakers.

As with a diverse working group, purely undirected discussion generally has 
the effect of defeating proposals, since there are always objections to any 
interesting idea.  Countering this requires disciplined management of the 
discussion, with the clear and strong goal of making real progress.  

Changes involving the IESG will only take place when the IESG takes a 
constructive role in making this happen. So far, the only change that it has 
encouraged has been to make working group chairs handle more of the 
administrative load. While that change is a good one, it has nothing to do 
with the kinds of authority changes that need to take place.

It is certainly true that working groups sometimes produce bad 
specifications.  It is therefore essential that the late-stage quality 
assurance process remain.

However the process is sometimes used for nit-picking, second-guessing, and 
personal preference. 

Since those using the process that way have a veto, this really is a 
problem.

  d/
  ---
  Dave Crocker

---

From: Sam Hartman <[email-masked]>
Date: Tue, 26 Apr 2005 12:13:52 -0400

>> I did not find claims that specific IESG members had used the
    >> discuss power to advance personal agendas.
    >> 

    Dave> Well, no, you probably did not miss the specifics that you
    Dave> are looking forward.

    Dave> No one is silly enough to stand up publicly, refer to AD X
    Dave> and declare "j'accuse".  That is sometimes done in private
    Dave> communications to the nomcom, but those are confidential of
    Dave> course.
I'm certainly silly enough to stand up and say such a thing if I
thought it were true and if I thought that I had no other recourse.

An argument made in this thread is that nomcom is ineffective.  I was
assuming, perhaps incorrectly,that you subscribed to that view.  If
you come to the conclusion that nomcom doesn't work and are still
interested in fixing the problem, it seems like you have to show
people that nomcom doesn't work.  I don't know how to do that in a
fair and reasonable way without specifics.  (I guess you might argue
that including specifics will not lead to a reasonable discussion.
Perhaps true; I think everything else will be more unreasonable).

If you do think nomcom is effective then you are completely right that
telling nomcom confidentially about problems with particular ADs is a
fine way to go about making change.

I think however that you and I disagree about how to conduct
organizational introspection.  It is certainly a topic on which
reasonable people disagree.

I'll close with one thought about RFC 3774.  As far as I can tell,
section 2.6 does not actually make the claim that ADs were abusing
their discuss power; it claims that there was a strong perception ADs
were doing so.  The solution to that problem is *not* to change how
discusses work.  The solution is to gather the necessary data to
determine whether the perception reflects reality or whether we have a
communication/education problem.

I think we're making strides in that direction.  The tracker helps a
lot.  Changes in IESG culture help too.  There is low tolerance for
poorly specified discusses right now and very low tolerance for
discusses that have not been written up.

--Sam

---

From: Keith Moore <[email-masked]>
Date: Tue, 26 Apr 2005 12:53:36 -0400

> otherwise-essential job of quality assurance has been taken to the
> extreme of  having ADs block completed work that has been developed in
> all the proper  ways.  This includes AD vetos (which are
> euphemistically called "discuss")  with non-normative issues or
> requirements for lists of adjunct -- ie, not  required -- work.

Speaking from personal experience as an AD for four years, I found it 
extremely difficult to use the "discuss" mechanism to block shoddy work 
by working groups.  There was considerable pressure from certain other
ADs to accept work that clearly did not meet RFC 2026 criteria, and 
if an AD tried to push back on such work with "discuss" the likely 
response was that the "shepherding" AD and/or WG would sit on it for 
several months or years, make no substantive changes, and blame the 
"discussing" AD for blocking their work.  In the end the document would 
usually get approved without the problems being fixed.

I understand that WGs that have been working for several years tend
to be exhausted and burned out and have difficulty doing good technical
work at all - and even more difficulty with a "reset" that would 
revisit old technical decisions even if those were poor decisions.
So in my mind, the problem isn't so much with the IESG process for
final approval of those documents - the problem is that WGs are allowed
to spend years pursuing approaches that have serious technical flaws
without any cross-area review or pushback. 

Keith

---

From: Dave Crocker <[email-masked]>
Date: Tue, 26 Apr 2005 09:56:46 -0700

>  I'm certainly silly enough to stand up and say such a thing if I thought
>  it were true and if I thought that I had no other recourse.
>
>  assuming, perhaps incorrectly,that you subscribed to that view.  

Most years, I put my name into the nomcom list.  I've been on nomcom twice. 

I believe in the process, but I also believe it needs significant changing.

The current design of the nomcom process works well when there is a large 
percentage of the volunteer pool that has extensive experience with both the 
technology and the IETF process. This makes it likely that a useful portion of 
the resulting nomcom will be savvy about IETF issues. 

Over the years, that percentage has gone down dramatically. This has resulted 
in recent nomcoms that have very little real IETF process experience among the 
voting members.  

That is a guarantee that a) those voting will be naive, and b) that the 
non-voting members -- ie, the representatives from existing IETF management -- 
will have undue sway.
 
Again, there have been proposals to make changes to the process, but they have 
gotten predictably derailed.

>If you
>  come to the conclusion that nomcom doesn't work and are still interested
>  in fixing the problem, it seems like you have to show people that nomcom
>  doesn't work.  I don't know how to do that in a fair and reasonable way
>  without specifics.  

I did that, Sam.  But I chose to do it without pointing at specific nomcom 
decisions that were a mistake, since I'd rather focus on the issues than get 
sidetracked by all the outrage at personal attacks.  (Do you see a 
double-bind, here?  Get criticized for personal attacks if I'm specific.  Get 
criticized for vagueness if I don't.)

So, instead, I am trying to point out both clear and direct design flaws, and 
citing patterns of concern that have been expressed.  

When there is a pattern of concern, there is a problem that needs fixing.

>  telling nomcom confidentially about problems with particular ADs is a fine
>  way to go about making change.

Actually, the degree of confidentiality in the current problem is a serious 
problem.  First of all, how can the average participant know who to make 
comments on?  Second, I will repeat that the social barrier against making 
explicit criticisms is really quite high.  The problem with your own 
willingness to express your views -- or mine, or many others -- is that it 
makes us entirely insensitive to the position of "average" participants. 

Further, the confidentiality ensures that a dialogue about concerns does not 
happen.

So, I chose to cite a common rationale used by the nomcom ("there were no 
other choices") and suggest that the regular occurrence of that rationale is 
sufficient indication of a deep, serious problem that needs addressing.

>  organizational introspection.  It is certainly a topic on which reasonable
>  people disagree.

I suspect our experience base is rather different.

>  I'll close with one thought about RFC 3774.  As far as I can tell, section
>  2.6 does not actually make the claim that ADs were abusing their discuss
>  power; it claims that there was a strong perception ADs were doing so.

SAM!! -- In the real world of organizational process, you are making a 
distinction without a difference.  I fully understand the nature of the 
distinction you are raising, and I think it is an important question to ask, 
but it is NOT reasonable to have that question derail efforts to make change.

When a community holds a perception that there is a problem, then there is a 
problem.

>  The solution to that problem is *not* to change how discusses work.  The
>  solution is to gather the necessary data to determine whether the
>  perception reflects reality or whether we have a communication/education
>  problem.

As I said, our base of experience with organizational and political processes 
is clearly quite different.

>  I think we're making strides in that direction.  The tracker helps a lot.

No we are not.  Not it does not.  

It has nothing at all to do with the kind of issues I've been raising.  

The predilection for confusing administrative change with changes to 
underlying process has been an effective way to avoid dealing with real 
issues.

>  Changes in IESG culture help too.  There is low tolerance for poorly
>  specified discusses right now and very low tolerance for discusses that
>  have not been written up.

Hmmm.  Well, now that you cite that, I do realize that those issuing a Discuss 
are far more accountable now than they used to be.  In olden times, it 
sometimes was not even possible to find out who issued the Discuss!

However, the mere fact of a single-AD veto is the problem.  

That we now are assured of knowing who is doing the veto merely means that we 
can engage in infinite, unproductive exchanges about their concerns.  There is 
no way to really counter the veto, when it is silly, other than an appeal.  

At the least, an appeal should occur only after extensive effort to resolve 
issues in other ways.  The result is that, at the least, there will be -- and 
is -- significant delay.

But again, please remember that going against IETF management is extremely 
intimidating for most folks.

  d/
  ---
  Dave Crocker

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Tue, 26 Apr 2005 14:23:22 -0400

On Tuesday, April 26, 2005 09:56:46 AM -0700 Dave Crocker 
<[email-masked]> wrote:

> percentage of the volunteer pool that has extensive experience with both
> the  technology and the IETF process. This makes it likely that a useful
> portion of  the resulting nomcom will be savvy about IETF issues.
>
> Over the years, that percentage has gone down dramatically. This has
> resulted  in recent nomcoms that have very little real IETF process
> experience among the  voting members.

That is indeed a problem.  I think a nontrivial part of the solution should 
be to encourage people who do have that experience to volunteer.  One thing 
I've noticed in organizing various kinds of volunteer activities is that 
you get better response when you ask or encourage people individually to 
participate, rather than relying solely on broadcast announcements such as 
the ones sent to this list and ietf-announce soliciting nomcom volunteers. 
Of course the announcements are necessary, but they are not sufficient.  I 
think we might get a better mix if people with an interest in a well-run 
IETF (but who are NOT members of the current managemet) would personally 
encourage qualfiied, experienced individuals to volunteer for nomcom, as 
well as doing so themselves.

> When there is a pattern of concern, there is a problem that needs fixing.

True.  But the problem may not be the one there is concern about.  If there 
is a pattern of concern that the sky is falling, the problem may be a lack 
of education concerning the nature of the atmosphere.  There is far too 
much tendency to make changes just to be seen to be "doing something", 
regardless of whether the changes are appropriate or effective.

> other choices") and suggest that the regular occurrence of that rationale
> is  sufficient indication of a deep, serious problem that needs
> addressing.

This is indeed a problem.  The time commitment required for these positions 
is so large that it is essentially a full-time job.  That means the nomcom 
has to try to find someone who is qualified, is willing to set aside 
whatever they're working on for 2 years, and has a source of funding for 
that period of time.  Few employers are willing to provide that kind of 
support, and few individuals (or at least, few individuals qualified to 
serve in the IETF leadership) can afford to do so on their own.

This is a problem that has been mentioned many times, with many proposed 
solutions of varying practicality and effectiveness.  I'm not going to 
comment here on any of those proposals, but I do think the problem needs to 
be addressed.  This is not easy, though, and will require some study and 
significant involvement from present and past holders of those jobs.

> That we now are assured of knowing who is doing the veto merely means
> that we  can engage in infinite, unproductive exchanges about their
> concerns.  There is  no way to really counter the veto, when it is silly,
> other than an appeal.

We rarely have appeals, and yet my impression is that most documents which 
reach the IESG do progress.  Maybe you're right, and the reason is that 
people are so intimidated by the prospect of going up against the 
management that they manage to address whatever the AD in question is 
unhappy about.  And maybe, in some cases, that's even a good thing.

However, I'm not entirely convinced that the unrestricted veto really 
exists.  Before I can think about solutions to this problem, I need to 
reexamine the process and convince myself that it really is a problem.

-- Jeff

---

From: Lakshminath Dondeti <[email-masked]>
Date: Tue, 26 Apr 2005 14:42:17 -0400

Hi,

Some comments on the Nomcom related topics:

"Inexperienced people on Nomcom"

My experience was that most, if not all, of the people in this year's 
Nomcom are quite experienced in the IETF processes and care about the 
IETF.  We have, I think, at least 4 WG chairs, and most if not all 
members have gone through the RFC process as an editor/author.   The 
IETF could consider revising the selection process to ensure that 
similarly experienced people will be on future Nomcoms, but I think that 
has the potential downside of decreasing the pool even more, and might 
also result in same old faces being on Nomcom again and again!

Note that many people are not even interested in providing 
input/feedback on current ADs or potential candidates, much less 
interested in serving on Nomcom (the common answer to not providing 
feedback: I am not interested in the politics; I work mostly with the 
other AD :-)).

"Influence of Liaison Members"

Yes, I do agree that the liaisons have a good amount of influence.  But, 
there is also a counterbalancing process, namely, providing input to 
Nomcom via email (anonymously if necessary): I for one would give equal 
importance to the input received from the community, especially if the 
input is backed up with examples: e.g., AD x had Zillion Discuss's and 
most of his/her comments are only disruptive, for instance eg1: "   ", 
eg2: "  ".  That will raise a flag, so Nomcom members can pay extra 
attention to the AD in question.

That leads into this most important point:

"Complain about an AD, but also nominate candidates who are interested 
in the job" or 'there were no other choices' might well be true

As the title indicates, it is not sufficient to just complain about an 
AD (I guess it might be sufficient in the "Recall" process), it is also 
necessary to provide a pool of, or just one for that matter, candidates 
who are interested and qualified.  Yes, I have real examples.  (May I 
suggest that Nomcom procedures be revised to make the final candidate 
list, or at least the number of interested candidates for each position, 
be made public?)

Even when the Nomcom does not replace an AD, I am told that the Nomcom 
chair provides a compilation of the feedback from the community to the 
IESG.  In the future, it might be worthwhile to store a copy of the 
feedback for consideration of the Nomcom considering the same candidates 
for a future term:

% Complaints about an AD, but not enough to replace him/her ---> 
feedback provided ---2 yrs later ---> work with the previous "state" as 
opposed to starting from scratch.

Perhaps it is time to tweak the Nomcom process a little bit and write 
down some of the advice that chairs seem to be handing down (recall that 
past chair is one of the non-voting members).

regards,
Lakshminath

---

From: Edward Lewis <[email-masked]>
Date: Tue, 26 Apr 2005 15:05:31 -0400

I'll chime in on this, having been a NomCom'er at one time.

>"Inexperienced people on Nomcom"

In my year, there were quite a few who would fit this category.  I 
found these folks to be quite objective and a refreshing source of 
questions.

The one hindering factor was that they, because of not having the 
history, did not know the direction in which to "search."

>"Influence of Liaison Members"

And that is where the liaison members came in.  Liaison is kind of a 
misnomer, the data transfer is one way - into the NomCom.  NomCom 
Liaisons do not (hopefully) take NomCom data back to the other 
organization.

>in the job" or 'there were no other choices' might well be true

Especially for NomComs that are thin on experience.  NomCom'ers don't 
have an obvious way to recruit candidates.  The only approach is 
based on "who you can think of" (besides those whose name are 
submitted) - which winds up being a popularity contest.  A 
meritocracy it isn't.
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis                                                +1-571-434-5468
NeuStar

If you knew what I was thinking, you'd understand what I was saying.

---

From: John C Klensin <[email-masked]>
Date: Tue, 26 Apr 2005 16:21:21 -0400

--On Tuesday, 26 April, 2005 14:23 -0400 Jeffrey Hutzelman
<[email-masked]> wrote:

>> So, I chose to cite a common rationale used by the nomcom
>> ("there were no other choices") and suggest that the regular
>> occurrence of that rationale is  sufficient indication of a
>> deep, serious problem that needs addressing.
> 
> This is indeed a problem.  The time commitment required for
> these positions is so large that it is essentially a full-time
> job.  That means the nomcom has to try to find someone who is
> qualified, is willing to set aside whatever they're working on
> for 2 years, and has a source of funding for that period of
> time.  Few employers are willing to provide that kind of
> support, and few individuals (or at least, few individuals
> qualified to serve in the IETF leadership) can afford to do so
> on their own.
> 
> This is a problem that has been mentioned many times, with
> many proposed solutions of varying practicality and
> effectiveness.  I'm not going to comment here on any of those
> proposals, but I do think the problem needs to be addressed.
> This is not easy, though, and will require some study and

Jeff,

There are some ways to fix, or at least improve, parts of this,
but there doesn't seem to be any will in the community to do so.
I'm not going to say much that wasn't said during the Problem
Statement exercise, but perhaps a review is in order.

First, the IESG job has become so burdensome in part because of
decisions by the IESG about how much work they need to do and to
what level of detail they need to address documents.  To take a
handy, but deliberately non-specific, example, if the IESG
considers it appropriate to hold a document up over the wording
of a title or abstract, and to debate that wording, the cycles
that go into the discussion are, however indirectly, a
contribution to the IESG's becoming a full time job.   I would
rather see the IESG fix that than having the community apply
whatever blunt instruments it has --and there are some signs
that the current IESG might make some steps in the right
direction.  But, to the extent to which the Nomcom selects
people who are happy having the IESG be a full time job, rather
than dedicated to getting that workload down, the Nomcom and the
community are endorsing what we then complain about.

Second, there are well-known phenomena in many political systems
that lead people to say "well, that body consists mostly of
scum, but my representative is a good guy".  The US Congress has
turned into a particularly striking example, but it is by no
means unique.    The attitude creates a very strong bias toward
reselection of incumbents.   I suggest that the community, and
the Nomcoms, end up suffering from the same phenomenon and that,
if we don't like the symptoms that seem to come with people who
have been on the IESG for too long, we need to start thinking
about ways to adjust the system to prevent the "too long"
situation.

Finally (at least for this note), the rate at which incumbents
are returned if they are willing to put up with more punishment
suggests that Nomcoms are asking not whether someone new could
do the job about as well but whether they could do a better job
than the incumbent.  Unless an incumbent is perceived as having
really screwed up, the answer to that question will almost
always be "no".  If nothing else, IESG roles are complex and
anyone new will lose some number of weeks or months reading into
the role.  Also someone new is always an uncertainty and it is
rational and normal for, e.g., a WG Chair to advise the nomcom
that a particular AD should be returned based on logic of "I
have figured out how to work with this one, even though it is
sometimes difficult; I wouldn't look forward to breaking in a
new one, risking a worse relationship, etc."   In addition to
the "too long in position" issues raised above, this interacts
with the workload and volunteers problem: Indicating
availability for one of these positions, and then staying
available, is itself a big commitment.  That is a commitment not
just for an individual but for a company.  It is tolerable if
there is a reasonable expectation of getting the position and
having an opportunity to serve.  But, if the odds are high that
the incumbent will be returned no matter what, the incentives to
get the needed approvals and resources lined up are really very
poor.  We are hence seeing problems convincing people to put
their names in the pot against the name of an incumbent who is
doing even a passable job... and I predict that will get worse.

There are ways to fix, or at least significantly improve, things
along each of those dimensions.  But I don't think the community
is really interested enough to consider them seriously, much
less push them through.  And that, I suggest, gives us exactly
the leadership structures and behaviors we deserve and
presumably want.

     john

---

From: Lakshminath Dondeti <[email-masked]>
Date: Tue, 26 Apr 2005 16:44:51 -0400

Nomcom again, so I'll take the bait.

* On the workload aspects, I for one, as a Nomcom member, am guilty as 
charged.  We get a guidance at the beginning of the process from the IAB 
and the IESG on what the work entails.  If I recall correctly, an AD's 
job is more or less full-time, whereas an IAB member spends about 20% of 
time or more (chair and 1 or 2 more people spend, we are told, about 50% 
or more).  The general complain we hear is that the processes are 
generally slow, so I am inclined to select someone who can put the hours. 

I did make a suggestion that for some areas, it might make sense to 
appoint 3 ADs.  That is of course not the Nomcom's decision, and we were 
told that that idea was considered and rejected since it might result in 
too much time wasted in coordination between the ADs (makes sense to me, 
to an extent).

* On term limits, I would use the following rules of thumb (I am *not* 
saying I or the current or the past Nomcoms used these guidelines, but 
putting forth ideas for potential consensus):

For the 2nd term, unless there is substantially more negative feedback 
than positive, reup the AD (we are told it takes about 1 year to learn 
the ropes so to speak, so it is disruptive to keep replacing people 
before they learn to do it well).

For the 3rd term, I would place the incumbent at the same level as a new 
candidate (assuming neutral feedback on the sitting AD, or even if 
he/she is doing a very good job).

For the 4th term, I would favor new candidates than the incumbent (so 
unless the sitting AD is excellent and the area absolutely needs 
him/her, and no one else can do half-the-job etc. :-)).

If there is consensus around something like this, I think future Nomcom 
members might very well follow the guidelines.

(Disclaimer: Again reasserting that this does not mean that I or others 
are currently using the above guidelines.)

regards,
Lakshminath

---

From: Dave Crocker <[email-masked]>
Date: Tue, 26 Apr 2005 14:07:24 -0700

>  more).  The general complain we hear is that the processes are generally
>  slow, so I am inclined to select someone who can put the hours.

A significant alternative to consider is to select someone who wants to find
ways to reduce the hours needed.  

Note the kinds of examples that John cited; there are plenty of opportunities
for the IESG to do a good job, but without taking so much time out of an AD's
life. However this requires a) prioritizing what ADs will spend their time on
and, almost certainly, b) giving up some power.

>  than positive, reup the AD (we are told it takes about 1 year to learn the
>  ropes so to speak, so it is disruptive to keep replacing people before
>  they learn to do it well).

1. Again, we should consider whether it is reasonable to have "volunteer" jobs
that take a year to learn.  Or, rather, we should consider the changes
necessary to alter that.  Note that a job taking less effort might be able to
obtain candidates with more IETF experience, thereby reducing the learning
curve.

2. The idea of rough consensus is to seek a strongly dominant view. It turns
out that it also looks for a significant opposition.  The opposition does not
have to be anywhere near as much as 50%.  I suggest that the same principles
apply.  If an AD cannot garner enough support to look like rough consensus in
their favor -- or, equivalently, if the AD garners significant opposition --
then they should not be an AD.  It is not enough that there are some folks who
love the AD.  And I am not suggesting that one or two IETF participants should
have a "veto" on AD selection.  

I am suggestion that an AD (or candidate) who has developed a significant
constituency of opponents should not be selected.

>  For the 3rd term,

One of the underlying problems with the administration of the nomcom is the
serious lack of institutional memory.  Even with the liaisons and the
previous-time chair participating, each nomcom a) reinvents itself, and b) has
essentially none of the history of rulemaking, guidelines or candidate detail
that was obtained by the previous nomcom.

>  members might very well follow the guidelines.

A Guidelines for Nomcom Participants would almost certainly be helpful.

  d/
  ---
  Dave Crocker

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Tue, 26 Apr 2005 17:20:37 -0400

> community are endorsing what we then complain about.

All of this is true.  And, it's part of why I think that present IESG 
members are an essential part of any attempt to correct the full-time-job 
problem.  Note, however, that reducing the workload is not the only 
possible solution.  I suspect that even partial funding for these positions 
would make it easier for people to volunteer.

> about ways to adjust the system to prevent the "too long"

I think the US Congress could benefit from term limits.  I'm not so sure 
about the IESG.  I think a certain amount of continuity is a good thing, 
and I start to worry when we have both AD's in an area turn over in less 
than a year.  And perhaps I'm in the minority, but on the whole I tend to 
believe the IESG is doing a reasonable job, and that its biggest problem is 
being too slow.  But then, I tend to believe the IESG should be somewhat 
conservative about insisting that documents produced by the IETF be of the 
highest technical quality (which is not the same as nitpicking about the 
abstract, or references, or document formatting, or other nits.  Those 
things should be done too, but before the document reaches the IESG and not 
by the IESG).

> new one, risking a worse relationship, etc."

I'm not sure this is a bad thing.  Sure, we could decide that we want term 
limits, in which case when the term limit is up the outgoing AD is simply 
not eligible.  But lacking that, I think continuity is good, and the bar to 
remove an incumbent _should_ be a bit higher.  After all, while most 
employers do periodic performance reviews, I know of very few that 
periodically look for applicants with the intention of firing someone who 
is doing a decent job on the grounds that they found someone who might be 
better.

> In addition to

This is a problem, and it's not.  On the one hand, it's poor that the 
nomcom doesn't have a ready pool of qualified volunteers if they should 
decide to remove an incumbent.  On the other hand, if qualified people are 
not willing to put their names in the pot, perhaps it's because they don't 
think the incumbent needs to be replaced, and that's worth considering.

-- Jeff

---

From: Dave Crocker <[email-masked]>
Date: Tue, 26 Apr 2005 14:35:16 -0700

>  Note, however, that reducing the workload is not the only possible
>  solution.  I suspect that even partial funding for these positions would
>  make it easier for people to volunteer.

not really.

most people have full-time jobs.  giving them outside funding does not change
the impact on that full-time job, unless they give it up.

if the candidate is to be as knowledgeable about the profession as we would
like, then they should be valuable to their employer.  

So the real requirement is to reduce the load the IETF places on an AD.

This will permit a broader range of candidates.

  d/
  ---
  Dave Crocker

---

From: Sam Hartman <[email-masked]>
Date: Tue, 26 Apr 2005 17:38:22 -0400

>>>>> "Jeffrey" == Jeffrey Hutzelman <[email-masked]> writes:

    Jeffrey> On Tuesday, April 26, 2005 04:21:21 PM -0400 John C
    Jeffrey> Klensin
    Jeffrey> <[email-masked]> wrote:

    >> decisions by the IESG about how much work they need to do and
    >> to what level of detail they need to address documents.  To
    >> take a handy, but deliberately non-specific, example, if the
    >> IESG considers it appropriate to hold a document up over the
    >> wording of a title or abstract, and to debate that wording, the
    >> cycles that go into the discussion are, however indirectly, a
    >> than dedicated to getting that workload down, the Nomcom and
    >> the community are endorsing what we then complain about.

    Jeffrey> All of this is true.  And, it's part of why I think that
    Jeffrey> present IESG members are an essential part of any attempt
    Jeffrey> to correct the full-time-job problem.  Note, however,
    Jeffrey> that reducing the workload is not the only possible
    Jeffrey> solution.  

I'd just like to say that I'm not at all sure being an AD is a full
time job.  It certainly sometimes is.

I really haven't settled in yet and my day job and personal life has
been in enough flux I'm not sure how much time being an AD takes up or
how much time I can depend on being able to make for my day job.  I
suspect the steady state answer is less than full time and not zero.
However I'm going to need to do a bit more research to find out;)

---

From: Keith Moore <[email-masked]>
Date: Tue, 26 Apr 2005 18:36:03 -0400

> So the real requirement is to reduce the load the IETF places on an AD.

This seems like an extremely difficult problem to me.    Most of IESG's workload is in reviewing technical specifications.  I don't see any way to provide good quality technical specifications without a final review by a group of technically adept individuals with a broad range of expertise.  It's clear (to me, anyway) that we can't simply trust WGs to produce good output - most do, but maybe half of the documents have serious flaws that need some correction before publication, and maybe 10% produce documents that are so bad they are hard to salvage into something that meets standards-track criteria.  And while I'm all in favor of giving WGs earlier feedback to reduce the number of late surprises (and to help them finish their work on time), this seems more likely to increase IESG workload than to decrease it. 

I do suspect that improvements in tools have made life a bit easier since I was on IESG.  But I don't think the tools can help ADs review documents.  As for external reviewers, I generally found that using external reviewers on WG documents just made it harder to get a compromise. 

To significantly reduce the load that the IETF places on an AD basically means one or more of the following:

- fewer WGs (and a higher bar for chartering new WGs)
- fewer documents (say, a quota on the number of documents a WG can produce)
- shorter documents (say, a per-document page quota)
- fewer WG participants
- a further degradation in document quality

...none of which are particularly attractive for an organization that derives its operational funds from meeting fees and whose reputation (and therefore effectiveness) depends on the number and quality of the documents it produces.

Even if you can somehow reduce the time required for an AD to do things that are more-or-less mandatory (reviewing specifications, reviewing charters, participating in telechats and face-to-face meetings, communicating with WG chairs and individuals) there are still likely to be a number of things that don't get enough attention - like keeping up with WG mail traffic and WG documents, trying to get errant WGs back on track, etc.

So to me it looks like being an AD is going to continue to be a full-time job.  We can and should try to arrange things so that ADs spend more of their time doing _useful_ work.  But while it might make the AD job less frustrating, it probably won't make it less time-consuming.

Keith

---

From: Joe Touch <[email-masked]>
Date: Tue, 26 Apr 2005 16:06:07 -0700

> like, then they should be valuable to their employer.  

Sure - but most people have to pay for their time out of some sort of
pot. Doing AD work while being on fed out of other pots puts the AD in a
tough spot of doubling their work. The solution is to be relieved of
some of their other work.

Reducing the load helps, but not that much; there aren't other jobs,
including TPC chairs of large conferences, that take anywhere near this
much time.

> So the real requirement is to reduce the load the IETF places on an AD.

IMO, this is definitely a requirement, but funding support wouldn't hurt
either.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 26 Apr 2005 16:08:23 -0700

Keith Moore wrote:
> This seems like an extremely difficult problem to me. Most of IESG's
> workload is in reviewing technical specifications. I don't see any way
> to provide good quality technical specifications without a final review
> by a group of technically adept individuals with a broad range of
> expertise. 

Other groups solve this by distributing that load - which also solves
the problem of individuals with bones to pick. They get outnumbered ;-)

> - a further degradation in document quality

Why isn't a larger number of "ADs" - or, more specifically, removing the
review process from the ADs and having a real review group, the solution
here?

Joe

---

From: Dave Crocker <[email-masked]>
Date: Tue, 26 Apr 2005 16:20:40 -0700

>  review process from the ADs and having a real review group, the solution

1. the repeated assessment has been that the aggregate size of the iesg is at 
its limit, for getting anything done.

2. if the work being done has too much effort on the wrong tasks, it does not 
help things to have more people doing the wrong things.

  d/
  ---
  Dave Crocker

---

From: Keith Moore <[email-masked]>
Date: Tue, 26 Apr 2005 19:52:02 -0400

> review process from the ADs and having a real review group, the solution

The more ADs there are, the more things get bogged down at the IESG
level. ADs need to come to a mutual understanding about what are
reasonable criteria for acceptance of a document (in general and/or for
a specific document), and this is harder when you have more people who
have to agree.  More ADs also mean that telechats take longer.

Also, the more ADs there are to provide some kind of "guidance" to a WG,
the more confusing things become for that group.  It would be a bit
like a business having multiple managers with direct authority over the
same employee - if the managers don't coordinate well the employees get
caught in the middle.

Moving the process to a real review group at best just moves the
problem - since the WG is ultimately interested in getting the
documents by the review group, those individuals become the people whom
the WGs will consider their managers.  So the IESG either becomes
irrelevant (in which case we have essentially moved the IESG's
functionality to the review group) or it starts trying to produce meta-
advice - something close to architectural guidance (in which case it
overlaps IAB's role).

Note that I didn't say that it couldn't be done, I said it was a
difficult problem.

For our current management structure there's an optimal number of ADs.
Too few and they're spread too thin, too many and they spend too much
time coordinating, trying to compromise, and generally trying to discuss
things.  I think we're pretty close to that number, perhaps a bit over
that number.  Two ADs per area seems to be almost managable and to 
have some advantages over a situation where some areas have only one
AD, though if the area is small and non-contentious a single AD can
probably do the job.

If you try to create a different structure that involves more people,
it's a losing game unless you figure out a way to minimize the need for
those people to coordinate with one another.  If you divide them up too
much, you risk inconsistent review or inadequate cross-area review.
You might end up with some benefits - like higher quality documents -
because you end up with a structure that forces the organization to
establish better and more specific criteria for standards acceptance.
You might also end up creating more delay in producing standards output
because it takes time to establish and document those criteria.  But
it's not the case that adding more people will inherently reduce
workload, and it's quite easy to have everyone's workload increase
without actually producing better output or getting more throughput.

And again, folks who are serious about reducing organizational overhead
need to think seriously about whether we really can afford to support 
every working group that some group of people want to establish.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 26 Apr 2005 20:00:59 -0400

> help things to have more people doing the wrong things.

the overall statement is true.  it's not clear that the premise is true.

Keith

---

From: Joe Touch <[email-masked]>
Date: Tue, 26 Apr 2005 17:02:27 -0700

> its limit, for getting anything done.

I wasn't advocating for more ADs, but for more 'virtual' ADs, i.e., to
move the work of reviewing out of the ADs, and let the ADs distrbute the
reviews and collect and interpret the results.

> help things to have more people doing the wrong things.

I agree on this; IMO, if the ADs spend too much time reviewing, then
that's self-correcting - review less. I've never understood the 'review'
portion of this; I do understand the coordination with the IETF, but the
review is supposed to happen at the lower levels inside the WGs.

I.e., either the ADs are "PC chairs" or not involved in a PC at all; I
don't see their role as PC members.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 26 Apr 2005 17:06:25 -0700

> Note that I didn't say that it couldn't be done, I said it was a

Only if we view this top-down. If the role of the ADs is to make sure a
doc gets reviewed, they can hand it to a WG or a separate "non WG set of
reviewers".

Guidance is just that; not control. The IESG doesn't become irrelevant,
but it ceases to be a program committee for documents, which is, I
believe, a good thing.

Joe

---

From: Keith Moore <[email-masked]>
Date: Tue, 26 Apr 2005 20:42:22 -0400

> doc gets reviewed, they can hand it to a WG or a separate "non WG set of

as I said before, this doesn't solve the problem, it just moves it.

there absolutely needs to be a program committee for documents if we care
at all about quality.   it doesn' t have to be IESG, but any program committee
is going to have most of the problems that IESG has.  and I don't believe
that you can entirely separate management/oversight and review.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 26 Apr 2005 21:41:21 -0400

> reviews and collect and interpret the results.

This is _more_ work for the ADs, not less, because the ADs have to read the reviews in addition to the documents.   You might ask why the ADs'
reviews are better than the virtual ADs' reviews.  It's not inherently the case, of course, but the "real" ADs will be exposed to more information that informs such reviews, and the "real" ADs are in a better position to work out technical compromises that helps keep different protocols from fighting with one another.  (for instance, keeping linklocal addressing or site-local addressing from doing harm to application-level interoperability).

What you are advocating would help if the majority of documents presented to IESG were not close to being acceptable for standards track.  But that's not the case.  The majority of documents are close to being acceptable - maybe 10% are not close.  But it can take a lot of work for those documents that are "close" to be made acceptable.  Some of the work required to craft technical compromises could perhaps be offloaded to virtual ADs.  And when you have truly non-controversial documents that don't require cross-area review you can offload those reviews.  But the IESG still needs to know what is in the document before it can make a determination as to whether cross-area review is necessary, and you can't really ask (say) a layer 3 person to make a determination as to whether this protocol will affect layer 7 concerns - particularly when layer 7 concerns are so diverse.

There's also the problem of how to provide incentives for those virtual ADs.  The Apps area had a directorate for many years with the idea that some reviews could be offloaded to the directorate.   It didn't work very well, because those people were busy too, and it was hard to find people who could reliably turn around a review of a difficult document in time for the IESG telechat.  They could (and did!) review the short/easy/noncontroversial documents, but there aren't that many of them, and they're not the ones that take up all of an AD's time.

(Aside: I really do think that a limit on the number of documents, and a page limit on normative technical specifications, might go a long way toward lessening AD workload and also helping WGs produce well-written specifications.  But there would need to be some way of making exceptions, as there will be a few documents that really do need to be long.)

> review is supposed to happen at the lower levels inside the WGs.

In my entire time in IETF, it's never worked that way.  Nor can I understand how it could work that way and produce reasonable output.  WGs are too focused on their immediate goals to think about the wider impact of their work, and they tend to interpret difficult technical requirements in such a way as to make them seem conveniently irrelevant.   It's easier to ignore problems than to solve them; it's easier to design a protocol that only works in a limited set of conditions than to design one that works on the global Internet.

Again - if you want to lower IESG workload, figure out how to encourage WGs to produce shorter and better quality output.

Keith

---

From: Doug Royer <[email-masked]>
Date: Tue, 26 Apr 2005 20:31:29 -0600

> reviews and collect and interpret the results.

I would agree on one point. Document reviewers seem to me would
help. Most of the initial feedback (at least for my '1' case)
was editorial and not technical. The technical feedback came
later.

So perhaps some kind of procedural reviewer would help the AD's
from what might be repetitive and common feedback? When
those are fixed, then the AD's read them for technical content.
The AD's would not have to worry about reading a hundred or so
pages more than needed.

Or were my initial submission issues rare?

-- 

Doug Royer                     | http://INET-Consulting.com
-------------------------------|-----------------------------

               We Do Standards - You Need Standards

---

From: John C Klensin <[email-masked]>
Date: Tue, 26 Apr 2005 23:34:59 -0400

Keith,

With the understanding that I completely agree that this
wouldn't be easy, especially in the transition, I think parts of
your discussion miss a key option or two.  I was struck by an
earlier comment of Dave Crocker's to the effect that we've had
problems since all of the authority got handed to the IESG.  I'm
not sure whether I completely agree with that, although, in
retrospect, I can identify at least the seeds of some of these
issues from back when Dave and I were both on the IESG (i.e.,
even before your time there).  

What follows is not a recommendation that we do anything, only
an outline of a different way of thinking about the issues that
might be worth considering, if only just because it generates
more options.

What we did post-Kobe was to take the IESG, which had previously
just been a "manage the WGs and IETF procedures"  group and add
standards approval, process change approval, and so on.  There
was some discussion during the Problem Statement effort to the
effect that the IESG had then gone forward and invented new
responsibilities and authority for itself, thereby adding
further to the workload, but, even if that were true, the
increment was insignificant compared to the immediate post-Kobe
changes.  And, for better or worse, the administrative
reorganization has now explicitly added a certain amount of
external administrative process oversight to the mix.

There was also some discussion during the Problem Statement
effort that this aggregation of responsibilities onto the IESG
created some conflict of interest problems.  To the credit of
everyone who has served on the IESG in recent years, it hasn't
happened as much, or had as serious an impact, as one might have
feared, but having an IESG member as manager, shepherd, and
advocate for a WG and its products involves an inherent conflict
with the type of cold-hearted and objective review that is
desirable for quality-focused reviews.

I don't know if it is the right thing to do, or if this is the
right time, or how to make the transition, but we could rethink
the notion that a single "IESG" is the right body to manage WGs
and process, to make decisions about process and processing
rules, and do manage and conduct review of documents proposed
for standardization or other forms of publication.  If we
thought it was appropriate and split some of those things up
into separate bodies, we would almost immediately change the
IESG's workload and might, in addition, change the job
description to the point that a larger range of people would be
qualified to serve on any given one of those bodies.

But maybe these wider ranges of options are, at some point,
worth thinking about rather than assuming that any effort to
spread the load out would inherently result in lower quality.

     john

--On Tuesday, 26 April, 2005 21:41 -0400 Keith Moore
<[email-masked]> wrote:

>> I wasn't advocating for more ADs, but for more 'virtual' ADs,
>> i.e., to move the work of reviewing out of the ADs, and let
>> the ADs distrbute the reviews and collect and interpret the
>> results.
> 
> This is _more_ work for the ADs, not less, because the ADs
> have to read the reviews in addition to the documents.   You
> might ask why the ADs' reviews are better than the virtual
> ADs' reviews.  It's not inherently the case, of course, but
> the "real" ADs will be exposed to more information that
> informs such reviews, and the "real" ADs are in a better
> position to work out technical compromises that helps keep
> different protocols from fighting with one another.  (for
> instance, keeping linklocal addressing or site-local
> addressing from doing harm to application-level
> interoperability).
> 
> What you are advocating would help if the majority of
> documents presented to IESG were not close to being acceptable
> for standards track.  But that's not the case.  The majority
> of documents are close to being acceptable - maybe 10% are not
> close.  But it can take a lot of work for those documents that
> are "close" to be made
>...

---

From: Dave Crocker <[email-masked]>
Date: Tue, 26 Apr 2005 20:36:44 -0700

>  The majority of documents are close to being acceptable - maybe 10% are
>  not close.  

there is a small question about the criteria that you might be using.

the current ietf's track is quite poor, both with respect to timeliness and 
quality.  quite simply we are taking a long time to turn out lots of 
specifications that tend not to get used very much.

so it is unclear that the considerable costs of the current 'quality' 
mechanisms are working very well.

you might say that that means we need to get the 10% to a smaller number, but 
my point is that the 90% is being approved without much benefit to the 
community (or, apparently, much harm) so it's not clear that all that effort 
to enforce quality criteria imposed by individual area directors is all that 
useful...

except for frustrating the heck out of all those hard-working working group 
participants who took so long to create the now-delayed specification...

  d/
  ---
  Dave Crocker

---

From: Keith Moore <[email-masked]>
Date: Wed, 27 Apr 2005 00:22:34 -0400

I'm not sure how useful it is to compare today's situation with the 
pre-Kobe days.  A lot of circumstances have changed since then.  In 
particular,  IETF is slightly larger and much more diverse, its 
audience is much larger and more diverse,  the target environment is 
several orders of magnitude larger and considerably more diverse, and 
the set of threats to applications and the network has also grown 
considerably.   Internet protocols are subject to much more pressure of 
every kind these days than they were then.  Engineering for today's 
environment is considerably more difficult than it was 10+ years ago, 
and even if we considered our way of managing the process good back 
then (which is debatable) it's not clear that it would serve us well 
now.

Actually I think our biggest problems stem from a deeply entrenched 
institutional culture that believes that endless unstructured 
discussion among a group of self-selected narrowly focused 
participants, without significant external supervision, and typically 
without any of the other steps associated with other disciplines of 
engineering, is a good way to do protocol design.  To the extent it 
worked in the pre-Kobe days, it might have been more because of 
favorable circumstances than because of the superiority of that way of 
doing things:

- the Internet was small and relatively uniform and easy to understand;
- the IETF was small, with less turnover, and most of us knew each 
other;
- if you were talented and wanted to do Internet work, IETF 
participation was almost a no-brainer - the barriers to entry 
(financial and otherwise) were low and the working atmosphere was 
friendly;
- working groups were fewer and focused on less ambitious tasks;
- there was far less time-to-market pressure;
- the target environment was much less demanding and hostile, making 
the design job much easier;
- there was less pressure to bend/subvert the architecture; and
- there were a lot more academics and generalists involved in IETF who 
were naturally inclined to think about cross-area issues.

All of this discussion reminds me of the Nasrudin story where he's 
looking for his keys on the ground outside his house, because the light 
was better there, even though they were inside his house when he last 
saw them.  People keep focusing on IESG when the biggest problems are 
at the WG level.

Keith

---

From: Keith Moore <[email-masked]>
Date: Wed, 27 Apr 2005 01:24:44 -0400

> was editorial and not technical. The technical feedback came

I can imagine that virtual ADs might be very useful for reviewing early 
WG drafts (long before the WG thinks it is done), identifying 
potentially contentious issues and needed cross-area review, giving 
feedback to the authoring WGs about those, and informing the real ADs 
about the state of the document.

Keith

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 27 Apr 2005 09:18:30 +0200

Dave Crocker wrote:
...
> observing that we have done not one thing to attend to them.

That's pretty unfair to my predecessor and his colleagues.

Here's a quick list of things that have been done. It's written in somewhat high
level terms, but there is substance behind each of these items. It doesn't
mean that we're done or that we're complacent, but accusing ourselves of
inaction is plain wrong.

Tools team in place to specify and help provide or enhance various efficiency tools

Many operational details of document review process have been and will be improved
(in other words, the ID tracker is being constantly improved; the PROTO
shepherding system and the General Area review team are operational.
But I, Dave and ICAR blew the early review issue so far.)

Education team in place to continuously educate leaders and participants

Regular reviews of RFC Editor and IANA performance in place

New procedures for liaison handling defined

Administrative support unit being created by ISOC
(the IAOC is up and running and the IAD hiring process is
moving right along)

Proposals for upgrading/streamlining standards track in discussion
(i.e. newtrk and specifically the ISD proposal, but there's
certainly more to do in newtrk)

    Brian

---

From: Jari Arkko <[email-masked]>
Date: Wed, 27 Apr 2005 13:54:44 +0300

I would like to support Brian's point that we have actually
done quite a lot in recent years to improve the IETF. Not
saying that we shouldn't do more -- we really need to -- but at
least from my perspective there has been a significant,
observable change for the better.

--Jari

Brian E Carpenter wrote:

> Dave Crocker wrote:
> ...
>
>> 1.  Apparently you missed the extended, public exchanges about these 
>> issues, over the last 3 years.  I am not suggesting anything new or 
>> different, merely observing that we have done not one thing to attend 
>> to them.
>
>

---

From: Jari Arkko <[email-masked]>
Date: Wed, 27 Apr 2005 13:59:38 +0300

Hi Lakshminath,

> AD (I guess it might be sufficient in the "Recall" process), it is 
> also necessary to provide a pool of, or just one for that matter, 
> candidates who are interested and qualified.  Yes, I have real 
> examples.  (May I suggest that Nomcom procedures be revised to make 
> the final candidate list, or at least the number of interested 
> candidates for each position, be made public?)

I like this suggestion. But first: I'd rather call this thread "feedback"
than "complaining", because I hope the nomcom gets a lot of input
and not just when someone is doing badly.

But back to the suggestion. I have beeing trying to send a lot of
input on various positions and candidates to the nomcom in recent
years. But from the point of view of a regular IETF participant this
isn't always easy. Basically, the problem is that we have a lot of
input to give you, but we lack the data about the candidates!

Of course, we can easily give you feedback on the current AD.
But we've had a large number of people leave the IAB and IESG
recently, and it isn't easy to provide feedback about potential
candidates. Sometimes I tried to do that, just to be surprised that
the people I commented on weren't even running or someone
I didn't know or didn't consider as a potential candidate was in
the process. The nomcom goes out to the area chairs and other
contributors and solicits feedback, revealing at least some of
the potential candidate names. This helps, but its fairly limited.
Or at least I would have wanted to give more feedback on more
areas than I received questions from the nomcom.

I would suggest that (agreeing) candidate lists be made public
early in the process, in order to make it easier for the IETFers to
provide you feedback. This would also increase the transparency
of the process. And yes, I am aware of the argument that some
candidates might be shy to reveal that they are running for the
job. But we are a major organization, and I would suggest that
the benefits for the organization outweigh benefits (if any) for
the candidates.

--Jari

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 27 Apr 2005 14:07:46 +0200

> reexamine the process and convince myself that it really is a problem.

A DISCUSS isn't a veto. I've seen numerous cases even in my short experience
as an AD of DISCUSSes leading to, er, discussion and a new draft (or edits)
that solves enough of the issues that the AD removes their DISCUSS. I don't
think I've seen any new DISCUSSes that I would categorize as frivolous or
individualistic.

What we do have is a long tail on the distribution of the dwell time of
DISCUSSes. I haven't had a chance to analyze the root causes of the very
old DISCUSSes.

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 27 Apr 2005 14:26:20 +0200

...
> the benefits for the organization outweigh benefits (if any) for

Just a fact: this point *was* debated at length during the discussion
of what became RFC 3777. And the WG Chair called the consensus. Of course,
the consensus may be different when we next revise it.

    Brian

---

From: Eliot Lear <[email-masked]>
Date: Wed, 27 Apr 2005 14:36:40 +0200

> specifications that tend not to get used very much.

I think we can each find examples on either side of this assertion. 
MIME took quite some time to get right, for instance, but it's a mighty 
fine standard.  SIP similarly so.  IPv6?  Remains to be seen.  There are 
others that are less tested and some that are probably unused, but if 
everything we did was used it would mean that our bar was so high to get 
stuff through that even more good stuff could have been done.  It would 
be better to have some sort of analysis of this.

> mechanisms are working very well.

Fair enough assertion, and an area in which I'm concerned having been 
told by IESG members previously that they've held up documents just 
because they were concerned about premature ossification.  However, the 
other side of this argument is that it is onerous to update a standard, 
and I think it is more due to real installed base matters that IETF 
procedures.  Again, IPv6 probably shows the the most extreme example of 
this.

> to enforce quality criteria imposed by individual area directors is all that 

You need to back up your estimate.  It's not reasonable to just accept 
those numbers without some analysis.

> and, almost certainly, b) giving up some power.

Depends on who gets that power.  If it's the WG chair we have any number 
of examples where a WG really doesn't do the job well enough for 
release, the most famous being the one I related earlier.

Eliot

---

From: "Spencer Dawkins" <[email-masked]>
Date: Wed, 27 Apr 2005 07:43:21 -0500

FWIW, there was the separate suggestion that NOMCOM publish the NUMBER 
of candidates who agreed to be considered, and this seems helpful 
without setting off the usual alarms...

From: "Brian E Carpenter" <[email-masked]>
To: "Jari Arkko" <[email-masked]>
Cc: <[email-masked]>; "Dave Crocker" <[email-masked]>
Sent: Wednesday, April 27, 2005 7:26 AM
Subject: Re: Complaining about ADs to Nomcom (Re: Voting (again))

---

From: Edward Lewis <[email-masked]>
Date: Wed, 27 Apr 2005 08:58:34 -0400

>and not just when someone is doing badly.

 From my experience - I'd call it complaining. ;)

Rarely are compliments offered without a prompt.
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis                                                +1-571-434-5468
NeuStar

---

From: Edward Lewis <[email-masked]>
Date: Wed, 27 Apr 2005 09:09:05 -0400

At 7:43 -0500 4/27/05, Spencer Dawkins wrote:
>FWIW, there was the separate suggestion that NOMCOM publish the NUMBER of
>candidates who agreed to be considered, and this seems helpful without setting
>off the usual alarms...

When I sat on the nomcom, we tried to get more information about 
willing candidates by submitting a list padded with non-willing 
candidates to obscure the true list.  If we are going to that length 
to protect those under consideration, I think giving the number of 
"competitors" might not be desirable.  (What if it's 1?  What if it's 
2?  What if you realize that you lost to so and so in a head-to-head 
race?)

If I had my naive druthers, having been on a committee with a lot of 
"fresh blood" - I would rather make the list of candidates known and 
open (like the IETF) to invite targeted comments.  Selections ought 
to be objective - that's the way we want the technology to be.  Given 
the context of "fresh blood" - this makes people who are diligent 
able to make better decisions without all of the guess work borne 
from not having experience (= pre-conceived notions).

In a really open and secure process, "losers" would be given a dump 
on why they lost.  Like a student getting an F - it's fair to tell 
someone how to improve for the next test.  (Another naive opinion of 
mine.)

I can understand why the list might be private - not willing to 
reveal who is available, who "lost", etc.
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis                                                +1-571-434-5468
NeuStar

---

From: Joe Touch <[email-masked]>
Date: Wed, 27 Apr 2005 07:41:14 -0700

Keith Moore wrote:
> This is _more_ work for the ADs, not less, because the ADs have to
> read the reviews in addition to the documents. 

That argues that there should be no PCs, only PC chairs, which isn't the
case.

The only way to releive work is to distribute it, not concentrate it.
The exceptions to this are based on expertise - and I don't buy that for
AD slots. There are too many non-expertise issues in their appointment.

> You might ask why the ADs'
> reviews are better than the virtual ADs' reviews. It's not inherently
> the case, of course, but the "real" ADs will be exposed to more
> information that informs such reviews, and the "real" ADs are in a
> better position to work out technical compromises that helps keep
> different protocols from fighting with one another. (for instance,
> keeping linklocal addressing or site-local addressing from doing harm to
> application-level interoperability).

ADs often don't have the time to track all the WGs and can end up
holding docs unnecessarily as a result. And they're not uniquely privvy
to the concept of compromise or negotiation; their "power" should come
from technical expertise, which is distributed in the IETF, not from
fiat (I didn't see white smoke from the Nomcom).

> What you are advocating would help if the majority of documents
> presented to IESG were not close to being acceptable for standards
> track.  But that's not the case.  The majority of documents are
> close to being acceptable - maybe 10% are not close.  But it can
> take a lot of work for those documents that are "close" to be made
> acceptable.  Some of the work required to craft technical
> compromises could perhaps be offloaded to virtual ADs.  And when
> you have truly non-controversial documents that don't require
> cross-area review you can offload those reviews.  But the IESG
> still needs to know what is in the document before it can make a
> determination as to whether cross-area review is necessary, and you
> can't really ask (say) a layer 3 person to make a determination as
> to whether this protocol will affect layer 7 concerns -
> particularly when layer 7 concerns are so diverse.

ADs are not the only ones who can look or work at mutliple layers; in
fact, their focus works against them in many cases in this regard.

And there are PLENTY of docs that need review that aren't standards
track that the IESG is just plain holding up.

> There's also the problem of how to provide incentives for those
> virtual ADs.

Ditto for PC members. Being a member is the incentive; the IETF is a
volunteer organization. If you can find a dozen or so people to stop
what they're doing and be ADs, you can certainly find more people who
will participate if they can do so at a lower level of commitment.

> The Apps area had a directorate for many years with the
> idea that some reviews could be offloaded to the directorate. It didn't
> work very well, because those people were busy too, and it was hard to
> find people who could reliably turn around a review of a difficult
> document in time for the IESG telechat. They could (and did!) review the
> short/easy/noncontroversial documents, but there aren't that many of
> them, and they're not the ones that take up all of an AD's time.

OK, so we have ONE example where it didn't work. Let's throw it out. ;-)

The issue with "directorates" is that they turn into papal conclaves (to
reuse the allusion). They need to be more open, but they can work fine.

> (Aside: I really do think that a limit on the number of documents, 
> and a page limit on normative technical specifications, might go a
> long way toward lessening AD workload and also helping WGs produce
> well-written specifications. But there would need to be some way of
> making exceptions, as there will be a few documents that really do
> need to be long.)

Page limits are fine, but won't lessen the work, just the chunksize.

> In my entire time in IETF, it's never worked that way. Nor can I
> understand how it could work that way and produce reasonable output. WGs
> are too focused on their immediate goals to think about the wider impact
> of their work, and they tend to interpret difficult technical
> requirements in such a way as to make them seem conveniently irrelevant.
> It's easier to ignore problems than to solve them; it's easier to design
> a protocol that only works in a limited set of conditions than to design
> one that works on the global Internet.

While I can't speak for all WGs, I can speak for the ones I participate
in more heavily, and they certainly never operate that myopically. What
DOES happen is that we solicit cross-area review, or send authors to
other WGs to present docs before the IESG review.

Then the IESG says "did you take this to X?" - when we had years
earlier. The excuse was basically "we can't track EVERYTHING".  Sadly,
this is exactly the thing you expect ADs to fix, and IMO it's they are
problem in this regard, not the solution.

The other issue is more of "hey, *I* have a technical problem with this
doc", regardless of whether it has already had cross-area review and
consensus. In which case we end up trying to appease the AD to sate
their personal perspective, not the IETF or the Internet as a whole, and
often at the expense (not to the credit) of the broad vision of the
global Internet.

Joe

---

From: Dave Crocker <[email-masked]>
Date: Wed, 27 Apr 2005 08:41:02 -0700

Brian,

> >  issues, over the last 3 years...  

>  Here's a quick list of things that have been done. It's written in
>  somewhat high level terms, but there is substance behind each of these
>  items. It doesn't mean that we're done or that we're complacent, but
>  accusing ourselves of inaction is plain wrong.

I did not say there had been no activity.  I said that we had done nothing 
about the issues currently under discussion.  

So i will ask you and everyone else to consider your list with respect to 
quality, timeliness and relevance. 

Most of the actions you cite may affect some of the problematic aspects of 
timeliness... sometimes. But that is all. 
  
Here's an exercise you might want to consider:  Take the list in the Problems 
RFC and try to reconcile your list against its.  

>  Tools team ...
>  Many operational details of document review process ...

These two are about better administration.  Improving administrative process 
is fine, but has nothing to do with problems in the "semantics" of the 
process.  

It does not affect quality or relevance.  And it does not do much about 
timeliness, in terms of making working groups go faster, reducing 
inappropriateintentional blocks by ADs, or any of the things that are the meat 
of IETF work.  

>  and the General Area review team are operational.

And since it is carefully targeted for the END of the working group effort, 
how will this improve quality, timeliness and relevance?

>  But I, Dave and ICAR blew the early review issue so far.)

Since this was an effort directly targeting quality and timeliness -- and 
especially since early reviews seem to have succeeded at gaining IETF rough 
consensus as a Good Thing to do -- do you have an theory about the failure to 
get this going, or better still, how to get it to succeed?

>  Education team in place to continuously educate leaders and participants

And you should note that I cited that elsewhere in the thread.  

Whether the actual content of the training is likely to have any impact on 
quality, timeliness and relevance is worth assessing.  Yes, it ought to, but 
how are we going to figure out whether it does?

>  Regular reviews of RFC Editor and IANA performance in place...
>  New procedures for liaison handling defined...
>  Administrative support unit being created by ISOC...

And you think these affect core issues of IETF utility to the Internet 
community?

You think that these changes will have any effect at all on better IETF 
participation or producing timely specifications of better quality and 
relevance?  

Please explain how.  

You might consider using the Problems RFC as a reference guide for directing 
such an assessment.

>  Proposals for upgrading/streamlining standards track in discussion (i.e.
>  newtrk and specifically the ISD proposal, but there's certainly more to do
>  in newtrk)

Another derailed activity.  Another activity that has nothing to do with 
quality, timeliness or relevance.

Go ahead, explain how it does.  

Explain how current document labeling practises hurt the IETF's utility to the 
Internet community.

Explain how the utility of the IETF to the community will be improved by our 
fixing this.

Brian, we need to distinguish activity from progress.  

We need to look for the issues that are at the core of the IETF's 
problems.  

As I said, in the last 3+ years, we have mostly chosen not to do 
that.

  d/
  ---
  Dave Crocker

---

From: Keith Moore <[email-masked]>
Date: Wed, 27 Apr 2005 11:59:23 -0400

> The only way to releive work is to distribute it, not concentrate it.

False.  You can also relieve work while keeping throughput constant by
reducing overhead.  

Distributing work often reduces throughput by creating more overhead.
Only a few applications are embarassingly parallel -  most do not scale
well as the number of PEs is increased.

> fiat (I didn't see white smoke from the Nomcom).

Just by doing their jobs, the ADs are inherently exposed to a wider
range of issues than other IETF participants, probably including those
in the IAB. Nobody else gets to concurrently see the problems that
exist at every level of the protocol stack. 

> fact, their focus works against them in many cases in this regard.

That's not "fact", that's just your conjecture.  And my experience
contradicts it.  ADs tend to become less focused over time, while these
days many individuals in IETF work only in a particular niche.  Every
AD sees considerably more documents from outside his area than inside.
They don't review every document of course, but they do participate in
discussions about every document.

> And there are PLENTY of docs that need review that aren't standards track that the IESG is just plain holding up.

Generally, those docs need review because they will be viewed as
standards or as conflicting with standards.  And not surprisingly,
working group and non-standards track documents have priority.

Something that people seem to miss is that for any document that goes
before IESG, only one AD has to review that document.  The other ADs
are free to say "no objection". Those who read the document do so
because they believe that the document needs scrutiny, not because
their jobs require them to do so.  IESG people are more acutely aware
of the potential for documents to create problems than most other IETF
participants, so they're going to take time to review documents that
seem to have the potential to create problems.

To the extent that ADs can entrust those reviews to other people, they
are able to do so and always have been able to do so.  And this works
well for small documents that only affect a particular niche.  But for
many documents there aren't many reviewers who have both the time/
energy and a sufficiently broad perspective to replace the AD's own
review and to make the IESG's deadlines.  Some of the documents I
reviewed while on IESG took ten to twenty hours of intensive time to
review - they were that long and complex and it required multiple
passes to see whether the loose ends were tied up.  IESG people get
committment from their employers to support that work.  Try saying to
someone whose employer hasn't made that committment - not a student,
but someone who is experienced and has a broad perspective, "I need you
to take ten or twenty hours away from your day job to review this
document, and I need your review by a week from this Thursday".  How
often do you think that will work?

> If you can find a dozen or so people to stop

No, it doesn't follow.

> OK, so we have ONE example where it didn't work. Let's throw it out. ;-)

sure, let's ignore the evidence and depend on conjecture :)
 
> reuse the allusion). They need to be more open, but they can work fine.

Directorates can be structured however the ADs want them to be
structured. We tried a variety of structures in Apps, including
completely open groups focused in particular areas (messaging,
directories, web).  They were even less useful than the "papal
conclave" style directorates that they replaced.  

> Page limits are fine, but won't lessen the work, just the chunksize.

which is why you need document limits also.

> other WGs to present docs before the IESG review.

If all WGs did frequent cross-area review, and documented that
review, IESG's job would be much easier.  In my experience, presenting
docs before other WGs doesn't work as effectively.  It requires a
continuous dialogue. A problem with both approaches is that there's not
always another WG to represent the area of concern - which is an
artifact of our "everything has to be a working group" mentality.

> earlier.

years earlier isn't good enough.  the Internet changes too frequently.

> often at the expense (not to the credit) of the broad vision of the

I've seen two kinds of cases - one where the AD has a legitimate 
technical problem with the document, often a problem that isn't visible
at the WG level because those people don't have a broad perspective
(or because the WG has actively engaged in denial about that problem).
I've also seen cases where an AD objected to a document on what 
appeared to be a mere whim, without stating technical reasons for
doing so, or without brooking any kind of technical discussion.  These
days where the ADs comments are visible to anyone who wants to
look at the document tracker, I suspect it's harder for an AD to get
away with that - and there are more grounds for appeal when the AD
does try that.  But because there are often cases where an AD sees
things that the narrowly-focused WG simply cannot see, I don't think
we can dispense with AD review.  And offhand I don't see how to 
build up a pool of reviewers who have that same level of perspective.
That doesn't mean we shouldn't try, but it probably does mean that
we shouldn't depend on it.

Keith

---

From: Joe Touch <[email-masked]>
Date: Wed, 27 Apr 2005 09:39:50 -0700

> well as the number of PEs is increased.

Agreed, but this isn't a computer; it runs very much like a program
committee, for which there are hundreds (if not thousands) of examples
of how to organize things more effectively.

> exist at every level of the protocol stack. 

Just by doing their jobs ADs focus on the area they direct; they can
be(and have been) just as myopic to that area as individuals working on
problems.

Very few of us mortals do NOT see the problems at other levels of the
protocol stack.

> contradicts it.

It's my experience that contradicts yours, not a conjecture. I wouldn't
bother with any of my observations were they hypothetical.

> ADs tend to become less focused over time, while these

So you're arguing for a single monolithic PC where only PC members
provide reviews (common among workshops), whereas I am arguing for
focused groups, which is the more common program committee model for
conferences, esp. where scale is an issue.

> working group and non-standards track documents have priority.

Those docs are reviewed to see IF they have such issues; it's not
possible to know whether such conflicts MIGHT exist until the doc is
reviewed.

> seem to have the potential to create problems.

The assertion that the IESG is more acutely aware than plenty of people
in the IETF is a nice assertion, but that's all it is. There's plenty of
evidence that they aren't aware of interactions or prior work of (in
some cases their own) WGs.

> often do you think that will work?

About 5,000 times a year for Infocom; plenty of us spend time on program
committees and provide plenty of reviews that take many hours in *some*
cases.

>>will participate if they can do so at a lower level of commitment.

It follows as much as other assertions that are being tossed about as
fact: reducing the level of effort increases the number of people who
are willing to volunteer that effort.

> sure, let's ignore the evidence and depend on conjecture :)

The evidence - overwhelming - is that this process works in plenty of
other places. The personal experience is that a single case has been
shown not to work.

We have cases where Nomcom failed, ADs failed, etc. - are we ignoring
the 'evidence' to keep those processes?

> conclave" style directorates that they replaced.  

Well the current system of ADs isn't useful in some cases either;
something has to change somewhere, and no system is going to be perfect.
The question is whether it spread the load and was *sufficient*, not
whether it was more useful than overloading a single AD. As noted
before, single ADs have other properties - some have been in the same
role for a decade - that give them context that isn't as much a function
of being an AD as spending 80% of their career for the past 10 years
reviewing IETF docs.

> which is why you need document limits also.

Sure - so who reads the document proposals and decides which ones we
review? Seems like a circular argument...

> review, IESG's job would be much easier.

If they believe the documentation. But when IDs get sent to the IESG
*with* documented cross-area review and get asked "did you ever present
this to WG X?" - when WG X is in the list - things get just plain
ridiculous. (and no, this isn't conjecture either)

> In my experience, presenting

So now you're arguing that the ADs are there for content review, rather
than to direct things to other WGs or groups. This is the problem I have
with the current structure - it basically says "rough consensus, running
code, AND the individual approval of the IESG".

We don't believe in kings, and IMO, the IESG have too much king-like
power in the current structure.

> years earlier isn't good enough.  the Internet changes too frequently.

In the case I'm citing it was "we had years earlier, and had been
working with ever since".

> (or because the WG has actively engaged in denial about that problem).

This is why ADs should participate in the WGs; I thought in general that
they did (or at least tried to). THAT is where the cross-area stuff
should occur, not later when the WG has finished (and sometimes disbanded).

> look at the document tracker

Not all of the detailed comments. Some comments refer to private
discussions.

>, I suspect it's harder for an AD to get

As I don't want to depend on an equally myopic pool of ADs either. I
wish I trusted the ADs perspective as much as you, but I simply don't;
there are too many preconditions on AD selection that have nothing to do
with expertise (availability, notably).

Joe

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Wed, 27 Apr 2005 19:32:33 +0200

Dear John and Brian,
thank you for this review. Brian quotes real efforts, John real life images 
(like US Congres). I feel the difference is the press coverage. Don't you 
think this debate would be different if what is debated and obtained at the 
IETF got press coverage and therefore public exposure and positive or 
negative recognition?

We are daily innundated by press technical hear-says from all over the 
world .... except from IETF. Don't you think that if IETF was really 
important and clear to the market it would not receive a proper press 
coverage with a dedicated magazine and columns in the specialized press all 
over the world. With articles in general press on important issues to the 
people.

I assume that the IETF work can be important. So the alternative is it is 
not clear to the press. The lack of relation with users, real world, etc. 
is often assumed as an IETF "plus" (I disagree with this: I think it is an 
inability to think how much it is important to communicate rather than a 
plus?). The debate on the IASA shown it: no interest in PR, no interest in 
local ISOC Chapter support, etc.

I will take a single example: http://www.ietf.org/iesg.html - if when you 
click on IESG Members names you got a resume, a picture and some vision 
statement it could be quite interesting for the press (and for members - 
all the more those not sponsored by a big employer and not being able to 
vacation three times a year at IETF meetings). I compare with 
http://iab.org/about/members-plus.html (which not easy to find) ... beut a 
beignning ... some are on both pages.

I submit that if press covered the IETF debate, we would discuss of what 
interests the markets, in the way the market is interested, at the speed 
the market is interested. Only because the payroll of some in here would 
depend on the image they would give of their employer.

And when I say "market" I also say Governments. IAB correctly identified 
that only them could sponsor the necessary Internet R&D (RFC 3869). I 
submit that a press coverage of their draft would have risen a few 
questions about the lack of allusion to multilingualism, while the USA are 
in the middle of the multilingualism year and 192 Governments made it a 
priority at the WSIS.

The 4th power is often our best protection in real life. I feel it could be 
the same at the IETF.
jfc

PS. Brian, I found an interesting English word. "particularism": "a 
principle of allowing each state in a nation or federation to act 
independently of the central authority, especially in promoting its own 
economic interests".

---

From: Pekka Savola <[email-masked]>
Date: Wed, 27 Apr 2005 21:18:04 +0300 (EEST)

> will participate if they can do so at a lower level of commitment.

Are you sure you can find the people?

What went wrong with ICAR (Improved Cross-Area Review)?  It was closed 
due to lack of activity; 12 people enlisted as reviewers, just a 
single review conducted, the reviewers didn't deliver.

See:

http://www1.ietf.org/mail-archive/web/icar/current/msg00293.html
http://www.machshav.com/~icar/
http://www.machshav.com/~icar/reviews/people/

It is not clear you can just simply find people to actively contribute 
in this kind of activity.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: "Adrian Farrel" <[email-masked]>
Date: Wed, 27 Apr 2005 19:30:03 +0100

> single review conducted, the reviewers didn't deliver.

I think your conclusion that "the reviewers didn't deliver" might be taken
poorly by the reviewers on the list who weren't asked to deliver.

Don't blame the reviewers if no I-D authors or WG chairs asked for
cross-area review. That only one review was conducted might be taken to
mean that only one review was requested. What does that tell us about the
process?

Adrian

---

From: Keith Moore <[email-masked]>
Date: Wed, 27 Apr 2005 14:50:45 -0400

okay, this is getting way too long, and starting to get repetitive and
even personal, so I'm going to summarize:

1.  A review structure that works for conferences doesn't necessarily
work for IETF.

Conferences and IETF have different goals.  IETF tries to do
engineering and produce technical consensus behind particular
solutions.  Conferences try to stimulate thought and discussion.  
IETF tries to get closure; while in a conference, some amount of
controversy is even desirable.  Conference paper reviews are just 
trying to keep the signal-to-noise ratio high; IETF reviews are trying
to see if the design is complete, appropriate design choices were made,
and if the various protocols can peacefully coexist in the network. 

Conferences and IETF also draw from different constituencies,
and IETFers may have more constraints on their time than those
who attend conferences and submit papers to conferences.

2.  IESG's scaling problems are a direct result of low-quality output
from working groups, and we can't do much to address that problem 
by changing how IESG works.  

3. I don't think we can make IESG significantly larger, I don't think
we can dispense with final document review and keep document quality
up, and I don't think that additional reviewers can signficantly
relieve IESG of the need to do final review.  I do think that
additional reviewers could be very valuable in giving WGs feedback from
early drafts, keeping them on the right track, and keeping IESG
informed about the status of the WGs.  I also think that a document
that has enjoyed such review and feedback throughout its life cycle
will be much easier for IESG to review, and that (without any changes
to IESG's organization or process) it will be harder for IESG to reject
such documents without sound technical justification.

4. Broad based (not just "cross-area") review is essential and IETF 
doesn't have adequate structures to do this at present.  WGs generally
need to have short lifetimes and to stay focused on deliverables,
while issues of conflict between concerns exist for years after a WG
has finished.

5. These days, most IETFers are narrowly focused.  What this means
is that we have to actively cultivate a broad view among people
entrusted to do broad review.  IESG turns out to (accidentally) be a
mechanism for cultivating broad views, but it's not the only possible
mechanism for doing so.

I suspect we would agree that the ADs alone aren't sufficient.  But I
think the primary benefit of additional reviewers is in early review
rather than late review - I think we want the early review both to give
timely feedback and correction to WGs and to give the ADs confidence
that the WGs have stayed on track and done their homework, thus making
the late reviews easier.

---

> power in the current structure.

Start giving IESG real rough consensus (rather than consensus by
exclusion and/or exhaustion) and real running code (or even
better, convincing analysis that the protocol will work well in the 
wild), and they won't feel the need to rule by edict.  IESG appears to
rule by edict because WGs demand that IESG provide them with very
concrete feedback. Simply saying "you failed to provide security" or
"you failed to address the concerns of this other group that you'll
harm their interoperability" doesn't work - either the WG will balk or
they'll sit on the document for months not being willing to fix the
problems. So ADs try to come up with good technical solutions in order
to get closure.  If the WG likes the solution all is well; if they
don't like the solution they complain about ADs who act like kings.
But mostly, WGs  get what they deserve.  ADs like nothing better than
to be presented with well-written, focused documents from groups that
have obviously done their background work and given it appropriate
consideration in their designs.  

Keith

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Wed, 27 Apr 2005 15:24:10 -0400

> they'll sit on the document for months not being willing to fix the

All true.  This is unfortunate.

> So ADs try to come up with good technical solutions in order
> to get closure.

But _this_ is the problem.  The AD's job should be to review and manage, 
not to personally try to find or design the missing pieces.  This task 
could and should be offloaded to a separate pool of volunteers.

-- Jeff

---

From: Keith Moore <[email-masked]>
Date: Wed, 27 Apr 2005 15:58:06 -0400

> could and should be offloaded to a separate pool of volunteers.

No, it's a symptom.  In the current environment, often the best thing
an AD can do is to come up with the solution.  Of course the AD can
ask volunteers to help if he/she can find people that can bring closure
to the effort in a short time (and this does sometimes happen when
the AD determines that a particular kind of expertise is needed).
But the AD is already in the loop and knows what the constraints 
are; the volunteers typically will not - unless you somehow find a way
to cultivate a new group of IETF participants with broad expertise.

We shouldn't blame the ADs for doing what (usually) works best in their
situation; instead we should find ways for WGs to produce better
documents so the ADs don't have to do that.

---

From: Joe Touch <[email-masked]>
Date: Wed, 27 Apr 2005 13:24:06 -0700

> even personal, so I'm going to summarize:

And I'll provide some final comments, since the summary isn't
particularly unbiased.

> 1.  A review structure that works for conferences doesn't necessarily

And it doesn't necessarily not work either. It has not been generally
applied, though.

> Conferences and IETF have different goals. 

Correctness is a part of conferences too; consensus not as much, but ADs
applying their personal feedback is not consensus.

...
> who attend conferences and submit papers to conferences.

In more than a few cases, they're the same groups. Where they are not, I
beleive IETFers tend to have less since many in that group are paid to
attend or participate.

> by changing how IESG works.  

IESG's scaling problems are also a result of taking on too much personal
responsibility to provide individual feedback rather than to delegate.

> such documents without sound technical justification.

All conjectures ;-)

The issue isn't whether there's sound technical justification, but
whether things get held up until that's addressed, even with ongoing
review and feedback from other WGs.

We either need to make the IESG larger (real or virtual by delegation)
or make their work smaller (by encouraging feedback to be directional -
as in 'take to WG X' - rather than technical review).

> while issues of conflict between concerns exist for years after a WG

I agree; a cross area review group would help, and certainly needs to be
a structure separate from the WGs.

> I suspect we would agree that the ADs alone aren't sufficient.

Indeed.

> But I

I agree, except that I don't see what the purpose of the late reviews
is; IMO, an ADs time is better spent pointing out which WGs might
overlap or have issue with a document than in providing specific
technical criteria for correcting things (unless that WG and the doc
editor disagree).

> wild), and they won't feel the need to rule by edict.

That isn't always sufficient. Sometimes they rule to appease
individuals, sometimes that individual is the AD. Either way, there's
more going on than just good protocol design.

> IESG appears to

While that is true for some ADs, others act for political reasons,
either based on the sway of individuals or a perceived need to 'play' in
a space, despite whether the WG has had consensus to do so.

I would prefer a bottom up organization that helped us create better,
more coordinated protocols than the top-down one that we have now.

Joe

---

From: Keith Moore <[email-masked]>
Date: Wed, 27 Apr 2005 16:50:16 -0400

> And it doesn't necessarily not work either. It has not been generally

Lots of things haven't been generally applied.  For instance, we haven't
tried evaluating IETF submissions by weight.

> applying their personal feedback is not consensus.

ADs apply their personal feedback out of the need for closure;
it's also not clear that an AD's personal feedback is worse than
a conference reviewer's personal feedback.  A conference reviewer
might not get to veto a paper, but an AD's DISCUSS isn't a veto
either.
 
> responsibility to provide individual feedback rather than to delegate.

No, that is a symptom.  The problem with ADs providing individual
feedback is exacerbated by working groups reaching the point of
exhaustion before they get significant external review and generally
being unwilling to accept significant changes at that point.
Delegating the reviews won't solve the exhaustion/denial problem, it
will just add delay. 

> > such documents without sound technical justification.

All informed by several years of experience, including 4 years on IESG.
(which you will probably want to dismiss as mere bias :)

> as in 'take to WG X' - rather than technical review).

Having WGs produce higher quality output will reduce IESG's workload
and reduce the time spent in final review.

By the time a document reaches IESG, the kind of feedback you 
recommend is often too late.

> technical criteria for correcting things (unless that WG and the doc

It's to make sure that the document adheres to 2026 criteria before 
publication.

> more going on than just good protocol design.

I am not saying that ADs will never misuse their power.  That's what
the appeals process is for.  I'm saying that under the current situation
the vast majority of AD "edicts" (as opposed to directed feedback)
are the result of WGs reaching the point of exhaustion without 
producing good designs.   Fix that problem and it becomes reasonable to
expect fewer and less onerous AD "edicts" and to push back on those
edicts more often.

> more coordinated protocols than the top-down one that we have now.

We already have a bottom-up organization.  It's because the bottom
is failing to do its job that the top has to play cop.

Keith

---

From: Joe Touch <[email-masked]>
Date: Wed, 27 Apr 2005 13:54:54 -0700

Keith Moore wrote:
...
> is failing to do its job that the top has to play cop.

Nobody died and made the IESG cop. They took it upon themselves, and
that's not how things (should) work in the IETF.

Joe

---

From: Fred Baker <[email-masked]>
Date: Wed, 27 Apr 2005 13:59:37 -0700

On Apr 27, 2005, at 11:50 AM, Keith Moore wrote:
> from working groups, and we can't do much to address that problem by 
> changing how IESG works.

I agree and disagree. We have rather a history of ADs sitting on 
documents and of ADs micromanaging working groups (not every AD and not 
every case, but enough of both that it has been something the IESG has 
actively had to address) that chairs feel un-empowered. This is in part 
what the "shepherding" thing is intended to address - widen the review, 
close loops, keep things out of cracks, etc. So changing the way the 
IESG works is in fact part of making the WG responsible for its output.

> IESG appears to rule by edict because WGs demand that IESG provide 
> them with very concrete feedback. Simply saying "you failed to provide 
> security" or "you failed to address the concerns of this other group 
> that you'll harm their interoperability" doesn't work - either the WG 
> will balk or they'll sit on the document for months not being willing 
> to fix the problems.

As stated, this sounds adversarial. While there have been adversarial 
relations with some WGs, I don't think that generalizes. In many cases 
where I have delayed updating a draft, it was because it wasn't clear 
to me what was being asked for, or there was no tickler that told me 
that the comments had been posted. "You failed to provide security" is, 
if you think about it, a pretty content-free statement. A better 
statement would be "I believe that this is open to a man-in-the-middle 
attack of this type" or "I don't see your threat analysis in the 
document".

Frankly, apart from a special cases, I think ADs sound like they are 
ruling by edict because they get a little frustrated saying the same 
thing a zillion times. The classic, which you noted earlier, is 
security. We have been trying to convince people to think about it for 
about 15 years now, and it is really only now that people are seriously 
considering taking other people to court for an egregious lack of 
thought on the topic that it is becoming an issue protocol developers 
think about. My issue with "security considerations" has always been 
that I personally am not a security expert, and dunning me for being 
open to this attack or that without informing me that the attack exists 
mostly feels to me like an attack.

You may remember an older version of the id-nits document in which I 
pulled a digest of common attacks (pulled from someone's doctoral 
thesis that I found on the net) and asked bluntly "have you asked 
yourself whether your protocol is vulnerable to ..."? I put that up 
because, frankly, several of those attacks were news to me. I notice 
that the current id-nits removes that set of questions; I think the net 
result is that people will not ask themselves about obscure forms of 
attack. But I think that approach is better than saying "you didn't do 
an adequate threat analysis"; tell people how to do a good one and what 
questions they are likely to need to answer.

> ADs like nothing better than to be presented with well-written, 
> focused documents from groups that have obviously done their 
> background work and given it appropriate consideration in their 
> designs.

Absolutely true.

---

From: Keith Moore <[email-masked]>
Date: Wed, 27 Apr 2005 17:58:37 -0400

> that's not how things (should) work in the IETF.

IESG is following RFC 2026, and it takes its responsibilities seriously.

Keith

---

From: Keith Moore <[email-masked]>
Date: Wed, 27 Apr 2005 18:13:25 -0400

Joe,

I don't agree with your interpretation of 2026.

Keith

---

From: Joe Touch <[email-masked]>
Date: Wed, 27 Apr 2005 15:07:15 -0700

> IESG is following RFC 2026, and it takes its responsibilities seriously.

Not Sec 4.2.3 for individual submissions; that one talks about checking
for conflict, not editing for content.

Ditto for Sec 6.1.1-6.1.2, which talks about checking for consistency
and clarity, but not editing for other reasons.

Ditto for Sec 6.5.1, which talks about what to do when WGs disagree. It
also mentions what to do when _individuals_ disagree about things or
raise technical points; the problem with current process is that the ADs
are acting as ADs when the speak about such issues, rather than individuals.

Were they to put their objections forth as individuals, it would be
necessary to recuse themselves from the process of conflict resolution,
because they have a vested interest in the outcome.

But I've pointed that out before, notably in Seoul.

I have no doubt that they take their responsibilities seriously; I have
doubt that there are appropriate checks and balances in this part of the
process outlined in 2026, as it is implemented by the IESG at present.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 27 Apr 2005 15:25:37 -0700

> I don't agree with your interpretation of 2026.

I knew that; the disagreement of the interpretation of 2026 is at the
heart of this issue overall.

Joe

---

From: Pekka Savola <[email-masked]>
Date: Thu, 28 Apr 2005 07:36:25 +0300 (EEST)

> poorly by the reviewers on the list who weren't asked to deliver.

That's true.. I was certainly on the reviewer list myself, and I 
was never asked to review anything :-).

However, as a WG chair asked 2-3 times through the ICAR chairs (just 
as an experiment to see if the process worked), unfortunately with 
little success.

But there may have been other reasons for the failure as well, such as 
not enough publicity, too low "rewards" for the reviewers, etc.

---

From: Pekka Savola <[email-masked]>
Date: Thu, 28 Apr 2005 07:47:57 +0300 (EEST)

> for conflict, not editing for content.

Have you taken a look at RFC 3922 ("The IESG and RFC Editor Documents: 
Procedures")?  While these were previously also dutily reviewed for 
individual RFC-editor submissions, they are no longer.  The IESG just 
produces a note to the RFC editor, and records its comments to the 
tracker (if any) -- in the interest of bringing up issues in the spec 
-- and it's then up to the RFC editor to check whether a revision (and 
what kind) is needed or not, as long as the IESG disclaimer is 
included.

> and clarity, but not editing for other reasons.

It also says, "technical quality".

> because they have a vested interest in the outcome.

If someone appeals the decision (this has happened), and it reaches 
the full IESG, the AD whose action is being appealed will recuse 
him/herself.

---

From: Joe Touch <[email-masked]>
Date: Thu, 28 Apr 2005 08:00:47 -0700

Pekka Savola wrote:
> tracker (if any) -- in the interest of bringing up issues in the spec --
> and it's then up to the RFC editor to check whether a revision (and what
> kind) is needed or not, as long as the IESG disclaimer is included.

Agreed; this was the update to the procedure that Harald noted in Seoul;
it's too early to see if it is having the desired effect on individual
submission, but I'm hopeful.

> It also says, "technical quality".

Agreed, but in a general "this is junk" way.

IMO, IESG review is taken less as a high-level quality control and check
for coordination with the IETF as a whole, and more as an opportunity
for the IESG to provide individual review. Perhaps 3992 will quash that
for individual submissions, but not for WG submissions. But, as has been
pointed out, IESG members are overloaded; IESG review is sometimes the
first time they see a document and they sometimes want input at that point.

> If someone appeals the decision (this has happened), and it reaches the
> full IESG, the AD whose action is being appealed will recuse him/herself.

Why then? IMO, the issue here is that when an AD wants to provide
individual input - not general "below threshold" or "please check with
WG X" feedback - at that point (which is MOST of what 'discuss' is about
these days, IMO) the AD should NOT be able to raise a discuss, but
rather need to forward their comments to another AD as an individual. If
the other AD feels that the comments are not sufficient to halt process,
they should not, just as for any other individual submission.

Putting the 'discuss' in the hands of ADs who are providing last-minute
individual input is a problem that can (and should) be fixed.

Joe

---

From: Dave Crocker <[email-masked]>
Date: Thu, 28 Apr 2005 08:53:41 -0700

Fred,

excellent comments.

>  where I have delayed updating a draft, it was because it wasn't clear to
>  me what was being asked for, or there was no tickler that told me that the
>  comments had been posted. "You failed to provide security" is, if you
>  think about it, a pretty content-free statement. A better statement would
>  be "I believe that this is open to a man-in-the-middle attack of this
>  type" or "I don't see your threat analysis in the document".

yes, on all counts.

>  ruling by edict because they get a little frustrated saying the same thing
>  a zillion times. 

Although I suspect there are a variety of reasons, the one you cite is 
particularly interesting, because it suggests that the iesg could generate a 
kind of 'semantic nits' document.  Of course, the issues are deeper than 
syntactic nits, but when they are consistently a problem, then dealing with 
them almost can be routinized.

>  My issue
>  with "security considerations" has always been that I personally am not a
>  security expert, and dunning me for being open to this attack or that
>  without informing me that the attack exists mostly feels to me like an
>  attack.

yup.

>  I notice that the
>  current id-nits removes that set of questions; I think the net result is
>  that people will not ask themselves about obscure forms of attack. But I
>  think that approach is better than saying "you didn't do an adequate
>  threat analysis"; tell people how to do a good one and what questions they
>  are likely to need to answer.

yup.

  d/
  ---
  Dave Crocker