Subject: Re: Re: Voting (again)

From: John Loughney <[email-masked]>
Date: Thu, 28 Apr 2005 9:12:12 +0300

Keith,

You've raised these points, over a number of years, but I wonder if it would be useful to explore implications of some of your comments:

> 2.  IESG's scaling problems are a direct result of low-quality output
> from working groups, and we can't do much to address that problem 
> by changing how IESG works.  
> 
> 3. I don't think we can make IESG significantly larger, I don't think
> we can dispense with final document review and keep document quality
> up, and I don't think that additional reviewers can signficantly
> relieve IESG of the need to do final review.  I do think that
> additional reviewers could be very valuable in giving WGs feedback from
> early drafts, keeping them on the right track, and keeping IESG
> informed about the status of the WGs.  I also think that a document
> that has enjoyed such review and feedback throughout its life cycle
> will be much easier for IESG to review, and that (without any changes
> to IESG's organization or process) it will be harder for IESG to reject
> such documents without sound technical justification.

Here, in the Problem WG and other places, you've raise the point that increasing the IESG probably won't help.  You've implied that we probably have too many working groups and too many drafts in the working groups.  The implications of these are that the IETF has too much work in too many areas to be effective. 

One remedy is to scale back the IETF; not accept so much work.   The implications could be that other SDOs pick-up the slack.  Instead of 3GPP(2)/IEEE/ITU-T bringing work to the IETF, they could do it themselves.  3GPP is doing a lot with SIP, let them handle it; 3GPP2 is using Mobile IP, let them continue the development; IEEE could handle RADIUS & EAP; ITU-T could handle AAA & QoS; let's let the ATM Forum handle all of the Sub-IP layer stuff. My feeling is that the IESG has been concerned that this would cause fragmentation and a lot of problems.  Another result of pushing back on work would mean that vendors might not document their work, or document it via vendor specifications.  We also see a rise of open source communities creating their own mechanisms for creating running code standards - stuff that is documented in Linux Kernel updates, SourceForge sites, or simply documented in running code.

I think all of the above scenarios are already happening, to some extent, today.  The question is, do we want the IETF to assume responsiblity for key pieces of the Internet or not.  If the answer is yes, then the real discussion is how to make it happen.  I think sticking with the current model because we choose not to think differently is not a good answer.

If I understand some of Dave's and John K's comments, they are willing to entertain thoughts on how to do things better (& differently) in order to ensure that the IETF remains relevant.

John L.

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Keith Moore <[email-masked]>
Date: Thu, 28 Apr 2005 09:17:20 -0400

On Apr 28, 2005, at 2:12 AM, John Loughney wrote:

> Keith,
>
> You've raised these points, over a number of years, but I wonder if it 
> would be useful to explore implications of some of your comments:
>
> Here, in the Problem WG and other places, you've raise the point that 
> increasing the IESG probably won't help.  You've implied that we 
> probably have too many working groups and too many drafts in the 
> working groups.  The implications of these are that the IETF has too 
> much work in too many areas to be effective.

I believe the IETF could perhaps take on more work if it improved the 
process by which working groups operate.  This would lessen IESG's 
burden by giving them better documents to work with; it might also 
reduce the average duration of a WG, making more room for others.  The 
industry wouldn't mind that either.  One of the problems that some WGs 
have is that they take on too many drafts, which both hinders the 
ability of the working group to finish its more important work and 
imposes additional burdens on IESG.

I also think that IETF could use its resources more effectively if it 
exercised more care about which working groups it chartered.  For 
instance, IMHO we've wasted a lot of effort trying to come up with 
short-term workarounds for NATs, without any of them providing a 
migration path away from NATs.

> If I understand some of Dave's and John K's comments, they are willing 
> to entertain thoughts on how to do things better (& differently) in 
> order to ensure that the IETF remains relevant.

As am I.  But I would like to see attention focused on working group 
operation, which I believe is our biggest source of problems.

Keith

---

From: John C Klensin <[email-masked]>
Date: Thu, 28 Apr 2005 15:59:19 -0400

Keith,

Let me offer a different perspective here as well and, in the
process, explain why I keep coming back to the IESG.

Going back almost to the dawn of IESG time, the IESG has had one
constant and primary responsibility.  That is to manage the WGs
and the WG process.  Under today's rules, they determine or
ratify which WGs get created, who chairs them and how they are
otherwise managed, what tasks and benchmarks go into charters,
how many and which documents a WG can work on at a time (and
whether they work on documents serially or in parallel), and
when to shut them down.  They have _huge_ latitude in how to
manage WGs and the decisions they make about that management
process, including a wide range of options about reporting,
review of benchmarks, actions or the lack thereof when targets
are not met, and so on.

WGs, and WG leadership, have no independent existence: the IESG,
and under some circumstances, individual ADs can dispose of them
as needed.

Personally, I think that is as it should be.  While the
community has periodically discussed constraints on WG behavior
and management (including one or two that I have written), none
have ever been approved: the IESG continues to be able to look
at WGs and their work on a case-by-case basis and to make
case-by-case decisions.  My personal view is that they (the
IESG) should have a little more guidance from the community to
aid them in making tough decisions and to assure them of backing
when such decisions are made, but that wouldn't change the
essential nature of the situation very much.

But, from that perspective, there is no "WG problem" or "problem
WG".  There is only an IESG management problem.  Either the
number and complexion of WGs is such that the IESG can manage
them effectively, or it isn't.  If it isn't, only the IESG can
be responsible.   Either WGs are sufficiently well chartered and
managed so that the review processes we have work adequately
well or they aren't.  Either way, the IESG bears ultimate
responsibility: they determine the charters, the management
structure, the review requirements or lack thereof at various
intermediate points, and so on.

I think we agree at least partially on this, because I certainly
endorse the position that we could use resources more
effectively if [we] exercised more care about which working
groups [were] chartered.  But, again, the IESG makes those
decisions: all the community can do, at least under the current
structure, is to give them advice and feedback, both about
individual proposed WGs and charters and about keeping the
totals tolerable.   But they are chartered to decide, and, if
they don't get it right, the problem belongs to them, not to the
participants in an ill-advised WG.

That isn't an easy job by any means.  I assume that, for every
WG we have, there are at least a handful of members of the
community who believe that WG contains the most important work
the IETF is doing.  But someone has to start making those hard
decisions and, as responsibilities are now handed out and groups
chartered, the IESG is the lucky group.

Now, it is reasonable to say "the IESG doesn't have bandwidth to
do that job well and still review documents for
standardization".  But that statement doesn't fix the WGs.  It
might be justification for moving document review to some other
body --presumably not delegated by the IESG but selected by the
Nomcom to fulfill that role.  Or it might be justification for
the IESG shrinking back the number of WGs to the point that they
did have time and bandwidth to do both jobs well (and maybe even
have lives and day jobs).  Or we might think about a management
structure that shifts the IESG's historical management function
wrt WGs elsewhere.

But it doesn't seem to me that saying "we have all of these out
of control WGs and need to concentrate on fixing them and not on
looking at the IESG" or even "focus our attention on WG
operation" is productive: if the WGs are not under control,
then, IMO, the IESG, as the body with the management
responsibility, needs to acknowledge that fact and then either
needs to fix it or make suggestions to the community as to how
we can fix it together.  If they are not convinced that "working
group operations" is the problem, then there is either no
problem or an IESG problem.

     john

--On Thursday, 28 April, 2005 09:17 -0400 Keith Moore
<[email-masked]> wrote:

> I believe the IETF could perhaps take on more work if it
> improved the process by which working groups operate.  This
> would lessen IESG's burden by giving them better documents to
> work with; it might also reduce the average duration of a WG,
> making more room for others.  The industry wouldn't mind that
> either.  One of the problems that some WGs have is that they
> take on too many drafts, which both hinders the ability of the
> working group to finish its more important work and imposes
> additional burdens on IESG.
> 
> I also think that IETF could use its resources more
> effectively if it exercised more care about which working
> groups it chartered.  For instance, IMHO we've wasted a lot of
> effort trying to come up with short-term workarounds for NATs,
> without any of them providing a migration path away from NATs.
> 
>> If I understand some of Dave's and John K's comments, they
>> are willing  to entertain thoughts on how to do things better
>> (& differently) in  order to ensure that the IETF remains
>> relevant.
> 
> As am I.  But I would like to see attention focused on working
> group operation, which I believe is our biggest source of
> problems.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 28 Apr 2005 17:40:26 -0700

>  I certainly
>  endorse the position that we could use resources more effectively if [we]
>  exercised more care about which working groups [were] chartered.  But,
>  again, the IESG makes those decisions:

More care about chartering is cited periodically, and I agree that it 
is needed. But we seem able to get neither a clear, concrete sense of 
what it means to use more care, nor the community resolve to pursue 
it.  Not really.

On John's latter point that the IESG makes those decisions, I suggest 
that it is at the core of at least two serious problems:  One is that 
is abrogates the community's responsibilities and the second is that 
it guarantees that the IESG remains overburdened.   

More and more, we see the general IETF community lacking a sense of 
responsibility for the health and utility of the IETF. Well, how can 
we feel responsible if we are disenfranchised? 

That is what it means to have others making the important decisions. 
The more we march down the path of having a classic, hierarchical 
authority structure, the more the IETF looks and acts like any other 
stiff, bureaucratic, unproductive standards group.  

The IETF's origins in rough consensus translated into community 
action and community responsibility.  That goes directly against the 
idea that it is some elite oligarchy's authority to make the 
decisions.

There is a long way between the highly centralized authority 
structure we now have, and the mayhem of an extreme literal 
democracy, where everybody 'votes' on everything.

But the original style of the IESG was to facilitate processes of 
developing community consensus.  Such a description is fundamentally 
at odds with a model that has that the IESG have primary authority 
for making the decisions.
  
On the question of burden, the issue is simple: ADs often feel 
sufficiently essential, to what they view as the necessary details of 
an outcome, that they are more and more inclined to inject themselves 
into it.  

(One of the more poignant examples of this is when an AD gets sucked 
into "ensuring" quality by participating in the working group as if 
they were a primary technical contributor. At that point they are an 
individual advocate for particular details, and they cease to be able 
to perform their higher-level job of oversight.)

Others in this thread have cited the IESG's job as assessing 
community consensus.  This, I think, should be viewed as the core job 
of the IESG:  coordinating macro processes based on assessments of 
community rough consensus (and looking for ways to develop it.)  

After all, any successful output from the IETF depends upon community 
adoption.  In that regard, the rough consensus model is really the 
process of acquiring community ownership of the output, with the 
expectation that the ownership makes it more likely they will 
actually use it.  Take away real community involvement in the 
decision and we lose the benefit of facilitating adoption.

For all this, the technical expertise of an AD remains extremely 
important, but not as the basis for an AD "decision".  Rather, it 
permits the AD to see problems and recruit community concurrence that 
there is a problem.  The AD may well have insight to its solutions, 
but frankly that should be viewed as an unexpected benefit. Good 
ideas come from lots of places; as good as any particular AD might 
be, no one can expect that they will have special insight that is 
better than all others.

Yet today we almost require it.  Certainly the recent structure and 
history of IETF process "trains" ADs to view their personal 
preferences as being an ultimate authority.  That is, after all, what 
the power of a veto communicates.

If an AD is unable to recruit rough consensus to their view, then 
what exactly is the benefit of their being given a veto?

  d/
  ---
  Dave Crocker
  Brandenburg InternetWorking
  +1.408.246.8253
  dcrocker  a t ...
  WE'VE MOVED to:  www.bbiw.net

---

From: Jari Arkko <[email-masked]>
Date: Fri, 29 Apr 2005 12:54:59 +0300

Keith, John C, John L,

I am in agreement with you that the main issue is how well
WGs operate in terms of their timeliness, quality, and
relevance. Just by observing what happens around, you
can see that most of the wall time spent in the creation
of specifications goes to the WG. Or at least that is my
perception in cases I have been involved. (There are exceptions
of course and I don't mean that we should stop shortening
the IESG, IANA, and RFC Editor lead times. They are still too
long.)

Another component that affects IETF's output a lot
is pre-WG time. This is harder to measure, as we don't
have an easy metric for the world-has-a-problem to ietf-
has-a-wg time.

I tend to agree with John C that the IESG shares a
part of the responsibility for the good or bad WG operation.
But I don't actually don't think IESG is that visible during the
normal operations of a WG. This may again by just my
perception, but the chairs get a lot of freedom and
responsibility for managing the group.

But where IESG and IAB have most effect is pre-WG time,
the scope and even the existence of the group. I feel
this is an area that affects perhaps most what will happen
later. If we allow things to be taken up too early, we end
up in having research results that may or may not be taken
into use. If we resist the creation of a WG too long, the
world moves on and creates other solutions or vendor
specific solutions are used. If the right problems are
chosen, we have interested contributors. If the wrong
problems are chosen, progress will be slow and on the
shoulders of one or two individuals. Etc. Sometimes
we do well here. Sometimes we don't, and I have personal
experiences from at least two cases where we really
blew it for no good reason.

Keith, you have been advocating a model where the IETF
would be stricter in allowing what work be taken up, in
order to ensure that we can actually deliver. But I share
the same opinion as John L that we should rather try to
shape the IETF so that it can deliver what the world needs.
Part of the reason why I believe so is that despite its problems,
I think the IETF produces the best technology and highest
quality. I want to use IETF multimedia, IETF network access
control mechanisms, IETF security and not something else.
This won't be easy of course, but I think we can do it. We
are extremely good engineers and we've been able to produce
scalable technology and useful, complexity reducing
abstractions. Maybe time to apply some of that for our
organization as well?

--Jari

---

From: Keith Moore <[email-masked]>
Date: Fri, 29 Apr 2005 08:21:40 -0400

> Keith, you have been advocating a model where the IETF would be 
> stricter in allowing what work be taken up, in order to ensure that we 
> can actually deliver. But I share the same opinion as John L that we 
> should rather try to shape the IETF so that it can deliver what the 
> world needs.

My primary criterion when arguing whether IETF should or should not 
take up a WG was always, in some sense, whether the Internet needed 
IETF to be involved in and supporting this effort.  It involved both an 
assessment of how much harm would result from a botched design (in 
particular, a design that didn't respect the Internet environment and 
other protocols on the net), and of whether IETF could expend the 
resources necessary to manage the group and whether it could bring the 
necessary expertise to the table.  It also involved an assessment of 
whether the proposed protocol would actually be of benefit to the 
Internet long-term.  What I didn't try to assess (much) was whether 
IETF's reputation would be enhanced by its involvement in that 
particular WG.

> Part of the reason why I believe so is that despite its problems, I 
> think the IETF produces the best technology and highest quality. I 
> want to use IETF multimedia, IETF network access control mechanisms, 
> IETF security and not something else. This won't be easy of course, 
> but I think we can do it. We are extremely good engineers and we've 
> been able to produce scalable technology and useful, complexity 
> reducing abstractions. Maybe time to apply some of that for our 
> organization as well?

I don't think that IETF inherently produces the best technology and 
highest quality in every area of Internet protocol design.  We cannot 
be good at everything.  I may be dated in my awareness of our 
participants' expertise, but I doubt we have enough of the best 
designers of cryptographic algorithms, audio or video codecs, forward 
error correction codes, radio transmission methods, etc.   There's a 
reason we leave valuable technical work to IEEE, 3GPP, W3C, etc.  We 
have to specialize, as they do.  The Internet is too vast and diverse 
for all of its technical work to be done by one organization.  For me 
the selection criteria (in brief) have to do with whether the protocols 
in question impact the core Internet protocols or protocols 
traditionally developed in IETF, or whether the protocols in question 
need input from those with the most expertise from core or traditional 
IETF protocols.  Those are fairly elastic criteria that cover a lot of 
ground, but not everything.  For instance, we don't need to be involved 
much in B2B transaction processing as long as those guys can use 
existing protocols like TCP or HTTP in a way that works well for them 
and doesn't adversely impact the Internet.  We might say things like 
"don't run everything over port 80" or "don't place too much faith in 
perimeter security" but we don't need to try to take over all of their 
protocol design.

---

From: Jari Arkko <[email-masked]>
Date: Fri, 29 Apr 2005 15:51:33 +0300

Hi Keith,

>> stricter in allowing what work be taken up, in order to ensure that 
>> we can actually deliver. But I share the same opinion as John L that 
>> we should rather try to shape the IETF so that it can deliver what 
>> the world needs.
>
>
> IETF to be involved in and supporting this effort.  It involved both 
> an assessment of how much harm would result from a botched design (in 
> Internet long-term.  

All good criteria! I would probably add "assessment of whether lack of 
the protocol would be of
harm to the Internet long-term" (assuming the protocol falls within our 
scope, as you correctly
point out below). Here's an example: a protocol that is within IETF 
scope, but we suddenly
stop maintaining it to respond to changing requirements, or open it up 
to vendor extensions
without providing good abstractions that maintain interoperability.

> the selection criteria (in brief) have to do with whether the 
> protocols in question impact the core Internet protocols or protocols 
> ground, but not everything.  For instance, we don't need to be 
> involved much in B2B transaction processing as long as those guys can 
> use existing protocols like TCP or HTTP in a way that works well for 
> them and doesn't adversely impact the Internet.  We might say things 
> like "don't run everything over port 80" or "don't place too much 
> faith in perimeter security" but we don't need to try to take over all 
> of their protocol design.

I am in agreement with all what you say here. Just pointing out that 
even with
specialization, we may have quite a lot to do.

--Jari

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Fri, 29 Apr 2005 15:29:27 +0200

Dear Keith and Jari,
this exchange is the first, where the real issue which interests me 
(user-centric approach) is discussed: the deliverables as such. "I am in 
agreement with all what you say here. Just pointing out that even with 
specialization, we may have quite a lot to do." says Jari to Keith, amen.

Jari says they should be the best the user expects, and Keith the best we 
can. I can certainly live with both approaches. But for years I 
bore  everyone in the IETF to obtain that kind of debate: I learn that due 
to the very nature of the RFC system, IETF is not interested in delivering 
and maintaining a technology (keeping a structured and updated consistent 
"how to develop/use documentation" - comprehensive for Jari, of high level 
for Keith), but in publishing Internet documents of interest to its 
members. I mesure that in particular through the lack of interest into a 
Multilingual Internet, however important and exciting are the architectural 
extensions it calls for, with falls back in every Internet areas.

So, I have two questions:

1. do you think that a third party effort extracting an "IETF Internet 
Technology" (Jari) or an "IETF and Others Internet Technology" (Keith) book 
and maintaining it would be of help? Do you think that after a probable 
sunrise despising period, RFC authors would contribute in proposing 
additions and updates corresponding to the RFCs they publish?

2. I see the IETF as a pool of authors, the IESG as a publisher and IAB as 
its technical adviser, managing a document collection. Some people are 
interested in authoring documents for various reasons (business, 
self-recognition, political, pet subject, etc.. The IESG makes them 
reviewed by peers before reviewing their text themselves (this process is 
not perfect, but is normal). I feel the recognition and exposure expected 
by the authors and the resulting seriousness filter and synthesis of the 
public comments is not important enough to fully play their role. This is 
why everyone is a little or much dismayed, depending on their personal 
interests and experience.

And this is why I think press recognition, exposure, synthesis, critics and 
feed-backs are necessary. I accept that other standardization entities may 
not have press coverage, but they usually have another source of external 
recognition, exposure, synthesis, critics and feed-back through public / 
international bodies and financing, Internet has not (cf. RFC 3869 
analysis). This is why I keep referring to "Nature" which addresses the 
same problem for general Research.

Question: is this evaluation correct? Should it not be a negotiation of the 
IASA with a publisher to barter a part of the IETF publication rights 
against press coverage and a possible technology synthesis? Or would such a 
third party initiative help? It could start with an official personal 
maintained bio+vision+activity report of all those interested in 
normalizing the Internet. F2F pay-meetings are otherwise a way to make the 
IETF an internuts veterans business sponsored association.

I would be interested in statistics on IETF Membership: how many are 
attending meetings on personal expenses? how many per country? how many per 
class of age? for how long have them been a member? This kind of 
information would help better understanding this debate, to know who is 
excited by belonging to a group of experts and who is excited by the 
quality of the resulting common deliverables.

jfc

At 14:21 29/04/2005, Keith Moore wrote:
>>Keith, you have been advocating a model where the IETF would be stricter 
>>in allowing what work be taken up, in order to ensure that we can 
>>actually deliver. But I share the same opinion as John L that we should 
>>rather try to shape the IETF so that it can deliver what the world needs.
>
>My primary criterion when arguing whether IETF should or should not take 
>up a WG was always, in some sense, whether the Internet needed IETF to be 
>involved in and supporting this effort.  It involved both an assessment of 
>how much harm would result from a botched design (in particular, a design 
>that didn't respect the Internet environment and other protocols on the 
>net), and of whether IETF could expend the resources necessary to manage 
>the group and whether it could bring the necessary expertise to the 
>table.  It also involved an assessment of whether the proposed protocol 
>would actually be of benefit to the Internet long-term.  What I didn't try 
>to assess (much) was whether IETF's reputation would be enhanced by its 
>involvement in that particular WG.
>
>>Part of the reason why I believe so is that despite its problems, I think 
>>the IETF produces the best technology and highest quality. I want to use 
>>IETF multimedia, IETF network access control mechanisms, IETF security 
>>and not something else. This won't be easy of course, but I think we can 
>>do it. We are extremely good engineers and we've been able to produce 
>>scalable technology and useful, complexity reducing abstractions. Maybe 
>>time to apply some of that for our organization as well?
>
>highest quality in every area of Internet protocol design.  We cannot be 
>good at everything.  I may be dated in my awareness of our participants' 
>expertise, but I doubt we have enough of the best designers of 
>cryptographic algorithms, audio or video codecs, forward error correction 
>codes, radio transmission methods, etc.   There's a reason we leave 
>valuable technical work to IEEE, 3GPP, W3C, etc.  We have to specialize, 
>as they do.  The Internet is too vast and diverse for all of its technical 
>work to be done by one organization.  For me the selection criteria (in 
>brief) have to do with whether the protocols in question impact the core 
>Internet protocols or protocols traditionally developed in IETF, or 
>whether the protocols in question need input from those with the most 
>expertise from core or traditional IETF protocols.  Those are fairly 
>elastic criteria that cover a lot of ground, but not everything.  For 
>instance, we don't need to be involved much in B2B transaction processing 
>as long as those guys can use existing protocols like TCP or HTTP in a way 
>that works well for them and doesn't adversely impact the Internet.  We 
>might say things like "don't run everything over port 80" or "don't place 
>too much faith in perimeter security" but we don't need to try to take 
>over all of their protocol design.
>
>

---

From: Keith Moore <[email-masked]>
Date: Fri, 29 Apr 2005 14:49:58 -0400

> Personally, I think that is as it should be.  
[...]
> WG".  There is only an IESG management problem. 

Without quoting your entire message, let me say that I do agree at least
partially with this.  But while at least in theory, IESG has all of the 
authority and mechanisms it needs to change working group behavior,
let me explain why, in practice, I don't think IESG can make the changes
that are necessary.

There are many things that IESG cannot do with working groups, for
a variety of reasons.  

1. WG participants are "volunteers".  For the purpose of this discussion
    it matters not whether they are volunteering their own personal 
    time and energy or whether their employers are "volunteering" the
    time and energy.  The IESG can name WG chairs (assuming it can
    find willing victims) but it cannot "hire" or "fire" participants
    based on their qualifications, nor can it do much to create incentives
    to fill particular positions with especially qualified people.  This is
    generally as it should be; however, it removes some of the mechanisms
    by which managers in the "real world" produce high-quality output.

2. There is a considerable cultural inertia within IETF that largely 
    dictates how working groups operate, and which is extremely 
    difficult to change.  For instance, community expectations include:

  - If there is a BOF held and sufficient constituency identified for
    a working group, the working group must be chartered.

  - The working group gets to pick at least some of its own chairs.

  - Charter prose that attempts to scope the working group or 
    dictate how the working group operates is irrelevant once the 
    working group is chartered.

  - Charter milestones are meaningless (in the sense that they are 
    so poorly defined as to be useless - like publish version -00 of 
    document X) and irrelevant (in that their dates can be ignored).

  - Certain charter requirements, such as (poorly named) 
    "requirements" documents, are viewed as meaningless hoops
    to jump through rather than tools to help the working group
    refine its scope and better understand its problem.  Whenever
    possible, requirements documents are written _after_ the 
    protocol specification, so that the requirements won't appear
    inconsistent with the protocol.

  - Discussion is held and decisions are taken on mailing lists, which 
    are largely unstructured.  Any issue can be raised at any time;
    any argument can devolve into a rathole and continue as long as 
    its participants want until it's explicitly terminated by the chair.

  - 90% of meeting time consists of powerpoint presentations which
    may or may not be relevant to the work currently being undertaken
    by the group.  The goal in scheduling the meeting is to allow as
    many presentations as time permites.  Any discussion taken during
    presentations should consist of questions about the presentations.
    Time remaining after presentations can be used for discussion, 
    if anyone is still awake.

  - It's normal and acceptable for meeting participants to occupy 
    themselves during meeting time reading their mail, browsing the 
    web, chatting, or playing games on laptops.  Wireless net access 
    is mandatory. 
   
  - Any document produced by a WG must be considered by IESG,
    and IESG is required to either approve the document or tell the
    WG, very specifically, what it takes to fix the document - even
    if the document violates the terms of the charter.

  - IESG should provide near-immediate turnaround on a WG's
    document, no matter how long or poorly written it is.

3. Relatively few working group participants seem to have engineering
    backgrounds or understand how to use engineering disciplines to
    straightforwardly develop and refine a specification to the point 
    that there is high confidence that 

    - the goals and requirements are understood and sufficiently
      precise,
    - the specification satisfies those goals and requirements, 
    - the specification is unambiguous and complete,
    - the specification is implementable, with reasonable effort,
      in a way that permits interoperation of independent
      implementations,
    - the protocol defined by the specification is deployable and can
      operate with reasonable efficiency for both the network and the
      hosts that support it
    - the behavior of the protocol when deployed on a large scale
      is understood, including its effects on the net and other protocols,
      and those effects are benign or beneficial

   and that the design rationale and support for that confidence are
   documented.  To the extent that the confidence exists and is
   justified, it is likely to have been developed via an awkward 
   and inefficient process that delayed the WG's output by months
   or years.

Now, again, in theory IESG can specify all of this to the Nth degree,
and there are occasions where IESG has specified most of these things
(one at a time) and made them stick.  But it is generally unable to
specify most of those things most of the time and make them stick.

It's possible to view this as a scaling problem, but I think it's more
enlightening to view it as a cultural or education problem.  Our culture
hasn't developed or accepted the skill set that it really needs to do 
work on this scale (even though some of the individuals do have
engineering skills, and others have intuition that serves them quite
well), and our culture has acquired a number of bad habits that are
counterproductive.

So I don't think that IESG is in a good position to fix these problems,
even though it is in a good position to implement the fixes once they
are understood and accepted by the community.

The real trick is getting the community to be willing to change its 
way of operating, and getting the community to understand that 
acquiring additional discipline in developing protocols will produce 
better output more quickly - once we get used to it.

Keith

---

From: John C Klensin <[email-masked]>
Date: Fri, 29 Apr 2005 16:28:42 -0400

Keith,

We largely agree on most of what is in your note, but let me
make a few additional observations...

--On Friday, 29 April, 2005 14:49 -0400 Keith Moore
<[email-masked]> wrote:

>> Going back almost to the dawn of IESG time, the IESG has had
>> one constant and primary responsibility.  That is to manage
>> the WGs and the WG process.  Under today's rules, they
>>...

> Without quoting your entire message, let me say that I do
> agree at least partially with this.  But while at least in
> theory, IESG has all of the  authority and mechanisms it needs
> to change working group behavior, let me explain why, in
> practice, I don't think IESG can make the changes that are
> necessary.
> 
> There are many things that IESG cannot do with working groups,
> for a variety of reasons.  
> 
> 1. WG participants are "volunteers".  For the purpose of this
> discussion     it matters not whether they are volunteering
> their own personal      time and energy or whether their
> employers are "volunteering" the     time and energy.  The
>...
> Now, again, in theory IESG can specify all of this to the Nth
> degree, and there are occasions where IESG has specified most
> of these things (one at a time) and made them stick.  But it
> is generally unable to specify most of those things most of
> the time and make them stick.
> 
> It's possible to view this as a scaling problem, but I think
> it's more enlightening to view it as a cultural or education
> problem.  Our culture hasn't developed or accepted the skill
> set that it really needs to do  work on this scale (even
> though some of the individuals do have engineering skills, and
> others have intuition that serves them quite well), and our
> culture has acquired a number of bad habits that are
> counterproductive.

We agree.  How do you propose that gets fixed (see below)?

> So I don't think that IESG is in a good position to fix these
> problems, even though it is in a good position to implement
> the fixes once they are understood and accepted by the
> community.
> 
> The real trick is getting the community to be willing to
> change its  way of operating, and getting the community to
> understand that  acquiring additional discipline in developing
> protocols will produce  better output more quickly - once we
> get used to it.

Ok, we have gotten ourselves into a bit of a mess in how we
handle some things.   Whether one agrees with your specific list
or not, or with the relative importance of particular points, it
is clear that no one (on or off the IESG) can wave a magic wand
and make things perfect.  It is also clear that at least some of
the issues you identify meet my criteria (and Dave's) of having
real impacts on the efficiency, timeliness, and quality of our
processes and results.  Others probably don't.

I suggest that whining, bemoaning our fate, generally sitting
around being miserable, or even casting blame without
identifying solutions, are not only not productive in getting us
to better and more timely results, but may actively distract us
and divert resources from improvements.

It seems to me that there are only three paths out of here:

(1) We know there are other standards bodies out there that, by
their definitions (although not necessarily by ours) are
"successful".  They have formulas for getting things done,
formulas that involve memberships (usually at a price), at least
presumed technical qualifications and approval mechanisms for
participation in working groups, very specific voting rules at
almost every level, including rules about voting by multiple
people from one company, institution, or country, and so on.
They have very specific management and organizational models and
rituals.  They also tend to be characterized by very strong
professional secretariats who end up doing a lot of the
management work that we have assigned to the IESG (or let the
IESG accumulate).  The IETF community has traditionally had
doubts about the quality and/or timeliness of the results from
those bodies, but they do manage to get documents out.   We
could stop what we are doing and emulate them in a greater or
lesser number of ways.

(2) We know that there are standards-producing consortia out
there.  The classic version involves a body organized for the
benefit of a particular cluster of companies, with rather
expensive memberships and participation by people who are not
affiliated with those companies only on a very restricted basis.
They often have elaborate rules for sharing of intellectual
property and about contributions to the consortium as well.
(Note that some consortia, by plan or incremental evolution,
have started to more strongly resemble the above standards
bodies, or other creatures, sometimes even the IETF -- don't get
confused by labels).  They typically have management
arrangements that, among other things, end up dictating their
work plans.  The IETF community has traditionally had doubts
about that model and its products, especially with regard to
openness and quality of review, but they do manage, usually, to
get work done and documents out.   We could stop what we are
doing and emulate them in a greater or lesser number of ways.

(3) We can conclude that we have had --and, I think, still do
have-- something a little bit unique here, with some history of
producing good work that actually permits decentralized networks
to hang together and function reasonably well.  While the IETF
has not been around that long, there are core principles in how
we produce and evaluate work that aren't much different, in
principle, from the way the core ARPANET protocols, TCP/IP,
etc., were produced.  We can even conclude that there is little
point trying to turn into one of those traditional standards
bodies or consortia: they do what they do pretty well, at least
under their definitions.  Maybe those who prefer to work that
way should go to them and we should continue to work with, and
work out, the IETF way of doing things.  If we conclude that it
doesn't work any more because <insert your favorite excuse
here>, then we can either identify that problem and get it
acceptably under control, or it is time to close shop and move
on.

That suggests to me that we need to be on a quest, not for
things to complain about or groups to cast blame on, but for
targets of opportunity that, if changed, might have high
leverage on getting better quality work done and in a more
timely way.   Dave and I may disagree about whether the ISD work
is likely to be worth the amount of trouble that has gone into
it, and others might disagree about likelihood of its success.
But the intent there was to go after something that would have
fairly high leverage on a specific and identified set of
problems that are, themselves, relevant to speed and quality of
output.   The leadership to identify those high-importance,
critical-path problems and possible things to do about them
needs to come from somewhere.  Several of us have been trying to
do just that work from among the masses, even though some
efforts (e.g., the tools one) have been lots more successful
than others (e.g., ISDs and some reorganization attempts).  But
some of that leadership needs to come from what is, you will
recall, called a "steering group"... both because they have a
different, and I pray better, perspective on what might work and
because, especially procedurally, they can block nearly anything
just by bogging it down or picking it to death.   

So I agree with you that there are many things that might be
done that the IESG can't do by themselves.  Some of those things
are just matters in which, if the IESG concludes something is
the right thing to do, they need to educate, and steer, and
lead... rather than having either them, or us, assume that they
can announce or declare something and have everyone salute.  It
seems to me that is A Good Thing, no matter how much more
efficient an imperial regime might be.

Looking at that matter of leverage and the critical path to
significant improvements in quality and timeliness, I've got
doubts about some of your points.  I want to pick on two as
examples, but the issue, for me, isn't whether you are right or
not, but how to think about this:

(i) People are sitting in meeting rooms, with wireless, paying
attention to email, not the work at hand.  I suggest that a
change to this is unlikely to be useful enough to be worth the
trouble.  If they weren't in the room doing email, they might be
asleep.  Or they might be somewhere else doing email.   Neither
case would appreciably increase the number of people in the room
who are making active and useful contributions to the work at
hand. Measured in terms of getting quality work done in a timely
way, those people are the only ones who count.  Sometimes people
who are mostly paying attention to email notice an issue in the
room and make a useful point, and we are better off for that,
however marginally.  Can the added people who aren't paying
attention be annoying?  Sure, but so would loud snoring be.
Would taking away the wireless make a noticeable improvement in
the quality or timeliness of our work (even ignoring arguments
about its value in making materials available, permitting better
off-site participation, etc.)?  Well, I'm unconvinced.

There is also the possibility that some of those who are doing
email are doing so because the S/N ratio in the meeting rooms,
especially for those who have read and thought about the
documents, is just not high enough to take up their full
attention.  You know, I do email in meeting rooms and often
follow the Jabber script of meetings on not in at the same time.
I stop when there is something to which I need to pay attention.
If a WG Chair wants an estimate of my opinion of the S/N ratio
of what is going on in the room, just watch how much I'm typing.
Gee... a clue.  Maybe that is an advantage of wireless :-(.
And that brings me to the Powerpoint problem.

(ii) The first thing to notice about the Powerpoint problem is
that, even more so than the wireless one, it is of fairly recent
origin.  The second is that the "presentations" issue is
actually separate from the powerpoint one, and that the former
has, to one degree or another, always been with us.  Now, for
that problem, the IESG has, from my observation, been letting
WGs do pretty much as they like.  But I'd predict that, if a WG
Chair went to his or her AD and said, "I'm not going to permit
any more presentations", there wouldn't be large pushback.  I'd
also suggest that, if a lot of us started going to ADs, or
standing up in plenaries, and saying "WG X has had nothing but
presentations for the last three meetings and has had no useful
discussions, should it really continue to exist", the ADs might
actually start pushing back on WG chairs about this sort of
thing.  As you know, I'm personally a pretty strong believer in
the "oh, you haven't read the documents, go to the back of the
room and be quiet" school of WG meeting leadership.  I've gotten
a lot of pushback for that, and been called several bad names,
but it has never come from the IESG.    So, on a WG by WG basis,
I suggest what we have met the problem and it is us.   But I
also think the IESG could be doing some more aggressive
leadership in the area _and_ that they would be doing so if more
of the community were obviously supportive.

PowerPoint itself is another issue.  I, personally, hate it for
IETF WG-like meetings, not because of all of the cliche reasons,
but because it discourages real interaction.   When I was able
to walk into a room with a few overhead projector foils and a
handful of colored markers, I could measure how useful and
effective a discussion was by how much more those foils were
marked up by the end of the meeting than they were coming in.
The same comment would apply to flip-chart pads, although they
work less well in larger rooms.  Powerpoint discourages that
sort of interactive markup and revisions.  The more
professional-looking the deck, the harder it becomes to mark up
on the fly.  As a sometime WG Chair, I don't know if I could get
an overhead projector or flip chart back if I asked for them --
I haven't tried and, after this conversation, probably will.
But I certainly know how to turn the projector off and/or
confiscate the connector cord.  Would it have a big impact on
productivity?   I don't really know... it might.  But it is a
really easy experiment and I can't imagine the IESG pushing back
on a WG Chair who made that choice or on an AD who made the
choice for an Area... or even having a lot of sympathy for
someone who came to them whining that the WG Chair had refused
to give them 20 minutes to make a presentation when no one else
was making presentations either.

Conclusion (and you, Keith, are not really my subject/target):
stop complaining and whining.  If the IETF's way of working,
even at its best, isn't an environment in which you think you
can and want to work effectively, go elsewhere.   If you see
ways to make the IETF work better --enough better to be worth
the trouble, the arguments, the debates, and the transition--
make specific suggestions, discuss them here, and discuss them
with the IESG (plenaries have been, IMO, _much_ too quiet lately
and maybe that is a symptom).   If there is a good suggestion
that would have significant positive effects, and that has
strong community support, and the IESG blocks it, then it is
time to get rid of the IESG... either en mass or one AD at a
time until they get the point.  I assume, and hope, it won't
come to that: while I have had, and have, a lot of disagreements
with the way the IESG and individual ADs do business, and I
think things are worse in that regard than they were a
half-dozen years ago, I believe that, on the average day, the
average AD understands that we are all on the same side and will
either succeed or fail together.

     john

---

From: John C Klensin <[email-masked]>
Date: Fri, 29 Apr 2005 16:49:08 -0400

> have an easy metric for the world-has-a-problem to ietf-

Yes.  And see below

> responsibility for managing the group.

They do indeed.   But the choice of how much freedom and
responsibility they get is an AD decision.  For example, those
who believe strongly in the value of benchmarks would argue
that, if a WG is seriously overdue on commitments, the WG and
Chair should find themselves on an extremely short leash if not
just shut down.  By contrast, WGs that are working well, at
least by that measure, should be left to work in whatever ways
they are working.

> blew it for no good reason.

I think this analysis is just right.  But I'd make a suggestion
for you and others to think about.

We've had a long tradition of WGs saying "you chartered us, we
did a lot of work, and therefore we are entitled to have
whatever we produce made a standard".  Such groups are usually
problematic.  Many of the worst examples of thing that Keith
(and I) believe the IESG needs to stop come out of WGs who, when
they stop being able to justify their work on a technical
quality basis, make just that argument.  Sometimes the IESG has
been better at pushing back than others, but pushing back is
rarely a rewarding activity: the WG participants get mad and no
one else seems to care, certain not enough to utter supporting
words.

Now, if we take the IETF down the path that you attribute to
Keith...

> order to ensure that we can actually deliver. 

then the review and WG approval process will tend to get ever
longer as we try harder and harder to be sure that no WG that
will fail to produce good results ever gets into the system.  I
suggest that is madness, but it is also our only choice unless
we strongly encourage the IESG, and individual ADs, to adopt a
different path.  "Encourage" here means both extensive kind
words and signs of approval when the do the right thing and loud
complaints to the nomcom and maybe even recall petitions when
they don't.

What is that different path?   I don't suggest that we should do
it for all cases, or even most cases, but the problem with these
pre-WG review and charter processes is that sometimes we get to
the point of diminishing returns in which a lot more marginal
time doesn't really make a better charter or better odds of
timely and high-quality success.   For those situations, as with
standards, we need to know where to stop.  For standards, part
of our stopping mechanism is to say "no convergence,
experimental" (a mechanism we may not be using enough of late,
but...) or "good enough for now, Proposed" (doing that
effectively may require other changes of attitude). We may need
a way to have an "experimental" or "probationary" WG: to say to
a group "we don't have much confidence in this, but you are
welcome to try to run with it and prove us wrong... you get a
fixed amount of time, after which the assumption is that we are
going to shut you down unless you have produced enough useful
results to justify rechartering".  

But, again, to even think about that, the IESG is going to need
a lot of support and bottom-up direction.  I don't know what
would happen if a group, after some months of negotiation, would
say to the IESG "look, we think we can do this, give us nine
months to produce a product that will pass standards review and,
if we can't do it, we will voluntarily shut down".  But it is
certainly an experiment I'd like to see and, if someone tries it
and the IESG says "no, we really like negotiating future
details", I hope that is reported on this list, with names.  If
we continue on the path in which shutting down a WG that wants
to continue produces nothing but abuse for the relevant ADs,
then we are dooming ourselves to long, probably ever-longer,
charter and WG negotiation processes.

And, yes, I agree that is part of the problem that is worth
trying to fix.

    john

---

From: Dave Crocker <[email-masked]>
Date: Fri, 29 Apr 2005 15:23:53 -0700

>  As you know, I'm personally a pretty strong believer in
>  but it has never come from the IESG.

I'll venture to suggest that anyone who thinks about both the scarcity of 
meeting time and the aggregate cost of such meeting -- include the cost of the 
time of the participants -- will agree with you that the meeting needs to 
focus on deliverables, rather than "education".  

There are, of course, exceptions, but they are just that, exceptional.

>  PowerPoint itself is another issue.  I, personally, hate it for IETF WG-
>  like meetings, not because of all of the cliche reasons, but because it
>  discourages real interaction.   

People can and do use powerpoint slides in many ways.  Some folks will rework 
text in real-time, based on interaction with the participants.  Some folks 
just talk their slides rather than actually engaging with the participants.

A thing to keep in mind is that slides and the jabber activity can be 
incredibly helpful to folks for whom English not their native language.

I think that, in fact, the issue is not powerpoint-vs-no-powerpoint.  I think 
it is exactly and only the concern you raise:  meetings need to be for working 
group interaction.  If that is the clear goal and if the meeting is run with 
that goal enforced, then none of the trappings matter.

 d/
 ---
 Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Fri, 29 Apr 2005 16:48:23 -0700

>  But, again, to even think about that, the IESG is going to need a lot of
>  support and bottom-up direction.

John,

Let me suggest that there has already been quite a bit of that.  

It has not been any sort of overwhelming, unified, shout-in-a-single-voice, 
but there really have been quite a few IETF participants calling for just what 
you suggest.

So far, there have been two problems:

As always, the IETF also has lots of voices objecting to any particular 
proposal.  So the ability to achieve any sort of meaningful progress requires 
much more, ummm... "active" management than we tend to prefer.  

(There is a wg chair technique of asserting what the specific consensus is, 
and then saying that that assertion will hold unless there is rough consensus 
AGAINST.  This is an example of such active management.  Of course, that 
sounds a lot like a wg chair dictating things and it only works if, in fact, 
the wg group is eager to make forward progress and gives the chair strong 
support to be assertive in this way.)

The IESG has not been willing to take a real leadership role in effective 
substantive structural or process changes, to address core issues.  It has, 
instead, deferred things to working groups and then mostly ignored them, and 
it has spent many months focusing on administrative changes.

We do not have the sort of cohesive, self-motivating community we had 10+ 
years ago.  We have a community that really does look to the IESG for 
leadership.  

After the Kobe revolt, change did come from the grassroots.  The IETF 
leadership merely had to avoid trying to stand in its way.  But we do not have 
that IETF.  We have one that is far more diffuse and, therefore, far less 
cohesive about how to get things done.

Absent any other grass-roots effort, real change is going to depend upon the 
IESG getting serious about it.

 d/
 ---
 Dave Crocker

---

From: John Leslie <[email-masked]>
Date: Fri, 29 Apr 2005 22:27:57 -0400

> review of benchmarks, actions or the lack thereof when targets

   You've covered an awful lot of territory there:

- creating / shutting down WGs -- clearly must be IESG responsibility.
- who chairs WG -- feels more like an Area Director responsibility.
  The IESG should require ADs to report on WG progress; the AD should
  have the right to deal with WG chairs s/he trusts.
- how they are otherwise managed -- whazzat?
  WG chairs should be responsible to ADs. End-runs around WGCs (other
  than formal appeals) tend to be harmful.
- what goes into charters -- the IESG must take responsibility for the
  initial charter, but not all updates should need IESG action.
- how many documents a WG can work on -- silly-season...
  Clearly, taking on a new task should be a charter revision, which
  might require IESG action; but splitting a task between two Document
  Editors shouldn't even require notice to the AD.
- whether documents can be worked on in parallel -- micromanagement.
  For the AD to specify this would be seriously unwise: for the IESG
  to even talk about it is a waste of time or worse.

> and under some circumstances, individual ADs can dispose of them

   I've never been comfortable with the idea that the IESG can "solve"
a problem by dissolving a WG. That said, there is a clear need for
something like a probationary status, where a WG loses (temporarily)
its right to submit documents as WG submissions until some defect is
corrected; and at some point, actually loses its right to any AD
oversight at all.

   IMHO, "surprise" dissolutions of WGs are not helpful.

> essential nature of the situation very much.

   I'm not really sure what the community _could_ do to offer much
guidance here; and I don't think community "support" of IESG decision
has much meaning. The question is whether an IESG action is going to
result in a long-winded appeal process; and there's remarkably little
that anyone can do about that.

> be responsible.

   Except that I see the bottleneck at the AD/WGC interface, I agree.

> Either WGs are sufficiently well chartered and managed so that the
> review processes we have work adequately well or they aren't.
> Either way, the IESG bears ultimate responsibility: they determine
> the charters, the management structure, the review requirements or
> lack thereof at various intermediate points, and so on.

   I'm not sure it's helpful to look at it that narrowly.

   I think I quite agree with John Klensin that if charters, structure,
and review requirements are done right, corrective actions will become
obvious. I don't agree, however, that our selection process for IESG
members does anything to ensure sufficient expertise to get charters,
structure, and review requirements right on the first try.

   I think we need a mechanism to notice that something needs correction,
and get independent review of charters, structure, and review requirements
when problems appear.

> do that job well and still review documents for standardization".

   Certainly a reasonable question: but I don't choose to give an
opinion on it at present...

> problem or an IESG problem.

   This view is entirely too "top-down".

   Obviously, both WG-problems and problem-WGs exist: almost as obviously,
the problems are not all the same. It's fine to say that in theory, all
these problems belong to the IESG: but it's not particularly helpful.
If we expect the IESG to fix all WG problems, we severly constrain the
number of WGs we can have -- leaving no satisfactory method for getting
increasing amounts of work done.

   The _best_ this view has to offer is stagnation. :^(

   If, instead, we view the situation as groups of folks that want to work
on problems, and the _issue_ as whether they should associate with IETF
or work somewhere else, I believe much more can be accomplished.

--
John Leslie <[email-masked]>

---

From: Keith Moore <[email-masked]>
Date: Sat, 30 Apr 2005 01:40:45 -0400

wow...I keep wanting to make terse replies, but there never seems to be 
a way to address the subject with a short answer.

I'm sorry you see these explanations as whining.  I believe that we 
have to recognize that part of our problem is how WGs operate before we 
can be willing to solve that problem.  So I try to describe that 
problem in a way that people will recognize it.  Maybe people already 
realize that we have this problem and I don't actually need to 
illustrate it.

As for solutions - I have been thinking about possible solutions for 
several years.  But I'm much better at protocol engineering than I am 
at engineering management or social structures, so I don't have much 
confidence in my ideas for how to solve the problem.

Recently I've begun to suspect that a good answer to some of these 
problems might involve a layer of management between IESG and working 
groups - a set of people who had responsibility to give some technical 
oversight to working groups, monitor their progress, and keep the ADs 
up-to-date on the state of things.  I say "oversight" rather than 
"direction" because "direction" would be too strong a term.  I don't 
see the supervisor (let's call him a supervisor for now) telling the 
group what technical decisions to make.  I do see the supervisor making 
sure that the group investigates important issues - setting short-term 
milestones and an agenda for the group.

What I'm thinking is that the person in charge of supervising a WG ends 
up consulting with the chair and/or document authors on a regular basis 
(say, every three or four weeks) and agreeing on the next set of 
near-term goals and deliverables for that WG.  The supervisor might say 
"you need to understand and document the nature of the disagreement 
between these two concerns" or "you need to pick a 
mandatory-to-implement encryption algorithm", or "you need to solicit 
cross-area review for this issue" or "you need to analyze whether this 
ABNF grammar can be implemented with only one symbol of look-ahead" or 
"defer that question for now, the thing you have to decide before you 
do anything else is how much you are going to constrain yourself to be 
backward compatible with this legacy protocol that we all know is 
broken".

As I see it, the supervisor would offload some of the present Chair's 
duties and some of the present Responsible AD's duties.  The chairs 
would still be responsible for managing the group discussion and making 
sure process rules were followed.  The ADs would still approve charter 
changes (overall goals, long-term milestones), review documents before 
publication, and be the first level of appeal.
The supervisor would try to make sure that important issues were 
identified and dealt with early, and in plenty of time to get them 
resolved before the specification is finalized; he would also make sure 
that measurable progress was being made every few weeks and report 
progress or lack thereof to the AD.

Another part of the supervisor's job would be to ensure that when the 
document finally goes to IESG, neither the WG nor the IESG are 
surprised by how the other reacts to it.  Along with the specification 
presented to IESG should be supporting material to give the IESG 
confidence in the document, for example: a list of goals and 
requirements (written before the specification); a list of changes to 
those goals and requirements with the rationale for each change; a list 
of issues and contentious design choices that arose, how each was 
resolved, and why; what kinds of review the specification was subjected 
to, the results of that review, and what changes (if any) were made 
based on that feedback; documentation of any analysis that was done on 
the protocol; description of early implementations (prototypes) and how 
they differ from the final specification.  This material would take the 
place of the "writeup" that an AD is now expected to do before a 
document from his working group goes to IESG ballot.

Similarly for face-to-face meetings - the Chair would preside, but the 
supervisor would work out the agendas of face-to-face meetings in 
consultation with the Chair.   The meeting time would be used to reach 
closure on divisive issues.

The supervisor would be expected to NOT have a strong interest in the 
outcome of the WG (other than to see that quality work is done), nor to 
closely follow or regularly participate in group discussion.  His job 
would be to keep a higher-level perspective on things.   When 
identifying issues that must be resolved he could state an opinion but 
must clearly separate his opinion from the description of the issue. 
WGs would be allowed to "push back" on an issue definition if they 
thought it unreasonably constrained the resolution, and suggest an 
alternate way of framing the issue.

The concept of Technical Supervisors could be tried on two or three 
working groups and refined based on experience.  Initially the 
supervisors might be appointed by the AD and confirmed by IESG; 
eventually there might be a separate mechanism for nominating and 
vetting potential supervisors.  Ideally a working group would keep the 
same supervisor as long as it is chartered, though it would be possible 
to change a WG's supervisor - particularly one who wasn't doing his job 
well.

Supervising a WG shouldn't require more than say 2 hours per week.

There would need to be some way to recognize the contributions of WG 
supervisors, and perhaps some incentives for taking on a job with low 
visibility and minimal creative input.

-- 

so there you have one half-baked idea.  I'm sure it's not too difficult 
to poke holes in it, but I think it's a more promising avenue of 
inquiry than one which tries to move the final technical review outside 
of IESG.  In a world with technical supervisors, the IESG mostly 
continues to do what it is doing now, but its job should get easier 
because it will get better specifications and more supporting material 
with which to evaluate the specifications.  it might let the IETF grow 
larger because ADs will be able to spend less effort keeping up with 
their working groups' progress and less effort reviewing documents - 
IESG's workload would be more closely related to the number of 
documents reviewed than the number of working groups.

Keith

---

From: Pekka Savola <[email-masked]>
Date: Sat, 30 Apr 2005 09:40:37 +0300 (EEST)

On Sat, 30 Apr 2005, Keith Moore wrote:
> The concept of Technical Supervisors could be tried on two or three working 
> groups and refined based on experience.  Initially the supervisors might be 
> appointed by the AD and confirmed by IESG; eventually there might be a 
> separate mechanism for nominating and vetting potential supervisors.  Ideally 
> a working group would keep the same supervisor as long as it is chartered, 
> though it would be possible to change a WG's supervisor - particularly one 
> who wasn't doing his job well.
>

What you write could probably be accomplished as a 'Technical Advisor' 
that some WGs have and is listed in the charter pages ?

The key point here is that such technical supervisors should have 
broad experience (preferably have been on the IESG, or have been 
exposed to the work, e.g., by being a chair of a WG which produced a 
lot of documents) to be able to have sufficient cross-area insight.

The most difficult issue would probably be the coordination between 
the supervisors and chairs, and to a lesser degree, the ADs -- of 
course depending how much authority and micromanagement of document 
editors the supervisor would be required to do.

Personally, if each WG had just one "Techical Advisor" which would be 
committed to reviewing the specifications early, and who would be at 
the disposal of the chairs if there are doubts, we could be much 
better off -- especially if a WG wouldn't have experienced chairs.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Keith Moore <[email-masked]>
Date: Sat, 30 Apr 2005 07:47:56 -0400

> that some WGs have and is listed in the charter pages ?

it could be an expansion of that role.  in my mind, such people would 
have explicit authority to specify the agenda for WG discussions, and 
some explicit responsibilities to the responsible AD that may not be 
defined at present.

> lot of documents) to be able to have sufficient cross-area insight.

yes, I'm generally thinking senior IETF people who are familiar with 
the processes, and who are also familiar with engineering disciplines 
(whether by education or experience using them)

> editors the supervisor would be required to do.

it certainly implies a significant change to the role of WG chair, and 
I can imagine some chairs not wanting to give up that much control.  
the thing to do is to cultivate an effective working relationship 
between the chair and supervisor, where the chair sees the supervisor 
as someone who helps the WG's progress more quickly and helps the WG 
make its case to IESG.  but I see the supervisor as ultimately 
responsible to the AD rather than to the WG, so there's some inherent 
conflict there.

Keith

---

From: John C Klensin <[email-masked]>
Date: Sat, 30 Apr 2005 08:44:44 -0400

Keith,

This _is_ going to be a terse reply, since others have covered
much of what I would have.  But the topics are complex.  Three
observations...

(1) My observation about whining had more to do with the general
tone of many of these discussions, in this round and earlier,
rather than anything about what you may or may not have said
specifically.  Please don't take that categorization personally
or, even generally, as more than a warning about S/N ratios and
ways to make progress (or the lack thereof).

(2) We have repeatedly tried variations on this theme.  The
"area advisor" got that title because the secretariat couldn't
modify the relevant templates to include both "responsible AD"
(as we moved to two-AD areas with split, rather than shared,
responsibility) and the notion of "someone senior who keeps a
close eye on a WG, advising the chair and WG but reporting to
the AD".   A different variation might have been called
"designated leadership developer/coach".   I think these ideas
have worked well sometimes and not at all in others.  The fact
that we have tried variations should not imply that we should
avoid trying another, but may call for some serious thought
about why the previous attempts have not always worked
effectively.

(3) I've commented earlier on my concerns about adding
intermediate layers of management or review and won't repeat
those remarks here.

    john

--On Saturday, 30 April, 2005 01:40 -0400 Keith Moore
<[email-masked]> wrote:

> wow...I keep wanting to make terse replies, but there never
> seems to be a way to address the subject with a short answer.
> 
> I'm sorry you see these explanations as whining.  I believe
> that we have to recognize that part of our problem is how WGs
> operate before we can be willing to solve that problem.  So I
> try to describe that problem in a way that people will
> recognize it.  Maybe people already realize that we have this
> problem and I don't actually need to illustrate it.
> 
> As for solutions - I have been thinking about possible
> solutions for several years.  But I'm much better at protocol
> engineering than I am at engineering management or social
> structures, so I don't have much confidence in my ideas for
> how to solve the problem.
> 
> Recently I've begun to suspect that a good answer to some of
> these problems might involve a layer of management between
> IESG and working groups - a set of people who had
> responsibility to give some technical oversight to working
> groups, monitor their progress, and keep the ADs up-to-date on
> the state of things.  I say "oversight" rather than
> "direction" because "direction" would be too strong a term.  I
> don't see the supervisor (let's call him a supervisor for now)
>...