Subject: The "Clerk" function and Standards throughput and quality

From: John C Klensin <[email-masked]>
Date: Sun, 03 Oct 2004 14:46:41 -0400

Since the discussion about scenarios for structuring the
administrative arrangements seem to be settling down, it is
probably time to try raising some questions that might really
impact the problems that get solved.

The issues raised below interact directly with the problems of
getting better-quality standards produced more quickly.  Unlike
some recent discussions, they address those problems directly,
rather than hoping that sufficient administrative reorganization
will help by raising the tide.

At a very high level, Carl's document splits the current
Secretariat task into a meeting planning function and a "Clerk"
one.  There has already been some discussion about nit-picking
the boundary between the two.  But, ignoring that, the Clerk
function can be summarized as "what the secretariat is supposed
to be doing now, if only if worked well".    Because the way in
which this is structured may have a profound impact on what we
can and cannot do in the future, I think the community ought to
consider what we really want from that function... or at least
which options we wish to close and which ones we wish to leave
open.

Two examples (there are others, but I am _not_ going to write
another 300+ line note):

(1) Standards Secretariat function: It has been observed
repeatedly that the IESG has too large a workload, with many bad
consequences.  The solution that other standards bodies have
adopted for similar problems involves developing strong
secretariat functions that can provide significant staff support
to the standardization effort itself.   For example, ADs spend
significant time reviewing and checking documents to be sure
that they are structurally ready for Last Call, developing
ballot statements and draft protocol action notices, etc.  In
other standards bodies, the equivalent tasks are done by
interactions between what we call WGs and secretariat staff: in
those models, the AD would see something only when all of the
non-substantive nits, and maybe some of the substantive ones,
were picked and the ducks lined up (or in response to a
complaint about secretariat behavior).  The review committees
called for in various proposals might be appointed or approved
by the ADs, but coordinated by the secretariat: documents could
be sent out for review, and reviews accepted and collated for
AD/ IESG review, with no AD interaction at all.  In extreme
versions of this model, the Secretariat might even have people
with engineering skills on staff who could perform preliminary
technical reviews, check for overlap with other IETF activities,
etc.

The good news about that sort of model is that it could
significantly reduce IESG workload and increase throughput,
especially if we selected ADs who were willing to trust the
Secretariat and let them do the job as outlined.  Doing that
might permit us to put people on the IESG who lacked sufficient
external support to devote most of their lives to the IETF,
which would almost certainly be good.    The bad news is that it
is very risky: a number of standards bodies have gone down this
path without adequate management controls and adequate
understandings of boundaries and authority and discovered that
they had ended up with Secretariats who could block anything
they didn't like (or even coming from people they didn't like)
and who might be able to put some proposals at considerable
advantage relative to others.   

I suppose it is also obvious that it wouldn't be cheap.  But the
various supporters of the IETF --in dollars or via the in-kind
contributions of the most of the time of senior people-- either
care about throughput and leveraging those dollars or they
don't. If I were the employers of most of the IESG members, I'd
be happy to put some additional number of dollars (Euro, Kroner,
Yen, ...) into the pot if it simultaneously improved IETF
efficiency and got me a significant fraction of the attention of
those people back.

It is a tradeoff, and it involves decisions that the current
IESG should not be expected to make, if only because most of
them have figured out how to work in the current environment and
presumably like it (at least to the extent needed to continue).

However, moving in that direction is incompatible with notions
of "it is a generic function and we can contract it out to
anyone who provides generic support functions to standards
bodies and consortia".  It wouldn't have to be done within the
IETF Administrative operation itself, but it would certainly
need to be very closely coordinated (much more closely and with
much more responsiveness than we usually think of with
"contractor") and it would have to be staffed by people who had,
at least, a good understanding of the subject matter and
relationships in IETF's work.  

If there is any chance that we want any version of this
"standards secretariat" function, we had best make certain that
the administrative structure can accommodate it, and that it can
do so with little disruption to its basic design parameters.

(2) Document editing prior to Last Call.  With more pressures on
everyone, and more documents being written and edited by people
who are not fluent and native speakers and writers of English,
we are getting an increasing number of documents submitted for
Last Call that are painful to read and sometimes more ambiguous
than we would prefer, especially in standards-track materials.
Then either the ADs try to fix them (resulting in considerable
delays and interactions) or they go out for Last Call, where
people who might skim through and check a focused and
well-written document sometimes conclude that they have better
ways to spend their time, reducing quality of review.  They then
cycle back to the IESG, where there is some chance of the
problem being repeated.   And then they go to the RFC Editor,
where there is an inevitable set of conflicts about whether
editing can be done without damaging meaning, whether they have
any business intensively editing something that the IESG has
approved, etc.  If they do make extensive editorial changes, we
may end up with a standard published as an RFC which the WG that
created it barely recognizes.  If they don't, we may end up with
one that no one except the author(s) and WG can understand.  And
the time between approval and publication keeps going up because
editorial review and correction take time.

If we think this is a problem, one obvious way to fix it would
be to include editorial checking and, if necessary, professional
technical editing, prior to Last Call.  That would give us a
situation in which the documents that are Last Called would be
more comprehensible, easier to read, and a closer approximation
to what would finally be published (if approved) than we have
today.  Structurally, the activity could either be part of the
RFC Editor function or complementary to it.   It might or might
not increase the time between WG signoff on a document and
actual Last Call -- today, the ADs sift through the documents
and negotiate clarifications before Last Calls are issued, and
that is rarely a fast process -- but it would almost certainly
improve quality.

Again, this would not be cheap.  It would require getting
management procedures in place to make sure that documents were
properly assigned and flowed smoothly.  A little less "Clerk"
and a little more "Standards Secretariat" would help with that,
but is not a requirement.   It may or may not be compatible with
the models anticipated by Carl's document.  But, if we might
want it, we had best be sure, now, that whatever administrative
model is adopted is compatible with it.

     john

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 05 Oct 2004 13:59:51 +0200

John,

I would like to question some of your assumptions below.

> will help by raising the tide.

I think that we cannot address the questions you raise below without the 
administrative reorganization - simply because under the current regime, we 
have running code that this does not get any better.

> presumably like it (at least to the extent needed to continue).

We have considered such a model; in fact the General Area Review Team 
(gen-ART) has proved to me that it is possible to get a great deal done on 
a volunteer basis - and even more if you get someone to act as "secretary" 
for it, such as Avri Doria is now doing (Thank you, team!)
(see http://www.alvestrand.no/ietf/gen/review-guidelines.html for details)

Such a model would obviously be more stable if we could have parts of it 
moved into a paid-for secretariat.

But - in the current model, we have been told by the CNRI President that 
the IESG is being unreasonable already in the demands made of the 
secretariat, that no new functions can be added without financing for them, 
and that the only means available for getting more resources into the 
current secretariat are raising the meeting fees, getting direct sponsors 
for CNRI's IETF activities, holding exhibitions at the IETF and US 
government funding. Allowing the sponsors who currently sponsor the IETF 
through ISOC to pay for it was not an option.

Again, we have to fix the underlying problems before we can pop back up to 
this level. But once we get there, I don't think the current IESG is the 
biggest obstacle to doing so.

> relationships in IETF's work.

I do not see that this follows. All of the items described above - checking 
that people are doing what they promised, following up when they do not, 
checking for adherence to formal criteria, sending back reasonably cogent 
notes when the checks fail, and following up - are fairly standard for any 
type of document-producing organization - whether it be a consortium, a 
standards organization or even a technical publishing house.

> do so with little disruption to its basic design parameters.

A "contractor" would obviously have to work closely with the people running 
the process to get this done "right". But I don't see a particular reason 
to believe that an organization doing this under contract would be less 
able to make it work than a set of people hired by an organization more 
closely linked to the IETF to do the job.

But I don't see the models so far proposed blocking either possibility.

> model is adopted is compatible with it.

I agree that we need a structure where it is possible to get there. And I 
think the administrative restructuring envisioned is a necessary step in 
getting there.

                     Harald

---

From: John C Klensin <[email-masked]>
Date: Tue, 05 Oct 2004 12:04:26 -0400

--On Tuesday, 05 October, 2004 13:59 +0200 Harald Tveit
Alvestrand <[email-masked]> wrote:

>> getting better-quality standards produced more quickly.
>> Unlike some recent discussions, they address those problems
>> directly, rather than hoping that sufficient administrative
>> reorganization will help by raising the tide.
> 
> I think that we cannot address the questions you raise below
> without the administrative reorganization - simply because
> under the current regime, we have running code that this does
> not get any better.

Sorry.  You misunderstood me.  I believe that the administration
reorg is a _necessary_ condition to being able to seriously
consider some of these other things.  It is not a sufficient
condition.  Also, whether the current admin reorg plan is the
minimum one needed to do the job is a separate question, but may
not be relevant at this point.
 
I believe that most of the rest of our apparent disagreement
stems from this misunderstanding.

>>...
>> them have figured out how to work in the current environment
>> and presumably like it (at least to the extent needed to
>> continue).
> 
> We have considered such a model; in fact the General Area
> Review Team (gen-ART) has proved to me that it is possible to
> get a great deal done on a volunteer basis - and even more if
> you get someone to act as "secretary" for it, such as Avri
> Doria is now doing (Thank you, team!)

We could quibble about this, and, in particular, about whether
it is/would be adequate for _standards_ processing as distinct
from the things that fall within the General Area.  If it were
both adequate and helpful (at least without fine tuning), the
fact that it is being done in this way only in the General Area
would suggest that the other ADs like the workload, are
incompetent, etc.  I don't believe that, so I have to believe
that the model you suggest won't generalize without tuning, and
that brings me back to me assertion above.

> Such a model would obviously be more stable if we could have
> parts of it moved into a paid-for secretariat.
> 
> But - in the current model, we have been told by the CNRI
> President that the IESG is being unreasonable already in the
> demands made of the secretariat, that no new functions can be
> added without financing for them, and that the only means
> available for getting more resources into the current
> secretariat are raising the meeting fees, getting direct
> sponsors for CNRI's IETF activities, holding exhibitions at
> the IETF and US government funding. Allowing the sponsors who
> currently sponsor the IETF through ISOC to pay for it was not
> an option.

Yes. See above.

>... 
> Again, we have to fix the underlying problems before we can
> pop back up to this level. But once we get there, I don't
> think the current IESG is the biggest obstacle to doing so.
> 
>> need to be very closely coordinated (much more closely and
>> with much more responsiveness than we usually think of with
>> "contractor") and it would have to be staffed by people who
>> had, at least, a good understanding of the subject matter and
> I do not see that this follows. All of the items described
> above - checking that people are doing what they promised,
> following up when they do not, checking for adherence to
> formal criteria, sending back reasonably cogent notes when the
> checks fail, and following up - are fairly standard for any
> type of document-producing organization - whether it be a
> consortium, a standards organization or even a technical
> publishing house.

We should have this conversation at another time.  But please
reread my comment above about the incompetence this implies
vis-a-vis you fellow ADs.

>> chance that we want any version of this
>> "standards secretariat" function, we had best make certain
>> that the administrative structure can accommodate it, and
>> that it can do so with little disruption to its basic design
>> parameters.
> 
> A "contractor" would obviously have to work closely with the
> people running the process to get this done "right". But I
> don't see a particular reason to believe that an organization
> doing this under contract would be less able to make it work
> than a set of people hired by an organization more closely
> linked to the IETF to do the job.
> 

I do, partially because... well, when I have more time to write.

    john

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 06 Oct 2004 13:07:36 +0200

> that brings me back to me assertion above.

actually the Gen-ART review is a late-review function, and applies to ALL 
documents, not just General area docs - and I intend to put it forward as a 
model for "late review" in the ICAR WG once that WG turns its attention to 
"late review".

As for generalizing it to other areas - the inspiration for it came from 
experience in the Ops and Routing area directorates, which again was based 
on some experience from the MIB doctors and the Security directorate. I'm 
more fond of writing than most, so the Gen-ART team is more documented than 
other processes. I assume we need to tune it for wider usage - I was 
mentioning it as an example of what we are able to do without formalized, 
paid-for support - which could be improved with such support.

I do think our thoughts run very much in parallel - I'll be interested to 
hear more of why you think the "scenario O" organizational format will make 
it hard to make those support functions work.

                      Harald

---

From: John C Klensin <[email-masked]>
Date: Wed, 06 Oct 2004 17:50:04 -0400

--On Wednesday, October 06, 2004 1:07 PM +0200 Harald Tveit 
Alvestrand <[email-masked]> wrote:

>
> I do think our thoughts run very much in parallel - I'll be
> interested to hear more of why you think the "scenario O"
> organizational format will make it hard to make those support
> functions work.

Again a misunderstanding -- I don't see "Scenario O" as being 
either better or worse in regard to the above than any other 
scenario.  My concern is with the definition of the Clerk 
function, which is scenario-independent.

    john

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 07 Oct 2004 07:53:42 +0200

--On onsdag, oktober 06, 2004 17:50:04 -0400 John C Klensin 
<[email-masked]> wrote:

>
>
> --On Wednesday, October 06, 2004 1:07 PM +0200 Harald Tveit Alvestrand
> <[email-masked]> wrote:
>
>>
> Again a misunderstanding -- I don't see "Scenario O" as being either
> better or worse in regard to the above than any other scenario.  My
> concern is with the definition of the Clerk function, which is
> scenario-independent.

Thanks for the clarification!

I thought you might be pointing at the "one staff member - rest of the work 
is contracts", which is a common feature to scenarios C and O. If "all" 
that is required is to modify the description of the "clerk" function, that 
needs to be done before we call for interested parties - it is not on the 
critical path to adopting one scenario for implementation (although all 
clarification early is good).

                                 Harald

---

From: John C Klensin <[email-masked]>
Date: Thu, 07 Oct 2004 08:23:55 -0400

--On Thursday, October 07, 2004 7:53 AM +0200 Harald Tveit 
Alvestrand <[email-masked]> wrote:

> I thought you might be pointing at the "one staff member -
> rest of the work is contracts", which is a common feature to
> scenarios C and O. If "all" that is required is to modify the
> description of the "clerk" function, that needs to be done
> before we call for interested parties - it is not on the
> critical path to adopting one scenario for implementation
> (although all clarification early is good).

Harald,

I think the "one staff member" story is unrealistic as well. 
But, as you say, it is shared between C and O.  It is, by some 
creative definition-making, a bit easier for O because of the 
possibility of borrowing/sharing staff from/with ISOC.  But the 
fact remains that the document calls out several  more staff 
roles than the one employee it asks for.  Whether those staff 
members are "employees" or contractors working on an individual 
commitment basis (rather than hiring an organization to do the 
job without picking individual personnel) is, IMO, 
hair-splitting to make the staff look smaller, not a matter of 
substance.  It would be a matter of substance if it 
significantly changed either costs of management effort, but 
whether such people are employees or contractors doesn't help 
with the management and may actually increase costs.

This is one of my more general objections to the report -- in 
areas like the personnel one and how staffing roles are 
presented, it appears (intentionally or not) to be organized in 
such a way as to impede community understanding of what is being 
proposed.  But, again, this doesn't impact which Scenario is 
more appropriate, nor does it change the urgency or priorities 
of getting the relevant issues resolved.

    john
b

---

From: Carl Malamud <[email-masked]>
Date: Thu, 7 Oct 2004 07:20:07 -0700 (PDT)

> proposed.  

I'm not sure what you're referring to here, John.  You understood
it.  I assume the rest of the community is at least as smart.
This is a pretty hard community to impede, so I didn't try.

Rather than tell you, or other members of the community, what
to think, I tried to give you some additional facts.  You, as
well as everybody else, are perfectly able to draw your own
conclusions.  (And, should you wish it, I am available to talk
to you or any other member of the community to futher elaborate
any of these issues ... I'm even able to make concrete recommendations
if you'd like to hear them.)

As you rightly pointed out, there are more than one staff roles
that support the IETF.  You can do that as contractors or as
employees.  It just doesn't matter in the long run, in theory.
In practice, it depends on who you are able to attract who might
want to work for you.  And, as you've stressed a few times,
the first step is to get the administrative director ("IAD")
hired.

If you have specific suggestions for that job description, that
would be quite useful.  You've mentioned several times you didn't
like the one in the report, so this would be a good time to fix
that flaw.

Thanks!

Carl
,

---

From: John C Klensin <[email-masked]>
Date: Thu, 07 Oct 2004 17:22:59 -0400

Carl,

You know that this is going to take more than one employee (or 
individual contractor) and that the difference is hair-splitting 
for most reasons.   I know that.  The document if read carefully 
is pretty clear about that.

But I keep seeing "one employee only" comments which lead me to 
believe that some people are still confused.

    john

--On Thursday, October 07, 2004 7:20 AM -0700 Carl Malamud 
<[email-masked]> wrote:

>> presented, it appears (intentionally or not) to be organized
>> in  such a way as to impede community understanding of what
>> is being  proposed.
>
> I'm not sure what you're referring to here, John.  You
> understood it.  I assume the rest of the community is at least
> as smart. This is a pretty hard community to impede, so I
> didn't try.
>
> any of these issues ... I'm even able to make concrete
> recommendations if you'd like to hear them.)
>
> In practice, it depends on who you are able to attract who
> might want to work for you.  And, as you've stressed a few
> times, the first step is to get the administrative director
> ("IAD") hired.
>
> would be quite useful.  You've mentioned several times you
> didn't like the one in the report, so this would be a good
> time to fix that flaw.
>
> Thanks!
>
> Carl
> ,

---

From: "Tony Hain" <[email-masked]>
Date: Thu, 7 Oct 2004 17:10:12 -0700

> believe that some people are still confused.

Well I didn't think I was confused, but maybe. As far as I can tell the
Clerk function could be one of the contracted out services. To a large
degree the task list for the Clerk will determine just how efficient / cost
effective it would be to do that. If the Clerk's primary tasks are handling
the paperwork for the IAD, then it probably doesn't make sense to outsource
that. If as others have suggested the Clerk functions include processing
I-Ds and other tasks closer to the Secretariat service, then it should be
included as part of that service contract. 

To be clear, I am not confused about the ability of a single person handling
all of the tasks that the IESG/IAB might wish for. That just won't happen,
because even if such a person existed we couldn't afford them. That said, it
is not at all clear to me that we need more than a single person managing
well structured contracts with clear deliverables. If that person needs
independent administrative support, that could be just another contract. 

In a way we endanger the future viability of the IETF if we allow the IAD
office to grow beyond a single person. Human nature will make everyone
reluctant to cut permanent staff, while cutting back on contracted services
is much easier to swallow. 

Tony

---

From: Carl Malamud <[email-masked]>
Date: Fri, 8 Oct 2004 06:49:50 -0700 (PDT)

Hi Tony -

> included as part of that service contract. 

The Clerk functions are roughly equivalent tot the current Secretariat service
and I heard a pretty strong feeling that this should be contracted out,
as you say.  So, you're not confused.

> independent administrative support, that could be just another contract. 

A single person is the going in assumption.  When growing from zero, one is
a great first step.

> is much easier to swallow. 

And, that's the exact reason ... you've got it correct.  :)

Carl