Subject: Post-hoc working group chartering

From: Dave Crocker <[email-masked]>
Date: Wed, 22 Jul 2015 09:07:41 +0200

The IETF tends to charter working groups based on a very large gamble.

Basically, we look for a modestly credible charter and some expressions
of community interest. Then we hope that the charter is sufficiently
substantive and the community interest is real.

Interest, however does not always translate into getting work done,
nevermind getting good-quality work done.  Sometimes it does not even
translate into people showing up on the mailing list.[*]

I suggest we alter our model for chartering in order to fix this.
Arguably the proposal is along the lines ofsimilar to what the IRTF is
now doing,

Before chartering, in addition to needing a credible draft charter:

     1.  There must be an active mailing list for the effort. The word
'active' is key.

     2.  There must be a credible base of participants; that is, not
just a couple of folk who are active.

     3.  The effort must demonstrate 'productivity' towards the goals of
the charter.

That is, there must be meaningful community involvement and it must show
that it can get work done.

The criteria of "meaningful involvement" and "productivity" should be
left as entirely subjective.  We could spend quite a bit of energy
trying to make them objective, but I don't think it's worth the effort.
 It's not that difficult to tell when a group has diverse participation
and is productive.

Simply put, therefore, we should charter a working group only after it
is a de facto working group and the group is already making progress on
its charter.

d/

[*] I've just had to shut down arcmedia for this reason, which is
serving as the trigger for this issue for me, but the issue is broader
and more long-standing, IMO

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Dave Cridland <[email-masked]>
Date: Wed, 22 Jul 2015 08:41:32 +0100

I'm not sure I entirely disagree with what Dave's written below, but I do
have some caveats.

Firstly, the prerequisites for working group formation under those rules
are going to be based on - essentially - forming a stealth working group,
which somewhat implies gathering people who know how this works, and I'm
not entirely sure that an external group would figure this out. Our
processes are fairly impenetrable to outsiders; I have a number of examples
of work that has "gone elsewhere" in part because of this. I don't think
making this worse has any positive effect.

Secondly, just as I-D is the new Proposed, I worry we're in danger of
"left-shifting" our process again, to paraphrase Scott Bradner. I make no
claim to know what the outcome of this particular case might be, but in the
case of the standards track process, the eventual outcome was ill-fitting
names and really not much else, and certainly hasn't gained us anything. I
have a horrible feeling we'd end up with the same discussions around
creating a mailing list as we do with creating a working group, or
something.

Thirdly, it's not actually clear to me that there is a problem with taking
on work - and even creating working groups - and then later dropping it if
the supposed interest doesn't materialize. I admit it must be quite
frustrating for the I-D authors and WG chairs in these cases, but I wonder
how much of that is in part because it's seen as a failure. As a radical
suggestion, perhaps if we lowered the bar for working group creation, but
felt more comfortable closing them, we might make a net gain.

This all said, the crux of Dave's suggestion here is that working groups
should essentially form somewhat organically, rather than artificially
created with much fanfare. I think there is a strong argument that working
groups should be able to form almost spontaneously from pre-existing
discussions on existing IETF mailing lists, with minimal friction, and that
pre-existing groups should be able to bring their work under the IETF
umbrella with similarly minimal roadblocks. I don't think this is in
opposition to Dave's proposal, but I admit to approaching this from a
different angle.

Most importantly, I think it would be fascinating to look at groups that
haven't brought their work to the IETF, and find out why, and decide
whether we wanted them to.

---

From: Eliot Lear <[email-masked]>
Date: Wed, 22 Jul 2015 09:50:08 +0200

I might add that Lars handles this rather differently for the IRTF.  He
advises prospective RGs to simply start acting like RGs, and so long as
there is activity that is within the IRTF's remit, things can progress
from there.  We can do that at the IETF too, but it probably changes the
way we would do room assignments a the face to face meetings, because
there might be an explosion of such protogroups.

Eliot

On 7/22/15 9:41 AM, Dave Cridland wrote:
> I'm not sure I entirely disagree with what Dave's written below, but I
> do have some caveats.
>
> Firstly, the prerequisites for working group formation under those
> rules are going to be based on - essentially - forming a stealth
> working group, which somewhat implies gathering people who know how
> this works, and I'm not entirely sure that an external group would
> figure this out. Our processes are fairly impenetrable to outsiders; I
> have a number of examples of work that has "gone elsewhere" in part
> because of this. I don't think making this worse has any positive effect.
>
> "left-shifting" our process again, to paraphrase Scott Bradner. I make
> no claim to know what the outcome of this particular case might be,
> but in the case of the standards track process, the eventual outcome
> was ill-fitting names and really not much else, and certainly hasn't
> gained us anything. I have a horrible feeling we'd end up with the
> same discussions around creating a mailing list as we do with creating
> a working group, or something.
>
> Thirdly, it's not actually clear to me that there is a problem with
> taking on work - and even creating working groups - and then later
> dropping it if the supposed interest doesn't materialize. I admit it
> must be quite frustrating for the I-D authors and WG chairs in these
> cases, but I wonder how much of that is in part because it's seen as a
> failure. As a radical suggestion, perhaps if we lowered the bar for
> working group creation, but felt more comfortable closing them, we
> might make a net gain.
>
> This all said, the crux of Dave's suggestion here is that working
> groups should essentially form somewhat organically, rather than
> artificially created with much fanfare. I think there is a strong
> argument that working groups should be able to form almost
> spontaneously from pre-existing discussions on existing IETF mailing
> lists, with minimal friction, and that pre-existing groups should be
> able to bring their work under the IETF umbrella with similarly
> minimal roadblocks. I don't think this is in opposition to Dave's
> proposal, but I admit to approaching this from a different angle.
>
> Most importantly, I think it would be fascinating to look at groups
> that haven't brought their work to the IETF, and find out why, and
> decide whether we wanted them to.
>
> On 22 July 2015 at 08:07, Dave Crocker <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     bbiw.net <http://bbiw.net>
>
>

---

From: "Eggert, Lars" <[email-masked]>
Date: Wed, 22 Jul 2015 08:45:56 +0000

Hi,

On 2015-7-22, at 09:50, Eliot Lear <[email-masked]<mailto:[email-masked]>> wrote:
I might add that Lars handles this rather differently for the IRTF.  He advises prospective RGs to simply start acting like RGs, and so long as there is activity that is within the IRTF's remit, things can progress from there.

yep, this is a good summary for what we currently do over on the IRTF side of things. I cooked up the IRTF process, because I wanted to make it easy for anyone to start new work and bring in new ideas, and at the same time make it easy for myself to end efforts that hadn't come together or had fizzled out.

When someone comes with an idea for a new RG, I ask them to write a page of descriptive text that looks much more like a call for participation for a conference than an IETF charter. If it fits underneath the IRTF's charter and goals, I explain to the proponents that they should go and reach out to the folks they need to have in the group to make it successful, start having meetings in the way they want to have them (during the IETF week, or at research conferences, interims, etc.), and do work in the way they want to work.

About a year later, I sit down with the folks and we review what trajectory the group has been on. It's usually very clear to all involved at that point whether something worthwhile has formed or not. That conversation is dramatically different (in a good way) than the conversation an AD has to have with proponents after a BOF.

Lars

---

From: Dave Crocker <[email-masked]>
Date: Wed, 22 Jul 2015 11:18:23 +0200

Dave,

Thanks for the thoughtful response.  Comments inline...

Meta point:

     BOFs serve multiple purposes and therefore are appropriate at
different times.  So the model I'm proposing doesn't cite BOFs
explicitly as part of the sequence.  That's not to deny their use but
rather means "do a BOF whenever it makes sense to do one"...

> working

It is already reasonably easy to get a non-working group IETF mailing
list.  I like that and suggest that that should be the usual entry point
for nascent wg efforts.  So:  not stealth and not outside the IETF.

> group, which somewhat implies gathering people who know how this
> works, and I'm not entirely sure that an external group would figure
> this out.

Required familiarity with IETF process is a basic and long-standing
issue, independent of my proposal.  There's been a theory of having one
of the co-chairs be IETF-savvy, but that's probably later in the
sequence than we need.

So I'll trivially suggest that when a non-wg mailing list is created, it
should be assigned an IETF-savvy mentor.  (This would carry no
implication that the mentor would become a co-chair, should the effort
get chartered.)

> Secondly, just as I-D is the new Proposed, I worry we're in danger
> of "left-shifting" our process again, to paraphrase Scott Bradner. I
> make

Certainly a fair concern, since I share your view on the overly-high bar
for Proposed.  (And I'm thankful we were, at least, able to make the
criteria for full Standard be simpler and more pragmatic.)

So, yes, I'm suggesting a higher bar for chartering.

But note that I am /not/ suggesting any change in what it takes to get a
mailing list.  I think that it should be pretty easy, requiring no more
than a basic Statement of Purpose and passing a basic 'smell' test with
the sponsoring AD. So the basic ability to conduct an Internet technical
discussion in the IETF and start doing work would not change.  To me,
that's the important step.

In fact, I claim this /lowers/ the bar for getting work done.  Here's my
logic:  right now, a group has to focus first on getting chartered.
That tends to take months.  Eventually they get to focus on the work.
But with my suggestion, they are expected to focus on the work immediately.

One of the current wg efforts /refused/ to pursue substantive technical
work prior to chartering.  This led to months of problematic technical
proposals being circulated, with no effort to deal with the problems.
(The problems did get attention, many months later.)  With my
suggestion, the expectation would be that substantive technical concerns
would be addressed immediately.

> dropping it if the supposed interest doesn't materialize.

We tend to ignore the economics of unsuccessful or problematic efforts,
essentially treating working groups as a cost-free entity.  And we
frequently produce useless specifications.

The reality is that these are extremely expensive.  They consume
expensive engineer time, scarce management resources, and scarce IETF
meeting time.

They often also impose an opportunity cost, while the world thinks that
something productive is being done about a relevant problem, when in
fact progress is not being made.  Alternatives are not pursued, based on
the mistaken belief that the problem is being worked on.

Successful IETF work requires clarity about the topic being pursued,
meaningful market need for pursuing it, and people willing to work on it.

Currently, the IETF criteria for chartering a working group pretends to
approximate measuring these, but in reality does not.  Rather it takes
only the vaguest expressions of interest as sufficient.  Expressing
interest is not the same as doing work.

I am suggesting fixing this disconnect by requiring at least
substantive, sustained demonstrations of interest in the topic and an
ability to make progress with it.

d/

---

From: Ofer Inbar <[email-masked]>
Date: Wed, 22 Jul 2015 11:06:17 -0400

What about a sort of "working group handfasting"?
Much lower bar for chartering, but the initial goal of the group is
very short term (like 6 months) and mainly designed to see if the
group seems likely to be productive.  With the expectation that,
by default, such a group would end after the initial period, unless
the group's members actively think it should continue; it would
take positive action to request a recharter at this point, with a
longer term mission, but it would be expected that such a recharter
be approved if the consensus of the group is behind it.

That seems similar to the idea of a higher bar for chartering, because
it makes the bar be "actually try having a working group for a number
of months and see what comes of it" - much like Dave's suggestion.
But, it also makes it much clearer and more transparent what is going
on, easier for new people to see it, and gives some support and
guidance to reaching that higher bar, making it more likely that
worthwhile efforts will clear the bar.
  -- Cos

---

From: Eric Burger <[email-masked]>
Date: Wed, 22 Jul 2015 11:55:46 -0400

I sort of like this better. The problem we have today was identified by Dave Cridland: it is already so hard to charter a work group, people give up and leave. You need an active mail list, an active BOF, and active BOF, run through the Dave Crocker and jck filter, and finally get chartered. Easily a year if not more.

Even with that heavyweight process, we manage to take on work that probably belongs elsewhere.

I like the I-D approach. Provide the resources to get a proto WG going, let it run for six months, and see if it is getting any traction.

I think that will let people relax that don’t want to see the IETF take on work that no one is interested in - if no one is really interested in the work, it will die in 6 months.

Conversely, if something that an AD or two thought would not be interesting takes off in that initial six months, I’m sure they will claim parentage :-)

Other thoughts?

> On Jul 22, 2015, at 11:06 AM, Ofer Inbar <[email-masked]> wrote:
>

---

From: John C Klensin <[email-masked]>
Date: Wed, 22 Jul 2015 15:02:20 -0400

--On Wednesday, July 22, 2015 11:55 -0400 Eric Burger
<[email-masked]> wrote:

> I sort of like this better. The problem we have today was
> identified by Dave Cridland: it is already so hard to charter
> a work group, people give up and leave. You need an active
> mail list, an active BOF, and active BOF, run through the Dave
> Crocker and jck filter, and finally get chartered. Easily a
> year if not more.

I can't speak for Dave (and wouldn't if I could) but, since you
called me out more or less by name...   I would be quite
enthused about setting up WGs on the basis of AD judgment,
without BOFs or other heavyweight processes, if the following
principles were agreed to and treated as conditions:

	(1) We did not hesitate to shut them down if they were
	not productive.  I don't want a rigid rule because I
	think we all know that, in practice, some WGs move at a
	different pace than others, but it is important that we
	be able to recognize "not meeting expectations and
	making reasonable progress" and act on it.  In the last
	several years, I believe we have one successful example
	of doing so and lots of examples to the contrary.  So,
	whether we are serious at that or not is still
	uncertain.  
	
	(2) There be evidence of enough committed people to do
	the work.  We cannot collect signatures in blood or
	otherwise enforce promises, but there must be more than
	an enthusiastic individual, company, or organization or
   two.
	
	(3) The initial list of proposed or likely participants
	or supporters must be sufficiently diverse to show broad
	community interest.  That is key to meaningful community
	consensus and even more important if there is
	involvement by one or more IESG members or the companies
	with which they are associated in the work.  
	
	(4) The IESG and the community have the be firmly behind
	the idea that "a WG produced this" is not a guarantee of
	either standardization or publication if the work turns
	out to be below the quality standards of the IETF or the
	RFC Editor.  We've seen a lot of "we spent a lot of time
	on this in a WG and therefore it is entitled to approval
	even if we will let the community tweak it a bit".  That
	is, IMO, a problem even with carefully designed and
	vetted WGs; it could be much worse if attached to a
	lightweight WG setup process.

I'd like to see WG-forming BOFs become a thing of the past
except when they are actually needed for problem definition,
workplan design, or similar activities.   It would be, IMO, much
more productive to allocate meeting time to a WG to see if it
can actually get something useful done than to spend the same
time discussing and theorizing about whether getting something
useful done is likely or certain.

Especially because I think (2) and (3) can only be realized by
being sure the IESG is on board with the principles and then
exercising discretion and good judgment and because the IESG has
to be signed up to make either (1) or (4) work, I don't think
this is feasible unless the IESG is committed to it and willing
to be held accountable if they fail to come through on those
principles. 

> that probably belongs elsewhere.

And _that_ is my one exception.   I'm concerned about the IETF
resenting and/or complaining bitterly when some other body takes
on work that we believe belongs to us.  I'm even more concerned
when we take on work that probably belongs elsewhere, especially
when there is evidence of forum-shopping.  So I think that, when
those issues exist, they do need to be aired before WG creation.
However, I think that review and any discussions that are needed
can be carried out very much more efficiently and with a higher
success rate than we have been doing/using.

> I like the I-D approach. Provide the resources to get a proto
> WG going, let it run for six months, and see if it is getting
> any traction.

Yes.  See above and note that I don't know that "six months" is
the right number for every group.  However, if it isn't, we
ought to be able to figure it out and set a more appropriate
topic at startup time or grant one extension and then stop.

> I think that will let people relax that don't want to see
> the IETF take on work that no one is interested in - if no one
> is really interested in the work, it will die in 6 months.
> 
> Conversely, if something that an AD or two thought would not
> be interesting takes off in that initial six months, I'm
> sure they will claim parentage :-)
> 
> Other thoughts?

Because it might prove illuminating, at least about a prior
round of this type of conversation but with a slightly different
focus, I recommend two long-expired I-Ds to anyone interested in
this topic:

 https://datatracker.ietf.org/doc/draft-huston-ietf-pact/   and
 https://datatracker.ietf.org/doc/draft-klensin-overload/

--john

---

From: Melinda Shore <[email-masked]>
Date: Wed, 22 Jul 2015 11:40:04 -0800

On 7/22/15 11:02 AM, John C Klensin wrote:
> 	making reasonable progress" and act on it. 

My mental model for how-things-work in the IETF is based on
the working group secretaries situation - there was a sense
that working group chairs were overloaded and in some cases
not doing their jobs, and rather than remove a non-functioning
chair or add another chair, a decision was made to add a new
role (wg secretary) and an effort to formalize that role.
That is to say, when things aren't working we tend to add
more structure and more layers rather than removing structure
and layers.  Because this tends to happen, I am going to find
it difficult to support a process change that is structured
around an assumption that when things go badly groups will
be shut down and chairs will be fired.

I basically like this proposal, although I'm going to tend
to like any proposal that reduces leadership workload and
that pushes a chartering decision back beyond the point at
which a typical working group transitions from initial
enthusiasm to committed core.  But I think we need to deal
with the IETF as it is, which means a growing core of
professional standardizers among both the participant and
leadership (suggesting that people will basically work on
anything, regardless of actual utility) and a predisposition
to add goo rather than to remove it and against saying "no"
to established efforts and chairs (which also goes to
your fourth criterion, that sunk effort is not a guarantee of
publication or standardization).

Melinda

---

From: John C Klensin <[email-masked]>
Date: Wed, 22 Jul 2015 16:29:05 -0400

> be shut down and chairs will be fired.

I was trying to be positive and optimistic and I hope the recent
experience with ARCHICE marks a turnaround, but I certainly
agree that, if we cannot, in practice, shut WGs down and do so
efficiently and with little fuss, this whole idea should be
considered a non-started.   FWIW, I personally believe that this
long and complex process for creating WGs evolved precisely
because we could not shut a WG, once created, down and because
people believed that having a WG created entitled them to
produce standards and have them approved.  If those are our
beliefs in actual practice, then the only choice is to try to
keep WGs that might turn disfunctional from being created.

> enthusiasm to committed core.

I would add Eric's desire to get WG's started before people run
out of steam and a desire to spend time getting work done (if
possible) rather than speculating on whether it will be
successful to that list.

>  But I think we need to deal

Yep.

thanks,
    john