Subject: Re: Comments on draft-aboba-sg-experiment-02

From: Bernard Aboba <[email-masked]>
Date: Wed, 10 Oct 2007 17:43:21 -0700 (PDT)

Sam Hartman said:

"We move on to evaluating its work.

Now, there is a question of what happens when an SG misses its
milestones.  I could see an argument that hints at us being wrong that
we had sufficient interest.  Or perhaps an argument that the task we
set the SG was harder than anticipated."

Right.  

Since the bar for SG creation is the expectation of progress on basic 
milestones, if the SG doesn't deliver, the AD would need to 
re-evaluate the situation.  For example, if there is both an absence of 
progress as well as lack of energy (e.g. minimal mailng list 
traffic), then an extension (or even a second BOF) might be hard to 
justify. 

It is possible that the SG could deliver substandard output (e.g. review 
of the Charter and 2418 criteria garners poor reviews from the IESG/IAB 
and IETF commmunity).  In that situation, the AD needs to decide to 
give up or extend the SG. 

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: John C Klensin <[email-masked]>
Date: Thu, 11 Oct 2007 04:38:35 -0400

Hi.

I've been trying to follow this discussion from a safe distance,
but, given the amount of traffic, I want to add three general
observations to the mix, possibly summarizing some earlier
comments including my own:

	(1) Drawing analogies from IEEE, or IEEE 802, procedures
	to the IETF and extrapolating what will work is a tricky
	business, if only because of different ground rules for
	participation and approval of projects and different
	assumptions about standing committees / project areas in
	the two groups.
	
	(2) As the discussion goes on, if appears to me that, in
	practice, an SG is little more than a normal WG with an
	unusual set of charter-time constraints.  While unusual,
	those constraints are well within the boundaries of what
	the IESG is permitted to do today.  Indeed, I believe
	that groups have been chartered under constraints and
	with responsibilities not appreciably different from
	those that would apply to an SG.  From that perspective,
	our real problem is that, in the last few years, IESG
	members have not felt that they have sufficient
	community support (or pressure) to quickly and
	efficiently shut down WGs that are drifting or otherwise
	not performing.   An SG model could provide a less
	painful shutdown point by permitting simply not
	extending the SG or not granting a WG charter, rather
	than having to shut down something that has a few
	advocates or some investment, even if it is not making
	progress.  But, as others have pointed out, that same
	opportunity exists in principle when WG benchmarks are
	not met.  Whether an SG model is effective in
	eliminating pre-project efforts that would go nowhere
	would seem to depend on precisely the same factors that
	make the possibility of WG shutdown effective: support
	by the community for identifying and shutting down
	disfunctional efforts and willingness by the ADs for
	doing so.
	
	(3) I am concerned about anything that creates new
	opportunities for making the process of getting work
	into, through, and out of the IETF even longer and more
	complicated.  The worst risk associated with introducing
	SGs is that, just as we have evolved from "A BOF, or
	sometimes two, are optional steps to help get a WG
	organized and focused"  to "BOFs are almost always
	required", we might evolve into the expectation of
	spending one or more IETF meetings in SGs, as well as an
	additional meeting or two in BOFs, before generating a
	typical WG charter.  In the notation of Lakshminath's
	recent chart, we have moved from "Idea -->  WG" being
	the normal case, to "Idea -->  BoF-1 --> BoF-2 --> WG"
	being the normal case, to some risk of making "Idea -->
	BoF-1 --> BoF-2 --> SG --> WG" normal.  Under some
	circumstances, that would be a year well spent.   Under
	others, it is just another year wasted in exploration,
	extended problem description, and procedures when the
	goal should be to do technical work and get useful
	results out (and to cut unfocused ideas and ideas with
	insufficient support out of the system with as little
	wasted time and energy as possible).

Nonetheless, it seems clear from the list discussions that there
are a number of people in the community who think this is worth
trying.  Precisely because it doesn't seem to create any
substantively new options or framework and because its use is
optional, it seems to me that it is worth trying, if only in the
hope of refocusing the energy that seems to be going into
discussing its details back onto technical work.   I would
suggest that the "experiment" model was designed with the
intention of permitting experiments to be flexible and that it
would be preferable to either 

	* give the IESG discretion to set and evolve the details
	or 
	
	* to establish an ad hoc experiment specification,
	modification, and evaluation group to fix and modify
	details and generally guide the experiment and advise
	the IESG and the community about successes and failures
	on an ongoing basis.  

Very detailed design --whether of technology or procedures -- on
the IETF list has almost never worked for us in the past, yet it
seems that is exactly what people are trying to do with this
proposal.

Just my opinion, of course.

    john

---

From: Spencer Dawkins <[email-masked]>
Date: Thu, 11 Oct 2007 09:16:31 -0500

John,

Thank you for upleveling. I plead guilty to wandering through the detailed 
design.

I agree without comment with the rest of your observations, but one stuck 
out.

> those that would apply to an SG.

If your understanding of the intention for SGs is correct (and I think it 
is), and if current process BCPs allow this mode of operation (and I think 
they would), I'm wondering if this draft should be published as an ION.

ftp://ftp.rfc-editor.org/in-notes/rfc3933.txt is intended for process 
CHANGES. As we both know, the goal was to describe a lightweight alternative 
to formal/two-BOFs-plus-a-WG change to formal BCPs - RFC 3933 experiments 
are still relatively heavyweight, compared to publishing an ION.

Perhaps all that is required is to describe a style of WG charter, since the 
IESG already has discretion to charter as many, or as few, WGs with this 
style of charter as seems appropriate, and has discretion to continue doing 
that, or to stop doing that, as results seem to dictate.

If there are formal process changes required (doesn't matter what change, 
only that SGs require something that is explicitly disallowed in RFC 2418), 
then a process experiment would make sense, but if we don't change formal 
processes we don't have to worry about unintended side effects that require 
formal process change AGAIN to fix (with 
http://www1.ietf.org/mail-archive/web/ietf/current/msg48434.html for a 
recent example of what "require formal process change AGAIN to fix" looks 
like, for anyone else who might have missed it).

Thanks,

Spencer

---

From: Lakshminath Dondeti <[email-masked]>
Date: Thu, 11 Oct 2007 10:03:19 -0700

Just for the record, if the norm ends up being "Idea --> BoF-1 --> BoF-2 
--> SG --> WG," I would be very disappointed and would chalk that up 
under the law of unintended consequences :).  I am hoping that "Idea --> 
SG --> WG" or "Idea --> BoF1 --> SG --> WG" in that order become the 
norm (where SG is involved of course), especially when proponents of new 
work are people who may not be regulars at the IETF.

regards,
Lakshminath

On 10/11/2007 1:38 AM, John C Klensin wrote:
  In the notation of Lakshminath's

---

From: John C Klensin <[email-masked]>
Date: Thu, 11 Oct 2007 14:02:40 -0400

--On Thursday, 11 October, 2007 10:03 -0700 Lakshminath Dondeti
<[email-masked]> wrote:

> Just for the record, if the norm ends up being "Idea --> BoF-1
> --> BoF-2 --> SG --> WG," I would be very disappointed and
> would chalk that up under the law of unintended consequences
> :).

Unfortunately, there is some history in the IETF that might lead
someone who is even mildly cynical about the way procedures
unfold and are used (and it is probably obvious that I passed
"mildly" long ago) to predict that would be exactly the outcome,
unintended or not.   To the extent to which the IESG tends to be
responsive to community input and sensitive to constraints about
meeting and management time --and I'd hope they would be
responsive to both -- it might take only a few loud voices
saying "not ready" to turn BOF1 into a requirement for BOF2
(arguably we see that already) and similar voices to turn the
outcome of BOF2 from "WG" into "SG and more consideration".  

In practice, that might turn at least some SGs into exactly what
I think you are trying to avoid: an indeterminable series of
BOF-list sessions with a charter and under a different name.
That would also be an unintended consequence, but I don't see
how to avoid it other than to trust the IESG to manage SGs more
aggressively than they have historically managed WGs and to
assume that the community would vigorously support them in
applying that level of management.

  I am hoping that "Idea --> SG --> WG" or "Idea --> BoF1
> --> SG --> WG" in that order become the norm (where SG is
> involved of course), especially when proponents of new work
> are people who may not be regulars at the IETF.

If the goal is really the education and socialization of
newcomers and refinement of their ideas, shouldn't we be
thinking about more direct, explicit, and efficient ways to do
that, rather than about new procedures and process steps?

    john

---

From: Lakshminath Dondeti <[email-masked]>
Date: Thu, 11 Oct 2007 11:25:03 -0700

My cynicism hasn't reached that level yet, although people tell me that 
it will get there very soon given some of the activities I have signed 
up for.  :)

That out of the way, the intention of the proposal is to provide a 
framework for people with reasonable ideas for work at the IETF to 
develop those ideas and eventually succeed in standardization of 
protocols in that area, and sooner than later.  (the venue for 
developing the ideas for work is the SG whereas standardization of 
protocols is to happen in WGs that may or may not have been formed as a 
result of the SGs).

regards,
Lakshminath

---

From: Pekka Savola <[email-masked]>
Date: Fri, 12 Oct 2007 07:47:23 +0300 (EEST)

On Thu, 11 Oct 2007, Lakshminath Dondeti wrote:
> under the law of unintended consequences :).  I am hoping that "Idea --> SG 
> --> WG" or "Idea --> BoF1 --> SG --> WG" in that order become the norm (where 
> SG is involved of course), especially when proponents of new work are people 
> who may not be regulars at the IETF.

One of the reasons for having a BoF is that the BoF proponents need to 
convince the rest of the IETF that the idea is workable and there's 
sufficient interest to work on the topic.

If there is expectation that no BoF is held between the SG and WG 
phase, how can we guarantee that the IETF as a whole thinks the 
charter and the other deliverables the SG worked on are convincing and 
worth doing?

As for the timeslot scheduling, I'd say SGs should have a precedence 
over IRTF research groups, given that we're talking about IETF 
meetings, not IRTF meetings.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Pekka Savola <[email-masked]>
Date: Fri, 12 Oct 2007 07:47:23 +0300 (EEST)

---

From: Lakshminath Dondeti <[email-masked]>
Date: Thu, 11 Oct 2007 22:31:57 -0700

On 10/11/2007 9:47 PM, Pekka Savola wrote:
>> Just for the record, if the norm ends up being "Idea --> BoF-1 --> 
>> BoF-2 -->  SG --> WG," I would be very disappointed and would chalk 
>> that up under the law of unintended consequences :).  I am hoping that 
>> "Idea --> SG --> WG" or "Idea --> BoF1 --> SG --> WG" in that order 
>> become the norm (where SG is involved of course), especially when 
>> proponents of new work are people who may not be regulars at the IETF.
> 
> If there is expectation that no BoF is held between the SG and WG phase, 
> how can we guarantee that the IETF as a whole thinks the charter and the 
> other deliverables the SG worked on are convincing and worth doing?

Hi Pekka,

Section 2.3 of 2418 would apply.

regards,
Lakshminath
> over IRTF research groups, given that we're talking about IETF meetings, 
> not IRTF meetings.
>