Subject: RE: I-D ACTION:draft-etal-ietf-analysis-00.txt

From: "Michel Py" <[email-masked]>
Date: Sun, 14 Apr 2002 11:07:31 -0700

> Dave Crocker wrote:
> and, by the way, there is plenty of experience suggesting that
> time pressure often improves quality.  it focuses the group
> and emphasizes near-term utility.  within discussions about
> project management, it is usually recognized that milestones
> are not merely for measuring progress but also for motivating
> it.

I agree. In the case of the IETF, the worst part is that the milestones and/or deadlines are self-chosen. I have complained myself about some marketing bozo fixing arbitrary or unrealistic deadlines just because the product has to ship that day. This is not the way the IETF works. First, we can choose our own deadlines and second we can move them if we under-evaluated the time it would take. What is wrong is the generalized feeling that missing a deadline by two years (for a 6-month project) is ok. And the same is true for WG charters. Why do we have charters and milestones if nobody tries to meet them? Is it the way IETFers work at their day-time job?

And please, don't give me the crap "we do it for free". I do it for free, too. Quality has a little to do with money and a lot with engineers being interested, motivated, intellectually challenged, that kind of thing.

Michel.

---

From: Henning Schulzrinne <[email-masked]>
Date: Sun, 14 Apr 2002 14:59:39 -0400

Also, as efforts become more interconnected, working groups have
"customers" even within the IETF. It's not good if working group A can't
proceed in publishing a spec because of normative references from
working group B that can't get its act together. In that sense,
milestones are a contract between a working group and the larger
community. In particular, it's a contract between the spec
authors/editors and the larger community. A spec in a WG work item
generally gets some amount of exclusivity, in that the working group
isn't going to consider a similar spec by a competing set of authors.
(There are exceptions in highly contentious working groups, obviously.)
However, that only works if the authors deliver on time. Otherwise, an
effort should have been made to find (cajole, entice, browbeat,
sweet-talk) authors that have more time.

Milestones that are enforced are also useful to make resources
available. Speaking from recent working group experience, if a deadline
matters, employers are suddenly that much more willing to let people
work on the spec. Managers tend to understand the notion of a deadline.

Michel Py wrote:
> -
> This message was passed through [email-masked], which
> is a sublist of [email-masked]. Not all messages are passed.
> Decisions on what to pass are made solely by Raffaele D'Albenzio.

---

From: Robert Elz <[email-masked]>
Date: Mon, 15 Apr 2002 15:18:22 +0700

Date:        Sun, 14 Apr 2002 14:59:39 -0400
    From:        Henning Schulzrinne <[email-masked]>
    Message-ID:  <[email-masked]>

  | A spec in a WG work item
  | generally gets some amount of exclusivity, in that the working group
  | isn't going to consider a similar spec by a competing set of authors.
  | (There are exceptions in highly contentious working groups, obviously.)
  | However, that only works if the authors deliver on time.

It is also a large part of the reason why things don't get delivered on
time.  Not (usually, though even here sometimes) new versions of drafts,
but the completed product, that is what the milestone is about.

That is, because there is just one spec being written, it has to actually
catch the consensus of the group, and sometimes that is very hard to
determine.   Writing one set of words is easy, but if half the working
group then object, it has achieved little.

In a commercial environment, there'd be someone "higher up" who can use
one mechanism or another to resolve those differences "OK, we'll do it
this way", which everyone would then have to go along with (or leave).

The IETF doesn't have, and can't really have, any such arbiter.   On
occasion when things are very deadlocked, for very long times, various
schemes have been used to break out, but most of the time, it is just
time itself that eventually erodes one side's opposition, and allows things
to eventually proceed (either because of eventual frustration, or because
other events eventually show one solution or the other to be superior).
Of course, this means that the milestones (which generally couldn't have
predicted the impasse in advance) tend to fall by the wayside.   I'm not
sure there is any very good solution to this.

I suppose that we could do what most other similar bodies have done, and
simply write specs that satisfy everyone, with every conceivable option
included, and interoperability only by random chance - but I don't really
believe that any of us wants to start down that path.

So, it is all very well to tell working groups that they have to meet
their milestones, but it doesn't work unless some kind of (internal working
group) conflict resolution mechanism exists, and as much as possible, we
don't want that to turn into trade offs.

Shutting down working groups when they fail to meet milestones is an option,
but it certainly doesn't help that other working group you mentioned, which
is waiting on the results of the one which is deadlocked.

kre

---

From: Dave Crocker <[email-masked]>
Date: Tue, 16 Apr 2002 08:14:45 -0700

>is waiting on the results of the one which is deadlocked.

That other working group is already being not served.  Holding a working 
group to its milestones makes the situation more explicit.

Query to the group:  If we believe we should not hold working groups to 
their milestones, why bother to have those milestones?

d/

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

---

From: Aaron Falk <[email-masked]>
Date: 16 Apr 2002 10:47:15 -0700

> their milestones, why bother to have those milestones?

Dave-

Milestones are useful not just for the when but also for the what.  In
my opinion, working group charters have their impact on the quality of
the IETF output through scoping what the working group should and
(especially) should not do.  In the charter discussions I've
participated in this is made most explicit through the specification of
a set of deliverables (e.g., protocols, recommendations, etc).  This is
of enormous benefit from a managerial standpoint in that the key players
agree a priori on what is in and out of scope and the nature of the work
to be done.  

I'd be very concerned that, if we pushed working groups to deliver
faster, the first thing we'd lose would be the review from
cross-functional experts. These folks are already  very busy but their
commentary can be of enormous value.  It may contribute to longer
development of protocols but probably reduces the mess that would result
if protocols interact poorly on the Internet (which, as I mentioned in
an earlier email, is my criteria for IETF success).

One area I think we can improve is in finding ways to do more iterations
between IETF meetings.  My observation is that many working groups
operate on a four month clock, synchronized with the next face-to-face
meeting.  I think maybe we rely too much on working group meetings to
drive the schedule and to indicate consensus so as to move to the next
task.

--aaron

---

From: Dave Crocker <[email-masked]>
Date: Tue, 16 Apr 2002 10:54:51 -0700

>development of protocols but probably reduces the mess that would result

1.  We do not have any data to check against, but my impression is that the 
really serious delays are mostly due to issues internal to the working group.

2.  The problem with cross-functional review is that we do not do enough of 
it, no matter how much time there is.

d/

----------

---

From: Aaron Falk <[email-masked]>
Date: 16 Apr 2002 11:02:56 -0700

> really serious delays are mostly due to issues internal to the working group.

This highlights how important the choice of working group chair is. 
Building consensus can be hard.  

> it, no matter how much time there is.

No argument.  

--aaron

---

From: Henning Schulzrinne <[email-masked]>
Date: Tue, 16 Apr 2002 14:42:50 -0400

Dissolving a dysfunctional working group also allows for a reset, e.g.,
telling the first group that was waiting for a solution to develop a
more narrowly focused solution itself when the attempt at a broad
solution has failed.

---

From: Graham Klyne <[email-masked]>
Date: Tue, 16 Apr 2002 21:08:48 +0100

>their milestones, why bother to have those milestones?

I think there's a useful middle ground between slavery to milestones and 
completely ignoring them.

I think a group that has failed to deliver its milestones should reasonable 
expect to face some questions about why not, and absent any indication of 
reasonable cause then maybe shutting down is the appropriate response.

But (to be clear) I'm not sure that there's really anything here that calls 
for a change in the rules so much as maybe engendering a slight sense of 
urgency.  Sometimes, it's appropriate to nag folks to deliver on their 
promises in timely fashion (and I know I'm not entirely innocent of needing 
such prodding, on occasion).

#g

-------------------
Graham Klyne
<[email-masked]>

---

From: Dave Crocker <[email-masked]>
Date: Fri, 19 Apr 2002 14:23:53 -0700

>completely ignoring them.

There is a view of working group charters that they are a contract between 
the working group and the community.  As such, there is an argument in 
favor of respecting milestones diligently.  The the details of the 
milestones are part of the basis on which the IESG approves the group, so a 
working group's failure to meet the dates is, by definition, a failure to 
perform.

The piece of flexibility that you probably mean is that slippage happens 
and we need to deal with it constructively.

The normal way that is done in the rest of the world is to renegotiate the 
milestones.  This, of course, first requires paying attention to meeting 
milestones.  Since we largely do not do that, now, it would be a large 
advance in working group management to just pay attention.

>But (to be clear) I'm not sure that there's really anything here that 
>calls for a change in the rules so much as maybe engendering a slight 
>sense of urgency.  Sometimes, it's appropriate to nag folks to deliver on 
>their promises in timely fashion (and I know I'm not entirely innocent of 
>needing such prodding, on occasion).

Given current IETF cultural norms about meeting milestones, starting to 
take them seriously would constitute a very large change.

I started this thread trying to point out that we can deal with aggressive 
milestones in two different ways.  One is to move them later and the other 
is to take them seriously and try to meet them.

The current data show that we tend to take a long time to produce 
specifications. If we think that is fine, then there is nothing to fix.

Personally I believe that making and setting more aggressive milestones 
will make our work both more relevant to the current market and possibly 
even better technically.  (And, yes, the latter is counterintuitive.)

d/

----------