Subject: [proto-team] Re: PROTO - proceeding on adding PROTO shepherds to the tracker

From: Brian E Carpenter <[email-masked]>
Date: Sun, 14 May 2006 13:34:47 +0200

Thanks. Here are my comments on the two drafts.

One general question: has Michael Lee reviewed them
for gotchas?

One general suggestion: let's not waste time in the RFC queue
with these. Once they're agreed, just do it. You can post
<?rfc private ?> versions on the PROTO site for the record.

    Brian

draft-ietf-proto-wgchair-tracker-ext-00

> 2.  I-D Tracker Write Access
...
>    *  Identification of the actions and information which may not be
>       accessed by all users (R-002).  Such actions and information will
>       be called 'restricted features' in the following.  Some known
>       restricted features are:

It would be good to see an updated and marked-up state table and
state diagram, with the restricted features clearly identified.

> 3.1.  WG Document States
...
> 
>    *  WG Document Awaiting Reviews
>       This document needs reviews (possibly a certain number of reviews,
>       at a minimum) before a WG last call will be done.
> 
>       Possible next states: "Active WG Document", "Parked WG Document",
>       "Publication Requested", "In WG Last Call", "Dead"
> 
>       Permitted sub-states: "0 reviews", "1 reviews", "2 reviews", "3
>       reviews", "4 reviews", "5 reviews", "Awaiting MIB Doctor Review",
>       *** More special review states ***
>       (R-008)

This will be very useful for the proposed early cross-area review
mechanism. (Since the recent IESG retreat, Lisa owns that topic.)
Can we have a substate "Awaiting cross-area review"?

Linked to that, what happens about state change notifications?
For example, the dispatcher for cross-area reviews needs to get
a trigger when the sub-state "Awaiting cross-area review" is set.

> 5.  Modification of Existing States
> 
>    One existing sub-state in the tracker should be modified to reflect
>    the role of the WG document shepherds.
> 
>    The sub-state "AD Followup" is defined as generic and may be used for
>    many purposes by an Area Director.  However, the tracker
>    automatically assigns this sub-state when a document which has been
>    in the "Revised ID Needed" sub-state is updated.  The "AD Followup"
>    sub-state shall continue to exist for the first purpose, but when a
>    document is in "IESG Evaluation - Revised ID Needed" and an update
>    arrives, it shall receive an automatic state change to a new sub-
>    state instead: "Doc Shepherd Followup" (R-022).

But not for non-WG documents, which should still get "AD Followup."

draft-ietf-proto-iab-irtf-tracker-ext-00

You don't mention access control.

I'm surprised you don't expect "Revised ID Needed" to be used.

    Brian

_______________________________________________
proto-team mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/proto-team

---

From: Henrik Levkowetz <[email-masked]>
Date: Sun, 14 May 2006 14:47:26 +0200

Hi Brian,

Comments inline.  I've put up new working documents (-01.a and -01.b,
respectively, at

http://www1.tools.ietf.org/wg/proto/draft-ietf-proto-wgchair-tracker-ext/

and

http://www1.tools.ietf.org/wg/proto/draft-ietf-proto-iab-irtf-tracker-ext/

but note that more input is needed related to some comments below:

> One general question: has Michael Lee reviewed them

No, not yet.  I'll send a note asking him to check out the -00
versions.

> <?rfc private ?> versions on the PROTO site for the record.

Works for me.

> state diagram, with the restricted features clearly identified.

Ok.  What about an additional bullet and requirement:

   *  An updated state table and state diagram, with restricted
      features clearly identified in both (R-010).

> Can we have a substate "Awaiting cross-area review"?

Certainly. Added.

> a trigger when the sub-state "Awaiting cross-area review" is set.

If the tracker currently supports notification triggers, I'd add
that here - but I don't think it does, and in that case I'd suggest
we pull this information from the tracker in the review support tool
which Tero Kivinen is currently working on.  I'll forward a copy of
this mail to him, for information.

> But not for non-WG documents, which should still get "AD Followup."

Proposed revision, specifying 'working group document is in "IESG Eval...'
and adding a clarifying sentence at the end:

   The sub-state "AD Followup" is defined as generic and may be used for
   many purposes by an Area Director.  However, the tracker
   automatically assigns this sub-state when a document which has been
   in the "Revised ID Needed" sub-state is updated.  The "AD Followup"
   sub-state shall continue to exist for the first purpose, but when a
   working group document is in "IESG Evaluation - Revised ID Needed"
   and an update arrives, it shall receive an automatic state change to
   a new sub-state instead: "Doc Shepherd Followup" (R-022).  Non-WG
   documents continue to change state to "AD Followup" as before.

> You don't mention access control.

Not sure I know exactly what you think of here - this document
was only supposed to describe the additional IAB and IRTF states
needed, and having little knowledge of their process, I expect
someone else to provide the needed text adjustment for those.
But I get the impression that you also expect special access
control restrictions associated with the IAB and IRTF states?

> I'm surprised you don't expect "Revised ID Needed" to be used.

I'm sure it should - I've added it in a couple of places in rev. 01.b,
but my confidence that these states are accurate until we get text
from the IAB / IRTF is still low...

	Henrik

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 14 May 2006 16:03:23 +0200

[resend due to address problem the first time]

    Brian

...

...

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 14 May 2006 16:07:19 +0200

[resent to correct Ray's address again]

-------- Original Message --------
Subject: Re: PROTO - proceeding on adding PROTO shepherds to the tracker
Date: Sun, 14 May 2006 14:47:26 +0200
From: Henrik Levkowetz <[email-masked]>
To: Brian E Carpenter <[email-masked]>
CC: [email-masked], [email-masked], [email-masked], [email-masked],   Aaron Falk <[email-masked]>
References: <[email-masked]> <[email-masked]>

Hi Brian,

and

Works for me.

Certainly. Added.

	Henrik

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 14 May 2006 16:12:03 +0200

Henrik Levkowetz wrote:
> Comments inline.

ditto

>  I've put up new working documents (-01.a and -01.b,
> respectively, at
> 

Thanks

> that here - but I don't think it does,

It does for certain transitions - Michael can advise.

> and in that case I'd suggest

Exactly
> control restrictions associated with the IAB and IRTF states

Well, I hadn't really thought it through - but they presumably will
only have write access for IAB/IRTF drafts respectively.
> from the IAB / IRTF is still low...

:-)

      Brian

---

From: Henrik Levkowetz <[email-masked]>
Date: Sun, 14 May 2006 16:49:18 +0200

Hi again,

on 2006-05-14 16:12 Brian E Carpenter said the following:
...

> It does for certain transitions - Michael can advise.

Ok.  I've asked about this in the email to Michael. I've also
provisionally added this after the list of new states in -01.b :

3.2. Notifications	

   Under the assumption that the tracker already has the ability to send	
   email notifications when certain state transitions occur, the tracker	
   should be able to send out notification to one or more email	
   addresses when one of the followings states or sub-states are	
   entered(R-012):	

   * Awaiting MIB Doctor Review	

   * Awaiting Cross-Area Review

...
> only have write access for IAB/IRTF drafts respectively.

Ok.  For now I've added this to the requirements in Section 2 of
draft-ietf-proto-wgchair-tracker-ext:

   *  There should at least be individual groups corresponding to 'ADs',
      'Chairs', 'IAB' and 'IRTF', permitting per-group access control of
      actions and features with this granularity (R-011).

Regards,

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 15 May 2006 23:29:14 +0200

on 2006-05-15 20:51 Lisa Dusseault said the following:
> On May 14, 2006, at 7:49 AM, Henrik Levkowetz wrote:
>>
>> 3.2. Notifications	
>>
> It seems that there are other kinds of expert review besides MIB  
> doctor.  You could consider the security review to be an expert  
> review although naturally that can also be cross-area review.  The  
> MIME Types list does some kind of expert review which could be  
> handled the same way as MIB doctors. And it's interesting to consider  
> expert review for i18n issues.  Should this be simply "Awaiting  
> outside review" or, if we want to preserve the difference, "Awaiting  
> expert review" and "Awaiting cross-area review"?  If we preserve the  
> difference, what if both kinds of review happen simultaneously?

Yees, I've already been pondering how to capture the possibility of
more than one review being pending at the same time.  Either we can
reduce the state to something very general, or we can consider ways
to either have multiple instances of a sub-state, or having a sub-
state with an annotation filed.

Unfortunately, both of these are bound to result in larger extensions
to the tracker's capabilities than what was envisioned for this
round...

I think that with no constraint on the amount of change we could
introduce I'd advocate a fairly complete set of explicit review
actions, and the possibility of setting more than one sub-state
at the same time.  Then it's easy for other tools to pick up the
sub-state and use that to trigger other actions.  Any free-form
field makes that much harder and less predictable; and reducing
all the states to very generic ones makes it impossible to kick
off a particular review dispatch from the state change.

Whether we have the option to include this in the work package I
don't know, however.

> I do have an action item from the retreat to work more on the early- 
> review problem, so my questions here are somewhat influenced by  
> thinking how that might work.    (And by the way Allison, Aaron,  
> Henrik and proto-team, I welcome input from all quarters on that.)

Noted.  You should know (you probably do, but I mention it anyway)
that Tero Kivinen has started working on a support tool for review
groups.

Regards,

	Henrik

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 16 May 2006 09:35:09 +0200

> Whether we have the option to include this in the work package I

I would hate the result to be that the tracker artificially
serializes reviews.

I think that you should include the requirement, but with
flexibility for the implementor. If handling multiple parallel
review-waits is a challenge, the fallback as you say is annotation.
(You already have "5 reviews" as a substate. I was wondering
how that was going to be handled, except by annotation.)

     Brian

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 16 May 2006 11:19:01 +0200

Hi Brian,

...
> how that was going to be handled, except by annotation.)

How about adding this at the end of Section 3.1:

   "In several cases, such as for the 'WG Document Awaiting Reviews'
   state, it is highly desirable to be able to set multiple sub-states
   at the same time (R-013).  This is in order to avoid serialisation of
   reviews because the tracker doesn't support showing and indicating
   parallel reviews."

?

	Henrik

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 16 May 2006 11:59:54 +0200

Works for me.

    Brian

---

From: Lisa Dusseault <[email-masked]>
Date: Mon, 15 May 2006 11:51:55 -0700

thx,
Lisa

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 16 May 2006 14:48:53 +0200

>>    reviews because the tracker doesn't support showing and indicating

Ok.  Added in -01.e:

	Henrik

---

From: Lisa Dusseault <[email-masked]>
Date: Tue, 16 May 2006 10:13:55 -0700

Why would this have to be sub-states?  Why not just a state of  
"Document in external review" and the comments (or a review announce  
form) can indicate who the review request was sent to?

Trying to simplify, so if this isn't a real simplification, feel free  
to ignore.

thx,
Lisa

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 16 May 2006 19:24:32 +0200

Hi Lisa,

> Trying to simplify, so if this isn't a real simplification, feel free  

3, reasons, I think:

1. From my viewpoint, it looses state unnecessarily - "Document In
External Review" could happen at several different points in the
process, both before and after WGLC, and possibly after publication
request and ?? after IETF Last Call.  It would also loose the
difference between a WG document, an IAB document and an IRTF
document.

2. The need for multiple parallel states wouldn't be handled by this
especially well - and what's more, I'd not want to have the ability
to set multiple parallel states available for the major states - for
those, only one should be possible at any given time.

3. Using comments to indicate who the review request was sent to
is essentially equivalent with having a note field for the state -
since you could never be sure that everyone added the annotation
in the same way, you couldn't reliably automate other tools' use
of this information to trigger notifications, actions or whatnot.

Best,

	Henrik

---

From: Bill Fenner <[email-masked]>
Date: Tue, 16 May 2006 11:48:14 -0700

Here's a question:

Will the WG want to use the "in WG" states when the
IESG has said "Revise this document to address these
concerns" (e.g., AD Evaluation/Revised ID Needed, or
IESG Evaluation/Revised ID Needed)?

I think the answer could be "Yes", meaning that this is
actually a parallel state machine, not an extension to
the existing state machine.

  Bill

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 17 May 2006 10:29:20 +0200

Hi Bill,

> the existing state machine.

Mmm.  Either that (but then the two state machines are
bound together by constraints, for the WG states are
only meaningful in some IESG states) or we add WG states
for the cases you mention, which are different from the
pre-pub-request states.

Preferences?  I think I'd prefer the latter, to avoid
having to think about imposing constraints on two bound
state sets.

	Henrik

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 17 May 2006 10:58:55 +0200

> the existing state machine.

bzzzz... one of those state machines with multiple red dots
running round them. This could get interesting, depending
on how the tracker's basic model is coded. But I think
the requirement is fairly clear - parallelism is needed
at several points in the life cycle.

     Brian

---

From: Lisa Dusseault <[email-masked]>
Date: Thu, 18 May 2006 09:43:36 -0700

Parallelism could be hard for humans to keep track of.  I think there  
are other options but I'm starting to believe that writing tools  
requirements before we know what we're going to do is premature :)

I know I have the token on early review, and what I'm doing about  
that right now is talking to Ross and Bill about what kind of  
experiment they might run in Routing, and considering running an  
experiment myself.

lisa

---

From: Bill Fenner <[email-masked]>
Date: Thu, 18 May 2006 12:33:40 -0700

>having to think about imposing constraints on two bound

Well, I have another idea, which may address both the
"multiple review" issue and the "wg vs IESG states" issue,
but may be too much of a change of model.

The idea is still rough, but basically boils down to
thinking of a document as a ticket in a ticket system,
and blocking tasks as sub-tickets which get opened and
closed as needed.  E.g., in the current state machine,
"New ID Needed" might be replaced with the creation of
a blocking sub-ticket "New ID Needed", which is automatically
closed when the new ID is submitted.  This may solve the
problem of multiple reviews in the WG: create multiple
blocking sub-tickets, for review and WGLC and etc., and
we know the document can proceed when all the sub-tickets
are closed; similarly, the IESG "New ID Needed" sub-ticket
could move around in the WG states all it wants.

That's probably too much of a paradigm change to be the kind
of change to the tracker that we'd like to make now.

When we were designing the tracker states, we originally
put in occasional "pause" states, e.g., there were

AD Evaluation <----> New Version Needed (WG/Author)

and a huge complicated mess of states

Ready for Telechat
Evaluation: DISCUSS
Discuss writing received
Token @WG or Author
New Version coming
AD Re-review
Discuss fix received
Discuss Cleared, wait for approval

which ended up all being resolved with the "generic pause"
mechanism implemented by substates.

I'm concerned that we'll end up with a set of in-WG
states for several different in-IESG states (in
particular, AD Evaluation and IESG Evaluation) so it's
not clear who gets to do what when and use which state
for what and ...

One of the state machines during the design phase
(I don't remember what the colors meant) is at
http://rtg.ietf.org/~fenner/iesg/datatracker-3.1-diagram.ppt

  Bill