Subject: Author Incentives - Re: [Sipping] Re: [Sip] Thoughts on the PrincipleofDividingandConquering

From: Cullen Jennings <[email-masked]>
Date: Wed, 01 Mar 2006 09:30:48 -0800

These comments sure ring true to me. I leave every IETF with good intentions
of not waiting till the next deadline to rev my drafts but then I don't. I
have heard a few ideas come up:

1) Stuff Jonathan mentions below

2) Regular conference calls. The webdav group just did several months of 3
hour conference calls twice a week (yes 6 hours a week total). It was
painful but got the draft to WGLC.

3) Interim meetings.

4) Chair hounding authors

5) A fast track that was an agreement of the authors to submit more often
and the WG to review when they submit.

Clearly we could use any combination of these. Others I am missing? Do we
have some drafts that folks feel could benefit from one of the above?

On 2/27/06 6:09 PM, "Jonathan Rosenberg" <[email-masked]> wrote:

> 
> 
> Dan Wing wrote:
> 
>> The motivation for finishing I-Ds before the WG meeting is the
>> I-D cutoff date and the pressure to have made progress since the
>> last meeting.
> 
> Actually, I think the motivation really comes down to one thing: shame.
> The IETF meeting is where everyone meets and drafts that made progress
> get discussed and talked about. If you didnt revise your draft because
> you didn't get around to it, its embarassing. I know :)
> 
> Another aspect of this is the meeting blackout period. Because of it, if
> you don't update your draft NOW, you'll have to wait another three weeks
> to do so.
> 
> 
> 
>>  What you're describing is another way to create
>> pressure.  I agree that will help accelerate things.  We might
>> be best served by accelerating the important things -- a 'fast
>> track', if you will.  This would require editors to submit
>> documents 'between' the normal cutoff dates (IETF meetings)
>> and require substantive review and comments from the community.
> 
> There are two ways to create pressure- carrots and sticks.
> 
> The IETF draft deadlines are sticks, I think.
> 
> To find more sticks and carrots you need to look at what incents people.
> I think all of the following are real incentives:
> 
> 1. people are incented to get RFCs published with their names as
> authors; it brings recognition to their peers and the industry.
> 
> 2. people are incented to get RFCs published that benefit their day-job,
> because their day job requires problem X to be solved soon. The carrot
> here is the the day job = $$.
> 
> 3. people are incented to submit new drafts and work on new drafts
> because everyone likes working on the new cool stuff, rather than the
> tired-of-it stuff that is 90% complete, where the work is just getting
> that MUST/SHOULD correct.
> 
> 
> I'm sure there are others. Given those incentives, you need to tie them
> into making forward progress on our work in genreal.
> 
> One idea, for example, is to tie draft acceptance to reviews. Lets say I
> write draft X and I want it adopted by the WG. I can't get my draft
> accepted by the WG as a new item, until I commit to reviews and
> assistance in solving open issues for another draft that is already a WG
> item. Only once it is approved by IESG to I get to have my draft added.
> This is a carrot incentive.
> 
> A related one, is that if a draft author does not complete a set of
> assigned reviews in time, or does not produce a revision of the spec on
> an agreed-upon timetable, they are removed as author. This is a stick
> version of the same incentive.
> 
> No doubt there are others.
> 
> I personally think most of our problems are NOT with iesg or with WG
> chairs, but with the authors/editors and community. I know that I myself
> am not quick enough in turning around drafts, and that every time, I
> fall into the same trap of getting the updates done just before the
> deadline. And then, after IETF, I promise myself it won't happen again,
> and then day-job work that I put off comes up, and the previous IETF
> fades away, and each and every day there is always something more
> pressing to do than to update or work on an I-D that isn't really "due"
> until the next deadline....
> 
> 
> -Jonathan R.
> 

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: "Pete Cordell" <[email-masked]>
Date: Sat, 4 Mar 2006 16:56:57 -0000

My perception is that the main problem is getting third-party review of 
drafts and that's why they slow down.  So with that in mind, here's a 
proposal...

1) At each IETF (either at the meeting or around that time on the mailing 
list) ask for volunteers to become part of a SWAT team (co claim implementation.  And given the 
> strategic importance of SIP, these folks can frequently even get 
> management support for doing the work.
>
> At 09:23 AM 3/4/2006, Henry Sinnreich wrote:
>>Joel,
>>
>> > In general, with new work it is pretty hard to require implementation
>> > of the early stage ideas.
>>
>>If the authors of an idea do not invest and commit enough into writing 
>>code,
>>why should they SPAM the WG with I-Ds? Can anyone here stand this flood 
>>any
>>more?
>>
>>The idea could still be discussed on e-mail on the list (with a topic 
>>title)
>>without hijacking the IETF brand name to draw attention to themselves.
>>
>>The flood of I-Ds that Dean has signaled clearly needs a better control
>>mechanism and just legitimizing more I-Ds with more WGs is not the answer.
>>
>>Maybe there are better self control and stabilization mechanisms to 
>>control
>>the flood of I-Ds and it would be interesting to discuss various options.
>>
>>More WGs is not the answer IMHO, a self-regulating preselection mechanism 
>>is
>>the answer: Auto preselection by obligation and commitment to write code
>>first.
>>
>>Dean writes:
>>
>> >Lovely idea, but needs to be part of the IETF process, not the SIP 
>> >process.
>>
>>Yes, this is the original IETF process, before the _great flood_ swamped 
>>us.
>>
>> >The New Track working group, perhaps?
>>Great idea! The New Track will be a good place to experiment for progress.
>>
>>My two cents. Thanks, Henry
>>
>>
>>
>>-----Original Message-----
>>From: Marc Petit-Huguenin [mailto:[email-masked]]
>>Sent: Friday, March 03, 2006 7:29 PM
>>To: Joel M. Halpern
>>Cc: [email-masked]; 'sipping'
>>Subject: [Sipping] Re: [Sip] Re: Running code
>>
>>The way I see it, the I-D can start with an empty Running Code section,
>>like it is often the case with the IANA or Security section. As the
>>draft evolves, it can keep track of the various implementation of the
>>I-D in this section. This could be also a good indicator of the interest
>>in this work.
>>
>>Joel M. Halpern wrote:
>> > [Sorry, I couldn't let this go without comment.  I have trimmed the CC
>> > so the leadership won't receive 3 copies.]
>> >
>> > For SIP, maybe the problem space is in a state where that is practical
>> > (although I doubt it.)
>> >
>> > For many other working groups, it would be really strange to require
>> > running code before accepting a draft for discussion.  For example, in
>> > ForCES we needed to agree on what the structure of the ForCES protocol
>> > needed to be.  Requiring people to implement ideas that we wanted to
>> > discuss would have made it very hard to have clear discussion.
>> > Particularly since we had not agreed on the underlying transport, so it
>> > would have been easy to confuse transport properties with application
>> > properties.
>> >
>> > In general, with new work it is pretty hard to require implementation 
>> > of
>> > the early stage ideas.  It can sometimes even be counter-productive.
>> > Efforts to provide early implementations of alternative management
>> > structures seemed to cause those emplementors to become wedded to their
>> > particularl approach.
>> >
>> > I definitely like implementation.  The routing area is in fact
>> > increasing the emphasis on implementation before standardization.  But
>> > implementation before I-D seems a mistake.  We usually ask folks to
>> > write up an I-D on anything they want discussed.  And in most working
>> > groups, if the topic is on charter and the approach looks sane, the
>> > document is often made a WG document to create a clear trackable state
>> > for the work.
>> >
>> > Yours,
>> > Joel M. Halpern
>> >
>> > At 06:34 PM 3/3/2006, Dean Willis wrote:
>> >
>> >>> Documents that are only proposals or concepts, but are not backed
>> >>> up by
>> >>> running code can be very well exchaged as e-mails on the various
>> >>> lists.
>> >>> In this way, WG I-D overload would disappear and Dean's problem
>> >>> will be
>> >>> solved.
>> >>>
>> >>> If there is no running code, discussions on e-mail is just fine.
>> >>>
>> >>> Thanks, Henran't think of a 
better name at the moment!) to progress each key draft.  List the members of 
each draft's SWAT team on the softarmor web site.  You could also announce 
the members of the various SWAT teams in the final sessions at the IETF. 
(This is a bit of the carrot that Jonathan talks about.)  (Personally I 
think the teams should be self-appointing and not appointed by chairs etc. 
as has been the case for some task teams.)

2) If someone who volunteered to be on a team hasn't been sufficiently 
committed to the effort between IETF meetings, then they're off the team. 
(A bit of stick?)

3) Otherwise, those already on a SWAT team have first choice to stay on the 
SWAT team during the next inter-IETF period.

4) Acknowledge the SWAT team in the RFC.  (e.g. SWAT team members for this 
draft consisted of ...).

5) Ideally have some persistent record of who has worked on which drafts' 
SWAT teams so that it's possible to get an idea of who the 
completer/finishers are.

6) Encourage authors to say "I'll be on your swat team, if you'll be on 
mine!"

7) It goes without saying that if you can't get people to sign up to your 
draft's SWAT team it's not going to progress, so you don't need a rule to 
cover that case!

8) SWAT teams can be as big or as small as feels natural for the draft. 
They may initially start out small and gather members as the draft 
progresses.  Authors and chairs should have the authority to cull SWAT teams 
if they feel they are getting too big (but should avoid culling just because 
they have a difference of opinion!).

As to how the SWAT teams work I would see it as largely up to them. 
However, I think the default is that they should just communicate via the 
normal mailing list in the normal way.  (Then you have visibility of what 
they are doing.)  If they choose to arrange additional teleconferences that 
would be great, but their results should be captured on the mailing list as 
applies to the decisions made at IETF meetings.

That's the end of the proposal.

> painful but got the draft to WGLC.

 I'm not sure how effective this would be with large numbers of people. 
Could work well for the SWAT teams though.  (Effectively lots of small 
focussed calls would work better than general SIP(ING) working group calls.)

> 3) Interim meetings.

These can allow for great progress.  In my experience they tend to allow for 
developing and cementing ideas and developing a common consciousness, but 
they're generally not the right place for generating text.  As such they are 
not a total solution (not that anyone is suggesting that).  As things 
develop, and people are singing more from the same hymn sheet, the less they 
should be needed.  Also, bear in mind that some people may have to travel 
further than others to such meetings.  Avoiding a geographical bias to their 
scheduling is important.  (Every Monday and Wednesday morning in Boston may 
not be popular!)

Pete.
--
=============================================
Pete Cordell
Tech-Know-Ware Ltd
                         for XML to C++ data binding visit
                         http://www.tech-know-ware.com/lmx
                         (or http://www.xml2cpp.com)
=============================================

----- Original Message ----- 
From: "Cullen Jennings" <[email-masked]>
To: "sipping" <[email-masked]>
Cc: "Jon Peterson" <[email-masked]>; "Allison Mankin" 
<[email-masked]>
Sent: Wednesday, March 01, 2006 5:30 PM
Subject: Author Incentives - Re: [Sipping] Re: [Sip] Thoughts on 
thePrincipleofDividingandConquering

> hour conference calls twice a weeky
>> >>
>> >>
>> >> Lovely idea, but needs to be part of the IETF process, not the SIP
>> >> process. The New Track working group, perhaps?
>> >
>> >
>> >
>> > Sip mailing list  https://www1.ietf.org/mailman/listinfo/sip
>> > This list is for NEW development of the core SIP Protocol
>> > Use [email-masked] for new developments on the application of sip
>> >
>> >
>>
>>
>>--
>>Marc Petit-Huguenin
>>Home: [email-masked]
>>Work: [email-masked]
>>
> Sip mailing list  https://www1.ietf.org/mailman/listinfo/sip
> This list is for NEW development of the core SIP Protocol
> Use [email-masked] for new developments on the application of sip
>

---

From: "Pete Cordell" <[email-masked]>
Date: Sat, 4 Mar 2006 16:56:57 -0000

1) At each IETF (either at the meeting or around that time on the mailing 
list) ask for volunteers to become part of a SWAT team (c (yes 6 hours a week total). It was
>> until the next deadline....
an't think of a 

Pete.

----- Original Message ----- 
From: "Cullen Jennings" <[email-masked]>
To: "sipping" <[email-masked]>
Cc: "Jon Peterson" <[email-masked]>; "Allison Mankin" 
<[email-masked]>

> hour conference calls twice a week

 (yes 6 hours a week total). It was

---

From: Cullen Jennings <[email-masked]>
Date: Sat, 04 Mar 2006 20:57:30 -0800

Just to make sure I am clear ... the swat teams responsibility would be
promptly reviewing the draft when a new version came out?

On 3/4/06 8:56 AM, "Pete Cordell" <[email-masked]> wrote:

> list) ask for volunteers to become part of a SWAT team (can't think of a

---

From: "Pete Cordell" <[email-masked]>
Date: Sun, 5 Mar 2006 16:35:21 -0000

> promptly reviewing the draft when a new version came out?

I would say that fundamentally their job is to support the author(s) to get 
the draft completed.  Reviewing drafts would be a key part of this, but I 
think it should include helping to resolve issues raised against the draft. 
I don't think this should necessarily include non-author members coming up 
with solutions to problems (although if they did I don't think people would 
object), but when an author comes up with a bunch of potential solutions, 
the members of the team should be proactive in critiquing them and providing 
feedback etc.  (Call it pre-emptive reviewing if you like!)

By way of further explanation, I often see proposals that say we could do it 
this way, or that way.  There's either silence, or some people say I don't 
care either way.  In that case neither way seems to be accepted and things 
stall.  A SWAT team could help decide on a course of action so that drafts 
don't stall in this situation.

HTH,

Pete.

----- Original Message ----- 
From: "Cullen Jennings" <[email-masked]>
To: "Pete Cordell" <[email-masked]>; "sipping" <[email-masked]>
Cc: "Jon Peterson" <[email-masked]>; "Allison Mankin" 
<[email-masked]>
Sent: Sunday, March 05, 2006 4:57 AM
Subject: Re: Author Incentives - Re: [Sipping] Re: [Sip] Thoughts 
onthePrincipleofDividingandConquering

>> scheduling is important.  (Every Monday and Wednesday morning in Boston 
...cut...

---

From: Henning Schulzrinne <[email-masked]>
Date: Sun, 5 Mar 2006 11:42:53 -0500

You are describing what sounds fairly close to a design team.

On Mar 5, 2006, at 11:35 AM, Pete Cordell wrote:

> I would say that fundamentally their job is to support the author 
> (s) to get the draft completed.  Reviewing drafts would be a key  
> part of this, but I think it should include helping to resolve  
> issues raised against the draft. I don't think this should  
> necessarily include non-author members coming up with solutions to  
> problems (although if they did I don't think people would object),  
> but when an author comes up with a bunch of potential solutions,  
> the members of the team should be proactive in critiquing them and  
> providing feedback etc.  (Call it pre-emptive reviewing if you like!)
>
> By way of further explanation, I often see proposals that say we  
> could do it this way, or that way.  There's either silence, or some  
> people say I don't care either way.  In that case neither way seems  
> to be accepted and things stall.  A SWAT team could help decide on  
> a course of action so that drafts don't stall in this situation.

---

From: "Pete Cordell" <[email-masked]>
Date: Sun, 5 Mar 2006 17:38:40 -0000

----- Original Message From: "Henning Schulzrinne" <[email-masked]>>
> You are describing what sounds fairly close to a design team.

Close, yes.  But I think some of the details about who is on the team, how 
they're acknowledged (the carrots), how they're sacked(!) (the sticks) etc. 
are refinements.

(My experience of design teams is they are mainly used to boot-strap a 
complicated topic and then develop a way forward.  Once a way forward 
emerges, the work goes more over to the normal working group process.  A 
'swat' team could be considered to be under the overall umbrella of design 
teams but be geared more around some notion of a BCP for getting work 
completed.  e.g more the end game than the start game.)

Pete.

>> I would say that fundamentally their job is to support the author (s) to 
>> get the draft completed.  Reviewing drafts would be a key  part of this, 
>> but I think it should include helping to resolve  issues raised against 
>> the draft. I don't think this should  necessarily include non-author 
>> members coming up with solutions to  problems (although if they did I 
>> don't think people would object),  but when an author comes up with a 
>> bunch of potential solutions,  the members of the team should be 
>> proactive in critiquing them and  providing feedback etc.  (Call it 
>> pre-emptive reviewing if you like!)
>>
>> By way of further explanation, I often see proposals that say we  could 
>> do it this way, or that way.  There's either silence, or some  people say 
>> I don't care either way.  In that case neither way seems  to be accepted 
>> and things stall.  A SWAT team could help decide on  a course of action 
>> so that drafts don't stall in this situation.
>

---

From: Adam Roach <[email-masked]>
Date: Fri, 17 Mar 2006 18:57:22 -0600

Jonathan Rosenberg wrote:

> One idea, for example, is to tie draft acceptance to reviews. Lets say 
> I write draft X and I want it adopted by the WG. I can't get my draft 
> assistance in solving open issues for another draft that is already a 
> WG item. Only once it is approved by IESG to I get to have my draft 
> added. This is a carrot incentive.

This specific suggestion has received no comment so far. It sounds
potentially quite useful: if you're not doing work to help out the
working group, it's unreasonable to expect the working group to do work
to help you. I like it. Of course, it only really works when you have
very high-volume working groups, but then again, it's only *needed* when
you have very high-volume working groups.

/a

---

From: Adam Roach <[email-masked]>
Date: Fri, 17 Mar 2006 16:33:50 -0600

/a

---

From: Dean Willis <[email-masked]>
Date: Tue, 21 Mar 2006 16:52:30 -0600

On Mar 17, 2006, at 4:33 PM, Adam Roach wrote:

>> One idea, for example, is to tie draft acceptance to reviews. Lets  
>> say I write draft X and I want it adopted by the WG. I can't get  
>> my draft accepted by the WG as a new item, until I commit to  
>> reviews and assistance in solving open issues for another draft  
>> that is already a WG item. Only once it is approved by IESG to I  
>> get to have my draft added. This is a carrot incentive.
>
>
> working group, it's unreasonable to expect the working group to do  
> work to help you. I like it. Of course, it only really works when  
> you have very high-volume working groups, but then again, it's only  
> *needed* when you have very high-volume working groups.

It's a bit naive, I think. Perhaps we could define a pay-it-forward  
credit system where by doing reviews, you get credits that can be  
used to block other people's drafts from advancing until your draft  
has been reviewed.

Seriously, I think this is just something the chairs have to manage.  
I've tried to, and failed.

--
Dean

---

From: <[email-masked]>
Date: Wed, 22 Mar 2006 03:59:01 +0200

Hi,

In the RAI area meeting discussions today there were a couple of good proposals that I would like to see progress:

1. More organized reviews. I think reviews are needed especially at two specific points of the process. First, when someone comes up with a new draft and would like the WG to take it on the charter. Often work gets stalled at this point because it is hard to get any substancial comments on the draft, especially when the draft proposes some small thing. Second, around the time when the WG is wondering whether the draft is ready for WGLC. Again, time is often spent waiting for comments to push the draft forward. So, having some kind a pool of reviewers and the WG chairs using this pool to make sure that the reviews get done and posted on the lists.

2. More organized conference calls. I think it would be useful to resolve issues on the drafts at relevant points with using conference calls. It is true that nothing prevents doing this right now (and sometimes it happens too), but I believe having the WG chairs to assign specific people to actually take initiative on this would help. 

3. More organized distribution of editorships. It is clear that in this area drafts tend to concentrate on a few individuals. It might be good for WG chairs e.g. after each IETF meeting ping the people who have multiple editorships to see if they would be willing to give up on any of them. (I understand people like to keep the drafts that they have "invented" and are important to them, but there are probably also ones that can be more easily given up.) To take the work there should again be a pool of volunteers available.

Finally I would note that it seems that in order to get all this done would require quite much from the WG chairs, so if people agree on these approaches, that may give some input to the question that how many WGs/chairs would be needed.   

Regards,
	Markus   

-------------------------------
Markus Isomäki
[email-masked]
-------------------------------