Subject: [newtrk] the problem

From: Eliot Lear <[email-masked]>
Date: Mon, 06 Mar 2006 07:48:53 +0100

Brian, Ran,

I have two problems with the two step draft:

    * I question whether it would be better to more completely restate
      the process, so that people don't need to reconcile between this
      document and RFC 2026.  We have enough of that already.  Use of
      words like "roughly" are not appropriate, for instance.
    * There needs to be at least SOME rationale to argue that use of Two
      Step will actually solve the problem, as demonstrated by the
      statistics I posted last week.  Even if you add together DS and
      STD you get a rate of conversion that is well less than 10%.  Is
      the IETF producing that much useless stuff?  If so, then maybe the
      problem is that we are the wrong people to be writing standards. 
      If not, will changes to Interoperable Standard on their own open
      the flood gates?  For instance, the retiring of a working group is
      a process issue as well, and presumably has some impact, right?

Eliot
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 06 Mar 2006 08:38:21 +0100

>       words like "roughly" are not appropriate, for instance.

As the desirable end point, I fully agree - we need 2026bis and not
a further delta document. Hence draft-carpenter-rfc2026-critique-00.txt.
But first we need to agree on a direction.

>       the IETF producing that much useless stuff?

1. That's why I now prefer to name the first stage as Preliminary Standard.
There's no implication of uselessness in that name; quite the opposite.
But I've also stated a belief that having only one more stage to aim
at will actually increase the 'conversion rate' since the reward of
final status is nearer.

2. Why is it a problem that the Internet mainly runs on PS documents?

>       If so, then maybe the

Yes. I believe we also need 2418bis; but I haven't had cycles to analyse
2418 in detail yet.

     Brian

.

---

From: Eliot Lear <[email-masked]>
Date: Mon, 06 Mar 2006 08:45:37 +0100

Brian,
> 2. Why is it a problem that the Internet mainly runs on PS documents?
No problem at all.  But why bother with an additional step at all? 
What's the value?  By the way, before you answer "running code", I'd be
happy to see the bar raised for PS to show running code.

Eliot
.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 6 Mar 2006 06:22:49 -0600

To be fair to my co-author,

> happy to see the bar raised for PS to show running code.

When Charlie Perkins and I wrote 
http://www3.ietf.org/proceedings/03jul/I-D/draft-dawkins-pstmt-twostage-00.txt 
in 2003, this is actually one of the points we did not agree on - Charlie 
thought it was important to have implementation experience, even for 
Proposed Specifications (our name for the first standards-track stage). So 
this is not intuitively unreasonable.

At athat time, I was comfortable separating implementation reports from 
advancing on the standards track, but totally agree that implementation 
matters, and one (of several) things I liked about ISDs is that ISDs could 
contain implementation reports, even if we adopt a one-step standards track.

Thanks,

Spencer 

.

---

From: <[email-masked]>
Date: Mon, 6 Mar 2006 14:38:54 +0200

About running code, unless the IETF is going to get into the 
verification business, it would be awfully hard to set this
as a requirement, IMO.

John 

>-----Original Message-----
>From: [email-masked] 
>[mailto:[email-masked]] On Behalf Of ext 
>Spencer Dawkins
>Sent: 06 March, 2006 14:23
>To: [email-masked]
>Subject: Re: [newtrk] the problem
>
>> What's the value?  By the way, before you answer "running code", I'd 
>> be happy to see the bar raised for PS to show running code.
>>
>> Eliot
>
>in 2003, this is actually one of the points we did not agree 
>on - Charlie thought it was important to have implementation 
>experience, even for Proposed Specifications (our name for the 
>first standards-track stage). So this is not intuitively unreasonable.
>
>At athat time, I was comfortable separating implementation 
>reports from advancing on the standards track, but totally 
>agree that implementation matters, and one (of several) things 
>I liked about ISDs is that ISDs could contain implementation 
>reports, even if we adopt a one-step standards track.
>
>Thanks,
>
>Spencer 
>
>
>.

.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 6 Mar 2006 06:46:08 -0600

Hi, John,

I had assumed that Charlie was talking about collecting statements of the 
form "we implemented this draft", and possibly of the form, "we implemented 
this draft, except for the frabbits mechanism", but not talking about 
verifying those statements.

I am comfortable trusting individual IETF participants to make statements 
about what's been implemented, because we rely on individual integrity in 
other parts of the standards process (for instance, IPR). If someone says, 
"we implemented this specification" and others find that the specification 
can't be implemented because the state machine doesn't work (for example), 
we're likely to remember this for a very long time.

Thanks,

Spencer

From: <[email-masked]>
To: <[email-masked]>; <[email-masked]>
Sent: Monday, March 06, 2006 6:38 AM
Subject: RE: [newtrk] the problem

John 

.

---

From: <[email-masked]>
Date: Mon, 6 Mar 2006 14:56:01 +0200

Collecting is fine, requiring is a bit different.  I think that
providing implementation info is a good thing, and should be
considered A VERY GOOD THING during IESG review - but if
we make formal submission of interop reports, then we'll run
into more problems. 

>Sent: 06 March, 2006 14:46
>I had assumed that Charlie was talking about collecting 
>statements of the form "we implemented this draft", and 
>possibly of the form, "we implemented this draft, except for 
>the frabbits mechanism", but not talking about verifying those 
>statements.
>
>I am comfortable trusting individual IETF participants to make 
>statements about what's been implemented, because we rely on 
>individual integrity in other parts of the standards process 
>(for instance, IPR). If someone says, "we implemented this 
>specification" and others find that the specification can't be 
>implemented because the state machine doesn't work (for 
>example), we're likely to remember this for a very long time.
>
>Thanks,
>
>Spencer
>
>verification business, it would be awfully hard to set this as 
>a requirement, IMO.
>
>John 
>
>
>.

.

---

From: RJ Atkinson <[email-masked]>
Date: Mon, 6 Mar 2006 08:12:16 -0500

On  6 Mar 2006, someone wrote:
> What's the value?

	Demonstrated and verified interoperability is useful
information, but we ought not gate progress by demanding it
initially.

> By the way, before you answer "running code", I'd be

	I believe that increasing the burden on PS would just move
the blockage downstream, so that we were unable to move I-Ds to PS.
Then even more of the Internet would be running on unevaluated
and unapproved I-Ds (it does this somewhat today -- considering
draft-martini-* in the MPLS VPN world was widely implemented
and deployed 5 years ago).

	The PS state does have value as is.  Documents in that
state are known to all and sundry to have been evaluated formally
and approved by the community.  I-Ds do not have such properties.
Yet the PS state does not require more than that, so documents
really are able to move into PS in practice today.

	The other KEY difference with the twostep draft, that
maybe some folks overlooked, is in Section 6.  A LOT of the
current inability to move beyond PS is due to the inability to
reference downwards (e.g. IPsec keeps recycling at PS with
minor tweaks, which in turn blocks protocols that use IPsec
for security from moving to DS).  By removing that artificial
constraint against downward references, a lot of documents
will be able to move forward much faster.  This fix is entirely
due to Brian, as near as I can tell, but I fully agreed with it
upon first reading it.

	Thirdly, on a minor note, I had a lot of painful first-hand
experience as OTP WG Chair trying to move OTP documents to
Full Standard from Draft Standard.  It took over a year --
and all the documentation requirements were met when the RFCs
advanced to DS.  So I also see some value in eliminating the
3rd document state.

	Please note well:  This proposal is designed to fix
a small number of well-known process blockages.  I do not claim
it is a panacea.  My understanding [1] is that NewTrack WG would
continue work (on other fixes) even if this proposal were approved
by the WG.  I also think that the broader Internet can't really
afford for the NewTrack WG to wait a long time while trying
to come up with a single "ideal" solution.  The longer the WG
stalls, the worse the problems are going to be.  Some fixes
(or "fix attempts" if one prefers) need to be approved soon.

Yours,

Ran
[email-masked]

[1] Obviously I am not the WG Chair, so I might be confused in my
     understanding.

.

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 06 Mar 2006 14:16:58 +0100

> happy to see the bar raised for PS to show running code.

I think that is a shade too hard. There are clearly cases where a priori
standardization is useful.

     Brian

.

---

From: Eliot Lear <[email-masked]>
Date: Mon, 06 Mar 2006 14:29:35 +0100

> On  6 Mar 2006, someone wrote:
That would be me.
> information, but we ought not gate progress by demanding it

To whom?  Implementors?  If only implementors what's the motivation for
advancing?  Customers clearly don't know the difference.  If you can
provide me a satisfactory answer for the motivation for advancing I have
a much easier time swallowing the rest of this.  I think we got as many
through as we did because of egalitarian academicians.  Don't see too
many of those at the IETF any more.
> and deployed 5 years ago).
I'm having trouble parsing this paragraph.  Running code existed, so the
running code requirement would have been met.  How could a running code
requirement have been a bottleneck?  I must misunderstand you.

And Brian wrote:

> There are clearly cases where a priori standardization is useful.

Ok, but are they the general case?  I don't mind exceptions if there is
good reason.  I expect the IESG to use your judgment (albeit laying out
your reasoning).  That's why you're paid the big bucks!

> really are able to move into PS in practice today.

We keep talking about how the IESG is such a bottleneck.  It seems to me
only reasonable that at least in general there be SOME code before the
IESG is asked to consider this stuff.

Eliot
.

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 06 Mar 2006 14:28:21 +0100

What we require, formally, are implementation reports, but
RFC 2026 requires that they report on interoperability.

Some of them are pretty weak though; just scan
http://www.ietf.org/IESG/implementation.html

    Brian

.

---

From: RJ Atkinson <[email-masked]>
Date: Mon, 6 Mar 2006 11:49:31 -0500

On  6 Mar 2006, at 08:29, Eliot Lear wrote:
> advancing?  Customers clearly don't know the difference.

I disagree.  When I have been a user/customer (which I continue to be
even now :-), I have understood the difference and found the information
to be useful.  There are a number of end users who likewise care.
Many of my firm's {enterprise, SP} customers care.  Virtually every
SP who asked that draft-martini-* be implemented said that they wished
it were a standards-track RFC, so they would have more confidence in
specification stability.  (The SPs were telling me that they thought
the problem was that it had become too hard to publish a standards-track
RFC.)  Many governments around the world care.

Are there some users who don't care ?  Probably, because the pool
of total users is so very large.  (It is equally true, for the same
reason, that there are probably customers who do not like green
eggs and ham. :-)

You appear to be objecting to a proposal for process reduction,
which seems confusing since your arguments seem to argue for
some process reduction.

Why not start by supporting this proposal, then see whether it
is actually sufficient reduction or not, and leave open the
question of further process reduction -- which could occur
either concurrently or later on ?

> requirement have been a bottleneck?  I must misunderstand you.

More than half of documents that currently advance to PS do not have
(at time of IESG approval) any demonstrated interoperability or
demonstrated working software/hardware.   Requiring that to be
demonstrated (or documented) before PS would *increase* the process
burden for advancing from I-D to PS, which is contrary to the goal
of reducing process burden.

> your reasoning).  That's why you're paid the big bucks!

I think the general case is that we move documents to PS without
requiring or having interoperability or running software/hardware.

I object to proposals which would make it harder to move to PS
since the widespread belief is that there is currently too much
process burden.

> IESG is asked to consider this stuff.

Did you write up an I-D with your process change proposal someplace ?

I do not consider it reasonable to require documentation that
software/hardware exists that implements a draft before the
draft moves to PS.  Note that any requirement that "some code"
exist really boils down to a requirement for reasonable documentation
that "some code" exists -- because if there is no such documentation
then a document with "no code" could advance in violation of the
stated policy.

Yours,

Ran

.

---

From: Eliot Lear <[email-masked]>
Date: Mon, 06 Mar 2006 18:21:50 +0100

Ran,

You've suggested that I accept two step as perhaps an intermediate
step.  My support comes at what I hope is a   reasonable price: show me
an economic motivation that actually causes companies to want to spend
their employees' time to get specs to the 2nd step.  Put another way,
let's only have the second step if we can reasonably predict that people
will use it.

I think I wasn't clear in my earlier message.  It's not that customers
don't know the difference between an internet-draft and a standards
track document (some do - not all), but they clearly don't care about
the difference between a draft standard and a proposed standard - an RFC
# is sufficient most of the time.  And we know this to be generally true
because we'd at least see more attempts to get from proposed onward.

Ultimately the state I'd like to see us end up in is one where we're
using the practices as documented, as intended.  It seems we all agree
that three step isn't working.  Now we just have to sort between one and
two and how to encode them.  And I think you'd agree that we really
don't want to have to tinker again for a while, so let's try and get it
right.

Eliot
.

---

From: RJ Atkinson <[email-masked]>
Date: Mon, 6 Mar 2006 12:36:10 -0500

> let's only have the second step if we can reasonably predict that  

I don't think we can reasonably predict the behaviour of large groups
of people in this community.  I can say that I would use it.
I like to think that I don't knowingly waste my own time.
I don't think I am alone.  Reducing some of the process pain now
should by itself be worth undertaking without delay, even
if more reduction in process is made later on.

> because we'd at least see more attempts to get from proposed onward.

I don't think we have good measurements on "attempts to get from
proposed onward".  If you think you've got measurements, please
share.

In my own experience, the big blockage for moving from PS to DS
is the issue with downwards references being (at present) disallowed.
My document fixes that major issue.  I think that fixing that one
item alone will help things move along from PS.  I also think that
eliminating the 3rd step creates more incentive to advance from PS
to the (now final) next step.

> don't want to have to tinker again for a while, so let's try and  

In fact, I do not agree with your last sentences.  I think we should
start with some process reduction (if folks do not like twostep,
then pick another), see how things go, and then re-evaluate.

I specifically do NOT want to wait around trying to find a perfect
solution without trying some of the partial fixes that have been
proposed already.  The pain of the current process is too high to
wait for a perfect solution.  We can't be perfectionists here
-- maybe perfectionism is how this WG seems to have become wedged.

My proposal would be to advance twostep without closing this WG
and without precluding the WG from considering other proposals
-- either concurrently or later.

Cheers,

Ran

.

---

From: Eric Rosen <[email-masked]>
Date: Mon, 06 Mar 2006 13:03:52 -0500

What  difference does it  make whether  there's a  two-step process  that we
don't follow or a three-step process that we don't follow?  

In terms of real effort expended,  failing to follow a two-step process does
not  take  any  less time  than  failing  to  follow a  three-step  process.
However, there is considerable effort  required to make the formal change in
procedure. 

Therefore, it's  hard to  see why this  proposal should be  supported, given
that no one seems  to be able to state why anyone  would want to bother with
the second step.  If we're going to  have a process that we don't follow, we
might as well leave things as they are and save effort all around.

There have been some suggestions that  we can make an external audience care
about the difference between the two steps by changing the name of the first
step, but this seems silly in  the extreme.  The very fact that "Request for
Comments" has come  to mean "standard" shows that the  semantics of the name
has no bearing whatsoever on the perceived status of the document. 

.

---

From: RJ Atkinson <[email-masked]>
Date: Mon, 6 Mar 2006 16:10:59 -0500

On  6 Mar 2006, at 13:03, Eric Rosen wrote:
> What  difference does it  make whether  there's a  two-step process
> that we don't follow or a three-step process that we don't follow?

I think that a two step process would actually get followed.
So I disagree with your premise.  Maybe I am not cynical enough ?

> However, there is considerable effort  required to make
> the formal change in procedure.

I disagree.  I think there is not a lot of effort to make the
formal changes proposed in my draft (or any similar formal change).

> Therefore, it's  hard to  see why this  proposal should be supported,
> given that no one seems to be able to state why anyone would want
> to bother with the second step.

I would want to bother, because advancing documents from PS to IS
would address existing requests from numerous end users to have
higher assurance of interoperability and numerous stated preferences
for specifications that are full IETF standards.  Moreover, the
experience to date has almost always been that a specification did
have anomalies that were corrected as part of the process to advance
beyond PS.  I know of any number of end users who want IETF
assurances of interoperability (e.g. from DS or IS), but also
do not want to gum up the process of getting new PS documents out.

Now, the current user community is quite large.  So one can find
slices of that group that believe almost anything.  I have no doubt
that "RFC equals standard" is a misconception that is widespread.
That doesn't mean it is in reality or that the IETF ought not try
to do a good job of clarifying the difference between actual standards
and non-standards that are (informational, experimental) RFCs.

Yours,

Ran
[email-masked]

.

---

From: Larry Masinter <[email-masked]>
Date: Mon, 06 Mar 2006 14:27:57 -0800

I guess I will make a last pitch for
   draft-ietf-newtrk-interop-reports-00.txt

I like "three steps" and think that we're twisting ourselves
in knots because we don't follow it too often, and never
quickly enough.

I'd propose keeping "three steps" but also

* formalize interoperability reporting, 
* change 2026 to remove the timeouts 
* point out that while many documents remain officially
  at "Proposed Standard", and progression is infrequent, 
  it is still possible to find out the status of
  implementations and interoperability by looking at
  the interoperability reports.

If in the end most things stay at the first step, it's
OK.

Maybe we could also keep a list, for each standards track
documents, of the reasons why it isn't at the next level.
Is it because of down-level references, lack of documentation
of interoperable implementations, or just 'no one has
proposed upgrading'?

We might also need to do a better job of managing errata.

Larry
-- 
http://larry.masinter.net

.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 07 Mar 2006 16:46:46 +0100

> don't follow or a three-step process that we don't follow?  

But we do follow a two-step process today. What we don't follow is
the mandatory review of all PS documents. What hampers us is the
downref rule.

To a very minor extent we follow a three-step process too, but it's
essentially bogus.

    Brian

.

---

From: Eric Rosen <[email-masked]>
Date: Tue, 07 Mar 2006 10:59:26 -0500

> But we do follow a two-step process today

I don't think we  do.  Taking a spec to DS is  pretty unusual, no matter how
widely  deployed  it is.   The  gain  from doing  so  just  isn't worth  the
effort. Even  if we greatly reduce the  effort, it's still hard  to see what
the gain is.  What we really follow is a one-step process. 

> What we don't follow is the mandatory review of all PS documents. 

Lucky for us, that piece of process is impossible to follow. 

> What hampers us is the downref rule. 

While  that rule  does  create a  number  of amusing  situations,  I see  no
evidence that it is the bottleneck. 

.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 07 Mar 2006 17:01:00 +0100

Larry,

It's my recollection that the newtrk BOF already sort of
converged on two steps, and we've been arguing between
1, 2 and 3 ever since. I simply observe that the mean of
1, 2 and 3 is 2.

However, although you mention getting rid of the timeouts
as a precondition for a workable 3 step process, you don't
mention the other - getting rid of the downref rule. If we
don't abolish that, you will continue to see drafts stuck
in the queue for literally years.

    Brian

.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Tue, 7 Mar 2006 11:49:11 -0600

Hi, Brian,

I'm not sure that newtrk has converged on some number of steps yet. I look 
forward to the day when it does.

I'm reading Eliot's math as saying that we really don't use either the 
second stage or the third stage of the current standards track. JohnL and I 
liked Eliot's math enough that we included it in the revised one-step draft:

> By RFC, not by STD (obviously):
>
> Status 1999 2000 2001 2002 2003 2004 2005
> -------------------------------------------------------------
> PS 102 119 71 105 103 131 169
> DRAFT 6 6 2 4 7 7 3
> STD 3(*) 2 0 8* 3 0 1
>
> (*) 3 in 1999 were SMIv2 6 in 2002 were SNMP

My understanding of the other side of this dicussion is that Brian and 
others think collapsing the second and third stages into a single stage, 
plus punting on (most) downrefs, changes the playing field enough that we 
can't extrapolate from previous experience. This may very well be true, and 
I encourage the working group to consider this possibility.

As long as we either converge on something...

Thanks,

Spencer

From: "Brian E Carpenter" <[email-masked]>
To: "Larry Masinter" <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, March 07, 2006 10:01 AM
Subject: Re: [newtrk] the problem

.

---

From: Larry Masinter <[email-masked]>
Date: Tue, 07 Mar 2006 10:00:15 -0800

> in the queue for literally years.

I think it's possible to use the protocol feature sets
documentation to help expedite the analysis of downward
references. Perhaps a protocol feature set for protocol A
(which covers RFC X) could include the fact that
it references features P, D, Q of protocol B
(which is covered by RFC Y). If features P, D, Q
of protocol B have independent interoperable implementations,
then the fact that RFC Y is 'stuck' at Proposed shouldn't
impede the advancement of protocol A (and RFC X) to
Draft Standard. 

Similarly, change "independent interoperable implementations"
to "many widely deployed implementations" and "Draft Standard"
to "Standard": you can judge downward references by
using the feature sets.

However, I think it's also OK to stop thinking of it
as "drafts stuck in the queue", that they're not 
"drafts", they're permanent standards-track documents,
they aren't "stuck", it's just a slow process, and it
isn't really a "queue", it's just some documentation of
the organization's judgment about maturity.

Larry

.

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 08 Mar 2006 15:50:58 +0100

> evidence that it is the bottleneck. 

It's a very clear bottleneck for PS->DS. I believe we would see a lot
more PS->DS proposals if we didn't have that rule.

    Brian

.

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 08 Mar 2006 15:59:34 +0100

> impede the advancement of protocol A (and RFC X) to

But it does. That is the effect of the current rule, whose
granularity is RFC-sized. You're proposing a new rule with
much finer granularity (feature-sized). I think that is unworkably
complicated.

If we retain more than one stage the only workable solution
is to abolish the downref rule. If you don't want to do that,
you will rapidly convert me from twostep to one-size-fits-all.

     Brian

.

---

From: Pekka Savola <[email-masked]>
Date: Wed, 8 Mar 2006 17:29:19 +0200 (EET)

On Wed, 8 Mar 2006, Brian E Carpenter wrote:
>> While that rule does create a number of amusing situations, I see 
>> no evidence that it is the bottleneck.
>

My perception is that downref rules cause a lot of 1) waiting for 
others to progress specifications, 2) more work for you.

Based on this, it seems clear that we _could_ have more PS->DS 
proposals (because the amount of work and time in which PS->DS could 
happen would be much smaller) because "problem" 1) would be 
eliminated.

On the other hand, it's not clear whether we'd actually get a lot more 
PS->DS if we don't adjust the system in other way because 2) still 
holds -- which is what I think Eric is saying.  There's still little 
incentive for the WGs, people, and the vendors involved to carry out 
the extra steps of formal interop/feature testing/reporting and 
recycling the spec.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 08 Mar 2006 09:19:12 -0800

>more PS->DS proposals if we didn't have that rule.

I suspect we might see less PS->DS proposals if we removed the
rule.

The rule leads to those wanting to update protocol X but to
push for updates of X's normative references.  If you remove
the rule, there will be less pushing for updates on
referenced specifications, and hence less likelihood that
they will be updated.

When since undertaking the LDAPv3 revision effort, I done a
lot of pushing, and even some leading, in getting referenced
specifications updated.  If the rule didn't exist, I suspect
I would have done a lot less pushing (and likely less leading)
in getting referenced specifications updated, and the
pushing would have done been less effective.  That is, some
folks that are responsive to "Please update specification Y
for DS so that X can progress on DS" that wouldn't be
responsive to "Please update specification Y for DS."

I support keeping the present rule in place.

I should also note that I don't mind waiting for referenced
specifications to catch up.  The value is not in the status,
but in the resolution of protocol issues realized through
revision of the specification.

Kurt 

.

---

From: Larry Masinter <[email-masked]>
Date: Wed, 08 Mar 2006 12:04:37 -0800

I think extending the 'downref exception' rule in the following way
would be helpful:

Suppose draft-A makes a reference to RFC B.

Rather than require B be at the same or higher level than proposed for A,
require that the independent interoperable implementations (or widespread
deployment) of A also be independent, interoperable implementations
(or widespread deployment) of the relevant parts of B.

So you can advance to Draft even if you reference Proposed, as long
as you also report implementations of the (relevant parts) of the
documents referenced.

I agree that it might remove some of the incentive to update documents
(moving URI to Standard was in part motivated by wanting to get HTTP
to Standard), but it might reduce some of the silly states where things
that have been widely deployed for a long time have no hope of progressing
just because the only document that contains a mature critical piece
also has some parts that are not ready. (Example, HTTP references
MIME which is stuck at Draft, but otherwise we could move HTTP to
Standard.)

Larry

.

---

From: David Kessens <[email-masked]>
Date: Wed, 8 Mar 2006 15:47:46 -0800

Larry,

> I think extending the 'downref exception' rule in the following way

I believe that the downref rule is solving a problem but that the new
problems caused by the rule are a magnitude larger than the original problem.

Why don't we abolish the rule and accept that we cannot have
mathematically correct references rather than try to make rules even
more complicated. To me, the value of getting things published far
outweighs the delays that we are incurring due to the rule.

David Kessens
---

> MIME which is stuck at Draft, but otherwise we could move HTTP to

David Kessens
---
.

---

From: Eliot Lear <[email-masked]>
Date: Wed, 08 Mar 2006 17:44:28 -0800

Spencer Dawkins wrote:
> others think collapsing the second and third stages into a single
> stage, plus punting on (most) downrefs, changes the playing field
> enough that we can't extrapolate from previous experience. This may
> very well be true, and I encourage the working group to consider this
> possibility.
By all means if you could convince me of this I would support 2 step. 
I'm not convinced as of yet.  Show me why anyone would expend resources
on this and you've got a sale.  What's the benefit to a company?

Eliot
.

---

From: <[email-masked]>
Date: Thu, 9 Mar 2006 05:49:28 +0200

>I'm not convinced as of yet.  Show me why anyone would expend 
>resources on this and you've got a sale.  What's the benefit 
>to a company?

I partially agree with Eliot - with the caviat that if this type
of two-step process isn't successful, then we're back to the one-step.

I have nothing against a two-step process, I just want to have more
assurance that there is enough will to actually make it happen in
practice.  

John

.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 9 Mar 2006 08:23:49 -0600

I should take this opportunity to point out that RFC 3933 was intended to 
get us out of the "endless noodling" cycle on process change, as a whole 
bunch of really intelligent engineers endlessly try to predict the effect of 
process change on human behavior, by (for example) picking either two-stage 
or one-step as an experiment and running it for a year...

... since we've been trying to predict what the likely effect of a different 
number of stages on human behavior would be on THIS list since, roughly, 
Seoul (ignoring similar noodling in previous forums), we've had time to run 
two year-long experiments already, so it might be time for an experiment 
(gathering actual data could cut us to the chase).

Thanks,

Spencer

.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 10 Mar 2006 14:49:58 +0100

I'd add that we have an RFC 3933 proposal in Last Call for another few days
and support or criticisms of that proposal are needed. See
http://www1.ietf.org/mail-archive/web/ietf-announce/current/msg02175.html

     Brian

Spencer Dawkins wrote:
> I should take this opportunity to point out that RFC 3933 was intended 
> to get us out of the "endless noodling" cycle on process change, as a 
> whole bunch of really intelligent engineers endlessly try to predict the 
> effect of process change on human behavior, by (for example) picking 
> either two-stage or one-step as an experiment and running it for a year...
> 
> ... since we've been trying to predict what the likely effect of a 
> different number of stages on human behavior would be on THIS list 
> since, roughly, Seoul (ignoring similar noodling in previous forums), 
> we've had time to run two year-long experiments already, so it might be 
> time for an experiment (gathering actual data could cut us to the chase).
> 
> Thanks,
> 
> Spencer
> 
>>>> My understanding of the other side of this dicussion is that Brian
>>
>> and
>>
>>>> very well be true, and I encourage the working group to
>>>
>>> consider this
>>>
>>>> possibility.
>>>

.

---

From: Eliot Lear <[email-masked]>
Date: Fri, 10 Mar 2006 08:18:46 -0800

Brian, Spencer,

I'm not convinced that we need to or should do a 3933 experiment.  I
claim you have rough consensus to do away with 3 step.  I claim you can
get rough consensus for 2 step IFF you can provide a rationale for
companies to economically support the work.  Otherwise I'd urge that we
settle for one step so that we are documenting existing practice.

Eliot
.

---

From: John C Klensin <[email-masked]>
Date: Sat, 11 Mar 2006 08:36:51 -0500

--On Thursday, 09 March, 2006 08:23 -0600 Spencer Dawkins
<[email-masked]> wrote:

> I should take this opportunity to point out that RFC 3933 was
> intended to get us out of the "endless noodling" cycle on
> process change, as a whole bunch of really intelligent
> engineers endlessly try to predict the effect of process
> change on human behavior, by (for example) picking either
> two-stage or one-step as an experiment and running it for a
> year...
> 
> ... since we've been trying to predict what the likely effect
> of a different number of stages on human behavior would be on
> THIS list since, roughly, Seoul (ignoring similar noodling in
> previous forums), we've had time to run two year-long
> experiments already, so it might be time for an experiment

One observation and a proposal:

* I don't see how one uses 3933 in this case because I don't see
a practical way to get back.   If, for example, we go to
"one-step" as an experiment, we end out the year with a bunch of
full standards that don't meet our present (i.e., 2026) criteria
for those standards.  If we then decide it is a bad idea, we've
got a mess of the first order on our hands, possibly requiring
the IESG to spend a month or so reclassifying the documents as
to whether they should really have been Proposed, Draft, or Full.

* However, I think there is an experiment that could be
performed, independent of moving from three steps to two.  Let's
experimentally change the downref rule, from "either don't do it
or invoke the RFC 3967 procedure" to "if you make a downward
reference, you must annotate the reference itself by indicating
that the document referenced is at a lower maturity level".  

That is clearly less burdensome than the 3967 procedure.  I
think 3967 ought to be sufficiently lightweight in practice that
I'm skeptical about claims that eliminating the downref rule
would unleash a flood of advancement requests, but that is one
reason why it would make a good experiment.   At the same time,
for those of us who like the logic of the downref rule, the
reference annotation would provide an explicit warning about
normative references from "presumed more mature and stable" to
"presumed less mature and stable": that seems to me to be a
plausible middle ground with _very_ little extra effort.  For
documents produced during the experimental period, the warning
would be sufficient: if the experiment was deemed a failure, we
would neither need to "fix" documents produced in the interim or
expect readers to know that, during a particular time window,
references and status were a little less clear than usual.  And,
unlike the 3967 procedure, it would not require special notice
or approval by anyone, just clear author-editor awareness of
what was being done (which awareness, IMO, is a good thing).

      john

.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Sun, 12 Mar 2006 22:40:53 -0600

Sorry, I thought I had replied to this earlier...

I agree that John's proposal for downref would make sense in a two-stage or 
three-stage world. Beyond this...

John's comments about using 3933 experiments on the standards track do point 
out that the effect of some unsuccessful experiments may be messy, and I 
agree with his point. In general, comments of the form "it will be a lot 
easier to unravel a two-stage experiment than a one-stage experiment, so do 
the two-stage experiment first and see if you still need to do a one-stage 
experiment" seem very helpful.

Thanks,

Spencer

From: "John C Klensin" <[email-masked]>

.

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 13 Mar 2006 21:36:33 +0100

> that the document referenced is at a lower maturity level".  

I think this would be a very useful thing to try.

      Brian

.

---

From: John C Klensin <[email-masked]>
Date: Mon, 13 Mar 2006 16:02:34 -0500

--On Monday, 13 March, 2006 21:36 +0100 Brian E Carpenter
<[email-masked]> wrote:

>> performed, independent of moving from three steps to two.
>> Let's experimentally change the downref rule, from "either
>> don't do it or invoke the RFC 3967 procedure" to "if you make
>> a downward reference, you must annotate the reference itself
>> by indicating that the document referenced is at a lower
>> maturity level".  
> 

I'll try to get it written up into I-D form before Sunday unless
Spencer has more time than I do and wants to volunteer.

    john

.

---

From: Michael Richardson <[email-masked]>
Date: Tue, 21 Mar 2006 13:29:59 -0600

>>>>> "RJ" == RJ Atkinson <[email-masked]> writes:
    >> By the way, before you answer "running code", I'd be happy to see the
    >> bar raised for PS to show running code.

    RJ> 	I believe that increasing the burden on PS would just move
    RJ> the blockage downstream, so that we were unable to move I-Ds to PS.

  What he said.
  I want to see the bar on PS lowered. 

  I want to see single incarnations of working groups cycle on *PS* within a
reasonable time-span. (less than 2 years)

  I do not want to ever come to a WG that has a -05 draft that I just tried
to implement, discover it has multiple significant buffer overrun risks (for
no good reason other than poor document writting), and be told that the
specification can not be fixed because it has been deployed.

  That document should have been at PS, if the WG felt that way.

    RJ> 	The other KEY difference with the twostep draft, that maybe
    RJ> some folks overlooked, is in Section 6.  A LOT of the current
    RJ> inability to move beyond PS is due to the inability to reference
    RJ> downwards (e.g. IPsec keeps recycling at PS with minor tweaks, which
    RJ> in turn blocks protocols that use IPsec for security from moving to
    RJ> DS).  By removing that artificial constraint against downward
    RJ> references, a lot of documents will be able to move forward much
    RJ> faster.  This fix is entirely due to Brian, as near as I can tell,
    RJ> but I fully agreed with it upon first reading it.

  I agree. That's also why having *more* PS documents helps, if we can
reference things downwards. 

-- 
]       ON HUMILITY: to err is human. To moo, bovine.           |  firewalls  [
]   Michael Richardson,    Xelerance Corporation, Ottawa, ON    |net architect[
] [email-masked]      http://www.sandelman.ottawa.on.ca/mcr/ |device driver[
] panic("Just another Debian GNU/Linux using, kernel hacking, security guy"); [

---

From: Michael Richardson <[email-masked]>
Date: Tue, 21 Mar 2006 13:29:42 -0600

>>>>> "John" == John C Klensin <[email-masked]> writes:
    John> * However, I think there is an experiment that could be performed,
    John> independent of moving from three steps to two.  Let's
    John> experimentally change the downref rule, from "either don't do it or
    John> invoke the RFC 3967 procedure" to "if you make a downward
    John> reference, you must annotate the reference itself by indicating
    John> that the document referenced is at a lower maturity level".

  I agree. We should do this experiment.
  I think the hypothesis is:
    "the downref rules prevents specifications at DS from advancing to FS,
    and keeps documents with known interoperability reports at PS"
  
  The results would be to evaluate the numbers of new DS and FS at the end of
the year.   

  Would a document with a 1+ year old interoperability report that was at PS,
be able to move PS->DS->FS quickly?  I.e. at the minimum time given by 2026? 

  I have another hypothesis:
    "The number of downref's after the experiement will decrease"

  i.e. we may have some complicated cycles of downref dependancies, that once
  adjusted for, may mean that we have far fewer downrefs after the
  experiement.