Subject: Re: 2026, draft, full, etc.

From: Simon Josefsson <[email-masked]>
Date: Thu, 01 Nov 2007 09:36:37 +0100

Eliot Lear <[email-masked]> writes:

> [I'm changing the subject and cutting off the references list as we seem
> to have changed topic.]
>
> Simon,
>
>> DS designates a mature standard.  If you read the requirements in RFC
>> 2026 for a mature standard it is clear that few of the modern IETF
>> protocols live up to that standard -- you need to demonstrate
>> interoperability between two completely independent implementations of
>> _all_ features in the protocol standard.
>
>
> I think we can all agree that the calendaring standard is mature.  We
> are in the process of doing what I would consider to be a relatively
> minor update to it, and yet it is only PS.  IMAPv4 is only PS and yet
> has MASSIVE deployment.  LDAP is only PS and is MASSIVELY deployed.  SIP
> is all over the place and it is only PS as well.

I'm not convinced these protocols qualify for DS status.  DS status
requires a lot, specifically that ALL features in the document have been
demonstrated interoperable, and that their normative references are DS.
I implemented IMAP and wrote
<http://josefsson.org/nnimap/buggy-imap-servers.html>, I'm pretty sure
others implementing other protocols have had similar experiences and
frustration.

> And so it's pretty clear that nobody cares about DS or IS.  What's
> more, why should they?  What benefit does it bring to anyone to
> advance a standard to DS?  AND it's a whole lot of work.

Yes, it is a lot of work, and the benefit seems marginal.  But I think
that if you want to claim that all of those protocols are mature enough,
the onus is on you to do that work.

> So why are we even having an argument about what gets stuck into
> requirements for DS?

Because Brian wrote a draft...

> Shouldn't we instead be eliminating it entirely?

I'm not sure about this.  I used to think DS was useless, but it doesn't
seem actively harmful.  I think the problem is that we don't have a
replacement for it today.  If we can come up with a scheme to allow the
community to know which standards are mature and which are not, and that
scheme actually works, I think we could eliminate the DS way.  But until
that happens, I'm not sure.

/Simon

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 02 Nov 2007 10:14:31 +1300

> Eliot Lear <[email-masked]> writes:
...

> Because Brian wrote a draft...

Sorry ;-)

> that happens, I'm not sure.

One idea that was floated a couple of years ago, as part of a one-level
standards track, was to retain the register of implementation reports
(http://www.ietf.org/IESG/implementation.html) and mark the entries
that have been approved by the IESG. The RFC index could then point to
approved implementation reports, without any formal "promotion" needed.

    Brian

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 01 Nov 2007 15:29:06 -0600

> approved implementation reports, without any formal "promotion" needed.

In which case we might want to resurrect Larry Masinter's proposal about
feature sets:

http://tools.ietf.org/html/draft-ietf-newtrk-interop-reports-00

I've been following that in defining a feature set for XMPP:

http://tools.ietf.org/html/draft-saintandre-xmpp-feature-set-00

Whether I can convince XMPP developers to submit implementation reports
is another question. :)

Perhaps more experiments along these lines are in order?

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

> approved implementation reports, without any formal "promotion" needed.

In which case we might want to resurrect Larry Masinter's proposal about
feature sets:

http://tools.ietf.org/html/draft-ietf-newtrk-interop-reports-00

I've been following that in defining a feature set for XMPP:

http://tools.ietf.org/html/draft-saintandre-xmpp-feature-set-00

Whether I can convince XMPP developers to submit implementation reports
is another question. :)

Perhaps more experiments along these lines are in order?

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Eliot Lear <[email-masked]>
Date: Fri, 02 Nov 2007 07:31:01 +0100

Simon,

> others implementing other protocols have had similar experiences and

In the case of the base IMAP there are SO many implementations that I'm
going to guess (not being fully conversant in IMAP) that the vast
majority of the base specification is well covered.  We are aware of
some interoperability issues with Calendaring, and you're probably right
on that one.  My guess is that LDAP is closer to IMAP than Calendaring. 
But what you write is even true with HTTP, which *is* DS.  There are
lots of little nits with that protocol that the community has expressed
in interest in seeing corrected.

But let me turn it around.  If ALL it takes are two or even three
implementations to demonstrate interoperability, that really says
nothing about maturity.  Take for example SNMPv3.  It was rushed through
to full standard, and yet the deployment issues remainsubstantial, and
are only now being addressed in ISMS.  And SNMPv3 is a full standard. 
[I'm not really trying to pick on v3, but to make the point that what we
have by way of markers in RFC 2026 isn't useful].

> that happens, I'm not sure.

In the case of SNMPv3, I could argue that the premature declaration was
actually harmful because it stalled development of workable solutions
that integrated security on the device.  I was specifically told that
people wanted to let the standard bake in code before allowing for any
sort of updates.  Again, I am not saying that SNMPv3 is bad, but that it
needed to continue to evolve in some areas, and was prevented from doing
so by premature ossification.  Now one could argue that this is not the
general case with our process, but we have so few examples to pick from ;-(

I like the idea that we are reviewing proposals that have been put
before us in the past (Larry Masinter's was just mentioned), and I am
hopeful that we can DEossify our own process ;-)

Eliot

---

From: Ned Freed <[email-masked]>
Date: Thu, 01 Nov 2007 15:33:14 -0700 (PDT)

> others implementing other protocols have had similar experiences and

I don't see how this is in any way relevant. Implementation bugs are a fact of
life for any protocol you care to name, and IMAP is certainly no exception. But
unless those bugs indicate that the specification is ambiguous in some way, or
perhaps that some aspect of the protocol is too difficult for implementors to
get right, I fail to see why any of this would preclude a move to draft.

It is perhaps also worth point ouf that this information is seriously out of
date. Indeed, not only are several of these implementations no longer
available, at least one of them is so old it was no longer maintained at the
time the testing seems to be been done.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Fri, 02 Nov 2007 16:32:24 -0400

Almost makes one think that our standards labeling scheme needs some
simplification.

One might even imagine that the only label after Proposed should be one which
means "is now in widespread use".

So we would have Proposed mean "has undergone technical review" and Full means 
"has gained public acceptance".

d/

Ned Freed wrote:
> The main barrier to IMAP moving to draft is the large number of normative
> references that first need to move first. Getting RFC 2822 to draft is a
> necessary first step and we're working on that. But what about TLS? The
> now-widespread use of TLS+plain has solved a lot of problems for appplications
> but has created a serious obstacle to standards track advancement.
> 
> Perhaps a downreference exception needs to be made here, but if so that needs
> to be approved in advance because nobody is going to bother going through all
> the pain of documenting interoperability without first being sure that a
> normative reference issue isn't going to render their work meaningless.

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: John C Klensin <[email-masked]>
Date: Mon, 05 Nov 2007 01:52:26 -0500

> approved implementation reports, without any formal

Back in the days when words like "simple", as in "Simple foo
protocol" meant what one would expect, I think this would have
been a very useful alternative.  Today, most of our protocols
have many features, options, and characteristics.  I suggest
that only rarely is every one of them implemented,
comprehensively and faithfully, in every implementation that
might appear in an implementation report.   In addition, rarely
has an initial specification of a protocol turned out to be
completely clear and comprehensive.

Consequently, a properly-written and accurate implementation
report is likely to be a mixture of what was fully implemented
without problems, what was partially implemented or ignored
(with explanations), and what was implemented in some particular
way with other possible interpretations.   The only way to take
those reports and produce a clear description of what the
standard should be, as reflected by the combination of the
initial specification and practice experience, involves some
sort of consensus document.  That document could be either a
revision of the original or an extended description of which
parts of the original were still relevant together with
corrections and clarifications as needed.

A mere collection of implementation reports would be
significantly helpful to a reader trying to figure out what the
standard should be iff it was associated with an Applicability
Statement that did the latter job.   FWIW, I think our
experience has been that documents suitable for publication as
Draft or Full Standards are easier to write, and written more
often, than Applicability Statements of that type.

    john