Subject: draft-klensin-newtrk-8540style-harmful (and (and draft-roach-bis-documents-, etc.)

From: John C Klensin <[email-masked]>
Date: Fri, 07 Jun 2019 12:39:05 -0400

Hi.

After a number of discussions, some of them off-list, with Adam
Roach and others, in the last month or so, I've just posted an
I-D.  That I-D is a personal analysis and critique of the
practice of publishing Informational RFCs in the IETF Stream
(or, for that matter, any other Stream) that appear to update
standards track documents without actually being updates and a
few closely-related issues.   It seems to me that publication of
such documents in the IETF Stream creates a greater opportunity
for confusion about what is and is not a standard and with its
status is than almost anything that could be done in another
stream, particularly because the boilerplate of IETF Stream
documents asserts that the represent IETF Consensus and
documents coming from the other Streams explicitly disclaim that.

The I-D is intended to provide a more in-depth analysis of some
of the issues that I understand motivated
draft-roach-bis-document, but  the latter addresses only a
subset of the issues the I-D identifies.   It also raises the
question of whether the type of documents it critiques and their
implications indicate that it is time to reopen some of the
questions about identification about what is and is not part of
a standard that were discussed in NEWTRK more than a decade ago.

This is a personal critique.  While it makes some suggestions
that the community might want to examine, it does not attempt to
set any sort of policy.  My expectation is to hand it off to the
ISE after it has had a chance to ripen a bit.  If the IESG or
particular ADs believe it should be in the IETF Stream, I'm
happy to discuss that but with the understanding that, were it
adopted as part of an IETF policy, I'd expect the IESG to start
moving documents that violate its principles to Historic (at
least) and to not allow any new documents of that sort.  As an
Independent Stream critique, the question of whether policy
actions should be built on top of it and what those should be
remain, as it were, independent. 

Comments welcome but, unless they address the question of
whether or why it should be processed in the IETF Stream, they
should probably be addressed to me personally rather than to the
IETF list.  Please retain enough of the subject line that I can
spot comments easily

best,
    john

---

From: Michael Richardson <[email-masked]>
Date: Fri, 07 Jun 2019 14:51:16 -0400

John, I haven't read your whole document, and you mention
RFC 8540 specifically.  I'm guessing that this is the document that pushed
you to think about this?

>   The WG suggested in its summary for IETF
>   Last Call for what became RFC 8540 that an errata listing like that
>   provided by RFCs 4460 and 8540 is helpful in producing replacements
>   for the original documents [LC8540-Statement] but there is no
>   evidence that the same purpose could not be served by retaining the
>   same list as an Internet-Draft until the actual replacement document
>   is ready to be published and then either discarding that I-D or, if
>   the WG felt a need to do so, incorporating the errata listing as an
>   appendix in the final document.

As far as I can see, 8540 was produced by the tsvwg, and went through IETF
process. Yes, it's informational, rather than standards track ("Updates"),
but that seems somewhat immaterial to me.

I am not an expert in SCTP or the issues reported, but my guess is that the
situation is that the issues reported do not affect all users, but that they
affect enough that having some clear text is useful.

What you suggest, that it remain an ID would seem to me, to elevate IDs to be
equivalent to RFCs.

I couldn't puzzle out what your Conclusion was.  Maybe if you'd dealt with
another example, it would help.    I was involved in NEWTRK, and I think you
probably need to hit the reader over the head harder here.

||ugh Daniel's once lamented that it every software release needs a label so
that one can refer to it properly, but that it's often hard to know which
releases are good ones until after they get a label.   He described what he
wanted was a kind of "weather report", which tells you how things worked out
earlier in the "day".   I think that you (and NEWTRK) are really asking for
this.

It seems to me that RFC8540 needs to be a weather report, and that we really
need a new kind of document for this.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 8 Jun 2019 12:02:15 +1200

> need a new kind of document for this.

For me a big question is whether weather reports should use RFC2119 language,
which makes it easy for an outsider to mistake them for standards.

But yes, https://tools.ietf.org/html/draft-klensin-isdbis-00
I don't think that ignoring that idea has served the IETF well.

   Brian

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 8 Jun 2019 09:06:17 +0200

Another example to consider: RFC 4815.
This was not a weather report.  It was addressing a similar need:
Getting small corrections and clarifications out that came from experience gathered in half a decade of deployment of a very new protocol and framework.

It did stay at I-D stage (draft-ietf-rohc-rtp-impl-guide-00 was Feb 2002) for most of that time, until we decided it was time to wrap it up.

For RFC 4815, we ultimately went for standards-track, and with hindsight I have no idea why one would do otherwise.

I’m currently proposing to do a similar document for CoRE.

Grüße, Carsten

https://datatracker.ietf.org/doc/rfc4815/

4815 RObust Header Compression (ROHC): Corrections and Clarifications to
     RFC 3095. L-E. Jonsson, K. Sandlund, G. Pelletier, P. Kremer.
     February 2007. (Format: TXT=74819, HTML=0 bytes) (Updates RFC3095,
     RFC3241, RFC3843, RFC4019, RFC4362) (Status: PROPOSED STANDARD)
     (DOI: 10.17487/RFC4815) 

> On Jun 8, 2019, at 02:02, Brian E Carpenter <[email-masked]> wrote:
>

---

From: Michael Richardson <[email-masked]>
Date: Sat, 08 Jun 2019 18:21:23 -0400

> which makes it easy for an outsider to mistake them for standards.

Hmm. Good point.

    > I don't think that ignoring that idea has served the IETF well.

Maybe we are ready for this now?
If it eliminates the need to publish so many -bis documents, then that would
seem to reduce some of the same pain that roach-bis is trying to reduce.

---

From: Michael Richardson <[email-masked]>
Date: Sat, 08 Jun 2019 18:23:46 -0400

> For RFC 4815, we ultimately went for standards-track, and with
    > hindsight I have no idea why one would do otherwise.

and I think Klensin would be happy with that designation.

    > I’m currently proposing to do a similar document for CoRE.

sounds reasonable.  Care to comment on your views of the relative effort of
doing this vs a -bis document?  Would roach-bis change that for you?

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 9 Jun 2019 01:12:01 +0200

On Jun 9, 2019, at 00:23, Michael Richardson <[email-masked]> wrote:
> doing this vs a -bis document?  

The bis document is fine if you can ignore the implementer community for a while and then come back and do the bis.  This is not what we wanted to do; we wanted to have a live document that had the fixes dynamically added to it, and that is only in time getting the full IETF consensus process.

(We had a pretty interesting first ROHC interop [“bay cough”], where we had two camps that had implemented some boring protocol parser detail in a different way, where both camps could point to text that supported their position.  This is how getting clarifications agreed and made available in a timely way became part of the culture.  [And it also raised our interest in doing formal description techniques for the boring stuff as shown in RFC 4997.])

> Would roach-bis change that for you?

We could of course treat the I-D for the bis document as a living document for a few years.  Not sure that makes a lot of sense when you have a whole ecosystem of specs working together — do you run coordinated roach-bis documents?

The point of a corrections and clarifications document is that it calls out those corrections and clarifications specifically, not by simply changing the original document, but also providing context and rationale.

Grüße, Carsten

---

From: Christian Huitema <[email-masked]>
Date: Sun, 9 Jun 2019 16:29:05 -0700

> but that seems somewhat immaterial to me.

I have read both John's draft and RFC 8450. John makes essentially two
points: that informational documents should not "informally" update
standard track documents, and that structuring a document as a catalog
of erratas and updates makes for hard to read text.

I would agree with John on the first point. The document status is
important. RFC 8450 should have been published in the standard track,
and it should have formally updated RFC 4960. If it had, people reading
4960 would immediately notice the "updated by" reference, but currently
they don't.

As for the second point, I think it a classic trade-off between
timeliness and quality. This is best judged by the WG and the AD.

-- Christian Huitema

---

From: John C Klensin <[email-masked]>
Date: Sun, 09 Jun 2019 19:59:30 -0400

--On Sunday, June 9, 2019 16:29 -0700 Christian Huitema
<[email-masked]> wrote:

>> As far as I can see, 8540 was produced by the tsvwg, and went
>> through IETF process. Yes, it's informational, rather than
>> standards track ("Updates"), but that seems somewhat
>> immaterial to me.
> 
> I have read both John's draft and RFC 8450. John makes
> essentially two points: that informational documents should
> not "informally" update standard track documents, and that
> structuring a document as a catalog of erratas and updates
> makes for hard to read text.
> 
> I would agree with John on the first point. The document
> status is important. RFC 8450 should have been published in
> the standard track, and it should have formally updated RFC
> 4960. If it had, people reading 4960 would immediately notice
> the "updated by" reference, but currently they don't.
> 

Christian,

Thanks.   I'm trying to avoid discussing this non-IETF document
on the IETF list, but an observation about that tradeoff.
Assuming standards track, I think there is actually a three-way
tradeoff among:

(1) A list of so-called "verified" errata in the form taken by
8540.

(2) A list of changes (or, as some might say, patches) that have
IETF consensus, in the form of, e.g., RFC 4815 if the list is
intended to be comprehensive.  Absent a desire to be
comprehensive, most of our "updates" documents fall into this
category as well and there is a potentially large range between
an update or two and a comprehensive patch list.

(3) A revised document that replaces the original one.  That is
our classic "obsoletes" case and, with, AFAICT, a small
variation on draft-roach-bis-documents.  

I see many tradeoffs between the variations on (2) and (3) and
agree with you that the choice is best judged by the WG and the
AD.  However, the first raises some other issues.  In no
particular order, the two most important are that it does not
reflect well on the IETF to publish documents in the RFC Series
that are not internally consistent or that otherwise create
confusion about what the actual content of a standard is.  And
we actually have a rule (in RFC 2026) about known omissions: an
errata list cannot be comprehensive because whether or not
errata are filed is a bit serendipitous and different
corrections to the same section of text that are not obviously
consistent are certainly an omission of the language needed to
resolve those inconsistencies (8540 basically tells users to
sort it out themselves.   In addition, if you look at the
ballots for 8540, you will see places where one or more ADs
claimed that the solutions in the listed errata are incorrect
and proposed others.  That is a "known omission" if I've ever
heard of one and may be evidence of lack of IETF (not just WG)
consensus.  Now, 2026 allows the IESG to waive that "no known
omissions" rule.  If there were clear text in a standards track
version of 8540 that identified what was unresolved and the IESG
approved a waiver on that basis, I'd think that publishing it
would be a reasonable judgment call on the WG's part and,
assuming IETF consensus, that of the IESG.   But I think not
otherwise.

best,
   john

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 10 Jun 2019 21:43:56 -0500

I have opinions about RFC 8540 that disqualify me from detailed discussions
about specifics in this thread, but ...

> I don't think that ignoring that idea has served the IETF well.

I couldn't agree more about this one, and especially (in this case) about
the opportunity to do a better job of errata handling than we do today,
using something like
https://tools.ietf.org/html/draft-klensin-isdbis-00#section-11.2.2). I note
that it's only been within the past year or three that we've added "Errata
Exist" hyperlinks on the HTML version of RFCs, and that was a Great Leap
Forward at the time.

Spencer