Subject: Proposed IESG Statement Regarding RFC Errata for IETF Sream RFCs

From: The IESG <[email-masked]>
Date: Wed, 16 Apr 2008 08:16:59 -0700 (PDT)

The IESG is considering the following statement to guide the handling of
RFC Errata for IETF Stream RFCs.  Your review and comment on this policy
is encouraged.

Russ Housley
on Behalf of the IESG

- - - - - - - - - - - -

Proposed IESG Statement Regarding RFC Errata for IETF Sream RFCs

   These are strong guidelines and not immutable rules.  Common sense
   and good judgment should be used by the IESG to decide what is the
   right thing to do.  Errata are meant to fix "bugs" in the
   specification and should not be used to change what the community
   meant when it approved the RFC.  These guidelines only apply to
   errata on RFCs in the IETF stream.  They apply to new errata and
   not errata that had already been approved.

   After an erratum is reported, a report will be sent to the authors and

   Area Directors (ADs) of the WG in which it originated.  If the WG has
   closed or the document was not associated with a WG, then the
   report will be sent to the ADs for the Area most closely associated
   to the subject matter.  The ADs for the area will review it, either
   themselves or by delegating, and classify it as falling under
   one of the following states:

   o  Approved - The errata is appropriate under the criteria below and
      should be available to implementors or people deploying the RFC.

   o  Rejected - The errata is in error, or proposes a change to the RFC
      that is clearly inappropriate to do with an errata.  In the latter
      case, if the change is to be considered for future updates of the
      document, it should be proposed using other channels than errata,
      such as a WG mailing list.

   o  Archived - The errata is not a necessary update to the RFC.
      However, any future update of the document should consider this
      errata, and determine whether it is correct and merits including
      in the update.

   Guidelines for review are:

   1.  Only errors that could cause implementation or deployment
       problems or significant confusion should be Approved.

   2.  Things that are clearly wrong but could not cause an
       implementation or deployment problem should be Archived.

   3.  Errata on obsolete RFCs should treated the same as errata on
       non-obsolete RFC where there is strong evidence that some
       people are still making use of the related technology.

   4.  Trivial grammar corrections should be Archived.

   5.  Ugly typos that are clearly bogus typos but would not cause any
       confusions to implementation or deployments should be Archived.

   6.  Changes which are simply stylistic issues or simply make things
       read better should be Archived.

   7.  Changes that modified the working of a protocol to something that
       might be different from the intended consensus when the document
       was approved should be either Archived or Rejected. Deciding
       between these two depends on judgment. Changes that are clearly
       modifications to the intended consensus, or are of major
       importance, should be Rejected. In unclear situations, small
       changes can be Archived.

   8.  Changes that modify the working of a process, such as changing
       an IANA registration procedure, to something that might be
       different from the intended consensus when the document was
       approved should be Archived.

_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Marshall Eubanks <[email-masked]>
Date: Wed, 16 Apr 2008 13:06:33 -0400

Stripping out a raft of cc:s, a nit and a comment in line.

> RFC Errata for IETF Stream RFCs.  Your review and comment on this  

I would suggest

s/modified/modify/

>       importance, should be Rejected. In unclear situations, small

I don't know about this - strip out the "or" clause and you get

Changes that ... are of major importance, should be Rejected.

Are you intending to say that only unimportant errata will be accepted ?

Regards
Marshall

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Wed, 16 Apr 2008 13:11:26 -0400

I support adoption of these proposed guidelines, but have a couple of minor 
comments...

>    to the subject matter.

If the document was produced by a WG that is not closed, the report should 
be copied to the WG chairs as well.

>        confusions to implementation or deployments should be Archived.

The intent here is reasonable, but IMHO it's kind of poorly expressed.
I'd suggest "Clear typographical errors which would not cause...".

-- Jeff

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 17 Apr 2008 10:06:02 +1200

Seems good to me. Will the "archived" and "rejected" errata
be visible to the public? I think that should be mentioned.

BTW, "errata" is a plural noun. The singular is "erratum."

   Brian

> RFC Errata for IETF Stream RFCs.  Your review and comment on this policy
...

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 17 Apr 2008 01:37:54 +0200

> be visible to the public? I think that should be mentioned.

Ideally the IESG procedure is integrated in the errata draft
<http://tools.ietf.org/html/draft-rfc-editor-errata-process>
and published as RFC (it could be even a BCP).

> BTW, "errata" is a plural noun. The singular is "erratum."

But we rarely get away with the singular... ;-) 

 Frank

---

From: Pekka Savola <[email-masked]>
Date: Thu, 17 Apr 2008 09:04:55 +0300 (EEST)

...

>       approved should be Archived.

I do not understand an errata that suggests changing the defined 
process should be Archived.  Shouldn't this be flat out Rejected?

The problem I see with this proposed errata process is that "Archived" 
tries to fill the gap for the need of an issue tracker for substantial 
change suggestions (today these are sent to a subset of authors, WG 
chairs, and/or WG mailing list if active, but are rarely tracked 
systematically).

I don't think the errata process should be used to track substantial 
change proposals.  That procedure needs to be separate from the errata 
process, and it the best place for it would probably be at @ietf.org.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Jari Arkko <[email-masked]>
Date: Thu, 17 Apr 2008 10:39:50 +0300

Marshall, Pekka,

> process should be Archived. Shouldn't this be flat out Rejected?

Right, this seems to be a bug in the text.

> process, and it the best place for it would probably be at @ietf.org. 

Indeed. But please note that the satement is not to be taken as a
recommendation that substantial change proposals be submitted as errata.
You are quite correct that they should be pursued as WG work, as BOFs,
drafts be written, etc. However, in the event that someone does send an
errata about the redesign of the protocol we need to know how to deal
with it :-)

> Are you intending to say that only unimportant errata will be accepted ?

No, that was not the intent. What we wanted to convey was that the
magnitude of some changes may be inappropriate for being done as an
errata. A big redesign requires careful thought, review, probably a
number of revisions of proposals, last call review, etc. As an example,
one of my RFCs has a mistake where it uses the wrong protocol number
constant in a checksum calculation. The authors, RFC Editor, and IANA
failed to catch this error when the number allocations were done. This
is clearly an important issue, and getting it wrong would completely
prevent interoperability. But its also something that can be easily
fixed with an errata. But if we wanted to do something bigger, say,
remove a feature or redesign some aspect of the security solution it
would be inappropriate to do so in an errata.

Maybe the text was just wrong. Here's a suggested edit:

OLD:
Changes that are clearly modifications to the intended consensus, or are
of major importance, should be Rejected
NEW:
Changes that are clearly modifications to the intended consensus, or
involve large changes, should be Rejected

Jari

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 17 Apr 2008 09:53:59 +0100

>be copied to the WG chairs as well.
+1.

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 17 Apr 2008 07:13:19 -0700

>       errata, and determine whether it is correct and merits including

Assuming that both categories will be in the errata repository, the 
difference between these two may be clear to the IESG, but it will 
not be clear to readers of the errata. I suspect that the two 
categories were created so that the IESG only needs to consider 
"errors that could cause implementation or deployment problems or 
significant confusion", not the minor stuff, but this differentiation 
will simply cause more arguments about what errors would cause 
problems of what magnitude.

In the end, it is probably better for readers of the errata to have 
just one category, and for the IESG to not waste its time 
differentiating between the two categories.

--Paul Hoffman, Director
--VPN Consortium

---

From: Lisa Dusseault <[email-masked]>
Date: Thu, 17 Apr 2008 08:45:43 -0700

I can assure you, I at least was anticipating that the IESG (and other  
people handling errata) would be doing *more* work in classifying  
errata if we have the three categories.  The goal as I see it is to  
avoid presenting 50 errata on an RFC to a user, without any sorting or  
focus, when only three of them are crucial to interoperability.  If we  
overwhelm implementors with more than a page worth of errata, most of  
which are junk, implementors will be well justified in ignoring errata.

An important part of the errata handling, therefore, is to make the  
difference clear to the implementor.  When an implementor clicks  
"Errata" for an RFC, they should see the short-list of crucial errata  
and at the end, a link to "Other possible errata" (or other wording).  
With that kind of interface, I don't think readers of errata need to  
care about the exact difference between categories: the essential  
difference, to them, is which ones have been brought to their  
immediate attention.

Lisa

On Apr 17, 2008, at 7:13 AM, Paul Hoffman wrote:

> significant confusion", not the minor stuff, but this  
> differentiation will simply cause more arguments about what errors  
> would cause problems of what magnitude.
>

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 17 Apr 2008 09:25:26 -0700

At 8:45 AM -0700 4/17/08, Lisa Dusseault wrote:
>I can assure you, I at least was anticipating that the IESG (and 
>other people handling errata) would be doing *more* work in 
>classifying errata if we have the three categories.

OK, good. (Well, not good that you were asking for more work...)

>The goal as I see it is to avoid presenting 50 errata on an RFC to a 
>user, without any sorting or focus, when only three of them are 
>crucial to interoperability.  If we overwhelm implementors with more 
>than a page worth of errata, most of which are junk, implementors 
>will be well justified in ignoring errata.

That's a judgement call, one that I would disagree with. It is easy 
to skim a long errata list to weed out the typos; many of us do this 
all the time with the errata for important books we rely on. Even a 
list of 50 (which would be an outlier, I suspect) could be reviewed 
in less than half an hour.

>"Errata" for an RFC, they should see the short-list of crucial 
>errata and at the end, a link to "Other possible errata" (or other 
>wording). With that kind of interface, I don't think readers of 
>errata need to care about the exact difference between categories: 
>the essential difference, to them, is which ones have been brought 
>to their immediate attention.

That seems OK. However, it is far from clear that the amount of 
effort it will take for the IESG, document authors, WG chairs, and so 
on to make that differentiation is worth it: any serious implementer 
is going to look at both lists anyway to be sure that we didn't 
mis-categorize something important into the second list. I guess I'm 
arguing for less work for all of us at the expense of a bit more 
categorizing of importance for the implementers.

--Paul Hoffman, Director
--VPN Consortium

---

From: "David Harrington" <[email-masked]>
Date: Thu, 17 Apr 2008 09:39:35 -0700

Hi,

I read this document. On a quick read, this seemed very reasonable.

David Harrington
[email-masked]
[email-masked]
[email-masked]

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of The IESG
> Sent: Wednesday, April 16, 2008 8:17 AM
> To: IETF Announcement list
> Cc: [email-masked]; [email-masked]; [email-masked]; 
> [email-masked]
> Subject: Proposed IESG Statement Regarding RFC Errata for 
> IETF Sream RFCs 
> 
> IETF-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-announce
>

---

From: John C Klensin <[email-masked]>
Date: Thu, 17 Apr 2008 13:41:30 -0400

--On Thursday, 17 April, 2008 08:45 -0700 Lisa Dusseault
<[email-masked]> wrote:

> I can assure you, I at least was anticipating that the IESG
> (and other   people handling errata) would be doing *more*
> work in classifying   errata if we have the three categories.
> The goal as I see it is to   avoid presenting 50 errata on an
> RFC to a user, without any sorting or   focus, when only three
> of them are crucial to interoperability.  If we   overwhelm
> implementors with more than a page worth of errata, most of  
> An important part of the errata handling, therefore, is to
> make the   difference clear to the implementor.  When an
> implementor clicks   "Errata" for an RFC, they should see the
> short-list of crucial errata   and at the end, a link to
> "Other possible errata" (or other wording). 
>...

--On Thursday, 17 April, 2008 10:39 +0300 Jari Arkko
<[email-masked]> wrote:

>> change proposals. That procedure needs to be separate from
>> the errata process, and it the best place for it would
>> probably be at @ietf.org. 
> 
> Indeed. But please note that the satement is not to be taken
> as a recommendation that substantial change proposals be
> submitted as errata. You are quite correct that they should be
> pursued as WG work, as BOFs, drafts be written, etc. However,
> in the event that someone does send an errata about the
> redesign of the protocol we need to know how to deal with it
> :-)

Hi.  I had planned to just let this go by on the assumptions
that it was not likely to be terribly harmful and that it could
be fixed later if it turned out to not work.  However, your two
sets of comments have convinced me otherwise.

For standards-track materials, errata are part of the much more
general problem of "what do I need to know to implement and
understand this?"  The answer to that question includes several
types of complaints about putative errors in the text, related
documents and their relationships, statements about protocol
maturity (and implementations and deployment), comments about
how much particular provisions can be believed relative to
others, suggestions for a queue of possible changes to be
examined when the document is reopened, and so on.  

Each time we have looked closely at the broader problem, we have
concluded that trying to make a small list of categories and
then force things into them creates a lot of work, ties people
in knots about edge cases in the categories, and often doesn't
really help illuminate the situation.   We've seen that problem
with categories of "Standard", "Draft Standard", "Proposed
Standard", "BCP", etc. I predict that we will see it for
"Approved", "Rejected", and "Archived" and that the number of
"will X be available" and "what does Y mean" questions on the
list point strongly to that conclusion.

The last time we tried for a comprehensive solution to this
problem, the IESG declined to consider it, partially on the
grounds that it would be too much work for the IESG.  Now the
IESG has posted a proposal that is less clear about the
framework and how the pieces are put together, addresses only
one part of the issues, but that calls for the IESG to do a very
large fraction of the work  that was the putative cause of the
dead end we got into the last time.

Give that, I suggest that you consider two options:

(1) Divide recommended errata into only two categories:

(i) Changes that would affect the protocol definition in a
substantive way, even if they appear to be correcting a known
error and to already be reflected in deployed practice.  My
preference, in the current environment, is that this type of
change be reflected in a conventional, "updating" RFCs that is
processed according to the normal standards-track rules.  I can
see reasons for not doing that.  If the IESG does, then this
category should be described in a way that clearly makes it an
"interim update".  I also believe that such changes should be
subject to IETF Last Call for reasons I hope are obvious.

(ii) Everything else.  Like Paul Hoffman, I don't believe that a
major effort to classify things is likely to be worth the
trouble it would take, at least if we are viewing all of this as
"errata".  Whether you just lose the protocol changes that don't
qualify for (i) or identify them as "not considered as a
candidate" or "considered and rejected as a candidate" is up to
you.  Either way, there is a strong hint to the reader that
there is no demonstrated community consensus that the proposed
change has merit and that the specification should be followed
as written.

    --or--

(2) We reopen the broader question, figure out which parts of it
are important to attack first, and move forward.  I suspect
someone could even be found to post the most recent version of
the spec that covered this as an I-D if the IESG were interested.

Good luck in either event.
        john

---

From: Bob Hinden <[email-masked]>
Date: Thu, 17 Apr 2008 11:50:00 -0700

Russ and the IESG,

I generally support this proposal.  However, I think you have made it  
too complex.  Specifically, you have three states, where I think only  
two are required.

>      errata, and determine whether it is correct and merits including

I think that only "Approved" and "Archived" are required.

Approved is correctly for implementors to correct problems in the  
specification.

Everything else is for a working group to consider when the RFC is  
revised.  I see no value in making any distinction between proposed  
Errata if they are not Approved.  It will be for the working group to  
decide.  A working may decide to include changes to the protocol that  
would have been labeled as Rejected.  It seems harmful to give what  
might well be good, but non-interoperable, changes a negative label.   
A working group considering a revision can clearly sort out the good  
and bad suggestions, just as they do to ideas presented to the working  
group at a meeting or on the mailing list.  In essence, everything in  
the Archived queue, can be considered as input to the working group.

Keeping it to two states should also make it easier for the IESG to  
process the proposed Errata in a timely fashion.  I can imagine long  
debates on the differences between Reject and Archived.  As you say in  
the guidelines "Deciding between these two depends on judgment".  The  
is little to be gained by making this distinction.

Bob

---

From: Bill McQuillan <[email-masked]>
Date: Thu, 17 Apr 2008 12:22:47 -0700

> revised.  

I believe that this is a good way to go.

One quibble that I have is with the word "Archived". It merely describes
the mechanism to be used. (BTW, I hope that Approved Errata will also be
archived!)

"Archiving", IMO, implies "saving something valuable". Unfortunately, it
doesn't distinguish between items that are of value to be considered in the
next update discussion and items that may be of value to current
implementors.

I would propose that the two classifications be labeled: "Approved" and
"Not Yet Approved" with the clear understanding that *both* such types of
items will be archived so as to be available to the next document update
process.

-- 
Bill McQuillan <[email-masked]>

---

From: Russ Housley <[email-masked]>
Date: Thu, 17 Apr 2008 16:00:38 -0400

>process should be Archived.  Shouldn't this be flat out Rejected?

The intent was for the proposed change (submitted as an errata) to be 
considered if the document ever gets revised.

Russ

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 18 Apr 2008 10:13:49 +1200

> I believe that this is a good way to go.

I'm not convinced by any of the arguments against 3 categories.
There are proposed errata that are simply wrong, and there's no
reason to keep them around as potential future distractions.
So I think the "Rejected" category is useful.

> One quibble that I have is with the word "Archived". 

Yes, it carries unintended semantics.
...

> items will be archived so as to be available to the next document update

I would simplify that to "Not Approved." The "Yet" also carries
unintended semantics.

    Brian

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 17 Apr 2008 17:03:32 -0700

>I would simplify that to "Not Approved." The "Yet" also carries

 From what Lisa and Russ said, these minor changes *are* approved: 
they're just not considered as important as "Approved" ones. The 
semantics seem silly to me. Even on an RFC with 100 editorial nits 
and only a dozen significant errata, a developer reading the whole 
list would not spend more than an hour or two separating the wheat 
from the chaff.

--Paul Hoffman, Director
--VPN Consortium

---

From: "Sabahattin Gucukoglu" <[email-masked]>
Date: Fri, 18 Apr 2008 20:13:52 +0100

Hi,

How does Joe Implementer become aware of RFC errata available?  I don't 
recall seeing anything in the standard boilerplate to point them out.  
(And although I'm aware of them myself, I'd quite forgotten about them.)

Cheers,
Sabahattin

- -- 
Sabahattin Gucukoglu <mail<at>sabahattin<dash>gucukoglu<dot>com>
Address harvesters, snag this: [email-masked]
Phone: +44 20 88008915
Mobile: +44 7986 053399
http://sabahattin-gucukoglu.com/

---

From: "Frank Ellermann" <[email-masked]>
Date: Sat, 19 Apr 2008 21:34:37 +0200

Sabahattin Gucukoglu wrote:
 
> How does Joe Implementer become aware of RFC errata available?

1 - Joe found an RFC 4567 link in a default MediaWiki.  These
    links go to a page <http://tools.ietf.org/html/rfc4567>
    with a link to the errata at the end of the "tools" header.
2 - Joe found a <http://purl.net/net/rfc/4567> link somewhere,
    arriving at the RFC editor search form showing the meta
    data for RFC 4567, including a link to the (same) errata.
3 - Joe ran out of luck...

Abusing 4567, because 1234, 2345, and 3456 have no errata :-)

 Frank

---

From: "Yaakov Stein" <[email-masked]>
Date: Mon, 21 Apr 2008 23:01:13 +0300

All three categories are absolutely needed.

It is self evident, although unfortunate,  that the "accepted" category
will be used.
Even after WG, IESG, IETF LC, and the RFC editor, some errors make it
through.

From my experience with RFC errata, the "rejected" category will also
definitely be used.
I have seen some entirely erroneous comments
(for example claiming that pseudocode for packet processing could not be
correct
since the loop over packets never terminates - the proposed fix was to
decrement
the packet size each time and terminate when the packet size reached
zero!),
and some pretty useless rewordings.

Although we can quibble over the nomenclature, the "archived" category
has several uses.
One example is when the erratum submitter was not a WG participant,
and truly could not understand what was intended (at least without the
authors explaining what they meant).
If the RFC is ever obsoleted by a newer one, this will serve as a
reminder to rewrite that passage.
Another example is when the RFC suggests a method to handle an
exception, 
while a simpler method is inherent in the protocol itself.

Y)J(S

---

From: Marshall Eubanks <[email-masked]>
Date: Thu, 24 Apr 2008 11:49:12 -0400

Dear Jari;

Sorry I missed this on the 17th.

> would be inappropriate to do so in an errata.

I fully support your thinking here.

> are

I like this, but how about

s/involve large changes/involve large textual changes/

I am worried about cases like the one you mentioned above (or the  
infamous "missing not") where getting one character wrong or one word  
wrong makes
a huge difference. (In your checksum case, getting it wrong by one  
digit is as bad as getting it totally wrong.) That sounds like a  
proper errata to me.

Whereas, if the author says something like, "every mention of IS-IS  
should really be OSPF and Sections 4 and 5 need to be extensively  
rewritten to account for this,"
it may be an error, but it is not an errata.

Regards
Marshall