Subject: Errata Processing Stats/Queue?

From: Fernando Gont <[email-masked]>
Date: Tue, 6 Aug 2019 14:59:29 +0300

Folks,

Are there stats (whether global, per-area, per-wg, or whatever)
regarding the processing of Errata?

i.e., how many errata are in each state (Reported, verified, rejected,
etc.), how long it took them to be processed, etc.?

Thanks!

Cheers,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Brian Haberman <[email-masked]>
Date: Tue, 6 Aug 2019 08:08:54 -0400

https://www.rfc-editor.org/errata.php

---

From: Stewart Bryant <[email-masked]>
Date: Tue, 6 Aug 2019 13:20:43 +0100

Looking for status = reported by Area is a good way of seeing whether 
the ADs are processing them:

App = 113
Art = 228
Gen = 13
Int = 75
Ops = 23
Rai = 55
Rtg = 19
Sec = 169
Tsv = 58

I am please to see that RTG is still low (though could be lower!) 
considering the work it took to get them down to an acceptable number a 
few years back, but some of the other numbers look excessive.

- Stewart

---

From: Fernando Gont <[email-masked]>
Date: Tue, 6 Aug 2019 15:48:39 +0300

> few years back, but some of the other numbers look excessive.

Skimming through some of the errata, some areas seem to have technical
errata in their "Reported" Queue for over 5 years.

Is there an upper bound on how long errata can remain unprocessed, or
how long the queue may grow?

Any plan to, say, limit the amount of time errata can remain unprocessed
to, say, 6 months - 1 year?

How does this play when an AD leaves the role with a full queue of
unprocessed errata?

Thanks!

Cheers,

---

From: Stewart Bryant <[email-masked]>
Date: Tue, 6 Aug 2019 13:59:28 +0100

> How does this play when an AD leaves the role with a full queue of

The usual process is for the IETF Chair tell the ADs to address them and 
get the numbers down, and for the rest of us to have a discussion with 
the the Nomcom if neither of those happen.

- Stewart

---

From: John C Klensin <[email-masked]>
Date: Tue, 06 Aug 2019 09:55:41 -0400

--On Tuesday, August 6, 2019 13:59 +0100 Stewart Bryant
<[email-masked]> wrote:
 
>>> Looking for status = reported by Area is a good way of
>>> seeing whether the ADs are processing them:
>...
>>> I am please to see that RTG is still low (though could be
>>> lower!) considering the work it took to get them down to an
>>> acceptable number a few years back, but some of the other
>>> numbers look excessive.
>> 
>> Skimming through some of the errata, some areas seem to have
>> technical errata in their "Reported" Queue for over 5 years.
>> 
>> Is there an upper bound on how long errata can remain
>> unprocessed, or how long the queue may grow?
>> 
>> Any plan to, say, limit the amount of time errata can remain
>> unprocessed to, say, 6 months - 1 year?
>> 
>> How does this play when an AD leaves the role with a full
>> queue of unprocessed errata?

Probably the same thing that happens when an erratum is reported
long after the AD who managed the document or WG leaves the role.

>...
> The usual process is for the IETF Chair tell the ADs to
> address them and get the numbers down, and for the rest of us
> to have a discussion with the the Nomcom if neither of those
> happen.

Agreed, but let's take a step back.  First, large numbers can
also indicate that someone has gone on an errata rampage,
finding many trivial issues and filing errata on them, possibly
on the theory that it is an important contribution the IETF
(sometimes it is).  But if I were trying to draw strong
inferences from statistics, I'd try to look at controlling for
such efforts, especially in Areas where the subject matter makes
it easier for a non-expert to read the documents and flag
errors, especially editorial ones.

More important (applying primarily to standards track documents,
but generalizations may be possible)...

What I'm about to say interacts with both the "evolving
documents" (or whatever they are called this week) and WG
declarations of stability discussions, so maybe things will
change.  But right now and historically (including those
statistics), errata, even "verified" ones are nothing more than
placeholders for some change that could occur in the future.
"verification" generally doesn't indicate the consensus of a WG
if there is a relevant one and certainly does not represent IETF
consensus.

So, ignoring errata reports that are just wrong, what we have
are 

(1) many reports of editorial issues, some of which make the
documents less clear and others that are just annoying (the
distinction may depend on the reader's familiarity with the
subject matter and/or comfort with suboptimal English and so may
not be objective).   

(2) some reports of substantive problems where the document is
unclear about what it is requiring or recommending or just plain
wrong .. in both cases, at least in the opinion of the reporter
and any reviewers of the erratum.  These are all properly
classified as "save for update" because, again, there isn't
sufficient IETF consensus behind an erratum to say "the IETF has
concluded that the document is incorrect".

The _only_ solution to (2) and the best solution to (1) if the
problem is significant is an Internet-Draft and, eventually, a
new RFC, even if those documents say little more than "Section
N.M of RFC XXXX is wrong and should read ..." (ideally with an
explanation of why).  An erratum is a useful step along the path
to that result, but it not an endpoint.  

So, two questions:  Is putting energy into incremental
improvements in the errata system and errata processing
worthwhile or would the time spent doing that work be better
spent on updating or otherwise improved documents?  And, if we
want to measure errata processing and effectiveness, should we
be also measuring how many errata that identify substantive
issues lead to new documents on a timely basis?  As part of the
latter, if I need something to complain to the nomcome about an
AD, should it be how many errata get processed or about how many
documents get published and then generate substantive errata
and/or how many errata that address substantive issues lead to
new consensus documents.

best,
   john

---

From: Stewart Bryant <[email-masked]>
Date: Tue, 6 Aug 2019 15:33:03 +0100

On 06/08/2019 14:55, John C Klensin wrote:
> if I need something to complain to the nomcome about an

All of the above.

A lot of errata are relatively easy to dispatch, it is part of the job, 
and is one reason why we have technical experts as ADs.

- Stewart

---

From: Stewart Bryant <[email-masked]>
Date: Tue, 6 Aug 2019 15:38:10 +0100

> and is one reason why we have technical experts as ADs.

I would also note that there are at least two ADs per area, a 
directorate per area, and 2N WGCs, so there is a lot of expertise that 
the ADs can call on to address these issues as they are posted, and to 
work through the backlog.

- Stewart

---

From: Fernando Gont <[email-masked]>
Date: Tue, 6 Aug 2019 17:43:42 +0300

> long after the AD who managed the document or WG leaves the role.

THere seems to be a difference there. What I was referring to is that if
an AD leaves the role with 2 years worth of errata unprocessed, that's
probably an interesting "fresh start" for the incoming AD...

> errors, especially editorial ones.

For technical errata, I'd actually try to foster it. It's the equivalent
to "code review": you do want people to search for bugs. That tends to
improve things.

> concluded that the document is incorrect".

If only we were to revise specs frequently. That doesn't seem to be the
case. And in fact, there's people that find that undesirable...

> to that result, but it not an endpoint.  

I agree that errata are not and endpoint. On the other hand, documents
that patch other documents tend to end up in a mess. Because when you
read the base spec, you have to remember what parts have been obsoleted,
do a mental replacement, etc.

When the number of updating documents (or the number of parts of the
original document that need to be patched) becomes meaningful, the right
thing would be to coalesce all into a single document.

> spent on updating or otherwise improved documents?

One would expect that the two are not mutually exclusive. Many wgs
benefit from the errata processing, such that when a document is
revised, part of the bis effort is to incorporate all verified errata.

> And, if we
> issues lead to new documents on a timely basis? 

Yes, we should. Also low numbers for such stats could indicate either
not-so-meaningful errata, but also wgs simply failing to revise documents.

>  As part of the

All?

If RFCs were code, errata would be bug reports. Failure to process and
patch bug reports is bad. Shipping buggy code is not nice, but also happens.

Ideally, you'd expect that code is generally ok, but if a bug is found,
it is patched in a timely manner. Same for RFCs: one would expect that
the quality is high, and there's not much room for filing errata. BUt if
errors are found, that such error reports are processed in a timelier
manner.

Thanks,

---

From: Stewart Bryant <[email-masked]>
Date: Tue, 6 Aug 2019 16:05:13 +0100

> probably an interesting "fresh start" for the incoming AD...

It was.

- Stewart

---

From: Michael Richardson <[email-masked]>
Date: Tue, 06 Aug 2019 11:44:20 -0400

> https://www.rfc-editor.org/errata.php

That's the general search page.
I think that Fernando wants to know how many are in each state (on a per-WG,
and per-area basis), and to see this as time-series such that we can plot
things and see if we are doing better or worse.

---

From: Michael Richardson <[email-masked]>
Date: Tue, 06 Aug 2019 11:48:00 -0400

Stewart Bryant <[email-masked]> wrote:
    > The usual process is for the IETF Chair tell the ADs to address them
    > and get the numbers down, and for the rest of us to have a discussion
    > with the the Nomcom if neither of those happen.

I think that we need to do something to allow the work to be more visible to
WG chairs, leaving the ADs to deal with errata where there are no WG, and
then redirect it to an appropriate WG or XYZAREA-WG.
Some of this could be done by directorates too.

ADs are too valuable, and yet usually don't have the subject matter,
and at this point, authors don't have any ability to act.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Benjamin Kaduk <[email-masked]>
Date: Tue, 6 Aug 2019 10:52:42 -0500

> and at this point, authors don't have any ability to act.

Most of the time, if the authors reply back that an errata is (in)valid, I
am willing to act on it.  More input from a WG list is better, of course,
but if I'm left to my own devices, I tend to only act on errata related to
technologies that I consider myself to already be an expert at.

-Ben

---

From: Fernando Gont <[email-masked]>
Date: Tue, 6 Aug 2019 18:57:43 +0300

> things and see if we are doing better or worse.

Exactly.

To be able to quickly answer: how long, in average, does it take a a
technical errata in the X Area to be processed? Has this value changed
over time? etc etc

Thanks!

Cheers,

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 6 Aug 2019 12:05:09 -0400

What we know from history (that Stewart alluded to) is that errata 
processing times are quite variable.
The factor Stewart got hit by is the attention the AD(s) pay to it.
A factor John alluded to is, for example, when we had an interested and 
supportive individual read every RFC and note every grammatical error 
they found as an errata.
And yes, it does help if WGs are responsive to errata as well.

But trying to perform statistical analysis on the numbers is likely to 
appear to tell us things that just aren't so.

Yours,
Joel

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 6 Aug 2019 18:28:01 +0200

> technologies that I consider myself to already be an expert at.

If the WG that owns the technology is still around, the errata report should be tied to this WG.

In CoRE, we have recently started to make github issues out of our outstanding errata reports.  Not a guarantee of success, but a start.  Now we need to act on this:

— collect relevant information in the github issue
— once the collection appears reasonably complete, agree on a disposition
— check consensus on the mailing list

(This is on the chairs’ meager set of todos now.)

But there is also the underlying problem that the errata reporting process is so weirdly rigid, and so unintegrated into everything else we do.  That is maybe something the next RSE could invest effort on…

Grüße, Carsten

---

From: Fernando Gont <[email-masked]>
Date: Tue, 6 Aug 2019 19:28:06 +0300

> The factor Stewart got hit by is the attention the AD(s) pay to it.

The part I think is not right is when reported *technical* errata don't
get processed for years.

> supportive individual read every RFC and note every grammatical error

I probably know the person you are referring to. But he wasn't just
submitting errata. He would also review documents at early stages
(drafts) with a level of detail that would blow my mind (and energy ;-)
). I'm thankful for that, he did a lot to help improve documents I
co-authored before they got published.

Editorial errata on RFCs might be of less value. But I see *tecnical*
errata on RFCs as a lot of value -- and as a basis on what to patch
when/if a documents gets revised.

> And yes, it does help if WGs are responsive to errata as well.

FWIW, I do think when an appropriate WG is available, they should work
on emptying the errata queue.

Like bugs, if nobody cares in patching, why would folks care to report?

> appear to tell us things that just aren't so.

Not much of an analysis, but more of a simple way to spot where there
might be room for improvement.

Thanks,

---

From: Fernando Gont <[email-masked]>
Date: Tue, 6 Aug 2019 20:06:09 +0300

> technologies that I consider myself to already be an expert at.

What about the rest? Particularly for security, one may wonder what the
unprocessed (technical) errata are about, and what the impact may be.

---

From: Benjamin Kaduk <[email-masked]>
Date: Tue, 6 Aug 2019 12:17:15 -0500

> unprocessed (technical) errata are about, and what the impact may be.

I'm not sure I understand the question.  From memory, a qualitative sense
is that (leaving outside the substantial fraction of purely editorial
ones), many reports are a difference of opinion between the reporter and
the authors/WG, or pointing out things that we wouldn't do again based on
what is well-known now, but were intended at the time (and thus do not
really meet the qualifications for errata).  There are, of course, the
handful of cases that point out actual security-relevant or
interoperability-affecting flaws, and while I/we do try to process those
more reliably, it's still not particularly reliable.  That said, as John's
note alludes, from the perspective of a potential implementor wanting to do
the right thing, there's not a huge difference between having an errata in
the "reported" state vs. "hold for document update" (which is what many of
these would end up as, due to the nature of the proposed change) -- a new
document would likely be more effective.

-Ben

---

From: Michael Richardson <[email-masked]>
Date: Tue, 06 Aug 2019 16:50:46 -0400

> Like bugs, if nobody cares in patching, why would folks care to report?

1) If WG-chairs could change the state themselves that would help.
2) If WG sessions had a number of errata as part of the status slide,
   then that would probably help close the loop.

    > might be room for improvement.

Agreed.

---

From: Kathleen Moriarty <[email-masked]>
Date: Wed, 7 Aug 2019 07:31:34 -0400

>    then that would probably help close the loop.

I agree with this.  Errata needs to be delegated further to improve
processing.  If all of the ones that existed where there is an open WG
could be handled by the chairs, they would be more familiar with each
errata, who the experts were that could approve them, and may have the
history having been chairs for a long period of time.

Some ADs simply don't think errata is useful.  I handled some, but worked
with ADs who didn't care about them.  As a result, Security has the second
highest score, leaving Ben and Roman with a bit too much to deal with at
this point.

If there's no WG that remains or never was one, it would be nice if those
could be delegated too.  The usual process is for editors of the draft to
review the errata and respond on list as to their thoughts.  Then the AD is
able to press the buttons.  Maybe someone else could monitor the list
conversations and update the text in the errata filing accordingly with a
link to said email confirmations and press the buttons?

Best regards,
Kathleen

-- 

Best regards,
Kathleen

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 7 Aug 2019 12:49:03 +0100

Hiya,

On 07/08/2019 12:31, Kathleen Moriarty wrote:
> with ADs who didn't care about them. 

I guess I was one of those:-) I wouldn't say I considered
errata as not being useful, but I do consider most errata
are not useful and many are hugely time-consuming (it can
take an hour to acquire enough context to decide if moving
that comma is ok or not;-), and as a result my approach was
to leave errata alone unless one was discussed on one of my
WG's lists, or it was immediately apparent that the change
was good, or if someone other than the of the erratum mailed
me or a list to which I as subscribed (be that a WG
participant or RFC author or whomever) saying the erratum
was a good change.

Basically, I took silence as indicative of unimportance as
I didn't have time to process 'em all. And the world didn't
end, but yes the queue built up some more.

So another way to improve this might be to put new errata
into a "probably nobody cares right now" queue until it is
apparent that someone other than the poster of the erratum
cares that the change is good and ought be visible. At that
point an erratum could be moved into the "for processing"
queue. A WG later developing a bis draft could consider
those "nobody cared" errata just as much as ever, but in
the meantime they'd not bother the rest of the world (which,
again, would not end:-).

Cheers,
S.

---

From: Fernando Gont <[email-masked]>
Date: Wed, 7 Aug 2019 14:59:41 +0300

On 7/8/19 14:31, Kathleen Moriarty wrote:
> 
> 
> On Tue, Aug 6, 2019 at 4:51 PM Michael Richardson <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
> 
>     Fernando Gont <[email-masked] <mailto:[email-masked]>>
>     wrote:
>         >> And yes, it does help if WGs are responsive to errata as well.
> 
>         > FWIW, I do think when an appropriate WG is available, they
>     should work
>         > on emptying the errata queue.
> 
>         > Like bugs, if nobody cares in patching, why would folks care
>     to report?
> 

+1

> Some ADs simply don't think errata is useful. 

Isn't it their duty to process them? (no matter what their personal
opinion is)

> I handled some, but
> worked with ADs who didn't care about them.  As a result, Security has
> the second highest score, leaving Ben and Roman with a bit too much to
> deal with at this point.

I understand that that wasn't a fresh start for them. That said, it
would seem to me that some mechanism and control loop should be in place
to avoid this kind of thing from continuing. Particularly for security,
it's kind of ironic that the queue of unprocessed technical errata is so
long.

That's the equivalent of a vendor not processing vulnerability reports....

> If there's no WG that remains or never was one, it would be nice if
> those could be delegated too.  The usual process is for editors of the
> draft to review the errata and respond on list as to their thoughts. 
> Then the AD is able to press the buttons.  Maybe someone else could
> monitor the list conversations and update the text in the errata filing
> accordingly with a link to said email confirmations and press the buttons?

Is there something like AD "secretary" (parallel to wg secretaries) that
might help with this?

Thanks,

---

From: Fernando Gont <[email-masked]>
Date: Wed, 7 Aug 2019 15:06:28 +0300

On 7/8/19 14:49, Stephen Farrell wrote:
> that comma is ok or not;-), 

That may apply to editorial errata, but not to *techical* errata.

That doesn't mean that the errata might be of use, but it would seem to
me that it would be due dilligence to at least process the *technical*
errata.

> cares that the change is good and ought be visible. 

Isn't  part of IETF's mission to maintain its protocols? If nobody cares
about submitted errata, that would make one one wonder about the extent
to which that pat of the mission can be accomplished.

> At that

What I would expect is that, as with reported software vulnerabilities,
technical errata are processed, asap. It is not that the reporter is
telling you "review your spec, there is a bug, and i won't tell you
why", but normally the problematic text is quoted, and a solution proposed.

Get the author of the spec to process it, and/or else the wg chairs,
and/or the relevant wg, or, why not, a set of people from the Acks
section. Complete AD processing my happen as a last resort...

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 7 Aug 2019 13:31:21 +0100

Hiya,

On 07/08/2019 13:06, Fernando Gont wrote:
> It is not that the reporter is

I suggest you select a bunch of unprocessed errata (without
cherry picking:-) and record how long it takes you to write
'em up so that an AD could quickly read your email and then
hit approve or hfdu or whatever. I'd be interested in the
time it takes you to do 10 or 20 of those. And I'd not be
surprised if ADs were happy if you wanted to volunteer to
do that periodically.

Cheers,
S.

---

From: Fernando Gont <[email-masked]>
Date: Wed, 7 Aug 2019 15:39:05 +0300

On 7/8/19 15:31, Stephen Farrell wrote:
> hit approve or hfdu or whatever. 

Not sure what you mean. Are you arguing that the errata are usually
poorly written/submitted? That they are hard to process anyway?
Something else?

> I'd be interested in the

Doesn't the same analogy apply to processing vulnerability reports? And
continuing with the analogy, would it be fine to just leave the bugs
there because they take time to patch?

Cheers,

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 7 Aug 2019 13:44:27 +0100

> section. Complete AD processing my happen as a last resort...

I agree with Fernando here. We have a duty to maintain our protocols.

It is a dereliction of duty by the IESG if they fail to resolve the 
errata, which after all are assigned to them, in a timely manner.

If the IESG would prefer some method other than addressing them 
themselves, it is in their gift to implement a method or process for 
addressing them.

For example, a small group of competent area specialists sitting 
together at the equivalent of a hackathon table ought to be able to 
dispatch significant number, and where they could not, they could craft 
the right set of questions to gather the necessary information to 
resolve the issue.

- Stewart

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 7 Aug 2019 06:13:08 -0700

> participant or RFC author or whomever) saying the erratum

It's worth noting that the way we present errata significantly decreases
their usefulness, as they are not really that obvious when one reads the
RFC. The inline errata display project would significantly increase the
value of errata, and at least for me, would have increased the importance
of processing them.

Dear IESG: can you report on the status of that project and when the
tools/datatracker sites will show inline errata?

Thanks,
-Ekr

---

From: Fernando Gont <[email-masked]>
Date: Wed, 7 Aug 2019 16:22:27 +0300

On 7/8/19 16:13, Eric Rescorla wrote:
> 
> 
> On Wed, Aug 7, 2019 at 4:49 AM Stephen Farrell
> <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
>     Hiya,
> 
> value of errata, and at least for me, would have increased the
> importance of processing them.

While the non-IETFer may be unaware about the errata system, and simply
work from the RFC repo, processing the errata is important for at least
two reasons:

1) It relieves the folk that *Are* aware about the system to have to
figure out by themselves which of the reported errata make sense, and
which don't.

2) It serves as a basis for working on bis documents, or even triggering
bis documents.

Yes, having the tool you're referring to may be valuable. However, I
don't think it realy affects the importance of processing the errata.

Thanks,

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 7 Aug 2019 06:26:32 -0700

On Wed, Aug 7, 2019 at 6:22 AM Fernando Gont <[email-masked]> wrote:

> work from the RFC repo,

Not just the non-IETFer. I routinely work from the downloaded RFCs and even
when I go to the Web site, I don't click through to the errata.

processing the errata is important for at least
> 2) It serves as a basis for working on bis documents, or even triggering

Having prepared a number of bis documents, I am of the opinion that this
is not a major part of the effort in doing so.

Like many things, this is a matter of cost and benefit, and if the benefit
were
larger, then it would make the cost more worth bearing.

> don't think it realy affects the importance of processing the errata.

I guess this is just a topic on which we'll have to disagree.

-Ekr

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 7 Aug 2019 15:43:14 +0200

> errata in their "Reported" Queue for over 5 years.

Here is one more data point I forgot to mention:

There is at least one standard by another SDO that references an RFC **together with some unhandled errata report** to describe how they are using and interpreting the RFC.

I can’t remember where I saw this (sorry for not reporting it then), but I remember sitting there with an open mouth for a while.  In particular since the errata report seemed wrong at the time to me.  But I also thought we can’t blame the SDO; errata reports are presented in a quasi-official way by the RFC editor even before they are verified.

TL;DR: Unhandled errata reports do hurt.

Grüße, Carsten

---

From: Fernando Gont <[email-masked]>
Date: Wed, 7 Aug 2019 16:44:17 +0300

On 7/8/19 16:26, Eric Rescorla wrote:
> 
> 
> On Wed, Aug 7, 2019 at 6:22 AM Fernando Gont <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
>     > <[email-masked] <mailto:[email-masked]>
>     <mailto:[email-masked]
>     <mailto:[email-masked]>>> wrote:
>     >
[...]
> when I go to the Web site, I don't click through to the errata.

I routinely work from downloaded RFCs. However, when approaching a new
spec, it is nice to be able to check which errors have been reported and
verified, as opposed to having to go through them myself, and figure out
if they make sense or not -- particularly if I'm new to the spec.

> is not a major part of the effort in doing so.

Having worked on a number of this documents, a starting point for me has
been to address verified/hfdu errata, among other things.

Also, if you happen to have a document with tons of verified errata,
that might be an indication that a bis is warranted.

> Like many things, this is a matter of cost and benefit, and if the
> benefit were

I agree with you that the tool could be valuable and useful.

That said, if we have this:
<https://www.ietf.org/blog/iesg-processing-rfc-errata-ietf-stream/> ,
does it make sense for ADs to discretionally decide whether they want to
process errata?

If one is going to have long queues of unprocessed errata, and accept
that happily, then shut down the errata system, or don't pretend that
filled errata have a "status" (make them all "reported"). That's my
point.  (fwiw, no, i don't think that's the way to go).

Thanks,

---

From: Fernando Gont <[email-masked]>
Date: Wed, 7 Aug 2019 16:55:20 +0300

+1

(Thanks for the data, btw!)

---

From: Warren Kumari <[email-masked]>
Date: Wed, 7 Aug 2019 10:35:40 -0400

On Wed, Aug 7, 2019 at 7:32 AM Kathleen Moriarty
<[email-masked]> wrote:
>
>
>
> On Tue, Aug 6, 2019 at 4:51 PM Michael Richardson <[email-masked]> wrote:
>>
>>
> I agree with this.  Errata needs to be delegated further to improve processing.  If all of the ones that existed where there is an open WG could be handled by the chairs, they would be more familiar with each errata, who the experts were that could approve them, and may have the history having been chairs for a long period of time.

Counterpoint -- the relevant AD(s) can simply poke the chairs and / or
working group and ask them what should happen with the Errata.
As an example: https://mailarchive.ietf.org/arch/msg/dnsop/PFPRwLBE02SniDqaV6VaZ-Z4Uiw
this got some good discussion and feedback.

As another example:
https://mailarchive.ietf.org/arch/msg/opsec/jHa-VrrZzVkdgcPqL86lObiT-u8
This one was sent to the relevant WG (OpSec), and the chairs
responded. The actual "work" was opening the RFC, hitting "Command-F",
typing in a word or two, and marking the Errata as "Hold for Document
Update"[0].

Errata really are important (IMO!), and need to be treated judiciously
- from "RFC Editor Proposal for Handling RFC Errata" (
https://www.rfc-editor.org/materials/draft-rfc-editor-errata-process-02.txt
)
"We note that allowing technical errata is a slippery slope: there may
   be a temptation to use errata to "fix" protocol design errors, rather
   than publishing new RFCs that update the erroneous documents.  In
   general, an erratum is intended to report an error in a document,
   rather than an error in the design of the protocol or other entity
   defined in the document, but this distinction may be too imprecise to
   avoid hard choices.  For the IETF stream, these choices should be
   made by the IESG, and are discussed in their proposed guidelines on
   errata processing [IESG-Err-Proc]."

It can be *really* tempting to fix something obviously wrong as (or
outdated!) in a protocol design by simply filling an Errata - I
witnessed this in Montreal where a working group could have saved a
significant amount of work, and also made a protocol / the world
better by simply changing a "SHOULD do X" to "SHOULD NOT do X". The
chairs and (majority of the) WG were ready to do this through an
errata, but it would be a mistake -- the consensus when the document
was written was clearly "SHOULD do X" (there is even an explanation in
the next sentence), and as tempting as it is, using the Errata process
to "fix" this would violate our consensus process and weaken the RFC
series.
As the AD I have less skin in this particular game / no dog in the
fight[1], and so I'm (hopefully!) better able to avoid the temptation.

I agree that processing Errata can be annoying, and can require much
research, archeology, trying to figure out history, etc --  I keep
snippets so I can track what I've been spending my time on and there
are 101 instances of 'Errata' in my snippets - if each instance took
10 minutes (seems like a reasonable average, including context switch)
that's ~16 hours.

Errata notices get sent to (I believe) the WG, the authors and the AD
-- I'm quite certain that no AD would object to the working groups
(and authors) looking at the Errata notices as they arrive, discussing
it, writing up a summary / proposed resolution and sending it to the
AD to do the annoying administrivia bits of clicking the buttons.

W
[0]: "The erratum is not a necessary update to the RFC. However, any
future update of the document might consider this erratum, and
determine whether it is correct and merits including in the update."
(https://www.ietf.org/blog/iesg-processing-rfc-errata-ietf-stream/)

[1]: Sorry for the idioms -
https://en.wikipedia.org/wiki/Skin_in_the_game_(phrase) ,
https://www.usingenglish.com/reference/idioms/no+dog+in+this+fight.html

>
> Some ADs simply don't think errata is useful.  I handled some, but worked with ADs who didn't care about them.  As a result, Security has the second highest score, leaving Ben and Roman with a bit too much to deal with at this point.
>
> If there's no WG that remains or never was one, it would be nice if those could be delegated too.  The usual process is for editors of the draft to review the errata and respond on list as to their thoughts.  Then the AD is able to press the buttons.  Maybe someone else could monitor the list conversations and update the text in the errata filing accordingly with a link to said email confirmations and press the buttons?
>
> Best regards,
> Kathleen
>>
>>

--
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: Benjamin Kaduk <[email-masked]>
Date: Wed, 7 Aug 2019 09:52:41 -0500

> tools/datatracker sites will show inline errata?

My understanding is that the code is done and waiting to be deployed.

Alissa's note at
https://mailarchive.ietf.org/arch/msg/ietf/QDuj5ItTcY5nbVQVp7FMRjZiSZY may
be the most authoritative thing I can point to right now.

-Ben

---

From: Benjamin Kaduk <[email-masked]>
Date: Wed, 7 Aug 2019 09:58:53 -0500

> me that it would be due dilligence to at least process the *technical*

You keep emphasizing *technical* errata as if there is a crisp line between
technical and editorial.  But part of the work of processing an errata
report is to determine whether or not it is, in fact, technical.  I've lost
count of the number of reports I've seen where the submitter thought they
were technical but the WG/authors disagreed.

> to which that pat of the mission can be accomplished.

"maintenance" and "RFCs are immutable" seem somewhat at odds with each
other.

> why", but normally the problematic text is quoted, and a solution proposed.

I'm on the security team for a couple of pieces of open-source software.
Serious vulnerabilities are processed ASAP, yes.  But not all
vulnerabilities are serious, and not all reports are vulnerabilities.  Once
we have looked far enough to know that something is not a vulnerability or
not serious, it sometimes makes sense to remove the "security issue"
prioritization and handle it as part of the normal development workflow.

> section. Complete AD processing my happen as a last resort...

It remains unclear to me (and this is not directed at you) what would make
authors/WG more motivated to look at something if I forward a mail to them
versus just receiving it directly from the errata submission tool.  (The
latter being what already happens today.)  I already have more documents
blocking on me doing something than I can keep track of in my head (the
datatracker helps, a fair amount); attempting to add errata to that is not
something that I personally, at least, can manage.

-Ben

---

From: Robert Sparks <[email-masked]>
Date: Wed, 7 Aug 2019 10:30:21 -0500

> be the most authoritative thing I can point to right now.

This is currently in the rfc-editor's hands (as their plan is to deploy 
it on the rfc-editor website).

RjS

---

From: Fernando Gont <[email-masked]>
Date: Wed, 7 Aug 2019 19:09:31 +0300

> were technical but the WG/authors disagreed.

I'd expect that *all* errata gets processed. The reason I emphasize the
technical one is because that's supposed to be way more important and
relevant.

I never meant that processing errata is easy, quick, enjoyable, or
doesn't have false positives -- the same may also having for
vulnerability reports (continuing with the analogy).

What I do believe is that the system is important, processing the errata
is important, and a procedure should be devised such as errata are
handled in a timelier manner. All, or at the very least the ones flagged
as technical (even if they result to be false positives).

> "maintenance" and "RFCs are immutable" seem somewhat at odds with each

Not sure I follow. I'm all for updating or revising RFCs where needed.
In fact, most of the work I've done myself is exactly that.

In fact, I'm all for processing the errata under the assumption that
they may have flaws that require attention, sometimes in the form of a
verified errata, while others in the form of a bis document.

> prioritization and handle it as part of the normal development workflow.

I'd expect that when somebody reports a vulnerability, you handle the
report, and if it turns out that it actually isn't, you respond to the
submitter. -- that's handling the report (whether you actually patch
asap or not).

Errata in the "reported" state were simply not processed. Nobody knows
(neither the submitter, nor the community as a whole) if they make sense
or not.

> versus just receiving it directly from the errata submission tool.  

Agreed. THe question is how to respond when they don't respond. At the
end of the day, the document they authored stopped being *their*
document once the relevant wg adopted it.

So, it would be nice if they responded (I'd expect them to), but if they
don't, it's an IETF document that needs to be takn care of, anyway.

> (The

Could directorates help? Could something like a "AD secretary" help? An
Errata team for each area? Anything else?

FWIW, my point and intent is certainly not to overload ADs, but to find
a way in which somehow the work gets done.

Thanks,

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 7 Aug 2019 10:48:30 -0700

> it on the rfc-editor website).

I certainly agree it would be good to deploy it on the rfc-editor web site,
but lots of people go to datatracker and tools, so that shouldn't hold up
deploying it on the IETF sites.

-Ekr

---

From: Richard Barnes <[email-masked]>
Date: Wed, 7 Aug 2019 14:38:02 -0400

On Wed, Aug 7, 2019 at 1:50 PM Eric Rescorla <[email-masked]> wrote:

> I certainly agree it would be good to deploy it on the rfc-editor web
> site, but lots of people go to datatracker and tools, so that shouldn't
> hold up deploying it on the IETF sites.
>

Deploying it on rfc-editor.org seems fine, but of very little value
compared to datatracker and tools. Google, for example, seems to stick to
the following ranking when you search "RFC XXXX":

1. tools.ietf.org
2. datatracker.ietf.org
3. rfc-editor.org

(And indeed, sometimes puts other RFCs before rfc-editor.org, e.g.,
https://www.google.com/search?q=rfc+5280)

As an added benefit, datatracker and tools are controlled by the IESG, so
the changes can be deployed there without needing to trouble the RFC editor.

--Richard

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 8 Aug 2019 08:57:16 +1200

Richard,
On 08-Aug-19 06:38, Richard Barnes wrote:
> 
> 
> On Wed, Aug 7, 2019 at 1:50 PM Eric Rescorla <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
> 
>     On Wed, Aug 7, 2019 at 8:30 AM Robert Sparks <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
>     I certainly agree it would be good to deploy it on the rfc-editor web site, but lots of people go to datatracker and tools, so that shouldn't hold up deploying it on the IETF sites.
> 
> 
> Deploying it on rfc-editor.org <http://rfc-editor.org> seems fine, but of very little value compared to datatracker and tools. 

Since you've mentioned the elephant in the room...

> Google, for example, seems to stick to the following ranking when you search "RFC XXXX":
> 
> 1. tools.ietf.org <http://tools.ietf.org>
> 2. datatracker.ietf.org <http://datatracker.ietf.org>
> 3. rfc-editor.org <http://rfc-editor.org>
> 
> (And indeed, sometimes puts other RFCs before rfc-editor.org <http://rfc-editor.org>, e.g.., https://www.google.com/search?q=rfc+5280)

I don't think that the current de fact SEO of one search engine should determine our policy. Also, if the user comes in via the DOI, they are sent to rfc-editor.org.

> As an added benefit, datatracker and tools are controlled by the IESG, so the changes can be deployed there without needing to trouble the RFC editor.

But the errata mechanism is controlled by the RFC Editor, and applies to all streams. So it's natural that errata display in this new (and very welcome) form should originate at the RFC Editor site. I'd certainly be delighted to see the tracker and tools sites picking it up, but they are secondary sites as far as RFCs go, by definition.

As far as I can see, things are currently quite well integrated - if you click on the 'HTML' button in the RFC Editor's landing page for an RFC, you get a mirror of the tools site htmlized version. Presumably with the new format it will be the converse - the htmlized version will originate at the RFC Editor site.

(How the search engines will manage with the new format is also an interesting question.)

   Brian

---

From: Richard Barnes <[email-masked]>
Date: Wed, 7 Aug 2019 17:21:51 -0400

On Wed, Aug 7, 2019 at 4:57 PM Brian E Carpenter <
[email-masked]> wrote:

> >     I certainly agree it would be good to deploy it on the rfc-editor
> web site, but lots of people go to datatracker and tools, so that shouldn't
> > Deploying it on rfc-editor.org <http://rfc-editor.org> seems fine, but
> of very little value compared to datatracker and tools.
>
> > (And indeed, sometimes puts other RFCs before rfc-editor.org <
> http://rfc-editor.org>, e.g.., https://www.google.com/search?q=rfc+5280)
>
> I don't think that the current de fact SEO of one search engine should
> determine our policy. Also, if the user comes in via the DOI, they are sent
> to rfc-editor.org.
>

Wouldn't it be lovely if we had some metrics from these pages so we could
know which ones actually get used?

> > As an added benefit, datatracker and tools are controlled by the IESG,
> so the changes can be deployed there without needing to trouble the RFC
> editor.
>
> But the errata mechanism is controlled by the RFC Editor, and applies to
> all streams. So it's natural that errata display in this new (and very
> welcome) form should originate at the RFC Editor site. I'd certainly be
> delighted to see the tracker and tools sites picking it up, but they are
> secondary sites as far as RFCs go, by definition.
>

Just because the RFC Editor holds the data doesn't mean that other people
shouldn't display them differently.  Indeed, datatracker and tools
*already* display RFCs differently from the RFC Editor site.  Adding in
errata would just be a further tweak in how the underlying data are
displayed.

--Richard

>
> As far as I can see, things are currently quite well integrated - if you
> click on the 'HTML' button in the RFC Editor's landing page for an RFC, you
> get a mirror of the tools site htmlized version. Presumably with the new
> format it will be the converse - the htmlized version will originate at the
> RFC Editor site.
>

---

From: Clint Chaplin <[email-masked]>
Date: Wed, 7 Aug 2019 15:17:14 -0700

On Wed, Aug 7, 2019 at 11:39 AM Richard Barnes <[email-masked]> wrote:

> (And indeed, sometimes puts other RFCs before rfc-editor.org, e.g..,

Google, FOR YOU, ranks the search results as above.

There is no such thing anymore as a generic de novo naive search result
from Google.  What you get will very probably be different than what I get.

-- 
Clint Chaplin
Senior Principal Standards Engineer
Samsung Research America

---

From: Richard Barnes <[email-masked]>
Date: Wed, 7 Aug 2019 18:24:00 -0400

> from Google.  What you get will very probably be different than what I get.

Do you in fact get results different from the above?

--Richard

---

From: Clint Chaplin <[email-masked]>
Date: Wed, 7 Aug 2019 16:05:18 -0700

RFC 1510, for me, gave me the following:

RFC 1510 - The Kerberos Network Authentication Service ... - IETF Tools
https://tools.ietf.org/html/rfc1510
<https://tools.ietf.org/html/rfc1510>
<https://webcache.googleusercontent.com/search?q=cache:l6d5XaCxlRAJ:https://tools.ietf.org/html/rfc1510+&cd=1&hl=en&ct=clnk&gl=us>
<https://tools.ietf.org/html/rfc1510>
by C Neuman - ‎1993 - ‎Cited by 626
<https://scholar.google.com/scholar?rlz=1C1GCEU_enUS834US834&uact=5&um=1&ie=UTF-8&lr&cites=8430010103626316686>
-
‎Related articles
<https://scholar.google.com/scholar?rlz=1C1GCEU_enUS834US834&uact=5&um=1&ie=UTF-8&lr&q=related:jruZb4Zp_XRtyM:scholar.google.com/>
RFC 1510 Kerberos September 1993 Background The Kerberos model is based in
part on Needham and Schroeder's trusted third-party authentication
protocol ...
RFC 4120 - The Kerberos Network Authentication Service (V5)
https://tools.ietf.org/html/rfc4120
<https://tools.ietf.org/html/rfc4120>
<https://webcache.googleusercontent.com/search?q=cache:_vTctnD4oO0J:https://tools.ietf.org/html/rfc4120+&cd=2&hl=en&ct=clnk&gl=us>
<https://tools.ietf.org/html/rfc4120>
by C Neuman - ‎2005 - ‎Cited by 626
<https://scholar.google.com/scholar?rlz=1C1GCEU_enUS834US834&uact=5&um=1&ie=UTF-8&lr&cites=8430010103626316686>
-
‎Related articles
<https://scholar.google.com/scholar?rlz=1C1GCEU_enUS834US834&uact=5&um=1&ie=UTF-8&lr&q=related:jruZb4Zp_XRtyM:scholar.google.com/>
Abstract This document provides an overview and specification of Version 5
of the Kerberos protocol, and it obsoletes RFC 1510 to clarify aspects of
the protocol ...
RFC 1510 - The Kerberos Network Authentication ... - IETF Datatracker
https://datatracker.ietf.org/doc/rfc1510
<https://datatracker.ietf.org/doc/rfc1510>
<https://webcache.googleusercontent.com/search?q=cache:4WA0Doe40uYJ:https://datatracker.ietf.org/doc/rfc1510+&cd=3&hl=en&ct=clnk&gl=us>
<https://datatracker.ietf.org/doc/rfc1510>
The Kerberos Network Authentication Service (V5) (RFC 1510, September 1993;
obsoleted by RFC 4120, RFC 6649)
Information on RFC 1510 » RFC Editor
https://www.rfc-editor.org/info/rfc1510
<https://www.rfc-editor.org/info/rfc1510>
<https://webcache.googleusercontent.com/search?q=cache:BPjwqRwYaigJ:https://www.rfc-editor.org/info/rfc1510+&cd=4&hl=en&ct=clnk&gl=us>
<https://www.rfc-editor.org/info/rfc1510>
Canonical URL: https://www.rfc-editor.org/rfc/rfc1510.txt; File formats:
icon for text file icon for PDF icon for HTML; Status: HISTORIC (changed
from PROPOSED ...
[PDF]Notes on RFC 4120 superseding RFC 1510
https://lists.oasis-open.org/archives/wss/200508/pdf00000.pdf
<https://lists.oasis-open.org/archives/wss/200508/pdf00000.pdf>
<https://webcache.googleusercontent.com/search?q=cache:C7lmtMmxMI0J:https://lists.oasis-open.org/archives/wss/200508/pdf00000.pdf+&cd=5&hl=en&ct=clnk&gl=us>
<https://lists.oasis-open.org/archives/wss/200508/pdf00000.pdf>
existed in RFC 1510. It should be noted that both RFC's give an overview
and specification of Version 5 of the protocol for the Kerberos network
authentication.

Note the two entries from tools.ietf.org before the datatracker entry.

---

From: Eric Rescorla <[email-masked]>
Date: Wed, 7 Aug 2019 22:34:34 -0700

> determine our policy. Also, if the user comes in via the DOI, they are sent

I certainly agree that it would be good for rfc-editor.org to host
erratized RFCs.

> secondary sites as far as RFCs go, by definition.

Well, this is actually an interesting philosophical question: given that
the RFCs are immutable data, it's not clear to me that any particular Web
site has primacy for their ongoing display. However, that doesn't seem like
something we need to resolve now.

> get a mirror of the tools site htmlized version.

This point seems like the relevant precedent: the errata are displayed on
the HTML version and it's totally reasonable for the tools to just apply
them and show it, even if the RFC Editor has not yet done so.

Presumably with the new format it will be the converse - the htmlized
> version will originate at the RFC Editor site.
>

My understanding is that errata display for the v3 format is a different
project, so this seems like a separate question.

-Ekr

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 8 Aug 2019 18:26:48 +0200

Hi,

> the changes can be deployed there without needing to trouble the RFC editor.

Since 2004, tools.ietf.org has been running on hardware donated by me or
donated to me by others for the purpose or running tools.ietf.org.  The
same goes for the hosting.

I've written all the software in my off time over the years, as a pro bono
activity.  There's been basically no new development over the last 3 years,
though; I've simply not had free time to put into that.

During the last few years, the IETF has provided a little bit of money for
some maintenance backup (a different person standing by when I'm travelling
or offline, and also doing routine server maintenance tasks, a few hours a
month).

The IESG has no say on what I do and don't on tools.ietf.org, except that
over the years I've always tried to be responsive to ideas from the
community -- after all, everything that is on tools.ietf.org I've been
doing _for_ the community.

Regards,

	Henrik

---

From: "Adrian Farrel" <[email-masked]>
Date: Thu, 8 Aug 2019 17:40:48 +0100

Just for fun…

Deploying it on rfc-editor.org <http://rfc-editor.org>  seems fine, but of very little value compared to datatracker and tools. Google, for example, seems to stick to the following ranking when you search "RFC XXXX":

(And indeed, sometimes puts other RFCs before rfc-editor.org <http://rfc-editor.org> , e.g.., https://www.google.com/search?q=rfc+5280)

There is no such thing anymore as a generic de novo naive search result from Google.  What you get will very probably be different than what I get.

> Do you in fact get results different from the above?

I searched for “RFC5511”.

I got…

https://tools.ietf.org/html/rfc5511

https://www.rfc-editor.org/info/rfc5511

https://www.wikidata.org/wiki/Q47469159

https://www.arkko.com/tools/allstats/citations-rfc5511.html

https://www.tech-invite.com/y65/tinv-ietf-rfc-6510.html

https://standards.globalspec.com/std/1512671/ietf-rfc-5511

https://www.bortzmeyer.org/5511.html

http://citeseerx.ist.psu.edu/viewdoc/summary;jsessionid=CA1CB0A48C4222655367FC6CCF8EF789?doi=10.1.1.182.6713

…and then I got bored.

No datatracker there, I think.

Mind you, searching with a space (“RFC 5511”) gives different results…

https://tools.ietf.org/html/rfc5511

https://datatracker.ietf.org/doc/rfc5511/

https://www.wikidata.org/wiki/Q47469159

https://dephoto.biz/Search/Search.aspx?Path=Rugby%5CEaling%20RFC%5C2018%5CNovember%20Minis%20Festival%5CSun%2018th%20U8s%20and%20U10s%5CU12%5CPitch%20A2

https://www.arkko.com/tools/allstats/citations-rfc5511.html

https://books.google.co.uk/books?id=nMZnyp_zW8AC <https://books.google.co.uk/books?id=nMZnyp_zW8AC&pg=PA101&lpg=PA101&dq=RFC+5511&source=bl&ots=KbTq5x32_h&sig=ACfU3U3UWfgWdpZBciVKo3jSDbcliQTV8Q&hl=en&sa=X&ved=2ahUKEwi_49HM2vPjAhXbREEAHcY2DWQQ6AEwBXoECAkQAQ> &pg=PA101&lpg=PA101&dq=RFC+5511&source=bl&ots=KbTq5x32_h&sig=ACfU3U3UWfgWdpZBciVKo3jSDbcliQTV8Q&hl=en&sa=X&ved=2ahUKEwi_49HM2vPjAhXbREEAHcY2DWQQ6AEwBXoECAkQAQ

…No RFC Editor page there.

And this proves something incontrovertible and valuably pertinent to this debate. I just don’t know what it is.

Adrian

---

From: Sean Turner <[email-masked]>
Date: Thu, 8 Aug 2019 15:25:44 -0400

Kind of cherry picking here.

> On Aug 7, 2019, at 12:09, Fernando Gont <[email-masked]> wrote:
> 

just my two cents:

I remember when I thought processing errata was going to be oh so easy.  Man was I dead wrong.

As noted somewhere in this thread somebody read all of the RFCs (at the time) and submitted errata.   It was a _LOT_ of errata.  After trying to process them, there ended being even more because many of the errata were were multipart - like 15 parts.  One part could be verified, but others HFDU (hold for document update) or rejected.  To address this Tim Polk and I had to break most apart, which in turn made our “numbers” ballon*; but really the RFC editor did the separation for us.  Along the way, Tim and I had discussions with submitters about whether the errata fell into bucket 1, 2, 4, 5, 6, 7, or 8 from https://www.ietf.org/blog/iesg-processing-rfc-errata-ietf-stream/.

All of this was made a littler hard when the errata were against “legacy" RFCs because tracking down the authors for input was downright painful (Google was my friend here) and sometimes impossible.  When we could not find the authors or the WG chairs (sometimes of long closed WGs) were unresponsive, Tim and I would have to roll up our sleeves and do a deep dive.  This involved talking with members of the community, talking it out together, and running the proposed solution by community member(s).  We also ran things by the WGs, but frankly what we saw then is what I see now - there’s just not a lot of interest in processing errata.

I will note, AD-sponsored RFCs were even more painful, because, at least to me, it seemed like many of those authors got their RFC and were done with it/us.

It took many months to whittle the numbers down to 1.  We never did manage to get to zero.  Let’s say HFDU was really our friend here.

To get through this process Tim and I developed the following email which we shared with the chairs and the authors and wg chairs:

  The errata process is beginning to be used by more and more people.
  To ensure we (the ADs) properly process them, we need help from
  document authors and WG chairs.  We'd like your help with the
  following errata [insert #s].  We need your help with the following:

  #1) Ensure the reported errata is against the correct RFC.  If not,
  inform us.  If it is, continue with steps 2 through 5.

  #2) Determine whether the reported errata type is correct.  There are
  two types editorial and technical.

  Editorial errata are well editorial (i.e., commas, spelling, typos, etc. or
  subjective/stylistic edits) that would make the document clearer and easier
  to read, but do not really affect the utility of the specification.

  Technical errata are errors or inconsistencies that could introduce
  implementation or deployment problems.  Any errata that changes bits on the
  wire or processing algorithms is generally technical!

  #3) Use the criteria from
  http://www.ietf.org/iesg/statement/errata-processing.html to determine
  the correct next state: Approved, Hold For Document Update, or Rejected.
  Note that most editorial errata should be held or rejected.

  #4) If approved, determine whether the errata needs to be edited.  Sometimes
  the submitter has identified a real problem but the proposed edit does not
  entirely correct the problem.  If the proposed edit requires tweaking,
  please propose a correction!

  #5) Inform us whether you agree with the reported errata type, the
  next state, and whether any edits should be applied to the errata itself.

  As noted in the IESG statement, these are guidelines and not immutable
  rules. When in doubt, please rely on your common sense and good judgment! We
  greatly appreciate your support.

Hope this helps,

spt

* I remember Sandy giving reports about errata during IETF week’s Sunday IESG sessions and cringing at the SEC “tower” in the bar chart.

---

From: Sean Turner <[email-masked]>
Date: Thu, 8 Aug 2019 15:25:53 -0400

> On Aug 7, 2019, at 13:48, Eric Rescorla <[email-masked]> wrote:
> 
> 
> 

Great to see this is coming!  While I do sometimes go to the rfc-editor website (expired drafts) I mostly work from the  datatracker so having it there would be more helpful for me at least.

spt

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 9 Aug 2019 08:42:23 +1200

Yes, the whole "tools" effort started as a volunteer team (set up when
Harald was IETF Chair, I think) and that fact that tools.ietf.org has
become the site favoured by the major search engines** is a big win
for collaboration rather than top-down control.

** I checked Google, DuckDuckGo, Bing and Yahoo for rfc1234. tools.ietf.org
came first each time.

Regards
   Brian Carpenter

---

From: Randy Bush <[email-masked]>
Date: Thu, 08 Aug 2019 15:35:20 -0700

> ** I checked Google, DuckDuckGo, Bing and Yahoo for rfc1234. tools.ietf.org

i checked my fingers, and tools is where thry go

>> over the years I've always tried to be responsive

amazingly so, tyvm

randy

---

From: Henrik Levkowetz <[email-masked]>
Date: Fri, 9 Aug 2019 14:03:13 +0200

> i checked my fingers, and tools is where thry go

And to be clear, I'll be perfectly happy to work at deploying this on tools,
but I'm not assuming it's going to happen without putting a number of hours
into integrating the code from the inline errata display project for use
there.  I expect to look at that after I'm back from vacation, and can find
some free time.

>>> over the years I've always tried to be responsive

Thank you :-)

Best,

	Henrik

---

From: Michael Richardson <[email-masked]>
Date: Fri, 09 Aug 2019 11:27:44 -0400

> And to be clear, I'll be perfectly happy to work at deploying this on
    > tools, but I'm not assuming it's going to happen without putting a
    > number of hours into integrating the code from the inline errata
    > display project for use
    > there.  I expect to look at that after I'm back from vacation, and can
    > find  some free time.

From what I can see the rfc-editor.org HTML-ized version seems to be
almost identical to that from tools.  (On my screen the font is slightly
smaller, but that might be a local thing)

What if we put a redirect from tools.ietf.org/html/rfcXXXX to
rfc-editor.org/rfc/rfcXXXX.html?

The DT should probably point RFC documents to rfc-editor.org, rather than
tools.ietf.org.   For Internet-Drafts, it seems like we ought to bring the
HTML rendering into DT.  A reason why the tools.ietf.org copy of the
RFC (and Internet draft) gets a higher SEO is because it's the one that
is easiest to link to a particular section.

---

From: Henrik Levkowetz <[email-masked]>
Date: Sat, 10 Aug 2019 00:39:48 +0200

Hi Michael,

> smaller, but that might be a local thing)

That's because they are using the Python library I extracted from the
original htmlization code: https://pypi.org/project/rfc2html/

> rfc-editor.org/rfc/rfcXXXX.html?

That would remove my ability to continue to tweak the tools copy, which I
still do.  A new feature request popped up as recently as during IETF 105,
and I expect to address it soonish.  So no, I'm not up for that.

> HTML rendering into DT.

It's already there, but has a somewhat different look, e.g.:
  https://datatracker.ietf.org/doc/html/draft-lemon-homenet-babel-security-latest
(That copy also uses the htmlization module mentioned above.)

There's also a work item to make it look closer to what's on tools:
  https://svn.tools.ietf.org/svn/tools/ietfdb/trunk/PLAN
(Look for "* Polish the htmlization pages ...")

> A reason why the tools.ietf.org copy of the

Ack, but there are others too.  And efforts to spread the goodness.

	Henrik