Subject: How to get feedback on published RFCs [resending as plaintext]

From: Yaron Sheffer <[email-masked]>
Date: Tue, 19 Jul 2016 13:54:28 +0200

Once an RFC is published, there is essentially no way for readers to 
provide feedback: what works, what are the implementation pitfalls, how 
does the document relate to other technologies or even to other RFCs.

We IETF insiders usually know what is the relevant working group, and 
can take our feedback there. Non-insiders though don't have any contact 
point, and so will most likely keep their feedback to themselves. These 
non-IETFers are the target audience of our documents! Unfortunately, our 
so-called "Requests for Comments" are anything but an invitation to 
submit comments.

There is a number of tools now that allow "web annotations" (i.e., 
comments) on various published documents. I submitted a draft [1] 
recently that proposes to enable annotations on the "tools" version of 
our RFCs. Technically, this is a trivial change. From a process point of 
view it is more complicated and merits discussion on this list. Sec. 6 
of the draft allows you to see for yourself what such annotations would 
look like.

I am here in Berlin if people prefer to talk it over in person. 
Otherwise, please reply on this list.

Thanks,
     Yaron

[1] https://tools.ietf.org/html/draft-sheffer-ietf-rfc-annotations-00

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Tue, 19 Jul 2016 13:16:50 +0000

> -----Original Message-----
> From: ietf [mailto:[email-masked]] On Behalf Of Yaron Sheffer
> Sent: Tuesday, July 19, 2016 7:54 AM
> To: IETF
> Subject: How to get feedback on published RFCs [resending as plaintext]
> 
> Once an RFC is published, there is essentially no way for readers to provide
> feedback: what works, what are the implementation pitfalls, how does the
> document relate to other technologies or even to other RFCs.
> 
> We IETF insiders usually know what is the relevant working group, and can
> take our feedback there. Non-insiders though don't have any contact point,
> and so will most likely keep their feedback to themselves. These non-IETFers
> are the target audience of our documents! Unfortunately, our so-called
> "Requests for Comments" are anything but an invitation to submit
> comments.
> 

A simple solution would be to include a pointer to the relevant working group as a header or note to the RFC. There could be a standard "How to comment" section. No need for additional tools or process.

Mke

---

From: Miles Fidelman <[email-masked]>
Date: Tue, 19 Jul 2016 10:13:52 -0400

On 7/19/16 7:54 AM, Yaron Sheffer wrote:

> provide feedback: what works, what are the implementation pitfalls, 
> how does the document relate to other technologies or even to other RFCs.
>
> can take our feedback there. Non-insiders though don't have any 
> contact point, and so will most likely keep their feedback to 
> themselves. These non-IETFers are the target audience of our 
> documents! Unfortunately, our so-called "Requests for Comments" are 
> anything but an invitation to submit comments.
>
> our RFCs. Technically, this is a trivial change. From a process point 
> of view it is more complicated and merits discussion on this list. 
> Sec. 6 of the draft allows you to see for yourself what such 
> annotations would look like.
>

Maybe a silly observation, but we might look at the HTML "Living 
Standard" (https://html.spec.whatwg.org/multipage/) - as maintained by 
WHATWG.   Their process seems to have the benefit of working reasonably 
well over time.

Miles Fidelman

-- 
In theory, there is no difference between theory and practice.
In practice, there is.  .... Yogi Berra

---

From: Christer Holmberg <[email-masked]>
Date: Tue, 19 Jul 2016 14:28:55 +0000

Hi,

A couple of initial comments:

Q1:

In section3, the text says:

   "-  Providing feedback on correctness and pointing out errors.  This
      is a much easier process than submitting errata, and as such would
      likely yield a larger number of corrections."

I assume that, if something is to be fixed, an errata will eventually have to be created? I.e. the annotation will not be a formal correction.

Q2:

Keep in mind that some text in an RFC may not be valid anymore, if:

1)	It has already been changed in an errata; or
2)	It has been updated in another RFC ("this RFC updates section X of RFC Y")

Now, anyone who is about to give comments should obviously make sure whether the affected parts have been updated. But, assuming I want to comment on text that exists in an errata, how does that work?

Regards,

Christer

-----Original Message-----
From: ietf [mailto:[email-masked]] On Behalf Of Yaron Sheffer
Sent: 19 July 2016 14:54
To: IETF <[email-masked]>
Subject: How to get feedback on published RFCs [resending as plaintext]

Once an RFC is published, there is essentially no way for readers to provide feedback: what works, what are the implementation pitfalls, how does the document relate to other technologies or even to other RFCs.

We IETF insiders usually know what is the relevant working group, and can take our feedback there. Non-insiders though don't have any contact point, and so will most likely keep their feedback to themselves. These non-IETFers are the target audience of our documents! Unfortunately, our so-called "Requests for Comments" are anything but an invitation to submit comments.

There is a number of tools now that allow "web annotations" (i.e.,
comments) on various published documents. I submitted a draft [1] recently that proposes to enable annotations on the "tools" version of our RFCs. Technically, this is a trivial change. From a process point of view it is more complicated and merits discussion on this list. Sec. 6 of the draft allows you to see for yourself what such annotations would look like.

Thanks,
     Yaron

---

From: Ted Lemon <[email-masked]>
Date: Tue, 19 Jul 2016 16:30:21 +0200

Doing nothing is always an option.   These kind folks are proposing doing
something, though, rather than nothing.   This makes sense, because the
working group might no longer exist, and email archives are useless for
getting a summary of perhaps a few years of commentary that may have
occurred.

Do you have a specific objection to doing something, or do you just not
want to have to be the one to do it?   :)

On Tue, Jul 19, 2016 at 3:16 PM, MH Michael Hammer (5304) <[email-masked]>
wrote:

> provide
> A simple solution would be to include a pointer to the relevant working
> group as a header or note to the RFC. There could be a standard "How to
> comment" section. No need for additional tools or process.
>
> Mke
>

---

From: Yaron Sheffer <[email-masked]>
Date: Tue, 19 Jul 2016 16:34:14 +0200

> A simple solution would be to include a pointer to the relevant working group as a header or note to the RFC. There could be a standard "How to comment" section. No need for additional tools or pro…

Working groups are not forever, so giving a working group name or email 
address is not very useful. Besides, not all comments (e.g. 
implementation experience) are even appropriate to send to the WG list.

Thanks,
	Yaron

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Tue, 19 Jul 2016 14:38:07 +0000

I have no specific objection to doing something. I wasn’t planning on being the one to do anything anyways.

I do have a concern about your comment about getting a summary of perhaps a few years of commentary. The problem with tools that annotate comments against the RFC is that anyone looking at just those comments does not get the complete picture. Even if the working group is closed, pointing people to the group/archive gives someone the opportunity to get a more complete picture. Your comment may also be taken as a reason for keeping mailing lists open for an extended period even after the working group is no longer active. Every once in a while therew ill be a post to the DKIM or SPF lists by someone with a question or comment – even though those groups have been inactive for some time.

Mike

From: Ted Lemon [mailto:[email-masked]]
Sent: Tuesday, July 19, 2016 10:30 AM
To: MH Michael Hammer (5304)
Cc: Yaron Sheffer; IETF
Subject: Re: How to get feedback on published RFCs [resending as plaintext]

Doing nothing is always an option.   These kind folks are proposing doing something, though, rather than nothing.   This makes sense, because the working group might no longer exist, and email archives are useless for getting a summary of perhaps a few years of commentary that may have occurred.

Do you have a specific objection to doing something, or do you just not want to have to be the one to do it?   :)

On Tue, Jul 19, 2016 at 3:16 PM, MH Michael Hammer (5304) <[email-masked]<mailto:[email-masked]>> wrote:

> From: ietf [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Yaron Sheffer

Mke

---

From: Yaron Sheffer <[email-masked]>
Date: Tue, 19 Jul 2016 16:39:27 +0200

> I assume that, if something is to be fixed, an errata will eventually have to be created? I.e. the annotation will not be a formal correction.

I fully agree. Annotations are not curated by anybody. To make a formal 
change, you still need to follow the Errata process.

> Now, anyone who is about to give comments should obviously make sure whether the affected parts have been updated. But, assuming I want to comment on text that exists in an errata, how does that wo…

IMO we should ignore this issue for the initial experiment. Later on, we 
can think of ways to tie errata back into the original document (e.g. 
similar to "revision marks" in Word). Which would make it possible to 
annotate the latest version of the text. But that would be a much more 
extensive and complex change.

Thanks,
	Yaron

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Tue, 19 Jul 2016 14:45:33 +0000

> From: Yaron Sheffer [mailto:[email-masked]]
> Sent: Tuesday, July 19, 2016 10:34 AM
> To: MH Michael Hammer (5304); IETF
> >> feedback: what works, what are the implementation pitfalls, how does
> >> the document relate to other technologies or even to other RFCs.
> >>

We get plenty of inappropriate comments to WG lists already. The world has not come to an end. This may be an argument for having a parallel list that is kept open and which is specifically for implementation/ops issues. I have this great little cartoon over my desk with a picture of a building on fire. The caption is "Worked fine in Dev, ops problem now". - https://sethvargo.com/the-ten-myths-of-devops/worked-in-dev-ops-problem-now-7809f3cf.jpg

Mike

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 20 Jul 2016 19:58:37 +1200

On 20/07/2016 02:13, Miles Fidelman wrote:
>> Once an RFC is published, there is essentially no way for readers to provide feedback: what works, what are the implementation
>> pitfalls, how does the document relate to other technologies or even to other RFCs.
>>
>> We IETF insiders usually know what is the relevant working group, and can take our feedback there. Non-insiders though don't
>> have any contact point, and so will most likely keep their feedback to themselves. These non-IETFers are the target audience
>> of our documents! Unfortunately, our so-called "Requests for Comments" are anything but an invitation to submit comments.
>>
>> There is a number of tools now that allow "web annotations" (i.e., comments) on various published documents. I submitted a
>> draft [1] recently that proposes to enable annotations on the "tools" version of our RFCs. Technically, this is a trivial
>> change. From a process point of view it is more complicated and merits discussion on this list. Sec. 6 of the draft allows you
>> to see for yourself what such annotations would look like.
>>
> 
> Maybe a silly observation, but we might look at the HTML "Living Standard" (https://html.spec.whatwg.org/multipage/) - as
> maintained by WHATWG.   Their process seems to have the benefit of working reasonably well over time.

If that is your impression, I have to wonder whether you've ever tried to
"dialogue" with whatwg on a matter that challenges their received wisdom.
I don't think that is a good model for the IETF to follow.

    Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 20 Jul 2016 20:09:58 +1200

>       likely yield a larger number of corrections."

Really? I've never found submitting an erratum to be difficult, apart from
being annoyed each time by the collective failure to treat the word "errata"
as a plural.

> I assume that, if something is to be fixed, an errata will eventually have to be created? I.e. the annotation will not be a formal correction.

Clearly not - it will need filtering by somebody, and the identity of "somebody"
is a critical issue. With >7000 RFCs, are we going to need a list of >7000 designated
experts? But if nobody filters the annotations for useful stuff, this will just
become a garbage dump.

> 2)	It has been updated in another RFC ("this RFC updates section X of RFC Y")

Right. So actually we need each RFC to be pre-annotated with applicable errata
and updates from other RFCs. That's an interesting little AI project in itself.

(A side issue is that RFCs with status "Historic", "Obsoleted by" and "Unknown"
should probably be "greyed out" because annotating them is largely a waste
of time.)

(Another AI project is carrying forward relevant annotations to a new updating
or obsoleting RFC.)

> Now, anyone who is about to give comments should obviously make sure whether the affected parts have been updated. But, assuming I want to comment on text that exists in an errata, how does that wo…

That would be OK if the the erratum is already added as a annotation.

Rgds
   Brian

---

From: Donald Eastlake <[email-masked]>
Date: Wed, 20 Jul 2016 04:57:14 -0400

It does seem a bit absurd to me that there isn't a specific email
address given in RFCs for questions/comments, which should typically
be a WG email. Currently, the only choices for a random reader would
be the authors' email addresses or [email-masked]. True, WGs don't last
forever, but their mailing lists commonly have a longer lifetime than
the WG itself, questions and comments on earlier RFCs while the WG
that produced them was still active are particularly likely and
useful, and, when the WG email list is "closed", it could actually be
forwarded to a more general email address...

There are advantages and disadvantages to email and "annotation". I
would particularly note that both need spam filtration.

Thanks,
Donald
===============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 155 Beaver Street, Milford, MA 01757 USA
 [email-masked]

On Tue, Jul 19, 2016 at 10:34 AM, Yaron Sheffer <[email-masked]> wrote:
> address is not very useful. Besides, not all comments (e.g. implementation
> experience) are even appropriate to send to the WG list.
>
> Thanks,
>         Yaron
>

---

From: Michael Richardson <[email-masked]>
Date: Wed, 20 Jul 2016 12:06:00 -0400

I have read the document.
I strongly support the idea and of running the experiment.

I am concerned that one year may not be enough time to socialize this and get
useful feedback.

I propose that there be an option to renew for an additional year.
It can clearly also be used for Internet-Drafts (as some of us have done to
your draft...) ... I would like to suggest that the datatracker page include
a link to the stream page in section 5.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: [email-masked] (Martin Rex)
Date: Wed, 27 Jul 2016 19:49:50 +0200 (CEST)

Yaron Sheffer wrote:
> point, and so will most likely keep their feedback to themselves.

For a few years, the front pages of RFCs have been carrying a statement
like this on their front pages:

from RFC7901 (June 2016):

   Information about the current status of this document, any errata,
   and how to provide feedback on it may be obtained at
   http://www.rfc-editor.org/info/rfc7901.

from RFC5746 (Feb 2010):

   Information about the current status of this document, any errata,
   and how to provide feedback on it may be obtained at
   http://www.rfc-editor.org/info/rfc5746.

and the info on the RFC Editor website says

  Discuss this RFC: Send questions or comments to <....>@ietf.org

giving the IETF WG mailing list address (or [email-masked])

-Martin

---

From: Dave Taht <[email-masked]>
Date: Thu, 28 Jul 2016 07:13:02 +0200

> the one to do anything anyways.

These folk: https://hypothes.is/

have been at it a while and might be amiable to an ietf
standardization effort.  Adding in my contact there.

...

The history of web annotation is not very promising thus far, but who
knows what could happen if it emerges at the right time?

https://en.wikipedia.org/wiki/Web_annotation

>
>
>
> I do have a concern about your comment about getting a summary of perhaps a
> few years of commentary. The problem with tools that annotate comments
> against the RFC is that anyone looking at just those comments does not get
> the complete picture. Even if the working group is closed, pointing people
> to the group/archive gives someone the opportunity to get a more complete
> picture. Your comment may also be taken as a reason for keeping mailing
> lists open for an extended period even after the working group is no longer
> active. Every once in a while therew ill be a post to the DKIM or SPF lists
> by someone with a question or comment – even though those groups have been
> inactive for some time.
>
>
>
> Mike
>
>
>

-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

---

From: Dan Whaley <[email-masked]>
Date: Wed, 27 Jul 2016 22:29:01 -0700

Thanks Dave,

As coincidence would have it, Yaron @ IETF has recently published a thought
piece on this.

https://tools.ietf.org/html/draft-sheffer-ietf-rfc-annotations-00

We talk to him tomorrow morning.

Thanks for thinking of us.  We'll let you know if this goes anywhere.

Dan

On Wed, Jul 27, 2016 at 10:13 PM, Dave Taht <[email-masked]> wrote:

> > I have no specific objection to doing something. I wasn’t planning on
> being
> > the one to do anything anyways.
>
> can