Subject: [admin-discuss] On transparency of the road not taken (Re: Public archival of AUTH48 communications)

From: Eliot Lear <[email-masked]>
Date: Mon, 7 Mar 2022 10:40:12 +0100

Hi Francesca,

We have discussed transparency in AUTH48, the point at which a draft is 
about to be published.  I wonder if we could also consider the point at 
which it is decided *not* to publish a draft.  Documenting that decision 
process is important (a) when the organization has spent a lot of time 
on it or (b) when there is some reasonable expectation that the matter 
might arise again. This can help us preserve institutional memory 
without requiring people to sift through hundreds or thousands of email 
messages.

Two examples that come to mind are the work around NEWTRK that never got 
pushed through (this goes back to 2006) and comes up from time to time, 
and the work of sunset4, about which someone just queried me.

If a draft has been adopted by a working group but hasn't proceeded to 
publication, then I would argue that the IETF should take the time to 
establish a tombstone of some form that explains what happened, 
recording people's views, preferably succinctly.  I imagine this doesn't 
happen too often (I've cited two examples that go back 16 years), and I 
can think of two other cases that go back even further.

If a draft was not adopted and people would still like to explain the 
situation, then that is something that might be a valuable independent 
submission, assuming other approaches haven't been developed.

I am not suggesting a treatise in either case, nor a rant about how 
things went "wrong", but a simple summary of the discussions to explain 
what we learned that led to the decision to not publish.

Thoughts?

Eliot

On 24.02.22 23:24, The IESG wrote:
> The IETF prides itself on its open process and transparent
> communication, with one of our core principles being our commitment to
> making all materials related to our standards process and other
> activities publicly available.
>
> For a document undergoing publication within the IETF Stream, most of
> its history can be traced by exploring the mail archive - from the first
> submission, to Last Call comments, to IESG evaluation. However, once the
> document enters the RFC Editor queue, the communication between authors
> and RFC Production Center (RPC) is only visible to a specific set of
> people: RFC editors, authors, the responsible ADs and WG chairs (when
> applicable).
>
> In order to increase transparency during the final stages of document
> editing, the IESG, IAB, ISE, IRSG, Temporary RFC Series Project Manager,
> RPC, and Tools team are considering allowing anyone to search and read
> AUTH48 conversations about specific documents (or clusters of
> documents). It is not a goal to actively involve more people in the
> conversation between authors and RFC editors.
>
> Our proposal: to set up a public mailing list of AUTH48 conversations,
> for archival purposes only, i.e., read-only. The RFC editors, when
> initiating the conversation with the authors will CC this mailing list.
> All further responses usually maintain the CC addresses, and as a
> consequence will be archived in the mailing list archive.
>
> All AUTH48 discussions of all documents will be archived on the same
> mailing list. Searches and filtering will be available based on the
> mails’ content and metadata, including draft name, RFC-to-be number,
> cluster number, sender, and date.
>
> Opt out: the authors and RFC editors are able to opt out from archival,
> by removing the mailing list from the CC. Although we don’t envision
> that this will be necessary often, there may be cases where sensitive
> information needs to be shared.
>
> The initial AUTH48 mail from the RFC Editor will also include text about
> public archival, to make sure authors are aware. With this announcement,
> the IESG wants to inform the community and read feedback before we set
> anything up. Please send any comment [email-masked]  or reply to this
> thread by 24 March 2022.
>
> Francesca Palombini for the IESG
>
> _______________________________________________
> IETF-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-announce
>

---

From: John C Klensin <[email-masked]>
Date: Mon, 07 Mar 2022 13:39:58 -0500

Eliot,

The devil is, as usual, in the details, but I agree with the
general idea.  Three additions to your comments:

(1) Some of those "path not taken" or rejected documents have
ended up as Independent Submissions and published.  Especially
when an IETF WG picked a different solution to the same or a
similar problem, the approval process for those Independent
Submissions has been more or less insistent that revised
versions of those road-not-taken documents contain a careful
critical analysis of the two approaches and the differences
between them.

(2) Using the NEWTRK documents as an example, I think it would
be accurate to say that many of those who had participated in
the work found the IESG's explanation of what was done and why
to be completely unsatisfactory.   Were we to go down the path
you suggest, there should be some theory about how to resolve
such situations even if it were only that the IESG posts a draft
summary/tombstone but its text were subject to appeal.  However,
one of the lessons from that episode was that there were no
appeals from the IESG decision at least in part because the
animosity level was high enough that no one wanted to raise
temperatures further.

(3) Would you expect such a tombstone mechanism / process to
apply to Independent Submissions as well?  I can see strong
arguments both ways.

best,
    john

--On Monday, March 7, 2022 10:40 +0100 Eliot Lear <[email-masked]>
wrote:

> Hi Francesca,
> 
> We have discussed transparency in AUTH48, the point at which a
> draft is about to be published.  I wonder if we could also
> consider the point at which it is decided *not* to publish a
> draft.  Documenting that decision process is important (a)
> when the organization has spent a lot of time on it or (b)
> when there is some reasonable expectation that the matter
> might arise again. This can help us preserve institutional
> memory without requiring people to sift through hundreds or
> thousands of email messages.
> 
> Two examples that come to mind are the work around NEWTRK that
> never got pushed through (this goes back to 2006) and comes up
> from time to time, and the work of sunset4, about which
> someone just queried me.
> 
> If a draft has been adopted by a working group but hasn't
> proceeded to publication, then I would argue that the IETF
> should take the time to establish a tombstone of some form
> that explains what happened, recording people's views,
> preferably succinctly.  I imagine this doesn't happen too
> often (I've cited two examples that go back 16 years), and I
> If a draft was not adopted and people would still like to
> explain the situation, then that is something that might be a
> valuable independent submission, assuming other approaches
> haven't been developed.
> 
> I am not suggesting a treatise in either case, nor a rant
> about how things went "wrong", but a simple summary of the
> discussions to explain what we learned that led to the
> decision to not publish.
> 
> Thoughts?
> 
> Eliot
> 
>> communication, with one of our core principles being our
>> commitment to making all materials related to our standards
>> process and other activities publicly available.
>> 
>> For a document undergoing publication within the IETF Stream,
>> most of its history can be traced by exploring the mail
>> archive - from the first submission, to Last Call comments,
>> to IESG evaluation. However, once the document enters the RFC
>> Editor queue, the communication between authors and RFC
>> Production Center (RPC) is only visible to a specific set of
>> people: RFC editors, authors, the responsible ADs and WG
>> chairs (when applicable).
>> 
>> In order to increase transparency during the final stages of
>> document editing, the IESG, IAB, ISE, IRSG, Temporary RFC
>> Series Project Manager, RPC, and Tools team are considering
>> allowing anyone to search and read AUTH48 conversations about
>> specific documents (or clusters of documents). It is not a
>> goal to actively involve more people in the conversation
>> between authors and RFC editors.
>> 
>> Our proposal: to set up a public mailing list of AUTH48
>> conversations, for archival purposes only, i.e., read-only.
>> The RFC editors, when initiating the conversation with the
>> authors will CC this mailing list. All further responses
>> usually maintain the CC addresses, and as a consequence will
>> be archived in the mailing list archive.
>> 
>> All AUTH48 discussions of all documents will be archived on
>> the same mailing list. Searches and filtering will be
>> available based on the mails' content and metadata,
>> including draft name, RFC-to-be number, cluster number,
>> sender, and date.
>> 
>> Opt out: the authors and RFC editors are able to opt out from
>> archival, by removing the mailing list from the CC. Although
>> we don't envision that this will be necessary often, there
>> may be cases where sensitive information needs to be shared.
>> 
>> The initial AUTH48 mail from the RFC Editor will also include
>> text about public archival, to make sure authors are aware.
>> With this announcement, the IESG wants to inform the
>> community and read feedback before we set anything up. Please
>> send any comment [email-masked]  or reply to this thread by
>> 24 March 2022.
>>