Subject: Re: [rfc-i] Time to say "NO!!" to AUTH4200 (Re: AUTH48 checking the different formats (Re: Public archival of AUTH48 communications))

From: Carsten Bormann <[email-masked]>
Date: Tue, 1 Mar 2022 11:02:38 +0100

On 1. Mar 2022, at 00:24, Benjamin Kaduk <[email-masked]> wrote:
> 
> On Tue, Mar 01, 2022 at 12:17:24AM +0100, Carsten Bormann wrote:
>>> A full review should not be necessary. 
>> 
>> I wish that were true...
> 
> IMO a full review is necessary, not least to ensure that the document
> remains self-consistent and that any changes applied during editing are
> uniformly applied to all relevant parts of the document, not just in the
> subset of places that the editor noticed.

Absolutely.

(One more example for an API change, but not within the document: 
The most recent AUTH48 document I’m involved with has 69 <tt tags.  
The RFC editor rightly noted:

>> FYI, the text rendering of the <tt> element was changed in Sept. 2021 
>> (xml2rfc release 3.10.0). <tt> no longer yields quotation marks in the 
>> text rendering. 
>> 
>> It does yield fixed-width font in the HTML and PDF outputs. 
>> In light of this, please review and let us know if you want to 
>> make any changes (e.g., add quotation marks that would be in all 
>> three output formats).

The documents were of course mainly authored before the semantic change.
So one more task for the full reread: 
Check that the sentences still are understandable in the TXT form with the <tt information lost, and decide where quote marks need to be added.
This can only be decided by metamorphosing into a newly interested reader and reading through the text.
As noted in Star Trek 6, those metamorphoses can be exhausting.)

Grüße, Carsten

---

From: David Noveck <[email-masked]>
Date: Thu, 3 Mar 2022 07:08:58 -0500

On Mon, Feb 28, 2022, 6:24 PM Benjamin Kaduk <[email-masked]> wrote:

> IMO a full review is necessary,

My opinion is different.  If I followed your approach, a lot of documents
would have been delayed and probably abandoned.  I've been involved as
author/editor for a number of multi-hundred-page documents.  The results
were not perfect but the documents were improvements over the documents
they replaced and that is enough for me.

not least to ensure that the document
> subset of places that the editor noticed.

The changes made/proposed at editing are made by someone that is not an
author, is not part of the working group and should be rejected if they
raise potential difficulties such as you are worried about.

I cannot see how a large set of co-ordinated changes such as you are
worried about can be made by someone who is not an author.

    -  if a split infinitive is corrected, then the correction can be
approved without a further search for similar instances.  Each case is a
separate judgement call.
   - similarly with corrections of commas.
   - if a large set of co-ordinated changes is proposed, whose nature would
require a full rerevew, they should be rejected, as politely as possible.
If that is not possible for some reason, the working group should have the
opportunity to weigh in, but that option should be very rarely used.

> This is analogous to a situation that arises when doing code review.
> If you change an API's semantics,

"If it hurts when you do that, don't do that".

You should not be changing an APIs' semantics as part of a code review.
Those
Semantics should be agreed to before coding starts.

you had better go around and check
> all the callers of that API to see if they need adjustments, not just the
> callers of the API that your test suite happens to exercise.

Sound like a nightmare to me, since such adstments might require further
adjustments.

With
> specifications we do not have the rigid rules of function names to identify
> the places to check, and the safe thing to do is read the whole document
> with any changes that were made fresh in mind.
>

The safe thing is to limit changes after consensus to those that do not
require such extensive review by the authors who may, as someone pointed
out, have lost the necessary context.

> -Ben

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 3 Mar 2022 15:06:05 +0100

> My opinion is different.  If I followed your approach, a lot of documents would have been delayed and probably abandoned.  

The crucial observation is that different authors have different priorities here.  That is very much OK.
The responsible AD is part of the process to check that it doesn’t derail too much.

> I've been involved as author/editor for a number of multi-hundred-page documents.  The results were not perfect but the documents were improvements over the documents they replaced and that is enough for me.

The perfect is the enemy of the good…

>  should

Did I talk about process confabulation yet today?
I think we are still responsible for the outcomes, even if not everybody else keeps every SHOULD (or MUST).
But that doesn’t mean the result MUST be perfect.

Grüße, Carsten

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 4 Mar 2022 09:01:52 +1300

David,
On 04-Mar-22 01:08, David Noveck wrote:
...
> The safe thing is to limit changes after consensus to those that do not require such extensive review by the authors who may, as someone pointed out, have lost the necessary context.

This is a bit of a strawman, but it is entirely possible that an editor could resolve an ambiguous sentence in a way that effectively changes the meaning of the whole document; for example, changing a terminology definition that is used throughout the text.

We can't avoid this by issuing meaningless instructions to the editor like "don't change anything that matters". How we resolve it today is by the editor writing to the authors at AUTH48 time in terms like:

4) <!-- [rfced]  We're having difficulty parsing the following sentence
in Section 1.

Original:
    The foo operation includes deleting active state bars.

Perhaps:
    The foo operation consists of deleting all bars that are in the 'active' state.
-->

Only someone who understands the whole scenario can determine whether the second version is a clarification or a change of intent. Whatever the tooling for AUTH48, this sort of edit must be reviewed by a subject matter expert.

    Brian

---

From: John C Klensin <[email-masked]>
Date: Thu, 03 Mar 2022 21:37:38 -0500

--On Friday, March 4, 2022 09:01 +1300 Brian E Carpenter
<[email-masked]> wrote:

>> The safe thing is to limit changes after consensus to those
>> that do not require such extensive review by the authors who
>> may, as someone pointed out, have lost the necessary context.
> 
> This is a bit of a strawman, but it is entirely possible that
> an editor could resolve an ambiguous sentence in a way that
> effectively changes the meaning of the whole document; for
> example, changing a terminology definition that is used
> throughout the text.

Your supposed strawman is eventually what brought us to
pre-publication review of documents that, through a few stages
of review, brought us to AUTH48.  And problems about authors
either tuning out or taking advantage of questionable changes to
substitute their preferences for approving-body consensus.  In
other words "been there, tried that, and it did not work out
well".

Probably worth mentioning that those checks were first put in
place at a time when the editing team had considerably more
expertise and depth about the subject matter of RFCs than
today's RPC and any we are likely to see in the future.    Even
they could got get decisions about what did or did not change
meaning right 100% of the time.

> We can't avoid this by issuing meaningless instructions to the
> editor like "don't change anything that matters". How we
> resolve it today is by the editor writing to the authors at
> AUTH48 time in terms like:
>...

If the editor even notices that a change might be substantive.
The current RPC team gets that right almost all of the time, but
final review by subject matter experts is still important (if
nothing else, to keep this a team effort and avoid pressure on
the RPC for absolute accuracy and universal knowledge.

    john