Subject: Re: Back to the drawing board, was Re: Last Call: Registry Registrar Protocol (RRP) Version 1.1.0 to Informational

From: John C Klensin <[email-masked]>
Date: Wed, 05 Jan 2000 09:00:06 -0500

I've trimmed most of the cc's on the theory that everyone 
relevant except possibly Scott are on the IETF or IESG lists.

Ed, I've followed your comments very carefully, but, applying 
your reasoning to what I see as long-standing principles for 
handling Info RFCs, I reach nearly opposite conclusions. Please 
forgive the length of this note; I think it is important that we 
review and return to some principles here.

I don't have RFC 2223 in front of me and hope we can avoid 
hair-splitting and protocol-lawyering over this, but, in 
general, the RFC Editor has chosen to publish (and the IESG has 
chosen to not recommend otherwise) when

* The proposed document covers material that is at least 
potentially relevant and of interest to some reasonable fraction 
of the internet community.   E.g., a document that described the 
operation of a toaster that was not designed for connection to 
the network might well not be published.

* If the document covers a protocol or discussion about how to 
perform some protocol-related act, it isn't likely to create 
confusion with existing standards or standards-work-in-progress 
or reasonable ways to implement them.  E.g.,  a document that 
described itself as "HTTP 1.3", but was not compatible with HTTP 
1.1, and had not been through the standards process, would be 
unlikely to be published (at least under that name and without a 
lot of disclaimers).

* Whatever the document contains, it is not egregiously stupid. 
My impression is that, while this principle --rejecting drafts 
for publication on the basis of utter stupidity-- is discussed a 
lot, it is rarely actually applied.

More recently, there has been a lot of thought given to whether 
a particular document is part of a marketing plan to publish 
something as informaitional and then present it to customers as 
"an RFC" in a way that implies IETF approval and probably 
standardization.  My impression is that rejecting documents on 
this basis is, like the "no stupidity" criterion often 
discussed, but rarely applied.  These documents often go through 
with significant disclaimers attached, but they mostly go 
through.

By contrast, IETF has (I believe wisely) always steered clear of 
the certification process.   We never make statements --even for 
full Internet Standards-- that a given implementation works, or 
that a particular implementation faithfully reflects a given 
specification.   People and companies make those assertions, we 
don't try to stop them; we hope that the marketplace will 
appropriately reward those who lie (intentionally or otherwise). 
Usually that works, sometimes it doesn't, but, as you know, the 
IETF-authorized "protocol police" have always been a myth.

Now, in the case of this draft document, I was a little 
surprised that the IESG chose to Last Call this doc before 
letting it go to the RFC Editor.  They generally don't for 
informational documents (for which I think we must be thankful). 
I believe that the issues this has raised demonstrates that 
their decison was wise, even though many of the comments  have 
contained more heat than light.  But, if we look at the doc, it 
is a protocol description, supplied by one organization.    It 
doesn't make that clear enough, and it should, but that point 
was made and accepted long ago.  It is arguably not quite clear 
what NSI claims it describes, e.g.,

   * the protocol they are running today
   * the protocol they ran at one time
   * the protocol they wish they were running
   * the protocol they wish someone else were running, but have 
no intention of running themselves.

If that isn't clear to all reasonable readers, it ought to be 
fixed.   Within the traditional domain of info RFCs, any of 
those four (and others) are publishable, but we try to avoid 
confusion, especially in the case of the fourth.

But, I think an argument about whether or not the document 
actually conforms to some piece of code and making a decision to 
publish or not publish on that basis is *very* dangerous for us. 
As suggested above, we don't try to do that even for 
standards-track protocols and putatively-comforming programs.

If, as you suggest has happened here, NSI (or someone else) has 
submitted a document for info RFC publication, and it suggested 
things that aren't true, that is a problem... but it is a 
problem between NSI (again for example) and whomever tries to 
implement the thing, not an IETF problem.   I would expect that, 
if such a situation occurred, registrars who implemented the 
protocol as described in the RFC and then discovered that they 
needed to do something rather different to make it work, would 
be very annoyed.  They might rationally express that annoyance by
     * contacting their lawyers
     * contacting the press
     * writing proposed info RFCs with titles like "NSI RRP
       description considered harmful"
Just a personal opinion here, but I can't imagine the RFC Editor 
declining to publish documents of the latter type if they were 
technically coherent and well-reasoned.

In summary, I believe that our advice to the IESG should be

"make certain this document is clear about what it is and what 
it proports to be, and that the authors (or author organization) 
take responsibility for that being true.  Make certain that, 
should a RRP WG effort be launched, this document doesn't 
unreasonaby constrain it.   If areas are identified in which the 
document isn't clear about what it calls for, get those fixed. 
Consider attaching a disclaimer that indicates the list of 
unresolved issues contains some fairly serious problems and that 
there may be equally serious issues not on that list.   Then, 
since it is relevant and not obviously stupid, go ahead and 
publish the thing."

regards,
     john

Disclaimer:  I wasn't a member of the review committee (I was 
invited, but couldn't agree to the NDA in use at the time).   I 
did have a chance to look at a draft of this doc that preceeded 
the I-D.  My personal opinion is that, as a example of protocol 
design work, the I-D gets no better than a C- and that it may be 
an excellent example of why it is better to do protocol design 
in the open, rather than in closed groups tied up in NDAs.  But 
I can't get to "don't publish" from finding the thing a little 
distasteful: there are standards-track protocols I find a little 
distasteful and I believe the long-standing tradition of freedom 
to publish things that might be of use or interest is pretty 
important.

---

From: Ed Gerck <[email-masked]>
Date: Wed, 05 Jan 2000 06:32:02 -0800

John C Klensin wrote:

> handling Info RFCs, I reach nearly opposite conclusions.

Then, we have gone full circle because your reasoning is exactly
mine in its gist -- as I wrote in the subject line, NSI should go back
with the proposed RFC and make it clear before it is accepted to
be published as an info RFC.

I think your review is also very useful for future uses and it
helps IMO to stress the IETF role as a third-party in the process.
I am snipping it off for clarity here, not for lack of importance.

> since it is relevant and not obviously stupid, go ahead and

Yes, but IMO IETF RFCs, even informational, should not be
bureaucratic milestones in a chart but real contributions -- where
it is OK to be wrong since in Science a NO is also an answer.
So, an RFC should not be fictional or clearly wrong.

And this has nothing to do with "freedom of press" but with
old accepted rules of technical publication in whatever forum.

Cheers,

Ed Gerck

---

From: Ed Gerck <[email-masked]>
Date: Wed, 05 Jan 2000 06:32:02 -0800

Cheers,

Ed Gerck

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 Jan 2000 10:21:03 -0500

--On Wednesday, 05 January, 2000 06:32 -0800 Ed Gerck 
<[email-masked]> wrote:

> bureaucratic milestones in a chart but real contributions --
> where it is OK to be wrong since in Science a NO is also an
> answer. So, an RFC should not be fictional or clearly wrong.

Ed,

Assuming for purposes of argument that I agree with this 
distinction, the hypothetical chart and its milestones are a 
strawman.   I'm not especially attracted to the reasons some 
people participate in the IETF, but I think we need to judge the 
quality of the output almost independent of our guesses about 
the psychology of its sources.  Moreover, there has always been 
a distinction between Informational RFCs --which are expected to 
be an explanation or statement about something-- versus 
Experimental ones.   The latter should be subject to the 
scientific criteria you suggest.  Indeed, I've often felt that 
we would be better off if the authors of Experimental RFCs 
assumed an obligation to come back after a reasonable amount of 
time and write a new RFC explaining how the experiment came out 
and proposing the next experiment if appropriate.

If something is an opinion about how something should be done, 
one might disagree with it and it might even  be wrong 
technically, but that doesn't make it an invalid statement about 
that opinion.   And, if someone asserts that something they want 
to publish describes a partcular process, then I believe that we 
should publish that assertion along with the description.  But 
going further than that takes us dangerously close to certifying 
that the document and the implementation match, and that just 
isn't a good idea.

> old accepted rules of technical publication in whatever forum.

When I review things for a journal, I review them differently, 
and apply different criteria, depending on whether they 
represent opinion pieces or description on the one hand or, 
e.g., experimental results.

Let me try a strawman out on you: Suppose I came to the RFC 
Editor with a document entitled "a review of operational 
experience with the foobar protocol" that I wanted published as 
informational.   It would be entirely rational for the RFC 
Editor (or the IESG) to issue a Last Call whose key question was 
"does this description agree with the experience of others". 
Now, for many scientific publications, if the answer came back 
"no", the paper would go back to the authors, with a note that 
said "talk with these other folks and revise".  The tradition in 
the RFC series (and with some journals) has been that such 
things are suggested but, when the fail, the document is 
published, sometimes with an "opinions differ" disclaimer and 
the commenters are encouraged to write their own critiques. 
Either approach is scientifically reasonable and can be quite 
effective and illuminating; the second is arguably better at 
spreading knowledge of negative results.

But this document is, I believe, an NSI statement of what they 
think the protocol is, or would like the protocol to be,  or 
want people to think the protocol is (again, I think that needs 
to be clear).  To the extent that it is an accurate statement of 
their views in one of those categories, it can't be "fictional" 
or "clearly wrong".  To the extent that they claim it reflects 
the protocol and you believe it doesn't, your best recourse is 
to drop NSI a note that says, more or less, "if you publish that 
as if it were the protocol, I'll round up a bunch of registrars 
and sue you... partially on the theory that the Judge can lift 
the NDA that prevents our making the problems public".  But that 
isn't an IETF problem, and we put the IETF at risk by tryng to 
make it one.

     john

---

From: John C Klensin <[email-masked]>
Date: Wed, 05 Jan 2000 10:21:03 -0500

Ed,

     john

---

From: [email-masked]
Date: Wed, 05 Jan 2000 12:23:50 -0800 (PST)

> since it is relevant and not obviously stupid, go ahead and

I am in complete agreement with John on all of this. However, I would
like to add a couple of additional points:

(1) While it is true that publication as informational is sometimes
    misinterpreted as an endorsement, this is also something that is
    discussed a lot more than it actually happens. The RFC series
    contains many hundreds of documents describing protocols of dubious
    value and which don't receive a second glance by anybody, in some
    cases in spite of considerable marketing muscle being used to promote
    them.

    It isn't entirely clear why the publication == endorsement error
    happens in some cases, but it often involves some combination of people
    with no IETF standards experience, poor document labelling, a lack of
    a visible standards-track alternative or visible work underway on a
    standards-track alternative. In the present situation the first of these
    factors shouldn't apply and the second and third are things we control, so
    we should be able to minimize the risk of something bad happening.

(2) The publication of protocol specifications in order to document the
    history of Internet protocol evolution is actually pretty important when
    you're actually doing protocol development work. I find it incredibly
    useful, for example, to refer to RFC 733, RFC 788, RFC 1049, RFC 1154,
    FIPS PUB 98, and many other documents when tring to figure out why
    Internet mail has ended up where it has and what makes sense for the
    future.

(3) An alternative to putting a lot of comments about unresolved issues in
    given protocol document is to write a separate document critiquing the
    first and publish it as well. This approach used to be use quite a bit
    in the RFC series but has fallen into disuse in more recent times (probably
    because things move so fast these days). Perhaps this is an instance where
    the practice ought to be revived.

				Ned

---

From: [email-masked]
Date: Wed, 05 Jan 2000 12:23:50 -0800 (PST)

Ned