Subject: Last Call: draft-ietf-usefor-usepro (Netnews Architecture and Protocols) to Proposed Standard

From: The IESG <[email-masked]>
Date: Fri, 29 Aug 2008 17:06:45 -0700 (PDT)

The IESG has received a request from the Usenet Article Standard Update 
WG (usefor) to consider the following document:

- 'Netnews Architecture and Protocols '
   <draft-ietf-usefor-usepro-12.txt> as a Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action.  Please send substantive comments to the
[email-masked] mailing lists by 2008-09-12. Exceptionally, 
comments may be sent to [email-masked] instead. In either case, please 
retain the beginning of the Subject line to allow automated sorting.

The file can be obtained via
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usepro-12.txt

IESG discussion can be tracked via
https://datatracker.ietf.org/public/pidtracker.cgi?command=view_id&dTag=12256&rfc_flag=0

To: IETF-Announce <[email-masked]>
From: The IESG <[email-masked]>
Subject: Last Call: draft-ietf-usefor-usepro (Netnews Architecture and  Protocols) to Proposed Standard 
Reply-to: [email-masked]
CC: <[email-masked]>
Date: Fri, 29 Aug 2008 17:06:45 -0700 (PDT)
Sender: [email-masked]

The IESG has received a request from the Usenet Article Standard Update 
WG (usefor) to consider the following document:

- 'Netnews Architecture and Protocols '
   <draft-ietf-usefor-usepro-12.txt> as a Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action.  Please send substantive comments to the
[email-masked] mailing lists by 2008-09-12. Exceptionally, 
comments may be sent to [email-masked] instead. In either case, please 
retain the beginning of the Subject line to allow automated sorting.

The file can be obtained via
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usepro-12.txt

IESG discussion can be tracked via
https://datatracker.ietf.org/public/pidtracker.cgi?command=view_id&dTag=12256&rfc_flag=0

To: [email-masked]
Xref: clerew local.usefor:24874
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Niggles in usepro-draft-11
Date: Thu, 28 Aug 2008 17:05:58 GMT
Lines: 488
Sender: [email-masked]

I have now had a chance to read through the whole document, and was
pleased to see that it is now in a pretty tidy state.

The following niggles are mostly of a grammatical nature, or are textual
clarifications intended to ensure that our intentions are correctly
understood. I think there is only one item that might conceivably turn
into an Issue, although there are also a couple of cases where there is a
problem but I am not sure how to fix it, hence further discussion is
required.

3.2.1.  Constructing the Path Header Field

   ........  Note that the Path header
   field content is constructed from right to left by prepending
   elements.

   1.  ........

   2.  ........

   3.  A relaying or serving agent SHOULD prepend a <path-diagnostic> to
       the Path header field content, where the <path-diagnostic> is
       chosen as follows:

       *  If the expected <path-identity> of the source of the article
          matches the leftmost <path-identity> of the Path header
          field's content, use "!" (<diag-match>), resulting in two
          consecutive "!"s.

       *  If the expected <path-identity> of the source of the article
          does not match, use "!.MISMATCH." followed by the expected
          <path-identity> of the source or its IP address.

       *  If the relaying or serving agent is not willing or able to
          check the <path-identity>, use "!.SEEN." followed by the FQDN,
          IP address, or expected <path-identity> of the source.

The term "expected <path-identity>" has been used three times there. but
has not been properly explained. I would suggest adding, at the end of
these steps:

   In the above, the "expected" <path-identity> refers to that
   appropriate to the apparent source of the article (e.g. as determined
   by the channel whence it arrived, its IP address, or some
   authentication provided during connection setup).

3.2.2.  Path Header Field Example

   Here is an example of a Path header field created following the rules
   for injecting and relaying agents.

          Path: foo.isp.example!.SEEN.isp.example!!foo-news
            !.MISMATCH.2001:DB:0:0:8:800:200C:417A!bar.isp.example
            !!old.site.example!barbaz!!baz.isp.example
            !.POSTED.dialup123.baz.isp.example!not-for-mail

   This article was injected by baz.isp.example as indicated ....

   The article was relayed to the relaying agent known, at least to
   old.site.example, as "barbaz".

   barbaz relayed it to old.site.example, which does not support <diag-
   keyword> and therefore used the old "!" delimiter.  This indicates
   that the identity of "barbaz" was not verified and may have been
   forged.

But then there is no explanation of why barbaz used that "!!" delimiter
(all the other funnies in the example _do_ get accounted for in further
on in the text). So I would suggest:

   barbaz, being satisfied that it had indeed been recceived from
   baz.isp.example, inserted a "!!" <path-delimiter>. It then relayed it
   to old.site.example .........

3.3.  Article History and Duplicate Suppression

   o  ............  If this
      interval is shorter than the time it takes for an article to
      propagate through the network, the agent might reject an article
      it had not yet seen, so it ought not be aggressively short. .....
                                          ^
                                          to

   These are just two implementation strategies for article history,
   albeit the most common ones.  Relaying and serving agents are not
   required to use these strategies, only to meet the requirement of not
   accepting an article more than once.  However, these strategies are
   safe and widely deployed and implementors are encouraged to use one
   of them, especially if they do not have extensive experience with
   Netnews and the subtle effects of its flood-fill algorithm.
               ^^^
           with the more

3.4.  Duties of a Posting Agent

   If the proto-article already contains both Message-ID and Date header
   fields, posting agents MAY add an Injection-Date header field to that
   proto-article immediately before passing that proto-article to an
   injection agent.  They SHOULD do so if the Date header field
   (representing the composition time of the proto-article) is more than
   a day in the past at the time of injection.  They MUST do so if the
   proto-article is being submitted to more than one injecting agent;
   see Section 3.4.2.

We are now allowing (sometimes even requiring) posting agents to insert
an Injection Date.  Whereas clueful multiple injectors will probably do
this correctly, I am worried that incompetent MUA implementors (or which
there are many :-( ) will do it wrongly.

On the other hand, now it seems to have been decided only to allow
injection agents to insert it in limited situations, it is desirable
that posting agents should be encouraged (somewhere between MAY and
SHOULD) to do it routinely (otherwise this new header will never catch
on). So I suggest:
   
   When the proto-article already contains both Message-ID and Date
   header fields, posting agents MAY, as part of the injection process
   (i.e. immediately before passing it to an injection agent), add an
   Injection-Date header field to that proto-article (a useful practice,
   since it provides diagnostic information and can imrove propagation).
   Moreover, they SHOULD do so if the Date header field (representing
   the composition time of the proto-article) is more than a day in the
   past at the time of injection. and they MUST do so if the
   proto-article is being submitted to more than one injecting agent;
   see Section 3.4.2.

3.4.1.  Proto-articles

   A proto-article has the same format as a normal article except that
   the Injection-Info and Xref header fields MUST NOT be present; the
   Path header field MUST NOT contain a "POSTED" <diag-keyword>; and any
   of the following mandatory header fields MAY be omitted: Message-ID,
   Date, and Path.  In all other respects, a proto-article MUST be a
   valid Netnews article.  In particular, the header fields which may be
   omitted MUST NOT be present with invalid content.

I am worried about that 'MUST NOT contain a "POSTED" <diag-keyword>'. It
would cause no interoperability problems and cause no significant harm,
and hence a "MUST" cannot be justified under RFC 2119. Nothing breaks if
two "POSTEDs appear in one Path. Indeed, it could arise legitimately in
multiple injection "after the fact", and in other exotic gatewaying
situations.

Yes, some s[pc]ammers will preload the Path so as to disguise the true
origin of an article (which is indeed why POSTED was invented), but that
is a matter to be taken up by the netkops rather than the protocols. And
I would prefer to preserve all evidence left over from previous Paths in
order to facilitate bebugging of broken gateways, etc.

So, at the most, it ought to be a SHOULD, and I would be happy to omit
it altogether. Note that whatever change is made here, corresponding
changes would be needed in 3.4.2 and 3.5.2 Step 2.

3.4.2.  Multiple Injection of Articles

   Under some circumstances (posting to multiple disjoint networks,
                                                ^
                                             supposedly
   injecting agents with spotty connectivity, or for redundancy, for
   example), ........

   Whenever possible, multiple injection SHOULD be done by offering the
   same proto-article to multiple injecting agents.  The posting agent
   MUST supply the Message-ID, Date, and Injection-Date header fields,
   and the proto-article as offered to each injecting agent MUST be
           ^^^^^^^^^^^^^
           proto-articles
   identical.

   In some cases, offering the same proto-article to all injecting
   agents may not be possible (such as when gatewaying, after the fact,
   articles found on one Netnews network to another, supposedly
   unconnected one).  In this case, the posting agent MUST convert the
   article back into a proto-article before passing it to another
   injecting agent, but it MUST retain unmodified the Message-ID, Date,
   and Injection-Date header fields.  It MUST NOT add an Injection-Date
   header field if it is missing from the existing article.  It MUST
   remove any Xref header field and either rename or remove any
   Injection-Info header field and other trace fields.

I don't like the term "after the fact" (it has no intuitive meaning),
and I don't think "converting back to a proto-article" is the best way
to describe what needs to be done (since it can still contain various
headers not present in the first-time-around proto-article). Here is an
alternative wording:

   In some cases, multiple injection arises (perhaps inadvertently) when
   gatewaying articles found on one Netnews network to another,
   supposedly unconnected one.  In this case, the (re-)posting agent
   MUST remove any Xref header field and either rename or remove any
   Injection-Info header field and other trace fields (thus converting
   the article back into a proto-article) before passing it to another
   injecting agent, but it MUST retain unmodified the Message-ID, Date,
   and Injection-Date header fields.  It MUST NOT add an Injection-Date
   header field if it is missing from the existing article.

Note that wording might need further revision, depending on what we
decide regarding allowing "POSTED" to appear twice in a Path. And the
words "after the fact" would need rewriting in the following NOTE.

      NOTE: Multiple injection inherently risks duplicating articles.
      Multiple injection after the fact, by converting an article back
      to a proto-article and injecting it again, additionally risks
      loops, loss of trace information, unintended repeat injection into
      the same network, and other problems.  It should be done with care
      and only when there is no alternative.  The requirement to retain
      Message-ID, Date, and Injection-Date header fields minimizes the
      possibility of a loop and ensures that the newly injected article
      is not treated as a new, separate article.

   Multiple injection of an article listing one or more moderated
   newsgroups in its Newsgroups header field SHOULD only be done by a
                                                    ^^^^
						    ONLY
   moderator and MUST only be done after the proto-article has been
                      ^^^^
		      ONLY
   approved for all moderated groups to which it is to be posted and has
   an Approved header field (see Section 3.9).  .....

I think that is well within the spirit of RFC 2119, and will be much
clearer.

3.4.4.  Construction of the References Header Field

   The content of the new article's References header field MUST be
   formed from the content of the parent's References header field if
   present and the content of the Message-ID header field of the parent.
           ^^^
        followed by
   If the parent had a References header, FWS as defined in [USEFOR]
   MUST be added between its content and the Message-ID header field
   content.

If we don't make that change, then it does not actually make it clear
whether the new reference is to be placed to the left, or to the right,
of the existing ones.

3.5.  Duties of an Injecting Agent

   11.  If the proto-article already had an Injection-Date header field,
        it MUST NOT be modified or replaced.  If the proto-article had
        both a Message-ID header field and a Date header field, an
        Injection-Date header field MUST NOT be added, since the proto-
        article may have been multiply injected by a posting agent that
        predates this standard.  Otherwise, the injecting agent MUST add
        an Injection-Date header field containing the current date and
        time.

Well, you know that I don't like the second sentence in that. Taking the
preferences into account, we were split exactly 50:50 on the issue
(which is a perfect un-consensus :-) ). But I accept that it is better
for the Chair to make an arbitrary choice in this situation (absent
some unforeseen new consenus, of course).

3.6.  Duties of a Relaying Agent

   5.  It SHOULD reject any article that matches an already-received
       cancel control message or the contents of the Supersedes header
       field of an accepted article, provided that the relaying agent
       chose (on the basis of local site policy) to honor that cancel
       ^^^^^
       has chosen
       control message or Supersedes header field.

5.1.  Authentication and Authorization

   Control messages specify actions above and beyond the normal
   processing of an article and are therefore potential vectors of abuse
   and unauthorized action.  There is, at present, no standardized means
   of authenticating the sender of a control message or verifying that
   the contents of a control message were sent by the claimed sender.
   There are, however, some unstandardized authentication mechanisms in
   common use.

   Agents acting on control messages SHOULD take steps to authenticate
   control messages before acting on them, as determined by local
   authorization policy.  Whether this is done via the use of an
   unstandardized authentication protocol, by comparison with
   information obtained through another protocol, by human review, or by
   some other means is left unspecified by this document.  Further
   extensions or revisions of this protocol are expected to standardize
   a digital signature mechanism.

   Agents are expected to have their own local authorization policies
   for which control messages will be honored.  No Netnews agent is ever
   required to act on any control message.  The following descriptions
   specify the actions that a control message requests, but an agent MAY
   always decline to act on any given control message.

5.2.3.  The checkgroups Control Message

   The body of the message is an entity of type application/
                             ^
       (or contains, as part of a multipart/mixed)
   news-checkgroups.  It SHOULD be declared as such with appropriate
   MIME headers, but news servers SHOULD interpret checkgroups messages
   that lack the appropriate MIME headers as if the body were of type
   application/news-checkgroups for backward compatibility.

Cf. wording in 5.2.1.

5.3.  The cancel Control Message

   The cancel control message requests that a target article be
   withdrawn from circulation and access.  The syntax of its Control
   header field is:

         control-command     =/ Cancel-command
         Cancel-command      = "cancel" Cancel-arguments
         Cancel-arguments    = 1*WSP msg-id

   The argument identifies the article to be cancelled by its message
   identifier.  The body of the control message MAY contain anything,
   usually an explanatory text.

   A serving agent that elects to honor a cancel message SHOULD make the
   article unavailable to reading agents (perhaps by deleting it
   completely).  If the cancel control message arrives before the
   article it targets, news servers choosing to honor it SHOULD remember
   the message identifier that was cancelled and reject the cancelled
   article when it arrives.

   To best ensure that it will be relayed to the same news servers as
   the original message, a cancel control message SHOULD have the same
   Newsgroups header field as the message it is cancelling.

   Cancel control messages listing moderated newsgroups in their
   Newsgroups header field MUST contain an Approved header field like
   any other article in a moderated newsgroup.  This means that cancels
   posted to a moderated newsgroup will normally be sent to the
   moderator first for approval.  Outside of moderated newsgroups,
   cancel messages are not required to contain an Approved header field.

   Contrary to [RFC1036], cancel control messages are not required to
   contain From and Sender header fields matching the target message.
   This requirement only encouraged cancel issuers to conceal their
   identity and provided no security.

5.4.  The Supersedes Header Field

   The presence of a Supersedes header field in an article requests that
   the message identifier given in that header field be withdrawn in
   exactly the same manner as if it were the target of a cancel control
   message.  Accordingly, news servers SHOULD use the same
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   authentication and authorization checks for deciding whether to honor
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   a Supersedes header field as they use for cancel control messages.
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   If the Supersedes header field is honored, the news server SHOULD
   take the same actions as it would take when honoring a cancel control
   message for the given target article.

The bit I have marked with ^^^^^^^^ is all fine and dandy, except that
the preceding section 5.3 on Cancel Messages make no mention of
"authentication and authorization". I think the Cancel Message _should_
say something on the subject, but the question is What? (given that we
have nothing specific to offer). The old draft-06 contained the
following:

        NOTE: It is expected that the security extension envisaged in
        section 5.1 will make more detailed provisions for establishing
        whether honouring a particular "cancel" message is in order. In
        particular, it is likely that there will be provision for the
        digital signature of 3rd party cancels (i.e. those issued other
        than by the sender, the moderator, or the injector).

However, what is currently said in our section 5.1 is fine (so far as it
goes) for Group Control Messages, but has nothing to say of relevance to
Cancels and Supersedes. Moreover, we are hardly in a position now to
promise that a "Security Extension for Netnews" is in the offing,
desirable as such might be.

So what ought we to be saying as regards both Cancels and Supersedes? I
have quoted all the relevant texts above so that people can discuss
the possibilities.

And also, as regards Supersedes headers, we do not actually say that, in
addition to acting as a Cancel, they are also propagated and displayed
like an ordinary article. I therefore suggest the following additional
paragraph:

   The article containing the Supersedes header field is itself still
   intended to be made available to reading agents in the usual manner.

6.  Security Considerations

6.1.  Compromise of System Integrity

   Control messages pose a particular security concern since acting on
   unauthorized control messages may cause newsgroups to be removed,
   articles to be deleted, and unwanted newsgroups to be created.
   Administrators of news servers SHOULD therefore take steps to verify
   the authenticity of control messages as discussed in Section 5.1.
   Articles containing Supersedes header fields are effectively cancel
   control messages and SHOULD be subject to the same checks as
   discussed in Section 5.4.  Currently, many sites are ignoring all
   cancel control messages and Supersedes header fields due to the
   difficulty of authenticating them and their widespread abuse.

Agreed as far as Cancels, but does there in fact exist such "widespread
abuse" of Supersedes?

6.2.  Denial of Service

   Such articles intended to deny service, or other articles of an
   inflammatory nature, may also have their From or Reply-To addresses
   set to valid but incorrect email addresses, thus causing large
   volumes of email to descend on the true owners of those addresses.
                   ^
              ("mailbombs")
   Users and agents should always be aware that identifying information
   in articles may be forged.

Please add:

   The inclusion of a Disposition-Notification-To header field could
   also give rise to such mailbombs (hence its deprecation for Netnews
   in [RFC3798]).

Appendix A.  Changes to the Existing Protocols

   This document prescribes many changes, clarifications, and new
   features since the protocol described in [RFC1036].  Most notably:

   o  Addition of the new Injection-Date header field is required for
      injecting agents (Section 3.5) and MUST be used by news servers
      for date checks (Section 3.6).  Injecting agents are strongly
      encouraged to put all local trace information in the new
      Injection-Info header field.

Unless a sudden "unforeseen new consensus" happens, that first sentence
will need to be changed.

{Ah! I just spotted that Russ has fixed it in draft-12}

Authors' Addresses

   Charles H. Lindsey
   University of Manchester
   5 Clerewood Avenue
   Heald Green
   Cheadle
   Cheshire  SK8 3JU
   United Kingdom

I would suggest removing that "University of Manchester" line. I am long
retired from the University, and I would not like to people to think
that the address of the University was "5 Clerewood Avenue, Heald Green"
:-) .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

From: [email-masked]
To: [email-masked]
Cc: [email-masked]
Subject: I-D Action:draft-ietf-usefor-usepro-12.txt 
Date: Wed, 27 Aug 2008 13:15:02 -0700 (PDT)
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Usenet Article Standard Update Working Group of the IETF.

	Title           : Netnews Architecture and Protocols
	Author(s)       : R. Allbery, C. Lindsey
	Filename        : draft-ietf-usefor-usepro-12.txt
	Pages           : 54
	Date            : 2008-08-27

This document defines the architecture of Netnews systems and
specifies the correct manipulation and interpretation of Netnews
articles by software which originates, distributes, stores, and
displays them.  It also specifies the requirements that must be met
by any protocol used to transport and serve Netnews articles.Internet
Draft Comments

Comments are solicited and should be addressed to the Usenet Format
Working Group at [email-masked].

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usepro-12.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Message/External-body;
	name="draft-ietf-usefor-usepro-12.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:     <[email-masked]>

--NextPart--

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Wed, 27 Aug 2008 12:59:55 -0700 (PDT)
Sender: [email-masked]

    Date: Wednesday, August 27, 2008 @ 12:59:54
  Author: eagle
Revision: 4927

Updates to the changes since RFC 1036 around Path headers and
Injection-Date, reflecting restructuring and changes to the draft.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-27 19:49:43 UTC (rev 4926)
+++ docs/usefor/usepro.xml	2008-08-27 19:59:54 UTC (rev 4927)
@@ -2289,18 +2289,18 @@
           <t>A new, backward-compatible Path header field format that
           permits standardized embedding of additional trace and
           authentication information is now RECOMMENDED.  See <xref
-          target="injecting" /> and <xref target="relaying" />.  Folding
-          of the Path header is permitted.</t>
+          target="path" />.  Folding of the Path header is permitted.</t>
 
           <t>Trimming of the References header field is REQUIRED and a
           mechanism for doing so is defined.</t>
 
           <t>Addition of the new Injection-Date header field is required
-          for injecting agents (<xref target="injecting" />) and MUST be
-          used by news servers for date checks (<xref target="relaying"
-          />).  Injecting agents are strongly encouraged to put all
-          local trace information in the new Injection-Info header
-          field.</t>
+          in some circumstances for posting agents (<xref
+          target="multi-injection" />) and injecting agents (<xref
+          target="injecting" />) and MUST be used by news servers for date
+          checks (<xref target="relaying" />).  Injecting agents are also
+          strongly encouraged to put all local trace information in the
+          new Injection-Info header field.</t>
 
           <t>A new media type is defined for transmitting Netnews
           articles through other media (<xref target="transmission" />),

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (Makefile usepro.xml)
Date: Wed, 27 Aug 2008 12:49:44 -0700 (PDT)
Sender: [email-masked]

    Date: Wednesday, August 27, 2008 @ 12:49:43
  Author: eagle
Revision: 4926

Make usepro-12.

Modified:
  docs/usefor/Makefile
  docs/usefor/usepro.xml

Modified: docs/usefor/Makefile
===================================================================
--- docs/usefor/Makefile	2008-08-26 02:35:19 UTC (rev 4925)
+++ docs/usefor/Makefile	2008-08-27 19:49:43 UTC (rev 4926)
@@ -1,7 +1,7 @@
 # Generate new text and HTML versions of the USEPRO document and copy them
 # somewhere reasonable on my web site.
 
-DRAFT = draft-ietf-usefor-usepro-11
+DRAFT = draft-ietf-usefor-usepro-12
 WEB   = /home/eagle/web/eagle/usefor/usepro
 
 all: text html copy

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-26 02:35:19 UTC (rev 4925)
+++ docs/usefor/usepro.xml	2008-08-27 19:49:43 UTC (rev 4926)
@@ -28,7 +28,7 @@
     'http://xml.resource.org/public/rfc/bibxml/reference.RFC.5234.xml'>
 ]>
 
-<rfc ipr="full3978" docName="draft-ietf-usefor-usepro-11" obsoletes="1036"
+<rfc ipr="full3978" docName="draft-ietf-usefor-usepro-12" obsoletes="1036"
      category="std">
   <front>
     <title>Netnews Architecture and Protocols</title>

From: [email-masked]
To: [email-masked]
Subject: I-D Action:draft-ietf-usefor-usepro-11.txt 
Date: Fri, 22 Aug 2008 15:00:01 -0700 (PDT)
Cc: [email-masked]
Reply-To: [email-masked]
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Usenet Article Standard Update Working Group of the IETF.

	Title           : Netnews Architecture and Protocols
	Author(s)       : R. Allbery, C. Lindsey
	Filename        : draft-ietf-usefor-usepro-11.txt
	Pages           : 51
	Date            : 2008-08-22

This document defines the architecture of Netnews systems and
specifies the correct manipulation and interpretation of Netnews
articles by software which originates, distributes, stores, and
displays them.  It also specifies the requirements that must be met
by any protocol used to transport and serve Netnews articles.Internet
Draft Comments

Comments are solicited and should be addressed to the Usenet Format
Working Group at [email-masked].

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usepro-11.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Message/External-body;
	name="draft-ietf-usefor-usepro-11.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID: <[email-masked]>

--NextPart
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

--NextPart--

To: [email-masked]
Xref: clerew local.usefor:24870
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ABNF nits
Date: Tue, 26 Aug 2008 09:27:28 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>"Charles Lindsey" <[email-masked]> writes:

>> I also note that there is a "Restriction on usage" header in the RFC
>> 4288 template, and it might be useful to say that news-groupinfo and
>> news-checkgroups were only for use in newsgroup control messages (in
>> which case the "Intended usage" header should say "LINITED USE").

>I don't see a need to limit either to newsgroup control messages.  They're
>both  outside of that context (news-checkgroups more so,
>admittedly).  However, I don't feel strongly about news-groupinfo; if
>people think it's sufficiently weird that it doesn't make any sense to use
>it outside of control messages, I'm willing to make that change.  I feel
>more strongly that news-checkgroups is appropriate in a much wider variety
>of situations.

Yes, I take your point about possible use of news-checkgroups in regularly
posted articles in *.annouce groups. It's no big deal, because the
introductory text already present indicates that control messages are the
primary intended use for both of these.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24871
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Potential ISSUE: <utext> (was: ABNF nits)
Date: Tue, 26 Aug 2008 09:37:21 GMT
Lines: 37
Sender: [email-masked]

In <[email-masked]> "Frank Ellermann" <[email-masked]> writes:

>Charles Lindsey wrote:

>>> based on feedback from Charles, you, and me (among others)
>>> on the rfc822 list.
> 
>> Yes, we managed to get more changed in RFC 2822-bis than we
>> could ever have expected.

>The author successfully avoided the simple approach to adopt 
>our prior work wholesale.  His new <msg-id> (minus obs) is
>now simpler than our form, but allows ">".

And that is the problem. The USEFOR draft permits a whole raft of ugly
stuff that is no longer permitted by 2822-bis (and good riddance to it).
It would be much simpler to say that our message-ids are the same as those
of 2822 bis, but without the obs-stuff and without that ">".

>His <msg-id> plus obs is of course unacceptable for NetNews.
>All pieces are in place for a *future* final merger, but not
>for a sudden rush of AUTH48 modifications.

I doubt we are going to have any opportunity for any "*future* final
merger". If we don't do it now, it will never get done. I doubt any of it
would be the least bit controversial.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Mon, 25 Aug 2008 17:13:04 -0700 (PDT)
Sender: [email-masked]

    Date: Monday, August 25, 2008 @ 17:13:03
  Author: eagle
Revision: 4924

Move the Disposition bit for message/news to Interoperability
considerations.  In the other registrations, use Interoperability
considerations, not implementation considerations, to match the
template (it would help if I read carefully).  Re-add Frank's
correct Applications that use this media type for message/news.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-25 23:45:39 UTC (rev 4923)
+++ docs/usefor/usepro.xml	2008-08-26 00:13:03 UTC (rev 4924)
@@ -1455,15 +1455,15 @@
    MIME subtype name:        news
    Required parameters:      none
    Optional parameters:      charset, compare message/rfc822
-   Disposition:              by default, inline
    Encoding considerations:  compare message/rfc822 (RFC 2046)
    Security considerations:  OBSOLETE, use message/rfc822.
    Interoperability considerations:
                              Rarely used; and therefore often
-                             handled as application/octet-stream
+                             handled as application/octet-stream.
+                             Disposition should by default be inline.
    Published specification:  [SON-OF-1036], RFC&rfc.number;
    Applications that use this media type:
-                             None known.
+                             Some old mail and news user agents.
    Intended usage:           OBSOLETE
    Author:                   Henry Spencer 
    Change controller:        IETF
@@ -1548,7 +1548,7 @@
    Encoding considerations:  7bit or 8bit MUST be used to maintain
                              compatibility. 
    Security considerations:  None.
-   Implementation considerations:
+   Interoperability considerations:
                              Disposition should by default be inline.
    Applications that use this media type:
                              Control message issuers, relaying
@@ -1636,7 +1636,7 @@
                              should exist within a hierarchy.  Such
                              authorization must be accomplished by
                              other means.
-   Implementation considerations:
+   Interoperability considerations:
                              Disposition should by default be inline.
    Applications that use this media type:
                              Control message issuers, relaying

To: [email-masked]
Subject: Re: known application
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 25 Aug 2008 17:09:36 -0700
Sender: [email-masked]

"Frank Ellermann" <[email-masked]> writes:

>> +   Applications that use this media type:
>> +                             None known.
>
> I proposed "some old UAs", because netscape 3.x did.
> Maybe write "obsolete netscape UAs" (or similar)...

Oh, sorry, I didn't realize that.  Will change.  (I should have asked
first.)

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Mon, 25 Aug 2008 16:11:50 -0700
From: "Alexey Melnikov" <[email-masked]>
To: [email-masked]
Subject: Re: Deadlines (was: ABNF nits)
Sender: [email-masked]

On Mon, Aug 25, 2008 at 1:59 PM, Frank Ellermann
<[email-masked]> wrote:
>
> Charles Lindsey wrote:
>
>> I would suggest planning on doing it when your
>> "week or two" is over.
>
> Lisa's deadline is 08-29, and Russ' 08-28 fits.
> Integers for draft numbers are a cheap resource:
>
> The years old "usepro" has the same number of
> versions as the weeks old "idn-tlds" (2606bis).

Personally, I would like to see a new draft addressing all resolved issues ASAP.

Pushing another revision is indeed very cheap.

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: Potential ISSUE: <utext> (was: ABNF nits)
Date:  Mon, 25 Aug 2008 23:56:18 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 62
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Charles Lindsey wrote:

>> based on feedback from Charles, you, and me (among others)
>> on the rfc822 list.
=20
> Yes, we managed to get more changed in RFC 2822-bis than we
> could ever have expected.

The author successfully avoided the simple approach to adopt=20
our prior work wholesale.  His new <msg-id> (minus obs) is
now simpler than our form, but allows ">".

His <msg-id> plus obs is of course unacceptable for NetNews.
All pieces are in place for a *future* final merger, but not
for a sudden rush of AUTH48 modifications. =20

Harald already said that he has no energy to coordinate that.
Lisa also never committed to general 2822upd upgrades of the
NetNews RFCs.  And any last minute screwing with fundamental
concepts such as <msg-id-core> would deserve new Last Calls.

Leave this alone for this round.  Start the screwing in 2009
if you must.  Folks in the real world outside of USEFOR still
are forced to use RFC 1036 in their references (example, the
Archived-At standard).

The top priority is to get this stuff out now after 10 years,
not to start another round of <msg-id-core> embellishments.

Or to fix <toplabel> after the RFC 3696 author changed his
mind to "LDH starting with L".  (Foreseeable getting nothing,
his 1123-erratum killed my erratum, and the two DNSEXT folks
supposed to fix RFC 1123 in two lines went AWOL immediately
after volunteering for this job).

Or whatever else could be done in the approved RFCs above a
harmless AUTH48 fix of s/<path-delimiter>/path delimiter/ -
the <utext> problem of course also affects USEFOR, not only
USEPRO.

> I still think that, now 2822bis has been accepted by the
> IETF and will presumably become a Draft Standard, we should
> revisit USEFOR and rebase it on 2822-bis

NAK.  This is not "straight forward".  There are lots of at
first glance minor differences between 2822 and 2822upd with
subtle consequences for the NetNews RFCs.  And we even know
that 2822upd can't be the last word: =20

It has a "white space in quoted string local parts" concept
incompatible with 2821bis (found by Alexey, IIRC).  But for
this round nobody had the energy to analyse and fix it.

Oftwn when I tried "straight forward" in conjunction with
the IETF it ended up as disaster:  The toplabel task force
proposed on this list two years ago was so far unable to
fix a single word in RFC 1123.  The "4234 to STD" proposal
was supposed to be a no-brainer for four weeks, and ended
up as a major project with new drafts / stubborn authors /
interop testing and report / etc. until it ended as STD 68.
 =20
 Frank

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Deadlines (was: ABNF nits)
Date:  Mon, 25 Aug 2008 22:59:12 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 12
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Charles Lindsey wrote:

> I would suggest planning on doing it when your
> "week or two" is over.

Lisa's deadline is 08-29, and Russ' 08-28 fits.
Integers for draft numbers are a cheap resource:

The years old "usepro" has the same number of=20
versions as the weeks old "idn-tlds" (2606bis).

 Frank

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  known application (was: Commit in docs/usefor (usepro.xml))
Date:  Mon, 25 Aug 2008 22:42:17 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 16
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

<[email-masked]> wrote:

> +   Disposition:              by default, inline

Likely fixed later.

> +   Applications that use this media type:
> +                             None known.

I proposed "some old UAs", because netscape 3.x did.
Maybe write "obsolete netscape UAs" (or similar)...

...after all this was the mother of all Web browsers
(ignoring Mosaic and Lynx for a second)

 Frank

To: [email-masked]
Subject: Re: ABNF nits
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 25 Aug 2008 11:01:21 -0700
Sender: [email-masked]

"Charles Lindsey" <[email-masked]> writes:

> I "invented" the "Disposition" line in the template because it seemed
> like a good way to do it. However, on readng RFC 4288, I now think it
> would be better to include it under an "Additional Information" line in
> the template, since section 4.11 of that RFC seems to imply that you can
> put anything sensible in there. The change could easily be made.

I stuck it into Implementation considerations, which seemed even more
appropriate, but I can move it if people think that would be better.

> I also note that there is a "Restriction on usage" header in the RFC
> 4288 template, and it might be useful to say that news-groupinfo and
> news-checkgroups were only for use in newsgroup control messages (in
> which case the "Intended usage" header should say "LINITED USE").

I don't see a need to limit either to newsgroup control messages.  They're
both useful outside of that context (news-checkgroups more so,
admittedly).  However, I don't feel strongly about news-groupinfo; if
people think it's sufficiently weird that it doesn't make any sense to use
it outside of control messages, I'm willing to make that change.  I feel
more strongly that news-checkgroups is appropriate in a much wider variety
of situations.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Subject: Re: ABNF nits
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 25 Aug 2008 10:59:20 -0700
Sender: [email-masked]

"Charles Lindsey" <[email-masked]> writes:

> I am sure that there will be lots of niggles once people get a chance to
> read the latest draft (it is a long time since we had an up-to-date
> complete draft to consider as a whole). So, rather than rushing to
> produce instant drafts every couple of daye, I would suggest planning on
> doing it when your "week or two" is over.

I'd rather upload another draft before the AD deadline, just because.  I
probably won't continue to try to keep to that sort of quick upload
schedule after that.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Xref: clerew local.usefor:24859
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ABNF nits
Date: Mon, 25 Aug 2008 12:03:21 GMT
Lines: 51
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>"Frank Ellermann" <[email-masked]> writes:
>> Russ Allbery wrote:

>>> I didn't; I just didn't change it.  This has been in the
>>> draft for quite a while.  Should I remove it?  I don't see
>>> Disposition anywhere in the media type registration either.
>>
>> You could add an "Interoperability considerations" field, and
>> put the "default disposition inline" into this field.  Maybe
>> Charles still knows where he found this ?  (= not in RFC 4288)

>Charles, do you remember?

The reason it is there is that these are application MIME types, and as
such are normally expected to be processed by the receiving agent in some
application-specific manner (as indeed these are, for inserting
information into the local Newsgroups file). However, they are also
intended to be human readable (as indeed they are at present, since they
are currently not MIME types at all, and we have cunningly designed the
format so as to fool current software into recognising them). Moreover,
in the absence of a Content-Dispostion header, RFC 2183 is silent on
whether the default is "inline" or not; so we really need to say
something.

I "invented" the "Disposition" line in the template because it seemed like
a good way to do it. However, on readng RFC 4288, I now think it would be
better to include it under an "Additional Information" line in the
template, since section 4.11 of that RFC seems to imply that you can put
anything sensible in there. The change could easily be made.

I also note that there is a "Restriction on usage" header in the RFC 4288
template, and it might be useful to say that news-groupinfo and
news-checkgroups were only for use in newsgroup control messages (in which
case the "Intended usage" header should say "LINITED USE").

Note that none of these exotic headers existed in RFC 2048, which was the
relevant standard at the time I first wrote these templates.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24861
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: ABNF nits
Date: Mon, 25 Aug 2008 13:12:31 GMT
Lines: 23
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>I should probably upload another draft, but I'll give it a day or two for
>people to turn up any additional concerns that should be addressed in the
>next draft.  I will put out another version by the 28th, after which I
>will be unavailable to do working-group work for a week or two.

I am sure that there will be lots of niggles once people get a chance to
read the latest draft (it is a long time since we had an up-to-date
complete draft to consider as a whole). So, rather than rushing to produce
instant drafts every couple of daye, I would suggest planning on doing it
when your "week or two" is over.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24860
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Potential ISSUE: <utext> (was: ABNF nits)
Date: Mon, 25 Aug 2008 13:09:16 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> "Frank Ellermann" <[email-masked]> writes:

>Whatever the WG decided in 2007, the 2822upd modifications
>wrt <utext> and NO-WS-CTL are new (2008) events, based on
>feedback from Charles, you, and me (among others) on the
>rfc822 list.

Yes, we managed to get more changed in RFC 2822-bis than we could ever
have expected.

>Nobody here could foresee that there will be no <utext> in
>2822upd before the author pulled it.  Technically this is
>no problem, because you reference 2822, not 2822upd.  But
>my confidence that readers can figure this detail out is
>limited.  I changed the subject to attract the attention
>of Harald and Alexey. 

I still think that, now 2822bis has been accepted by the IETF and will
presumably become a Draft Standard, we should revisit USEFOR and rebase it
on 2822-bis (firmly resiting all temptations to make any technical
changes). It should be quite straightforward, and if Ken is not
able/willing to do it, then I will volunteer. Ken?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24858
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Commit in docs/usefor (usepro.xml)
Date: Mon, 25 Aug 2008 11:14:59 GMT
Lines: 44
Sender: [email-masked]

In <[email-masked]> "Frank Ellermann" <[email-masked]> writes:

>The version control bot wrote:

>> +        <seriesInfo name="Internet-Draft"
>> +          value="draft-ietf-usefor-useage-01" />

>That is the variant automatically interpreted as
>"work in progress" by xml2rfc.  If that is *not*
>what you want strike the "-Draft" in seriesInfo.

useage-01 is now way behind our current drafts, so I need to do another
draft sooner or later.

Formally speaking, the trio of UEFEFOR + USEPOR + USEAGE is still the
agreed program of work for this WG, but it is some years since that was
last set down in black and white.

But do remember that some famous battles in the past (such as what to do
about 'Re:' in the Subject header) were only resolved on the basis of a
split of the text between USEPRO and USEAGE, and the USEAGE draft
incorprates what was agreed then.

So what we actually do with USEAGE is a thing we need to decide when we
are agreed about USEPRO (which now seems to be in sight).

Possible options are:

1. Go ahead with USEAGE as originally planned
2. Leave me to continue it as a private effort
3. Forget it entirely

But we don't need to decide that now. It can wait a week or three.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Subject: Re: ABNF nits
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sun, 24 Aug 2008 21:17:57 -0700
Sender: [email-masked]

"Alexey Melnikov" <[email-masked]> writes:

>> There are only two references to USEAGE in the document (3.8 and 3.9).
>> Personally, I think they're expendible.  If USEAGE isn't going to be
>> published, I think the best resolution is to just remove them.  I'd
>> rather not cite an I-D in a published RFC, and I suspect the RFC Editor
>> would have similar qualms.

> See my reply to Frank's message.

Okay, I can leave it in.

>> I can also include the template in the document easily enough if the
>> working group thinks that's the right thing to do.  Does anyone else
>> have an opinion?

> I think that is the right thing to do.

Now included.

I should probably upload another draft, but I'll give it a day or two for
people to turn up any additional concerns that should be addressed in the
next draft.  I will put out another version by the 28th, after which I
will be unavailable to do working-group work for a week or two.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sun, 24 Aug 2008 21:11:18 -0700 (PDT)
Sender: [email-masked]

    Date: Sunday, August 24, 2008 @ 21:11:17
  Author: eagle
Revision: 4918

<path-delimiter> is not an ABNF rule used anywhere.  USEFOR uses
literals.  Change <path-delimiter> to "path delimiter".

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-25 04:10:24 UTC (rev 4917)
+++ docs/usefor/usepro.xml	2008-08-25 04:11:17 UTC (rev 4918)
@@ -464,7 +464,7 @@
 
           <t>old.site.example relayed it to a news server using the
           &lt;path-identity> of bar.isp.example and claiming (by using the
-          "!!" &lt;path-delimiter>) to have verified that it came from
+          "!!" path delimiter) to have verified that it came from
           old.site.example.</t>
 
           <t>bar.isp.example relayed it to foo-news which, not being

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sun, 24 Aug 2008 21:10:25 -0700 (PDT)
Sender: [email-masked]

    Date: Sunday, August 24, 2008 @ 21:10:24
  Author: eagle
Revision: 4917

Add references for ABNF rules used here but defined elsewhere.

Fix one mistaken marking of a Media Type registration as ABNF.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-25 04:04:58 UTC (rev 4916)
+++ docs/usefor/usepro.xml	2008-08-25 04:10:24 UTC (rev 4917)
@@ -136,6 +136,40 @@
         Backus-Naur Form (ABNF) notation (including the Core Rules)
         defined in <xref target="RFC5234" /> and constructs defined in
         <xref target="USEFOR" /> and <xref target="RFC2822" />.</t>
+
+        <figure>
+          <preamble>The ABNF rules defined elsewhere and used in this
+          document are:</preamble>
+
+          <artwork type="abnf"><![CDATA[
+      CRLF                = <see [RFC5234] appendix B.1>
+      DIGIT               = <see [RFC5234] appendix B.1> 
+      HTAB                = <see [RFC5234] appendix B.1>
+      SP                  = <see [RFC5234] appendix B.1>
+      WSP                 = <see [RFC5234] appendix B.1>
+
+      argument            = <see [USEFOR] section 3.2.3>
+      article-locator     = <see [USEFOR] section 3.2.14> 
+      component           = <see [USEFOR] section 3.1.4>
+      control-command     = <see [USEFOR] section 3.2.3>
+      diag-keyword        = <see [USEFOR] section 3.1.5>
+      diag-match          = <see [USEFOR] section 3.1.5>
+      diag-other          = <see [USEFOR] section 3.1.5> 
+      dist-name           = <see [USEFOR] section 3.2.4>
+      msg-id              = <see [USEFOR] section 3.1.3>
+      newsgroup-name      = <see [USEFOR] section 3.1.4>
+      path-diagnostic     = <see [USEFOR] section 3.1.5>
+      path-identity       = <see [USEFOR] section 3.1.5>
+      path-nodot          = <see [USEFOR] section 3.1.5>
+      tail-entry          = <see [USEFOR] section 3.1.5> 
+      verb                = <see [USEFOR] section 3.2.3>
+
+      display-name        = <see [RFC2822] section 3.4>
+      local-part          = <see [RFC2822] section 3.4.1>
+      mailbox             = <see [RFC2822] section 3.4>
+      utext               = <see [RFC2822] section 3.2.6>
+]]></artwork>
+        </figure>
       </section>
 
       <section title="Definitions">
@@ -1581,7 +1615,7 @@
           <preamble>The MIME media type definition of
           application/news-checkgroups is:</preamble>
 
-          <artwork type="abnf">
+          <artwork>
    MIME type name:           application
    MIME subtype name:        news-checkgroups
    Required parameters:      none

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sun, 24 Aug 2008 21:04:59 -0700 (PDT)
Sender: [email-masked]

    Date: Sunday, August 24, 2008 @ 21:04:58
  Author: eagle
Revision: 4916

Mark ABNF artwork as such.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-25 04:02:48 UTC (rev 4915)
+++ docs/usefor/usepro.xml	2008-08-25 04:04:58 UTC (rev 4916)
@@ -1529,7 +1529,7 @@
           <preamble>The content of the application/news-groupinfo body
           part is defined as:</preamble>
 
-          <artwork>
+          <artwork type="abnf">
       groupinfo-body      = [ newsgroups-tag CRLF ]
                                newsgroups-line CRLF
       newsgroups-tag      = %x46.6F.72 SP %x79.6F.75.72 SP
@@ -1581,7 +1581,7 @@
           <preamble>The MIME media type definition of
           application/news-checkgroups is:</preamble>
 
-          <artwork>
+          <artwork type="abnf">
    MIME type name:           application
    MIME subtype name:        news-checkgroups
    Required parameters:      none
@@ -1617,7 +1617,7 @@
           <preamble>The content of the application/news-groupinfo body
           part is defined as:</preamble>
 
-          <artwork>
+          <artwork type="abnf">
       checkgroups-body    = *( valid-group CRLF )
       valid-group         = newsgroups-line ; see 4.2
           </artwork>
@@ -1737,7 +1737,7 @@
           is:</t>
 
           <figure>
-            <artwork>
+            <artwork type="abnf">
       control-command     =/ Newgroup-command
       Newgroup-command    = "newgroup" Newgroup-arguments
       Newgroup-arguments  = 1*WSP newsgroup-name
@@ -1820,7 +1820,7 @@
           of its Control header field is:</t>
 
           <figure>
-            <artwork>
+            <artwork type="abnf">
       control-command     =/ Rmgroup-command
       Rmgroup-command     = "rmgroup" Rmgroup-arguments
       Rmgroup-arguments   = 1*WSP newsgroup-name
@@ -1841,7 +1841,7 @@
           its Control header field is:</t>
 
           <figure>
-            <artwork>
+            <artwork type="abnf">
       control-command     =/ Checkgroup-command
       Checkgroup-command  = "checkgroups" Checkgroup-arguments
       Checkgroup-arguments= [ chkscope ] [ chksernr ]
@@ -1903,7 +1903,7 @@
         header field is:</t>
 
         <figure>
-          <artwork>
+          <artwork type="abnf">
       control-command     =/ Cancel-command
       Cancel-command      = "cancel" Cancel-arguments
       Cancel-arguments    = 1*WSP msg-id
@@ -1962,7 +1962,7 @@
           <preamble>ihave and sendme control messages share similar syntax
           for their Control header fields and bodies:</preamble>
 
-          <artwork>
+          <artwork type="abnf">
       control-command     =/ Ihave-command
       Ihave-command       = "ihave" Ihave-arguments
       Ihave-arguments     = 1*WSP *( msg-id 1*WSP ) relayer-name

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sun, 24 Aug 2008 21:02:49 -0700 (PDT)
Sender: [email-masked]

    Date: Sunday, August 24, 2008 @ 21:02:48
  Author: eagle
Revision: 4915

Grammatical fix for the sendme control message reference in the
Security Considerations section.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-25 04:01:58 UTC (rev 4914)
+++ docs/usefor/usepro.xml	2008-08-25 04:02:48 UTC (rev 4915)
@@ -2143,8 +2143,8 @@
         advised to designate a small number of gateways to handle all news
         exchange with the outside world.</t>
 
-        <t>The sendme control message <xref target="sendme" />, insofar as
-        it is still used, can be used to request articles the requester
+        <t>The sendme control message (<xref target="sendme" />), insofar
+        as it is still used, can be used to request articles the requester
         should not have access to.</t>
       </section>
     </section>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sun, 24 Aug 2008 21:01:59 -0700 (PDT)
Sender: [email-masked]

    Date: Sunday, August 24, 2008 @ 21:01:58
  Author: eagle
Revision: 4914

Media Type registration cleanup.

Add a registration template for message/news as part of declaring it
obsolete.  Move Disposition to Implementation considerations.  Add
Applications that use this media type, Intended usage, and Change
controller to the other media type registrations.

Add a reference for son-of-1036 for the message/news registration and
cite that reference in the Acknowledgements section.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-24 20:58:03 UTC (rev 4913)
+++ docs/usefor/usepro.xml	2008-08-25 04:01:58 UTC (rev 4914)
@@ -1412,6 +1412,30 @@
       media type message/rfc822 (defined in <xref target="RFC2046" />,
       section 5.2.1) SHOULD be used in its place.</t>
 
+      <figure>
+        <preamble>The updated MIME media type definition of message/news
+        is:</preamble>
+
+        <artwork>
+   MIME type name:           message
+   MIME subtype name:        news
+   Required parameters:      none
+   Optional parameters:      charset, compare message/rfc822
+   Disposition:              by default, inline
+   Encoding considerations:  compare message/rfc822 (RFC 2046)
+   Security considerations:  OBSOLETE, use message/rfc822.
+   Interoperability considerations:
+                             Rarely used; and therefore often
+                             handled as application/octet-stream
+   Published specification:  [SON-OF-1036], RFC&rfc.number;
+   Applications that use this media type:
+                             None known.
+   Intended usage:           OBSOLETE
+   Author:                   Henry Spencer 
+   Change controller:        IETF
+        </artwork>
+      </figure>
+
       <section anchor="transmission" title="application/news-transmission">
         <t>The media type application/news-transmission is intended for
         the encapsulation of complete news articles where the intention is
@@ -1446,6 +1470,10 @@
    Body part:                A complete proto-article ready for
                              injection into Netnews or an article
                              being relayed to another agent.
+   Applications that use this media type:
+                             Injecting agents, Netnews moderators.
+   Intended usage:           COMMON
+   Change controller:        IETF
           </artwork>
         </figure>
 
@@ -1483,11 +1511,17 @@
                              Specifies the charset of the body part.
                              If not given, the charset defaults to
                              US-ASCII [ASCII].
-   Disposition:              by default, inline
    Encoding considerations:  7bit or 8bit MUST be used to maintain
                              compatibility. 
    Security considerations:  None.
+   Implementation considerations:
+                             Disposition should by default be inline.
+   Applications that use this media type:
+                             Control message issuers, relaying
+                             agents, serving agents.
    Published specification:  RFC&rfc.number;
+   Intended usage:           COMMON
+   Change controller:        IETF
           </artwork>
         </figure>
 
@@ -1558,7 +1592,6 @@
                              Specifies the charset of the body part.
                              If not given, the charset defaults to
                              US-ASCII [ASCII].
-   Disposition:              by default, inline
    Encoding considerations:  7bit or 8bit MUST be used to maintain
                              compatibility. 
    Security considerations:  This media type provides only a means
@@ -1569,7 +1602,14 @@
                              should exist within a hierarchy.  Such
                              authorization must be accomplished by
                              other means.
+   Implementation considerations:
+                             Disposition should by default be inline.
+   Applications that use this media type:
+                             Control message issuers, relaying
+                             agents, serving agents.
    Published specification:  RFC&rfc.number;
+   Intended usage:           COMMON
+   Change controller:        IETF
           </artwork>
         </figure>
 
@@ -2127,7 +2167,8 @@
       registration.</t>
 
       <t>IANA is also requested to change the status of the message/news
-      media type to "OBSOLETE".  message/rfc822 should be used instead.</t>
+      media type to "OBSOLETE".  message/rfc822 should be used instead.
+      An updated template is included in <xref target="media" />.</t>
     </section>
   </middle>
 
@@ -2179,6 +2220,16 @@
       &rfc2606;
       &rfc3798;
       &rfc3977;
+      <reference anchor="SON-OF-1036">
+        <front>
+          <title>News Article Format and Transmission</title>
+          <author initials="H." surname="Spencer"
+            fullname="Henry Spencer">
+            <organization>SP Systems</organization>
+          </author>
+          <date month="June" year="1994" />
+        </front>
+      </reference>
       <reference anchor="USEAGE"
         target="http://tools.ietf.org/html/draft-ietf-usefor-useage-01">
         <front>
@@ -2258,8 +2309,9 @@
       members of the USEFOR Working Group of the Internet Engineering Task
       Force (IETF) and the accompanying mailing list.</t>
 
-      <t>Special thanks are due to Henry Spencer, whose Son-of-1036 draft
-      served as the initial basis for this document.</t>
+      <t>Special thanks are due to Henry Spencer, whose <xref
+      target="SON-OF-1036" /> draft served as the initial basis for this
+      document.</t>
     </section>
   </back>
 </rfc>

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  MIME subtype review requests (was: ABNF nits)
Date:  Sun, 24 Aug 2008 19:36:58 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 23
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Alexey Melnikov wrote:

> I am happy for you to submit the review request
> on behalf of the WG.

Done for message/news, where I know the background.

For the application/news-* registrations it's better
when they're first fixed, and when somebody posts a
review request who could answer questions about the
application/news-* details. =20

>> I can also include the template in the document
>> easily enough if the working group thinks that's
>> the right thing to do.  Does anyone else have an
>> opinion?
=20
> I think that is the right thing to do.

+1   Of course it makes the text longer, but 4 or 3
templates is no big difference (less than one page).

 Frank

Date: Sun, 24 Aug 2008 07:08:48 -0700
From: "Alexey Melnikov" <[email-masked]>
To: "Russ Allbery" <[email-masked]>
Subject: Re: ABNF nits
Cc: [email-masked]
Sender: [email-masked]

>>> If it's not going to be published, we need to strip all
>>> references to USEAGE in this document.
>>
>> ...not required, and USEAGE won't be published anytime soon.
>>
>> IMO nothing is wrong with an informative reference to prior
>> related work.  But saying that it's "work in progress" when
>> that's not the case (at the moment) could be bad.  The folks
>> at rfc-editor.org could decide to treat USEAGE as MISSREF...
>
> There are only two references to USEAGE in the document (3.8 and 3.9).
> Personally, I think they're expendible.  If USEAGE isn't going to be
> published, I think the best resolution is to just remove them.  I'd rather
> not cite an I-D in a published RFC, and I suspect the RFC Editor would
> have similar qualms.

See my reply to Frank's message.

>>  [review requests]
>>> I don't have time to do this at present and can make no
>>> promises as to when I will have time to do so.
>>
>> Okay, sometimes the WG Chairs or the responsible AD do this
>> for WG drafts.  But doing it too late is sub-optimal, after
>> all a review request *could* trigger a no-nonsense review.
>
> I think it's definitely a good thing to do.

I think it is not only a good idea, but it is also a require step.

>  I just won't be able to do it
> soon.  I might be able to do it towards the end of September; failing
> that, I won't have a free block of time until November.  Unless there's
> likely to be no discussion other than just the message, which of course is
> simple, but usually that's not true of public IETF review.
>
>> Clearly I think it is worthwhile, because IANA cannot do much
>> to "obsolete" message/news without a template.  All they can
>> do without template is update their 1036 reference to USEPRO.
>
> That's a good point.
>
>> But it might be possible to submit the template individually,
>> should I try this ?  In that case I'd be at least confident
>> about posting a review request "on behalf of the USEFOR WG",
>> e.g. I'd know that I can't say "on behalf of".
>
> I'm very happy to either support that.

Frank, speaking as a chair I am happy for you to submit the review
request on behalf of the WG.

> I can also include the template in
> the document easily enough if the working group thinks that's the right
> thing to do.  Does anyone else have an opinion?

I think that is the right thing to do.

Date: Sun, 24 Aug 2008 07:04:53 -0700
From: "Alexey Melnikov" <[email-masked]>
To: "Frank Ellermann" <[email-masked]>
Subject: Re: ABNF nits
Cc: [email-masked]
Sender: [email-masked]

>> The status of USEAGE needs to be resolved before publishing
>> the document.
>
> The WG will be terminated, and the USEAGE reference is only
> informative...

Agreed.

>> If it's not going to be published, we need to strip all
>> references to USEAGE in this document.
>
> ...not required, and USEAGE won't be published anytime soon.

Indeed. An informative reference is not going to block publication of
the document.

> IMO nothing is wrong with an informative reference to prior
> related work.

IMHO, it is much better having such references in documents. This make
it much easier for people to find history of the document or related
documents. And if one google for the USEAGE document, one would find
the document long after it expires.

> But saying that it's "work in progress" when
> that's not the case (at the moment) could be bad.

An Internet draft has limited lifetime (6 months), so saying that is
is work in progress is appropriate (i.e. the work is not finished and
the document is not an RFC).

> The folks at rfc-editor.org could decide to treat USEAGE as MISSREF...

No, they wouldn't, unless you explicitly ask them to delay publication
till the informative reference is published as an RFC. They publish
RFCs with informative references to drafts all the time.

> [Actually I think they can't simply decide this, and that's
>  the core idea of "normative" vs. "informative" references.]

To: [email-masked]
Subject: Re: Import patch
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sat, 23 Aug 2008 22:54:03 -0700
Sender: [email-masked]

"Frank Ellermann" <[email-masked]> writes:

> I have no patch tool yet on the W2K box, but I can prepare an XML
> snippet for copy and paste, you can insert it where it fits.

This is perfect -- thank you very much!  I'll incorporate this probably
tomorrow.  Apologies for the misunderstanding originally, and thank you
for talking me through it.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Import patch (was: Potential ISSUE: <utext>)
Date:  Sun, 24 Aug 2008 06:37:16 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 59
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Russ Allbery wrote:

> since you've already tracked most of them down, I think I'm not
> adverse, and mechanical validation of ABNF is definitely useful.
=20
> Would you be willing to prepare a patch?  That would be even
> faster for me.

I have no patch tool yet on the W2K box, but I can prepare an XML
snippet for copy and paste, you can insert it where it fits.

Style counter, you start in column 10 with the "=3D" in column 30:
         newsgroups-line     =3D newsgroup-name
....5...10....5...20....5...30....5...40....5...50....5...60....5...70..
~~~ cut ~~~
<figure><artwork type=3D"abnf"><![CDATA[
         CRLF                =3D <see RFC 5234 appendix B.1>
         DIGIT               =3D <see RFC 5234 appendix B.1>=20
         HTAB                =3D <see RFC 5234 appendix B.1>
         SP                  =3D <see RFC 5234 appendix B.1>
         WSP                 =3D <see RFC 5234 appendix B.1>

         argument            =3D <see [USEFOR] section 3.2.3>
         article-locator     =3D <see [USEFOR] section 3.2.14>=20
         component           =3D <see [USEFOR] section 3.1.4>
         control-command     =3D <see [USEFOR] section 3.2.3>
         diag-keyword        =3D <see [USEFOR] section 3.1.5>
         diag-match          =3D <see [USEFOR] section 3.1.5>
         diag-other          =3D <see [USEFOR] section 3.1.5>=20
         dist-name           =3D <see [USEFOR] section 3.2.4>
         msg-id              =3D <see [USEFOR] section 3.1.3>
         newsgroup-name      =3D <see [USEFOR] section 3.1.4>
         path-delimiter      =3D <see [USEFOR] **KERNEL PANIC**>
         path-diagnostic     =3D <see [USEFOR] section 3.1.5>
         path-identity       =3D <see [USEFOR] section 3.1.5>
         path-nodot          =3D <see [USEFOR] section 3.1.5>
         tail-entry          =3D <see [USEFOR] section 3.1.5>=20
         verb                =3D <see [USEFOR] section 3.2.3>

         display-name        =3D <see RFC 2822 section 3.4>
         local-part          =3D <see RFC 2822 section 3.4.1>
         mailbox             =3D <see RFC 2822 section 3.4>
         utext               =3D <see RFC 2822 section 3.2.6>
]]></artwork></figure>
~~~ end ~~~

Strike <path-delimiter>, both memos use this only in prose.=20

BTW, if you are annoyed how xml2rfc splits some ABNF terms
in prose at the hyphen write &nbhy; (=3D no break hyphen).

Integration test, I grab this stuff and paste it in the
still open ABNF parser form, result:  No errors, good.

You'll get more than four allegedly unused ABNF terms now,
because I added all terms in angle-brackets in the prose.
But that's as it should be, ignore it.

 Frank

To: [email-masked]
Subject: Re: Potential ISSUE: <utext>
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sat, 23 Aug 2008 20:15:16 -0700
Sender: [email-masked]

"Frank Ellermann" <[email-masked]> writes:

> The "prose ABNF" was not invented here, it is pure STD 68.
> Maybe this WG invented the idea of using "prose ABNF" for
> ABNF imports, but IIRC Bill Fenner had the idea before us.

Oh, I see my mistake.  I thought the : that you were putting on those
lines was part of your suggested syntax, but you're just using it as a
quoting character.  I couldn't find that syntax in STD 68.

I'm sorry about that; that was my confusion.

So you think that we should put somewhere a list of all the ABNF labels
that we use from other documents into the introduction, probably in the
Syntax section.  Hm.  Well, since you've already tracked most of them
down, I think I'm not adverse, and mechanical validation of ABNF is
definitely useful.

Would you be willing to prepare a patch?  That would be even faster for
me.

>> I'd rather not cite an I-D in a published RFC, and I suspect the RFC
>> Editor would have similar qualms.

> Dunno - if they'd ask me to remove s-o-1036 or the Gilman draft
> references I won't comply.  But that is a different case, these drafts
> are very obviously "historic" (and no IETF WG drafts).

Yes, Son-of is something of a special case.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Potential ISSUE: <utext> (was: ABNF nits)
Date:  Sun, 24 Aug 2008 04:58:42 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 94
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Russ Allbery wrote:
=20
> I'm sympathetic that this might be interesting in general
> for the IETF, but in the absence of a general IETF policy
> or some standard about how this is supposed to be done, I
> think this is just about the worst possible working group
> to be innovating in how we present ABNF.

The "prose ABNF" was not invented here, it is pure STD 68.
Maybe this WG invented the idea of using "prose ABNF" for
ABNF imports, but IIRC Bill Fenner had the idea before us.

BTW, this WG was the first to adopt the RFC 3696 <toplabel>
concept in a standards track RFC.  IMO that was innovative,
the rest of the IETF still uses RFC 1123 2.1 (officially).

Harald even asked IAB and ICANN what they think about this
innovation.  RFC.usefor supported IDNA TLDs *before* ICANN,
one of those procedural stunts I love. =20

> Is there something like that available, such as
> documentation of the import syntax that you're using, in
> an RFC or a guideline for ABNF usage somewhere?

STD 68.  You can simply test it with Bill's ABNF validator:

| No errors during parsing.
[...]
| foo =3D <as specified in RFC 3092>
| bar =3D foo [ foo ]
| ; bar defined but not used

 [Expand <utext> because it's not defined in 2822upd]
> I'm going to decline to discuss this issue unless the
> chairs determine that it should be reopened.

My proposed <utext> expansion was wrong, it included WSP.
The "real thing" (adding 8bit, but removing obs- and WSP):

eightbit-utext =3D %d1-8 / %d11-12 / %d14-31 / %d33-255

<eightbit-utext> is used in exactly one production:

newsgroup-description =3D eightbit-utext *( *WSP eightbit-utext )

Renaming it to <eightbit-mime> is a matter of taste.  But
using the term <utext> after it was removed from 2822upd,
and while it contains NO-WS-CTL declared to be obsolete in
2822upd, is IMO a seriously confusing idea.

Whatever the WG decided in 2007, the 2822upd modifications
wrt <utext> and NO-WS-CTL are new (2008) events, based on
feedback from Charles, you, and me (among others) on the
rfc822 list.

Nobody here could foresee that there will be no <utext> in
2822upd before the author pulled it.  Technically this is
no problem, because you reference 2822, not 2822upd.  But
my confidence that readers can figure this detail out is
limited.  I changed the subject to attract the attention
of Harald and Alexey.=20

> I'd rather not cite an I-D in a published RFC, and I
> suspect the RFC Editor would have similar qualms.

Dunno - if they'd ask me to remove s-o-1036 or the Gilman
draft references I won't comply.  But that is a different
case, these drafts are very obviously "historic" (and no
IETF WG drafts).

 [message/news]=20
> I'm very happy to either support that.  I can also include
> the template in the document easily enough if the working
> group thinks that's the right thing to do.  Does anyone
> else have an opinion?

Okay, in both cases the template needs a review request, I'm
going to post that as proposed earlier...  One of four down.

> I think it's worthwhile to explicitly deprecate it in the
> text of a published RFC, personally, but maybe I'm too
> anal about that.

Sure.  Your "out-sourcing" idea was about the template, not
the place of the deprecation (=3D as is, RFCXXXX).  You said
the template is too long... <shrug /> =20

Maybe note the timestamp (today) of this review request in
the document history - I try to note all relevant facts for
a later writeup in the document history (so far for anal ;-)=20

 Frank
--=20
<http://article.gmane.org/gmane.ietf.types/1080>

To: [email-masked]
Subject: Re: ABNF nits
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sat, 23 Aug 2008 15:16:01 -0700
Sender: [email-masked]

"Frank Ellermann" <[email-masked]> writes:
> Russ Allbery wrote:

>> The ABNF in USEPRO is additive to the ABNF in USEFOR and the
>> Core Rules defined in RFC5234 as noted in 1.4.

> Tools like the ABNF extractor don't know this, I think it's a
> good idea to have an "import interface" also for human users.

I'm sympathetic that this might be interesting in general for the IETF,
but in the absence of a general IETF policy or some standard about how
this is supposed to be done, I think this is just about the worst possible
working group to be innovating in how we present ABNF.  What we're doing
right now seems to be pretty much in line with what other RFCs are doing,
although I may have missed recent changes.

Is there something like that available, such as documentation of the
import syntax that you're using, in an RFC or a guideline for ABNF usage
somewhere?

> I looked only at the ABNF, references, and stuff related to
> the IANA templates.  SHOULD ASCII and otherwise SHOULD UTF-8
> makes sense.  Checking the <utext> again I think you are not
> talking about some "Unicode text", but about "unstructured
> text" <utext> in RFC 2822 imported via RFC.ietf-usefor-usefor:
>
> | utext   =       NO-WS-CTL /     ; Non white space controls
> |                  %d33-126 /     ; The rest of US-ASCII
> |                 obs-utext
>
> Or actually about the non-obsolete part of RFC 2822 <utext>.

I'm sorry to be uninformative here, but the points you are raising are
points previously discussed in the working group on an issue that was
closed.  In the interests of trying to shut this down, I'm going to
decline to discuss this issue unless the chairs determine that it should
be reopened.

> I thought that you intend to keep [USEFOR] and [USEAGE] in the
> prose.  If that's not the case forget my idea - it makes only
> sense when you'd update [USEFOR] or rather RFCnnnn only in the
> syntax, keeping [USEFOR] elsewhere as is.

Ah, yes, the [USEFOR] string will definitely change.  [USEAGE] is, from my
perspective, rather up in the air.

>> If it's not going to be published, we need to strip all 
>> references to USEAGE in this document.
>
> ...not required, and USEAGE won't be published anytime soon.
>
> IMO nothing is wrong with an informative reference to prior
> related work.  But saying that it's "work in progress" when
> that's not the case (at the moment) could be bad.  The folks
> at rfc-editor.org could decide to treat USEAGE as MISSREF...

There are only two references to USEAGE in the document (3.8 and 3.9).
Personally, I think they're expendible.  If USEAGE isn't going to be
published, I think the best resolution is to just remove them.  I'd rather
not cite an I-D in a published RFC, and I suspect the RFC Editor would
have similar qualms.

>  [review requests]
>> I don't have time to do this at present and can make no
>> promises as to when I will have time to do so.
>
> Okay, sometimes the WG Chairs or the responsible AD do this
> for WG drafts.  But doing it too late is sub-optimal, after
> all a review request *could* trigger a no-nonsense review.

I think it's definitely a good thing to do.  I just won't be able to do it
soon.  I might be able to do it towards the end of September; failing
that, I won't have a free block of time until November.  Unless there's
likely to be no discussion other than just the message, which of course is
simple, but usually that's not true of public IETF review.

> Clearly I think it is worthwhile, because IANA cannot do much
> to "obsolete" message/news without a template.  All they can
> do without template is update their 1036 reference to USEPRO.

That's a good point.

> But it might be possible to submit the template individually,
> should I try this ?  In that case I'd be at least confident 
> about posting a review request "on behalf of the USEFOR WG",
> e.g. I'd know that I can't say "on behalf of".

I'm very happy to either support that.  I can also include the template in
the document easily enough if the working group thinks that's the right
thing to do.  Does anyone else have an opinion?

>> I didn't; I just didn't change it.  This has been in the
>> draft for quite a while.  Should I remove it?  I don't see
>> Disposition anywhere in the media type registration either.
>
> You could add an "Interoperability considerations" field, and
> put the "default disposition inline" into this field.  Maybe
> Charles still knows where he found this ?  (= not in RFC 4288)

Charles, do you remember?

> For "message/news" I'd remove it as not more relevant if we
> pick the "make it so" out-sourced IANA registration approach.

I think it's worthwhile to explicitly deprecate it in the text of a
published RFC, personally, but maybe I'm too anal about that.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: ABNF nits
Date:  Sat, 23 Aug 2008 23:43:29 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 171
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Russ Allbery wrote:

> The ABNF in USEPRO is additive to the ABNF in USEFOR and the
> Core Rules defined in RFC5234 as noted in 1.4.

Tools like the ABNF extractor don't know this, I think it's a
good idea to have an "import interface" also for human users.

The <control-command> continuation based on a start in USEFOR
is not obvious for folks not familiar with the details.  As
in say "IESG members, GenArt reviewers, or RFC-editor staff",
who might all try to check the ABNF "as is". =20

I did *not* check what happens if the USEFOR + USEPRO syntax
is simply aggregated, maybe that is good enough.

 [utext]
> No, I mean what the text says.
[...]

I looked only at the ABNF, references, and stuff related to
the IANA templates.  SHOULD ASCII and otherwise SHOULD UTF-8
makes sense.  Checking the <utext> again I think you are not
talking about some "Unicode text", but about "unstructured
text" <utext> in RFC 2822 imported via RFC.ietf-usefor-usefor:

| utext   =3D       NO-WS-CTL /     ; Non white space controls
|                  %d33-126 /     ; The rest of US-ASCII
|                 obs-utext

Or actually about the non-obsolete part of RFC 2822 <utext>.

>From my POV that's not intuitive, especially because 2822upd
has no <utext> at all.  I propose to spell out what you want
here, in essence any MIME-compatible octet, that is not NUL,
and lots of fine print for CR LF.  You could simply write:

:  eightbit-utext =3D %d1-9 / %d11-12 / %d14-255

The name "eightbit-utext" is arguably confusing, at least it
managed to confuse me.  Please use <eightbit-mime> for this,
it would also reflect the "NOTE" at the begin of chapter 2.

>| Regardless of the charset used, the constraints of the
>| above grammar MUST be met and the <newsgroup-name> MUST
>| be represented in that charset using the same octets as
>| would be used with a charset of US-ASCII.

I don't understand the last part "using the same octets as
would be used with a charset of US-ASCII".  What does this
mean ? =20

Let's say the newsgroup-name is example.=F6ko with an umlauted
o.  There are many ways to limit this to ASCII, oe, &ouml;,
percent-encoded UTF-8, numeric character references (NCRs,
i.e. Unicode points), IDNA punycode (but that won't fly for
some non-LDH ASCII group characters), RFC 2047, and more. =20

>> Technical detail, I omitted %d127 (DEL), because it is not
>> obviously needed.  Please spice it with a RFC 5198
>> reference
[...]
=20
> Likewise here.  I don't care one way or the other about DEL,
> but making substantive changes at this phase is just asking
> for trouble.  I'm willing to drop it if there's clear=20
> consensus and the chair okays making that change, but not
> otherwise, since we're trying to finish.

FWIW all but one %d127 uses in 2821bis drafts were removed.
We are quibbling about a non-issue, I didn't understand what
<utext> is.  See above for a better name and clearer syntax.

>> : msg-id                =3D <see [USEFOR] section 3.1.3>
=20
> I don't think it's necessary to note everywhere that we
> reference msg-id that we mean the version defined in USEFOR.

Agreed, I also don't think that it is necessary to note this
fact *everywhere*.  I proposed to note this *once* in syntax,
in an "import interface".  In prose you have it already, but
I didn't look at your prose for my ABNF validation.

> [USEFOR] is a much simpler search string to use to find all
> instances that need to be corrected once the other draft has
> been published.

I thought that you intend to keep [USEFOR] and [USEAGE] in the
prose.  If that's not the case forget my idea - it makes only
sense when you'd update [USEFOR] or rather RFCnnnn only in the
syntax, keeping [USEFOR] elsewhere as is.

Just in case, the drafts will be published simultaneously, they
have bidirectional normative references.

  [<verb> e.a.]
>> When you add an "ABNF import interface" in chapter 1 please
>> import also those terms while you are at it.
=20
> I don't, at present, intend to add an import interface.

I recently added an import interface to a draft that needs no
syntax at all.  After that I finally began to understand what
this draft is about (=3D the other "u", not as in <utext>, but
as in U-label).

> The status of USEAGE needs to be resolved before publishing
> the document.

The WG will be terminated, and the USEAGE reference is only
informative... =20

> If it's not going to be published, we need to strip all=20
> references to USEAGE in this document.

...not required, and USEAGE won't be published anytime soon.

IMO nothing is wrong with an informative reference to prior
related work.  But saying that it's "work in progress" when
that's not the case (at the moment) could be bad.  The folks
at rfc-editor.org could decide to treat USEAGE as MISSREF...

[Actually I think they can't simply decide this, and that's
 the core idea of "normative" vs. "informative" references.]

> I didn't think that 2822upd breaks anything that wasn't=20
> already broken. What are you worried about having break
> here?

The <utext> detail is interesting.  NO-WS-CTL was firmly
moved to "obs-cene".  IIRC no more <dcontent> (that was a
nit in 2821bis, no nit in usepro).  Various minor fixes
related to old 822 #-lists (all "obs" =3D> irrelevant here).

 [review requests]
> I don't have time to do this at present and can make no
> promises as to when I will have time to do so.

Okay, sometimes the WG Chairs or the responsible AD do this
for WG drafts.  But doing it too late is sub-optimal, after
all a review request *could* trigger a no-nonsense review.

 [message/news template]
> I'm inclined not to bother with this unless people feel it
> would be worthwhile.  It makes the draft longer and I don't
> think anyone's going to care.

Clearly I think it is worthwhile, because IANA cannot do much
to "obsolete" message/news without a template.  All they can
do without template is update their 1036 reference to USEPRO.

But it might be possible to submit the template individually,
should I try this ?  In that case I'd be at least confident=20
about posting a review request "on behalf of the USEFOR WG",
e.g. I'd know that I can't say "on behalf of".

>> BTW, where did you find "Disposition" in RFC 4288 ?  It is
>> a good idea, but maybe it belongs to the "Interoperability".
=20
> I didn't; I just didn't change it.  This has been in the
> draft for quite a while.  Should I remove it?  I don't see
> Disposition anywhere in the media type registration either.

You could add an "Interoperability considerations" field, and
put the "default disposition inline" into this field.  Maybe
Charles still knows where he found this ?  (=3D not in RFC 4288)

For "message/news" I'd remove it as not more relevant if we
pick the "make it so" out-sourced IANA registration approach.

 Frank

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sat, 23 Aug 2008 12:18:36 -0700 (PDT)
Sender: [email-masked]

    Date: Saturday, August 23, 2008 @ 12:18:34
  Author: eagle
Revision: 4901

Add an explicit section reference to RFC 2046.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-23 18:23:59 UTC (rev 4900)
+++ docs/usefor/usepro.xml	2008-08-23 19:18:34 UTC (rev 4901)
@@ -1409,8 +1409,8 @@
       IANA, is hereby declared obsolete.  It was never widely
       implemented, and its default treatment as application/octet-stream
       by agents that did not recognize it was counter-productive.  The
-      media type message/rfc822 (defined in <xref target="RFC2046" />)
-      SHOULD be used in its place.</t>
+      media type message/rfc822 (defined in <xref target="RFC2046" />,
+      section 5.2.1) SHOULD be used in its place.</t>
 
       <section anchor="transmission" title="application/news-transmission">
         <t>The media type application/news-transmission is intended for

To: [email-masked]
Subject: Re: Commit in docs/usefor (usepro.xml)
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sat, 23 Aug 2008 12:18:00 -0700
Sender: [email-masked]

"Frank Ellermann" <[email-masked]> writes:
> The version control bot wrote:
>
>> + media type message/rfc822 (defined in <xref target="RFC2046" />)
>> + SHOULD be used in its place.</t>
>
> Yes, but adding "section 5.2.1" might help, because it is the
> precise place where RFC 2046 already said that message/news is
> a pointless subtype.  Not in these words of course, but still.

Oh, sorry, I missed that detail.  Fixing.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: Commit in docs/usefor (usepro.xml)
Date:  Sat, 23 Aug 2008 21:09:28 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 10
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

The version control bot wrote:

> + media type message/rfc822 (defined in <xref target=3D"RFC2046" />)
> + SHOULD be used in its place.</t>

Yes, but adding "section 5.2.1" might help, because it is the
precise place where RFC 2046 already said that message/news is
a pointless subtype.  Not in these words of course, but still.

 Frank

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: Commit in docs/usefor (usepro.xml)
Date:  Sat, 23 Aug 2008 21:01:06 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 17
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

The version control bot wrote:

> +        <seriesInfo name=3D"Internet-Draft"
> +          value=3D"draft-ietf-usefor-useage-01" />

That is the variant automatically interpreted as
"work in progress" by xml2rfc.  If that is *not*
what you want strike the "-Draft" in seriesInfo.

IMO we're not interested in a MissRef discussion,
and maybe Charles renames the draft after the WG
termination.  He could also decide to let it rot
until the USEFOR dust settled.

Out of curiosity, is USEFOR older than DNSEXT ?

 Frank

To: [email-masked]
Subject: Re: ABNF nits
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sat, 23 Aug 2008 11:47:18 -0700
Sender: [email-masked]

"Frank Ellermann" <[email-masked]> writes:

> | 11: moderation-flag     = SP "(" %4D.6F.64.65.72.61.74.65.64 ")"
> |                                  ^
> | 11:30: error: Illegal character '%' - skipping to end of line
>
> ITYM %x, I manually fixed the input to check the effect.

Thanks, fixed.

> | ; CRLF UNDEFINED
> | ; SP UNDEFINED
> | ; newsgroup-name UNDEFINED
>
> That's IMO no okay.  Please import it:

The ABNF in USEPRO is additive to the ABNF in USEFOR and the Core Rules
defined in RFC5234 as noted in 1.4.

> : newsgroup-description
> :                     = eightbit-utext *( *WSP eightbit-utext )
>
> | ; WSP UNDEFINED
> | ; utext UNDEFINED
>
> That's a real issue in the draft.  ITYM that it MUST be
> <UTF8-non-ascii> as specified in RFC 3977, but servers
> MUST in practice accept any octet.

No, I mean what the text says.

   While a charset parameter is defined for this media type, most
   existing software does not understand MIME header fields or correctly
   handle descriptions in a variety of charsets.  Using a charset of US-
   ASCII where possible is therefore RECOMMENDED; if not possible, UTF-8
   [RFC3629] SHOULD be used.  Regardless of the charset used, the
   constraints of the above grammar MUST be met and the <newsgroup-name>
   MUST be represented in that charset using the same octets as would be
   used with a charset of US-ASCII.

> How about this:
>
> - eightbit-utext      = utext / %d127-255
> + eightbit-utext      = UTF8-non-ascii / %d128-255
> + UTF8-non-ascii      = <see RFC3977 section 9.8>
> +
> + Note that <eightbit-utext> MUST be <UTF8-non-ascii>,
> + but for backwards compatibility servers MUST other
> + non-ASCII octets.

You'll have to convince the chairs to reopen this topic if you want to
make a change to the specification in this area, since that's not the
policy that we previously reached consensus on for how to handle Unicode.

> Technical detail, I omitted %d127 (DEL), because it is
> not obviously needed.  Please spice it with a RFC 5198
> reference to get NFC.  The EAI drafts do this, it is no
> new untested idea.  BTW, they also use <UTF8-non-ascii>.

Likewise here.  I don't care one way or the other about DEL, but making
substantive changes at this phase is just asking for trouble.  I'm willing
to drop it if there's clear consensus and the chair okays making that
change, but not otherwise, since we're trying to finish.

> Back to the ABNF parser output:
>
> | ; DIGIT UNDEFINED
> | ; msg-id UNDEFINED
>
> That is critical, various different definitions exist.
> Please import the (here) relevant <msg-id> definition:
>
> : msg-id                = <see [USEFOR] section 3.1.3>

I don't think it's necessary to note everywhere that we reference msg-id
that we mean the version defined in USEFOR.  I think the introduction to
the document covers that adequately.

> Subtle point, instead of "[USEFOR]" please use "RFCnnnn"
> in ABNF prose definitions, and add an RFC-editor note to
> replace "yyyy" by the RFC number.
>
> The very popular "rfcmarkup" tool can identify "RFCnnnn
> section X.Y" as link to a section X.Y in RFCnnnn.  We're
> not supposed to cuddle simple-minded tools in RFCs, but
> in that case I think it is worth it, rfcmarkup-format is
> used in RFC links by almost all MediaWiki installations.

I don't think this is useful.  [USEFOR] is a much simpler search string to
use to find all instances that need to be corrected once the other draft
has been published.

> You use a few ABNG terms like <verb> in the prose.  When you add an
> "ABNF import interface" in chapter 1 please import also those terms
> while you are at it.

I don't, at present, intend to add an import interface.

> Related nits:  Please upgrade RFC 4234 to 5234 (STD 68).

Done.

> The USEAGE reference should get a proper URL.

Done, thanks.

> I found a trick to bypass the xml2rfc "work in progress" blurb:

They should have a work in progress blurb.  They're Internet-Drafts.  I
missed the <seriesInfo> tag here and for USEFOR; fixed.

The status of USEAGE needs to be resolved before publishing the document.
If it's not going to be published, we need to strip all references to
USEAGE in this document.  So having the work in progress tag is useful to
be sure that we do that.

> Maybe add an RFC editor note that "2822" instead of "2822upd"
> is as it should be for reasons they don't need to care about
> unless they are willing to read the complete USEFOR archive
> as prerequisite for real explanations.

I didn't think that 2822upd breaks anything that wasn't already broken.
What are you worried about having break here?

> Last sentence in the intro of chapter 4, please add an exact
> reference to [RFC2046] section 5.2.1.

Done.

> Please post review requests for application/news-checkgroups,
> application/news-transmission, and application/news-groupinfo
> on the types review list.
>
> If you already did that years ago I missed it, but I guess
> we simply forgot to do this.  It is mostly "pro forma", but
> sometimes the experts ask good questions resulting in fixed
> registration templates.
>
> Clearly I should post the review requests, because I think
> they are required.  But I could not answer questions about
> many application/news-* details.  AFAICT Harald is also in
> the "required" camp.

I don't have time to do this at present and can make no promises as to
when I will have time to do so.

> For message/news I think you *could* add a new registration
> template, because you modify not only the status, but also
> the semantics ("use message/rfc822"), and the owner (IETF).
>
> But maybe that is irrelevant and/or too late now.

I'm inclined not to bother with this unless people feel it would be
worthwhile.  It makes the draft longer and I don't think anyone's going to
care.

> Looking in RFC 4288 I found that you did not use some fields
> in the other templates.  I can't tell if that is okay, e.g.,
> IMO "change controller: IETF" is implied by standards track.

Yeah, I don't know if this is okay or not either.  I can do whatever the
right thing to do here is.  Should I add all of the other template
attributes there?

> BTW, where did you find "Disposition" in RFC 4288 ?  It is
> a good idea, but maybe it belongs to the "Interoperability".

I didn't; I just didn't change it.  This has been in the draft for quite a
while.  Should I remove it?  I don't see Disposition anywhere in the media
type registration either.

> That's the point where you might need a proper [son-of-1036]
> reference, grab it from one of the XML sources.  I think you
> should not say "this specification" in any IANA template, 
> they are supposed to store these templates on their server.
>
> (What they do in practice is a diferent question, I did not
>  find the old original message/news template spontaneously).
>
> You can use the magic &rfc.number; in xml2rfc sources for
> this purpose.  It expands to RFCXXXX in drafts, and to the
> RFC number in AUTH48.  (Hopefully, I can't test this theory
> before RFC.ietf-usefor-usefor RFC got its number.)

Thank you.  Charles had this right and I broke it; I'm now using
&rfc.number;, which seems like a good solution.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sat, 23 Aug 2008 11:24:00 -0700 (PDT)
Sender: [email-masked]

    Date: Saturday, August 23, 2008 @ 11:23:59
  Author: eagle
Revision: 4900

Use &rfc.number; in specification references in media type registrations
rather than "This specification" since these registrations will be
excerpted from the document.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-23 18:20:05 UTC (rev 4899)
+++ docs/usefor/usepro.xml	2008-08-23 18:23:59 UTC (rev 4900)
@@ -1442,7 +1442,7 @@
                              which if processed could have effects on
                              the recipient host's system beyond just
                              storage of the article.
-   Published specification:  This specification.
+   Published specification:  RFC&rfc.number;
    Body part:                A complete proto-article ready for
                              injection into Netnews or an article
                              being relayed to another agent.
@@ -1487,7 +1487,7 @@
    Encoding considerations:  7bit or 8bit MUST be used to maintain
                              compatibility. 
    Security considerations:  None.
-   Published specification:  This specification.
+   Published specification:  RFC&rfc.number;
           </artwork>
         </figure>
 
@@ -1569,7 +1569,7 @@
                              should exist within a hierarchy.  Such
                              authorization must be accomplished by
                              other means.
-   Published specification:  This specification.
+   Published specification:  RFC&rfc.number;
           </artwork>
         </figure>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sat, 23 Aug 2008 11:20:06 -0700 (PDT)
Sender: [email-masked]

    Date: Saturday, August 23, 2008 @ 11:20:05
  Author: eagle
Revision: 4899

Reference RFC 2046 for the definition of message/rfc822 when
obsoleting message/news.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-23 18:17:00 UTC (rev 4898)
+++ docs/usefor/usepro.xml	2008-08-23 18:20:05 UTC (rev 4899)
@@ -1409,7 +1409,8 @@
       IANA, is hereby declared obsolete.  It was never widely
       implemented, and its default treatment as application/octet-stream
       by agents that did not recognize it was counter-productive.  The
-      media type message/rfc822 SHOULD be used in its place.</t>
+      media type message/rfc822 (defined in <xref target="RFC2046" />)
+      SHOULD be used in its place.</t>
 
       <section anchor="transmission" title="application/news-transmission">
         <t>The media type application/news-transmission is intended for

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sat, 23 Aug 2008 11:17:01 -0700 (PDT)
Sender: [email-masked]

    Date: Saturday, August 23, 2008 @ 11:17:00
  Author: eagle
Revision: 4898

Improve the citation for USEFOR and USEAGE to add URLs and properly
cite both as works in progress.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-23 17:58:49 UTC (rev 4897)
+++ docs/usefor/usepro.xml	2008-08-23 18:17:00 UTC (rev 4898)
@@ -2150,7 +2150,8 @@
       &rfc3629;
       &rfc4288;
       &rfc5234;
-      <reference anchor="USEFOR">
+      <reference anchor="USEFOR"
+        target="http://tools.ietf.org/html/draft-ietf-usefor-usefor-12">
         <front>
           <title>Netnews Article Format</title>
           <author initials="K." surname="Murchison"
@@ -2165,6 +2166,8 @@
             <organization>FlyDash, Inc.</organization>
           </author>
         </front>
+        <seriesInfo name="Internet-Draft"
+          value="draft-ietf-usefor-usefor-12" />
       </reference>
     </references>
 
@@ -2175,14 +2178,20 @@
       &rfc2606;
       &rfc3798;
       &rfc3977;
-      <reference anchor="USEAGE">
+      <reference anchor="USEAGE"
+        target="http://tools.ietf.org/html/draft-ietf-usefor-useage-01">
         <front>
           <title>Usenet Best Practice</title>
           <author initials="C." surname="Lindsey"
             fullname="Charles H. Lindsey">
             <organization>University of Manchester</organization>
           </author>
+          <date month="March" year="2005" />
         </front>
+        <seriesInfo name="Internet-Draft"
+          value="draft-ietf-usefor-useage-01" />
+        <format type="TXT"
+          target="http://tools.ietf.org/id/draft-ietf-usefor-useage-01" />
       </reference>
     </references>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sat, 23 Aug 2008 10:58:50 -0700 (PDT)
Sender: [email-masked]

    Date: Saturday, August 23, 2008 @ 10:58:49
  Author: eagle
Revision: 4897

RFC 5234 replaced RFC 4234.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-23 17:54:34 UTC (rev 4896)
+++ docs/usefor/usepro.xml	2008-08-23 17:58:49 UTC (rev 4897)
@@ -22,10 +22,10 @@
     'http://xml.resource.org/public/rfc/bibxml/reference.RFC.3798.xml'>
   <!ENTITY rfc3977 PUBLIC '' 
     'http://xml.resource.org/public/rfc/bibxml/reference.RFC.3977.xml'>
-  <!ENTITY rfc4234 PUBLIC '' 
-    'http://xml.resource.org/public/rfc/bibxml/reference.RFC.4234.xml'>
   <!ENTITY rfc4288 PUBLIC '' 
     'http://xml.resource.org/public/rfc/bibxml/reference.RFC.4288.xml'>
+  <!ENTITY rfc5234 PUBLIC '' 
+    'http://xml.resource.org/public/rfc/bibxml/reference.RFC.5234.xml'>
 ]>
 
 <rfc ipr="full3978" docName="draft-ietf-usefor-usepro-11" obsoletes="1036"
@@ -134,7 +134,7 @@
       <section title="Syntax Notation">
         <t>Syntax defined in this document uses the Augmented
         Backus-Naur Form (ABNF) notation (including the Core Rules)
-        defined in <xref target="RFC4234" /> and constructs defined in
+        defined in <xref target="RFC5234" /> and constructs defined in
         <xref target="USEFOR" /> and <xref target="RFC2822" />.</t>
       </section>
 
@@ -2148,8 +2148,8 @@
       &rfc2119;
       &rfc2822;
       &rfc3629;
-      &rfc4234;
       &rfc4288;
+      &rfc5234;
       <reference anchor="USEFOR">
         <front>
           <title>Netnews Article Format</title>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sat, 23 Aug 2008 10:54:36 -0700 (PDT)
Sender: [email-masked]

    Date: Saturday, August 23, 2008 @ 10:54:34
  Author: eagle
Revision: 4896

Fix ABNF syntax error.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-23 02:00:46 UTC (rev 4895)
+++ docs/usefor/usepro.xml	2008-08-23 17:54:34 UTC (rev 4896)
@@ -1507,7 +1507,7 @@
                                [ *WSP moderation-flag ]
       newsgroup-description
                           = eightbit-utext *( *WSP eightbit-utext )
-      moderation-flag     = SP "(" %4D.6F.64.65.72.61.74.65.64 ")"
+      moderation-flag     = SP "(" %x4D.6F.64.65.72.61.74.65.64 ")"
                                ; SPACE + case sensitive "(Moderated)"
       eightbit-utext      = utext / %d127-255
           </artwork>

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  ABNF nits (was: I-D Action:draft-ietf-usefor-usepro-11.txt)
Date:  Sat, 23 Aug 2008 19:13:28 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 217
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

> Filename        : draft-ietf-usefor-usepro-11.txt
[...]
> Comments are solicited and should be addressed to
> the Usenet Format Working Group at [email-masked].

New tools trick, you can extract the ABNF as seen by=20
http://tools.ietf.org/abnf/draft-ietf-usefor-usepro-11

Skipping two lines which are no ABNF I copied the
output and pasted it "as is" in Bill's ABNF parser.

| 11: moderation-flag     =3D SP "(" %4D.6F.64.65.72.61.74.65.64 ")"
|                                  ^
| 11:30: error: Illegal character '%' - skipping to end of line

ITYM %x, I manually fixed the input to check the effect.

| 17: control-command     =3D/ Newgroup-command
| 18: Newgroup-command    =3D "newgroup" Newgroup-arguments
|   ^
| 18:0: error: Rule control-command does not yet exist; treating /=3D as =
=3D

I manually added an "import" rule at the top like this:

: control-command  =3D <see [USEFOR] section 3.2.3>

After that it was at "no errors" listimg all undefined
and unused terms.  Bill's "canonical format" is a matter
of taste, but always intersting to see:

| ; CRLF UNDEFINED
| ; SP UNDEFINED
| ; newsgroup-name UNDEFINED

That's IMO no okay.  Please import it:

: newsgroup-name        =3D <see [USEFOR] section 3.1.4>

| ; HTAB UNDEFINED
| ; newsgroup-description UNDEFINED

That's an artefact of the ABNF extraction tool, I copied
the missing term manually to the ABNF input form field:

: newsgroup-description
:                     =3D eightbit-utext *( *WSP eightbit-utext )

| ; WSP UNDEFINED
| ; utext UNDEFINED

That's a real issue in the draft.  ITYM that it MUST be
<UTF8-non-ascii> as specified in RFC 3977, but servers
MUST in practice accept any octet.  How about this:

- eightbit-utext      =3D utext / %d127-255
+ eightbit-utext      =3D UTF8-non-ascii / %d128-255
+ UTF8-non-ascii      =3D <see RFC3977 section 9.8>
+
+ Note that <eightbit-utext> MUST be <UTF8-non-ascii>,
+ but for backwards compatibility servers MUST other
+ non-ASCII octets.

Technical detail, I omitted %d127 (DEL), because it is
not obviously needed.  Please spice it with a RFC 5198
reference to get NFC.  The EAI drafts do this, it is no
new untested idea.  BTW, they also use <UTF8-non-ascii>.

Back to the ABNF parser output:

| ; DIGIT UNDEFINED
| ; msg-id UNDEFINED

That is critical, various different definitions exist.
Please import the (here) relevant <msg-id> definition:

: msg-id                =3D <see [USEFOR] section 3.1.3>

Not RFC 3977 without "@", not RFC 2822 with NO-WS-CTL,
not RFC 2822upd allowing ">" (IIRC), and not RFC 1036.

It is a zoo, and for this round I hope to arrive at a
consistent story.  To align this with 2822upd I think
that a later RFC.ietf-usefor-usefor update would cover
all cages in the NetNews department of this zoo (down
to the news: URIs).

| ; path-identity UNDEFINED

: path-identity          =3D <see [USEFOR] section 3.1.5>

| ; control-command defined but not used
| ; groupinfo-body defined but not used
| ; checkgroups-body defined but not used
| ; sendme-body defined but not used

That's as it should be.  When you add the imported terms
you could also import the used STD 68 terms, as we did
it in RFC.ietf-usefor-usefor

Subtle point, instead of "[USEFOR]" please use "RFCnnnn"
in ABNF prose definitions, and add an RFC-editor note to
replace "yyyy" by the RFC number.

The very popular "rfcmarkup" tool can identify "RFCnnnn
section X.Y" as link to a section X.Y in RFCnnnn.  We're
not supposed to cuddle simple-minded tools in RFCs, but
in that case I think it is worth it, rfcmarkup-format is
used in RFC links by almost all MediaWiki installations.

You use a few ABNG terms like <verb> in the prose.  When
you add an "ABNF import interface" in chapter 1 please
import also those terms while you are at it.  IMHO this
helps to read (and maybe understand) non-trivial memos.

--------------------------------------------------------
Related nits:  Please upgrade RFC 4234 to 5234 (STD 68).
The USEAGE reference should get a proper URL.  I found
a trick to bypass the xml2rfc "work in progress" blurb:

<reference
  anchor=3D'USEAGE'
  target=3D'http://tools.ietf.org/html/draft-ietf-usefor-useage-01'>
  <front>
    <title>Usenet Best Practice</title>
    <author initials=3D'C.' surname=3D'Lindsey'
            fullname=3D'Charles H. Lindsey'>
      <organization>University of Manchester</organization>
    </author>
    <date month=3D'March' year=3D'2005' />
  </front>
  <seriesInfo name=3D'Internet' value=3D'draft-ietf-usefor-useage-01' />
  <format type=3D'TXT'
    target=3D'http://tools.ietf.org/id/draft-ietf-usefor-useage-01' />
</reference>

The trick in this reference is <seriesInfo name=3D"Internet" ...
(Tested in the news: URI RFC for the historical Gilman draft.)

Maybe add an RFC editor note that "2822" instead of "2822upd"
is as it should be for reasons they don't need to care about
unless they are willing to read the complete USEFOR archive
as prerequisite for real explanations.

I'm surprised that you got away without a s-o-1036 reference,
but I'm not going to investigate how you pulled that stunt.

If it was only the lack of a ready made xml2rfc <reference>
check out the news: URI I-D xml source under .../home/test

------------------------------------------------------------
IANA business

Last sentence in the intro of chapter 4, please add an exact
reference to [RFC2046] section 5.2.1.  After EAI introduced
a new message/global these details matter.   Not being exact
could require an essay in the security considerations, but
we have no time for this potential fun.

Please post review requests for application/news-checkgroups,
application/news-transmission, and application/news-groupinfo
on the types review list.

If you already did that years ago I missed it, but I guess
we simply forgot to do this.  It is mostly "pro forma", but
sometimes the experts ask good questions resulting in fixed
registration templates.

Clearly I should post the review requests, because I think
they are required.  But I could not answer questions about
many application/news-* details.  AFAICT Harald is also in
the "required" camp.

For message/news I think you *could* add a new registration
template, because you modify not only the status, but also
the semantics ("use message/rfc822"), and the owner (IETF).

But maybe that is irrelevant and/or too late now.  If not:

: MIME type name:           message
: MIME subtype name:        news
: Required parameters:      none
: Optional parameters:      charset, compare message/rfc822
: Disposition:              by default, inline
: Encoding considerations:  compare message/rfc822 (RFC 2046)
: Security considerations:  OBSOLETE, use message/rfc822.
: Interoperability considerations:
:                           Rarely used; and therefore often
:                           handled as application/octet-stream
: Published specification:  son-of-1036, RFCXXXX
: Applications that use this media type:
:                           Some old mail and news user agents
: Intended usage:           OBSOLETE
: Author:                   Henry Spencer=20
: Change controller:        IETF

Looking in RFC 4288 I found that you did not use some fields
in the other templates.  I can't tell if that is okay, e.g.,
IMO "change controller: IETF" is implied by standards track.

That's the point where you might need a proper [son-of-1036]
reference, grab it from one of the XML sources.  I think you
should not say "this specification" in any IANA template,=20
they are supposed to store these templates on their server.

(What they do in practice is a diferent question, I did not
 find the old original message/news template spontaneously).

You can use the magic &rfc.number; in xml2rfc sources for
this purpose.  It expands to RFCXXXX in drafts, and to the
RFC number in AUTH48.  (Hopefully, I can't test this theory
before RFC.ietf-usefor-usefor RFC got its number.)

BTW, where did you find "Disposition" in RFC 4288 ?  It is
a good idea, but maybe it belongs to the "Interoperability".

 Frank

From: [email-masked]
To: [email-masked]
Cc: [email-masked]
Subject: I-D Action:draft-ietf-usefor-usepro-11.txt 
Date: Fri, 22 Aug 2008 15:00:01 -0700 (PDT)
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Usenet Article Standard Update Working Group of the IETF.

	Title           : Netnews Architecture and Protocols
	Author(s)       : R. Allbery, C. Lindsey
	Filename        : draft-ietf-usefor-usepro-11.txt
	Pages           : 51
	Date            : 2008-08-22

This document defines the architecture of Netnews systems and
specifies the correct manipulation and interpretation of Netnews
articles by software which originates, distributes, stores, and
displays them.  It also specifies the requirements that must be met
by any protocol used to transport and serve Netnews articles.Internet
Draft Comments

Comments are solicited and should be addressed to the Usenet Format
Working Group at [email-masked].

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usepro-11.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Message/External-body;
	name="draft-ietf-usefor-usepro-11.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:     <[email-masked]>

--NextPart--

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (Makefile usepro.xml)
Date: Fri, 22 Aug 2008 14:51:03 -0700 (PDT)
Sender: [email-masked]

    Date: Friday, August 22, 2008 @ 14:51:02
  Author: eagle
Revision: 4892

Make usepro-11.

Modified:
  docs/usefor/Makefile
  docs/usefor/usepro.xml

Modified: docs/usefor/Makefile
===================================================================
--- docs/usefor/Makefile	2008-08-22 21:50:29 UTC (rev 4891)
+++ docs/usefor/Makefile	2008-08-22 21:51:02 UTC (rev 4892)
@@ -1,7 +1,7 @@
 # Generate new text and HTML versions of the USEPRO document and copy them
 # somewhere reasonable on my web site.
 
-DRAFT = draft-ietf-usefor-usepro-10
+DRAFT = draft-ietf-usefor-usepro-11
 WEB   = /home/eagle/web/eagle/usefor/usepro
 
 all: text html copy

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-22 21:50:29 UTC (rev 4891)
+++ docs/usefor/usepro.xml	2008-08-22 21:51:02 UTC (rev 4892)
@@ -28,7 +28,7 @@
     'http://xml.resource.org/public/rfc/bibxml/reference.RFC.4288.xml'>
 ]>
 
-<rfc ipr="full3978" docName="draft-ietf-usefor-usepro-10" obsoletes="1036"
+<rfc ipr="full3978" docName="draft-ietf-usefor-usepro-11" obsoletes="1036"
      category="std">
   <front>
     <title>Netnews Architecture and Protocols</title>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Fri, 22 Aug 2008 14:50:30 -0700 (PDT)
Sender: [email-masked]

    Date: Friday, August 22, 2008 @ 14:50:29
  Author: eagle
Revision: 4891

Resolution of #1416.

Introduce the concept of multiple injection, replacing reinjection.  Allow
posting agents to add Injection-Date and require them to do so in the
multiple injection scenario.  Prohibit injecting agents from changing
existing Injection-Date headers.  Prohibit injecting agents from adding
Injection-Date if Date and Message-ID are present but Injection-Date is
missing.  Better explanation of multiple injection and more
recommendations about how to do it properly.

Add an example of a header deprecated for netnews.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-20 04:58:28 UTC (rev 4890)
+++ docs/usefor/usepro.xml	2008-08-22 21:50:29 UTC (rev 4891)
@@ -18,6 +18,8 @@
     'http://xml.resource.org/public/rfc/bibxml/reference.RFC.2822.xml'>
   <!ENTITY rfc3629 PUBLIC '' 
     'http://xml.resource.org/public/rfc/bibxml/reference.RFC.3629.xml'>
+  <!ENTITY rfc3798 PUBLIC '' 
+    'http://xml.resource.org/public/rfc/bibxml/reference.RFC.3798.xml'>
   <!ENTITY rfc3977 PUBLIC '' 
     'http://xml.resource.org/public/rfc/bibxml/reference.RFC.3977.xml'>
   <!ENTITY rfc4234 PUBLIC '' 
@@ -165,8 +167,9 @@
 
         <t>"Injecting" an article is the processing of a proto-article by
         an injecting agent.  Normally this action is done once and only
-        once for a given article.  "Reinjecting" an article is passing an
-        already-injected article to an injection agent.</t>
+        once for a given article.  "Multiple injection" is passing the
+        same article to multiple injecting agents, either serially or in
+        parallel, by one or several posting agents.</t>
 
         <t>A "gateway" is software which receives news articles and
         converts them to messages of some other kind (such as <xref
@@ -519,10 +522,35 @@
 
         <t>Posting agents SHOULD ensure that proto-articles they create
         are valid according to <xref target="USEFOR" /> and any other
-        applicable policies.  They MUST NOT create any Injection-Date or
-        Injection-Info header fields; these headers will be added by the
-        injecting agent.</t>
+        applicable policies.  They MUST NOT create any Injection-Info
+        header field; this header field may only be added by the injecting
+        agent.</t>
 
+        <t>If the proto-article already contains both Message-ID and Date
+        header fields, posting agents MAY add an Injection-Date header
+        field to that proto-article immediately before passing that
+        proto-article to an injection agent.  They SHOULD do so if the
+        Date header field (representing the composition time of the
+        proto-article) is more than a day in the past at the time of
+        injection.  They MUST do so if the proto-article is being
+        submitted to more than one injecting agent; see <xref
+        target="multi-injection" />.</t>
+
+        <t>The Injection-Date header field is new in this revision of the
+        Netnews protocol and is designed to allow the Date header field to
+        hold the composition date (as recommended in section 3.6.1 of
+        <xref target="RFC2822" />), even if the proto-article is not to be
+        injected for some time after its composition.  However, note that
+        all implementations predating this specification ignore the
+        Injection-Date header field and use the Date header field in its
+        stead for rejecting articles older than their cutoff (see <xref
+        target="history" />), and injecting agents predating this
+        specification do not add an Injection-Date header.  Articles with
+        a Date header field substantially in the past will still be
+        rejected by implementations predating this specification,
+        regardless of the Injection-Date header field, and hence may
+        suffer poorer propagation.</t>
+
         <t>Contrary to <xref target="RFC2822" />, which implies that the
         mailbox or mailboxes in the From header field should be that of
         the poster or posters, a poster who does not, for whatever
@@ -544,48 +572,75 @@
           agent.</t>
 
           <t>A proto-article has the same format as a normal article
-          except that the Injection-Date, Injection-Info, and Xref header
-          fields MUST NOT be present; the Path header field MUST NOT
-          contain a "POSTED" &lt;diag-keyword>; and any of the following
-          mandatory header fields MAY be omitted: Message-ID, Date, and
-          Path.  In all other respects, a proto-article MUST be a valid
-          Netnews article.  In particular, the header fields which may be
-          omitted MUST NOT be present with invalid content.</t>
+          except that the Injection-Info and Xref header fields MUST NOT
+          be present; the Path header field MUST NOT contain a "POSTED"
+          &lt;diag-keyword>; and any of the following mandatory header
+          fields MAY be omitted: Message-ID, Date, and Path.  In all other
+          respects, a proto-article MUST be a valid Netnews article.  In
+          particular, the header fields which may be omitted MUST NOT be
+          present with invalid content.</t>
 
           <t>If a posting agent intends to offer the same proto-article to
-          multiple injecting agents, the header fields Message-ID and Date
-          MUST be present and identical in all copies of the
-          proto-article.</t>
+          multiple injecting agents, the header fields Message-ID, Date,
+          and Injection-Date MUST be present and identical in all copies
+          of the proto-article.  See <xref target="multi-injection" />.</t>
         </section>
 
-        <section anchor="reinjection" title="Reinjection of Articles">
-          <t>A given article SHOULD be processed by an injecting agent
-          once and only once.  The Injection-Date or Injection-Info
-          header fields are added by an injecting agent and are not
-          permitted in a proto-article.  Their presence (or the presence
-          of other unstandardized or obsolete trace headers such as
-          NNTP-Posting-Host, NNTP-Posting-Date, or X-Trace) indicates
-          that the proto-article is instead an article and has already
-          been processed by an injecting agent.  A posting agent SHOULD
-          normally reject such articles.</t>
+        <section anchor="multi-injection"
+                 title="Multiple Injection of Articles">
+          <t>Under some circumstances (posting to multiple disjoint
+          networks, injecting agents with spotty connectivity, or for
+          redundancy, for example), a posting agent may wish to offer the
+          same article to multiple injecting agents.  In this unusual
+          case, the goal is to not create multiple independent articles
+          but rather to inject the same article at multiple points and let
+          the normal duplicate suppression facility of Netnews (see <xref
+          target="history" />) ensure that any given agent accepts the
+          article only once, even if supposedly disjoint networks have
+          unexpected links.</t>
 
-          <t>In the exceptional case that an article needs to be
-          reinjected for some reason (such as transferring an article from
-          one Netnews to another where those networks have no relaying
-          agreement), the posting agent doing the reinjection MUST convert
-          the article back into a proto-article before passing it to an
-          injecting agent (such as by renaming the Injection-Info and
-          Injection-Date header fields and removing any Xref header field)
-          and MUST perform the date checks on the existing Injection-Date
-          or Date header fields that would otherwise be done by the
-          injecting agent.</t>
+          <t>Whenever possible, multiple injection SHOULD be done by
+          offering the same proto-article to multiple injecting agents.
+          The posting agent MUST supply the Message-ID, Date, and
+          Injection-Date header fields, and the proto-article as offered
+          to each injecting agent MUST be identical.</t>
 
-          <t>Reinjecting articles may cause loops, loss of trace
-          information, and other problems and should only be done with
-          care and when there is no available alternative.  A posting
-          agent that does reinjection is a limited type of gateway and as
-          such is subject to all of the requirements of an incoming
-          gateway in addition to the requirements of a posting agent.</t>
+          <t>In some cases, offering the same proto-article to all
+          injecting agents may not be possible (such as when gatewaying,
+          after the fact, articles found on one Netnews network to
+          another, supposedly unconnected one).  In this case, the posting
+          agent MUST convert the article back into a proto-article before
+          passing it to another injecting agent, but it MUST retain
+          unmodified the Message-ID, Date, and Injection-Date header
+          fields.  It MUST NOT add an Injection-Date header field if it is
+          missing from the existing article.  It MUST remove any Xref
+          header field and either rename or remove any Injection-Info
+          header field and other trace fields.
+            <list style="empty">
+              <t>NOTE: Multiple injection inherently risks duplicating
+              articles.  Multiple injection after the fact, by converting
+              an article back to a proto-article and injecting it again,
+              additionally risks loops, loss of trace information,
+              unintended repeat injection into the same network, and other
+              problems.  It should be done with care and only when there
+              is no alternative.  The requirement to retain Message-ID,
+              Date, and Injection-Date header fields minimizes the
+              possibility of a loop and ensures that the newly injected
+              article is not treated as a new, separate article.</t>
+            </list>
+          </t>
+
+          <t>Multiple injection of an article listing one or more
+          moderated newsgroups in its Newsgroups header field SHOULD only
+          be done by a moderator and MUST only be done after the
+          proto-article has been approved for all moderated groups to
+          which it is to be posted and has an Approved header field (see
+          <xref target="moderator" />).  Multiple injection of an
+          unapproved article intended for moderated newsgroups will
+          normally only result in the moderator receiving multiple copies,
+          and if the newsgroup status is not consistent across all
+          injecting agents, may result in duplication of the article or
+          other problems.</t>
         </section>
 
         <section anchor="followups" title="Followups">
@@ -710,23 +765,27 @@
 
             <t>It MUST reject any proto-article that does not have the
             proper mandatory header fields for a proto-article; that has
-            Injection-Date, Injection-Info, or Xref header fields; that
-            has a Path header field containing the "POSTED"
-            &lt;diag-keyword>; or that is not syntactically valid as
-            defined by <xref target="USEFOR" />.  It SHOULD reject any
-            proto-article which contains a header field deprecated for
-            Netnews.  It MAY reject any proto-article that contains trace
-            header fields indicating that it was already injected by an
-            injecting agent that did not add Injection-Info or
-            Injection-Date.</t>
+            Injection-Info or Xref header fields; that has a Path header
+            field containing the "POSTED" &lt;diag-keyword>; or that is
+            not syntactically valid as defined by <xref target="USEFOR"
+            />.  It SHOULD reject any proto-article which contains a
+            header field deprecated for Netnews (see, for example, <xref
+            target="RFC3798" />).  It MAY reject any proto-article that
+            contains trace header fields (e.g., NNTP-Posting-Host)
+            indicating that it was already injected by an injecting agent
+            that did not add Injection-Info or Injection-Date.</t>
 
-            <t>It SHOULD reject any article whose Date header field is
-            more than 24 hours into the future (and MAY use a margin less
-            than 24 hours).  It SHOULD reject any article whose Date
-            header appears to be stale (more than 72 hours into the past,
-            for example, or too old to still be recorded in the database
-            of a relaying agent the injecting agent will be using) since
-            not all news servers support Injection-Date.</t>
+            <t>It SHOULD reject any article whose Injection-Date or Date
+            header field is more than 24 hours into the future (and MAY
+            use a margin less than 24 hours).  It SHOULD reject any
+            article whose Injection-Date header field is too far in the
+            past (older than the cutoff interval of a relaying agent the
+            injecting agent is using, for example).  It SHOULD similarly
+            reject any article whose Date header field is too far in the
+            past, since not all news servers support Injection-Date and
+            only the injecting agent can provide a useful error message to
+            the posting agent.  In either case, this interval SHOULD NOT
+            be any shorter than 72 hours into the past.</t>
 
             <t>It SHOULD reject any proto-article whose Newsgroups header
             field does not contain at least one &lt;newsgroup-name> for a
@@ -770,8 +829,14 @@
             the source of the article and possibly other trace information
             as described in Section 3.2.8 of <xref target="USEFOR" />.</t>
 
-            <t>The injecting agent MUST then add an Injection-Date header
-            field containing the current date and time.</t>
+            <t>If the proto-article already had an Injection-Date header
+            field, it MUST NOT be modified or replaced.  If the
+            proto-article had both a Message-ID header field and a Date
+            header field, an Injection-Date header field MUST NOT be
+            added, since the proto-article may have been multiply injected
+            by a posting agent that predates this standard.  Otherwise,
+            the injecting agent MUST add an Injection-Date header field
+            containing the current date and time.</t>
 
             <t>Finally, the injecting agent forwards the article to one or
             more relaying agents, and the injection process is
@@ -1068,8 +1133,7 @@
             for reasons understood by the moderator (such as delays in the
             moderation process) in which case they MAY substitute the
             current date.  Any Injection-Date, Injection-Info, or Xref
-            header fields already present (though there should be none)
-            MUST be removed.</t>
+            header fields already present MUST be removed.</t>
 
             <t>Any Path header field MUST either be removed or truncated
             to only those entries following its "POSTED"
@@ -2109,6 +2173,7 @@
       &rfc1036;
       &rfc2045;
       &rfc2606;
+      &rfc3798;
       &rfc3977;
       <reference anchor="USEAGE">
         <front>

From: Ralph Babel <[email-masked]>
To: [email-masked]
Subject: Re: #1416 When to add Injection-Date
Date: Mon, 18 Aug 2008 13:34:50 +0200 (CEST)
Sender: [email-masked]

Richard Clayton wrote:

> I think the decision should be all about trust issues,
> not about duplication or over/under-enthusiastic expiry.

If headers generated elsewhere cannot be trusted
anyhow, and if servers are to rely on their own data
instead (e.g. an article's arrival date), this pretty
much implies that "Injection-Date:" is useless.

Gee, a severe case of deja vu, it seems.

> But trust is so weak on Usenet, I don't mind being ignored !

Welcome to the club!

To: [email-masked]
Xref: clerew local.usefor:24827
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416 When to add Injection-Date
Date: Mon, 18 Aug 2008 10:54:19 GMT
Lines: 38
Sender: [email-masked]

In <[email-masked]> Richard Clayton <[email-masked]> writes:

>In message <[email-masked]>, Harald Alvestrand
><[email-masked]> writes

>>ALTERNATIVE C:
>>
>>-            <t>The injecting agent MUST then add an Injection-Date header
>>-            field containing the current date and time.</t>

>My view (albeit weakly held) is that header fields are either added by
>servers (vaguely trusted) or by clients (not trusted in the slightest)

>This header field is clearly meant to be added by a server, so it should
>be added by a server and not by a client under any circumstances ... so
>I prefer C  (then A then B)

I think the problem with C is that it permits to override a previously
present Injection-Date (which can arise due to all sorts of strange
scenarios including loops involving exotic gatewaying, and possibly
leading to the continuation of those loops).

So if some guy has already put an Injection-Date header there (for
whatever reason, good or bad) then he is saying, in effect, "the clock for
deciding when to expire this article has already started ticking". In
which case he has only himself to blame if his action causes some
premature expiry because he did it wrong.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Mon, 18 Aug 2008 12:40:16 +0200
From: Harald Alvestrand <[email-masked]>
To: Usenet <[email-masked]>
Subject: Conclusion and summary of responses, opinion poll #1416
Sender: [email-masked]

It's still a few hours early, but it's a convenient time....

ALTERNATIVE A:

+            <t>If the proto-article already had an Injection-Date header
+            field, it MUST NOT be modified or replaced.  Otherwise,
+            the injecting agent MUST add an Injection-Date header field
+            containing the current date and time.</t>

Stanley
Charles Lindsey
Frank Ellermann

ALTERNATIVE B:

+            <t>If the proto-article already had an Injection-Date header
+            field, it MUST NOT be modified or replaced.  If the
+            proto-article had both a Message-ID header field and a Date
+            header field, an Injection-Date header field MUST NOT be
+            added, since the proto-article may have been multiply injected
+            by a posting agent that predates this standard.  Otherwise,
+            the injecting agent MUST add an Injection-Date header field
+            containing the current date and time.</t>

Forrest J. Cavalier
Russ Allbery
Thorfinn
Dan Schlitt

ALTERNATIVE C:

-            <t>The injecting agent MUST then add an Injection-Date header
-            field containing the current date and time.</t>

Richard Clayton

Some people gave a ranking of alternatives:

Stanley: A, C, B
Frank: A, C, B
Richard: C, A, B

CONCLUSIONS
==========

There isn't a single opinion that has the consensus of the group, or 
even a clear majority opinion.

Nonetheless, it's clear that the option of leaving the text that was in 
the -10 draft (alternative C) definitely does not have the support of 
the group as a first alternative.

Thus, it's a matter of picking one change to do.

I'm hereby instructing the editor to proceed based on alternative B, and 
produce an -11 version incorporating this text (and the other changes 
from the August 4 delta set).

                          Harald Alvestrand, speaking as chair.

Date: Sun, 17 Aug 2008 07:58:45 +0100
To: Harald Alvestrand <[email-masked]>
Cc: Usenet <[email-masked]>
From: Richard Clayton <[email-masked]>
Subject: #1416 When to add Injection-Date
Sender: [email-masked]

In message <[email-masked]>, Harald Alvestrand
<[email-masked]> writes

>ALTERNATIVE C:
>
>-            <t>The injecting agent MUST then add an Injection-Date header
>-            field containing the current date and time.</t>

My view (albeit weakly held) is that header fields are either added by
servers (vaguely trusted) or by clients (not trusted in the slightest)

This header field is clearly meant to be added by a server, so it should
be added by a server and not by a client under any circumstances ... so
I prefer C  (then A then B)

I cannot get excited about multiple injection, this is exotic and so the
people doing it can look out for themselves.

I cannot get excited about ancient articles reappearing after days in
the wilderness. Propagation time across the active parts of Usenet is
generally measured in seconds except when queues build up.. and there is
some attempt to avoid that...

I cannot get excited about servers that look at Dates to decide when to
expire things -- the trend is to hold the last <n> that arrived (why
trust someone else to tell the truth). Apart from anything else, if you
delete in FIFO order it simplifies your filing system.

>So in order to see if we have a strongly held position in the group, please 
>reply to this message with ONE of the following statements:
>
>* I prefer Alternative A
>* I prefer Alternative B
>* I prefer Alternative C
>* I have another opinion, which is.....

I think the decision should be all about trust issues, not about
duplication or over/under-enthusiastic expiry.  But trust is so weak on
Usenet, I don't mind being ignored !

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: #1416
Date:  Sat, 16 Aug 2008 00:15:22 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 21
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Charles Lindsey wrote:

> it wouldn't happen anymore once all the OLD servers had been
> pensioned off

I'd agree with anybody claiming that our chances that this ever
happens are somewhere between limited and lousy.  But I think
we should stick to the vision that it *could* happen, because
changing the Injection-Date course midway between usefor-usefor
and usefor-usepro is messy.

There was a reason why Russ proposed this header field, we had
a conflict with the RFC 2822 Date semantics, and we decided to=20
emulate the slightly different Netnews semantics with the new
Injection-Date.

If folks want to revert that decision let them do it later in
the successors of usefor-usefor and usefor-usepro.  But *that*
would be a discussion which I'd try to avoid, WG or elsewhere.

 Frank

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: Opinion poll: #1416 When to add Injection-Date
Date:  Fri, 15 Aug 2008 23:50:18 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 12
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Harald Alvestrand wrote:

> please reply to this message with ONE of the following statements:
=20
> * I prefer Alternative A
> * I prefer Alternative B
> * I prefer Alternative C
> * I have another opinion, which is.....

...a slight preference for A before C before B.

 Frank

To: [email-masked]
Xref: clerew local.usefor:24823
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416
Date: Fri, 15 Aug 2008 11:12:38 GMT
Lines: 29
Sender: [email-masked]

In <[email-masked]> [email-masked] writes:

>If a server yesterday looks at the Date header and deletes the message 
>(expiration), and today it gets the same message with the same Date header 
>and accepts it, it is broken. It is ridiculous to accept articles that are
>just going to be deleted in the next expiration pass. I don't know if 
>server authors actually considered that when they wrote their code, but it 
>seems like a useful optimization. "Don't accept things older than I'm 
>already expiring..."

Yes, but the whole point of Injection-Date is that NEW servers use it when
deciding whether to consider it stale. That way an article that was
injected 24 hours after it was composed has an extra 24 hours before it
gets considered stale, thus improving its chances of propagation.

The (slight) downside of that is that NEW servers can be hoodwinked in
some highly peculiar scenarios such as the one I showed (but it wouldn't
happen anymore once all the OLD servers had been pensioned off).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Thu, 14 Aug 2008 11:16:59 -0700 (PDT)
From: [email-masked]
To: [email-masked]
Subject: Re: #1416
Sender: [email-masked]

"Charles Lindsey" <chl@xxxxxxxxxxxxxxxx>:

>Sure, but in practice if you want to check whether a message is the same
>as some other ancient message, it is much simpler to have a "date" as 
>well,
>so you can just say "this is so stale that I shall assume it is a
>duplicate and ignore it anyway".

Yes, just like I already said, you reject it because it is too old, with 
the ASSUMPTION that "too old" means you've already seen it.

The only information you can get from the Date is the age of the article. 
You cannot get anything about whether it is a duplicate or not.

Our working group defined the message id and the message id alone as the 
globally, forever unique identifier for a message.

Perhaps we truly do need a statement to that effect somewhere in the 
draft, if even Charles is confused by the difference between "date prior 
to earliest history entry" and "duplicate". The assumption results in 
treating the article as if the two were the same, but they really are 
different cases.

>>> It requires, if I remember correctly, badly-timed multiple
>>> injection of the message such that it expires on the basis of its original
>>> Date and is then reaccepted on the basis of the modified Injection-Date.

>>If a message is expired based on the Date header, why would the server
>>then accept the same article with the same Date header later? If it does,
>>it's just going to expire it again. This just doesn't make sense. It
>>sounds too much like a bug and not a feature.

>If you now have both Date and Injection-Date headers, then which is the
>true "date" for the purposes of the test above?

If a server yesterday looks at the Date header and deletes the message 
(expiration), and today it gets the same message with the same Date header 
and accepts it, it is broken. It is ridiculous to accept articles that are
just going to be deleted in the next expiration pass. I don't know if 
server authors actually considered that when they wrote their code, but it 
seems like a useful optimization. "Don't accept things older than I'm 
already expiring..."

That server is looking at the Date header, not the Injection Date. That's 
why I wrote "If a message is expired based on the Date header...". It 
doesn't matter what you think the "true" date is. Hint: how do you have a 
"modified Injection Date" if there was no Injection Date to start with?

To: [email-masked]
Xref: clerew local.usefor:24820
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416
Date: Thu, 14 Aug 2008 11:25:58 GMT
Lines: 34
Sender: [email-masked]

In <[email-masked]> "Frank Ellermann" <[email-masked]> writes:

><[email-masked]> wrote:
> 
>> Hands up from everyone here who wants to be in the
>> next IETF USEFOR WG.

My hand is up.

>I'd look in Netnews drafts tackling topics related
>to gateways, i18n (EAI for News), any updates of the
>existing stuff, and maybe drafts in the direction of
>"dkim" or "useage".

However, it might be simpler to let the EAI WG do that extension, if we
are not in a fit state to do it.

EAI, as they propose it, will work fine with Netnews, so all that is
needed would be to define a UTF-8 version of the Newsgroups header (and
this WG did all the groundwork for that years ago), together with a
downgrade mechanism which would only ever have to be used when gatewaying
News to Mailing Lists, and then only at the point where some message on
the Mailing List encountered some non-UT8SMTP mail server.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24819
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416
Date: Thu, 14 Aug 2008 11:18:09 GMT
Lines: 76
Sender: [email-masked]

In <[email-masked]> [email-masked] writes:

>Russ Allbery <rra@xxxxxxxxxxxx>:

>> Briefly: the Message-ID plus the Date is the unique identifier for the
>> message.

>I'm with Frank in being confused by this statement. According to USEFOR:

>    A "message identifier" (Section 3.1.3) is a unique identifier...

>This clearly states that two articles with the same message id, even with 
>different dates, are the same article. "Every article ... ever". If you 
>say that "message id plus date" is the unique identifier, then you are 
>saying that two messages with different dates but identical message ids 
>are two different messages -- a violation of the definition of message id.

Sure, but in practice if you want to check whether a message is the same
as some other ancient message, it is much simpler to have a "date" as well,
so you can just say "this is so stale that I shall assume it is a
duplicate and ignore it anyway".

>> It requires, if I remember correctly, badly-timed multiple
>> injection of the message such that it expires on the basis of its original
>> Date and is then reaccepted on the basis of the modified Injection-Date.

>If a message is expired based on the Date header, why would the server 
>then accept the same article with the same Date header later? If it does,
>it's just going to expire it again. This just doesn't make sense. It 
>sounds too much like a bug and not a feature.

If you now have both Date and Injection-Date headers, then which is the
true "date" for the purposes of the test above? "OLD" (i.e. unupgraded
software) with use Date, and "NEW" (upgraded sotware) will use
Injection-Date.

So if two copies of an article are circulating, one with just Date and the
other with both Date and Injection-Date (say 24 hours later), and the
first arrives at some NEW server, is stored and eventually expires (say
afyer 10 days) and then 23 hours later the second turns up, it will be
accepted as if it were new.

So how can this happen? it requires the following scenario:

1. The article was generated by an OLD User Agent, and multiply injected
   to *two* Injection Agents (one OLD and one NEW).
2. The two injections were 24 hours apart.
3. The 2nd copy got delayed by ~10 days somewhere, and then somehow got
   routed to that FINAL agent which had just expired the 1st copy.
4. That 2nd copy just "happened" to arrive at the FINAL agent within the
   critical 24 hours after the expiry.
5. The flooding algorithm had somewhere failed to prevent the propagation
   of the of the delayed copy (i.e. the FINAL agent must have been in
   some rather poorly connected part of the network).

So that is five conditions to be satisied, all somewhat improbable. And,
at the end of the day, all that happens is that the client users of that
FINAL agent get to see the article twice, which is no Big Deal IMHO.

>For that reason, I think I have to object to the exclusion statement and 
>require injecting agents to put in an Injection-Date header unless that 
>header is already there, without regard to existing message id or date 
>headers.

Exactly!

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24821
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Opinion poll: #1416 When to add Injection-Date
Date: Thu, 14 Aug 2008 11:30:26 GMT
Lines: 21
Sender: [email-masked]

In <[email-masked]> Harald Alvestrand <[email-masked]> writes:

>OK, time to close this one (hopefully).

>* I prefer Alternative A

Alternative C would lead to all sorts of problems which we discussed way back.

Alternative B avoids the one rather improbably scenario which I have
described elsewhere, but brings other disadvantages.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 12 Aug 2008 18:19:16 -0400
From: Dan Schlitt <[email-masked]>
To: Harald Alvestrand <[email-masked]>
cc: Usenet <[email-masked]>
Subject: Re: Opinion poll: #1416 When to add Injection-Date
Sender: [email-masked]

I favor alternative B.

After listening to the discussion of this item over the years I feel
this will best serve the intended purpose.

/dan

-- 

Dan Schlitt
[email-masked]

Date: Tue, 12 Aug 2008 08:35:50 +0200
From: Harald Tveit Alvestrand <[email-masked]>
To: Usenet <[email-masked]>
Subject: CORRECTION Re: Opinion poll: #1416 When to add Injection-Date
Sender: [email-masked]

Harald Alvestrand skrev:
>
> I'll give this one week (until Monday, August 11, 2008, 12 noon UTC), 
> and then tally responses. 
As Stanley rightly commented, one week from yesterday is August 18, not 
August 11. Sorry about the error.

                    Harald

From: Thorfinn <[email-masked]>
To: Usenet <[email-masked]>
Subject: Re: Opinion poll: #1416 When to add Injection-Date
Date: Tue, 12 Aug 2008 12:15:07 +1000
Sender: [email-masked]

On 12 Aug 2008, at 00:01, Harald Alvestrand wrote:

>
> OK, time to close this one (hopefully).
>
> Taking the deltas posted by Russ Allbery on August 4, 2008 at 01:37  
> my time as a starting point, I think we are down to one essential  
> statement, which goes in bullet 11 of section 3.5 of draft-ietf- 
> usefor-usepro-10:

* I prefer Alternative B

-- 
<a href="http://tertius.net.au/~thorfinn/">[email-masked]</a>
~/ For those who've come across the seas, ~/ We've boundless plains to  
share, ~/
~/   With courage let us all combine,     ~/    To Advance Australia  
Fair.    ~/
      -- The end of the second verse of the Australian National  
Anthem. --

To: [email-masked]
Subject: Re: #1416
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 11 Aug 2008 13:51:02 -0700
Sender: [email-masked]

[email-masked] writes:

> No, there is no reason to add "date" to the message id. You keep a list
> of message ids and the starting date of the list. You make the statement
> in USEPRO that servers can ASSUME that any article with a Date prior to
> the starting date of the history has been seen and is thus a
> duplicate. End of problem. You assume the message id is unique, just add
> a different test for acceptance.

I *think* that this is what the current draft says, and hence we're just
arguing over wording of e-mail messages on the list and not actually about
the work product of the group.  At least, the above was certainly my
intention when writing the current document.  If the current document
*doesn't* say that in anyone's reading, please do raise where it's not
clear.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Mon, 11 Aug 2008 12:59:16 -0700 (PDT)
From: [email-masked]
To: [email-masked]
Subject: Re: #1416
Sender: [email-masked]

"Forrest J. Cavalier III" <mibsoft@xxxxxxxxxxxxxxx>:

>That is because the server's record of message IDs is a
>finite list in practice.

>No one should offer an opinion on how to resolve #1416
>unless and until they understand this point.

Ignore that other draft behind the curtain, _I_ am the great and powerful 
USEPRO! Believe what I say, even if we said something different before.

If this is point is so critical, then USEFOR should have said it, instead 
of what it does say.

If you want to reopen USEFOR and debate whether "message id" or "message 
id plus date" is the globally, chonologically unique identification for a 
message, I'm game. If you don't want USEFOR to say "forever", then let's 
debate it and make the change. We should not say "forever" if we don't 
mean it, just like we shouldn't say "MUST" unless there is a valid reason.

No, there is no reason to add "date" to the message id. You keep a list of 
message ids and the starting date of the list. You make the statement in 
USEPRO that servers can ASSUME that any article with a Date prior to the 
starting date of the history has been seen and is thus a duplicate. End of 
problem. You assume the message id is unique, just add a different test 
for acceptance.

Date: Mon, 11 Aug 2008 12:47:11 -0700 (PDT)
From: [email-masked]
To: [email-masked]
Subject: Re: Opinion poll: #1416 When to add Injection-Date
Sender: [email-masked]

I prefer alternative A, then C. I dislike B mainly because it adds 
needless complexity.

Harald Alvestrand <harald@xxxxxxxxxxxxx>

>I'll give this one week (until Monday, August 11, 2008, 12 noon UTC), and 
>then tally responses.

I assume you mean the 18th.

To: Usenet <[email-masked]>
Subject: Re: Opinion poll: #1416 When to add Injection-Date
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Mon, 11 Aug 2008 09:59:39 -0700
Sender: [email-masked]

I prefer alternative B.

Harald Alvestrand <[email-masked]> writes:

> ALTERNATIVE B:
>
> +            <t>If the proto-article already had an Injection-Date header
> +            field, it MUST NOT be modified or replaced.  If the
> +            proto-article had both a Message-ID header field and a Date
> +            header field, an Injection-Date header field MUST NOT be
> +            added, since the proto-article may have been multiply injected
> +            by a posting agent that predates this standard.  Otherwise,
> +            the injecting agent MUST add an Injection-Date header field
> +            containing the current date and time.</t>

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Mon, 11 Aug 2008 11:30:07 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To: Harald Alvestrand <[email-masked]>
CC: Usenet <[email-masked]>
Subject: Re: Opinion poll: #1416 When to add Injection-Date
Sender: [email-masked]

I prefer Alternative B, which is the text Russ has written.

> ALTERNATIVE B:
> 
> +            <t>If the proto-article already had an Injection-Date header
> +            field, it MUST NOT be modified or replaced.  If the
> +            proto-article had both a Message-ID header field and a Date
> +            header field, an Injection-Date header field MUST NOT be
> +            added, since the proto-article may have been multiply injected
> +            by a posting agent that predates this standard.  Otherwise,
> +            the injecting agent MUST add an Injection-Date header field
> +            containing the current date and time.</t>

Date: Mon, 11 Aug 2008 16:01:48 +0200
From: Harald Alvestrand <[email-masked]>
To: Usenet <[email-masked]>
Subject: Opinion poll: #1416 When to add Injection-Date
Sender: [email-masked]

OK, time to close this one (hopefully).

Taking the deltas posted by Russ Allbery on August 4, 2008 at 01:37 my 
time as a starting point, I think we are down to one essential 
statement, which goes in bullet 11 of section 3.5 of 
draft-ietf-usefor-usepro-10:

ALTERNATIVE A:

+            <t>If the proto-article already had an Injection-Date header
+            field, it MUST NOT be modified or replaced.  Otherwise,
+            the injecting agent MUST add an Injection-Date header field
+            containing the current date and time.</t>

ALTERNATIVE B:

+            <t>If the proto-article already had an Injection-Date header
+            field, it MUST NOT be modified or replaced.  If the
+            proto-article had both a Message-ID header field and a Date
+            header field, an Injection-Date header field MUST NOT be
+            added, since the proto-article may have been multiply injected
+            by a posting agent that predates this standard.  Otherwise,
+            the injecting agent MUST add an Injection-Date header field
+            containing the current date and time.</t>

ALTERNATIVE C:

-            <t>The injecting agent MUST then add an Injection-Date header
-            field containing the current date and time.</t>

I believe the other diffs are either uncontroversial (they are the terminology change from "reinjection" to "multiple injection") or are obvious how to change once we get this one settled.

I think I could easily infer some people's opinion from the postings already made, but let's make the call explicit.

So in order to see if we have a strongly held position in the group, please reply to this message with ONE of the following statements:

* I prefer Alternative A
* I prefer Alternative B
* I prefer Alternative C
* I have another opinion, which is.....

If you want to say anything EXCEPT state your preference, please change the subject line, and DO NOT reply to other people's statement of their opinion.

I'll give this one week (until Monday, August 11, 2008, 12 noon UTC), and then tally responses.

                       Harald

From: Ralph Babel <[email-masked]>
To: [email-masked]
Subject: Re: #1416
Date: Sat,  9 Aug 2008 23:08:53 +0200 (CEST)
Sender: [email-masked]

Lisa Dusseault wrote:

> I would be happy to hear arguments that the current
> draft is worse than the RFC it would replace.

*sigh* ... not _again_.

> If there is consensus around that point
> then I'd retract my promise to process
> the document by the end of the month.

I thought consensus was required to _publish_ an RFC, not
to keep a half-baked document from being advanced to RFC
status more or less by accident or general lack of interest.

Let the mailing-list archives and the drafts serve
as a memorial of what happens when you don't keep
idiots with a political agenda out of a technical WG.

> However, I'd still close the WG as there
> has been no progress for so long.

About time.

| Goals and Milestones:
|
| Apr 97      Identify areas where updates required and write
|             draft standards for inclusion in the main document.
|
| May 97      Produce a revised internet-draft
|             of the Usenet Article Standard.
|
| May-Sep 97  Produce further drafts as needed.
|
| Oct 97      Submit the revised Usenet Article Standard
|             to the IESG for Proposed Standard status

<[email-masked]>

From: Ralph Babel <[email-masked]>
To: [email-masked]
Subject: Re: #1416
Date: Sat,  9 Aug 2008 23:08:52 +0200 (CEST)
Sender: [email-masked]

Charles Lindsey wrote:

> But in this caase I think [...] I would be willing
> to accept a majority view if it was clear we had one.

Accept a majority?

You?

Since when?

> nobody is suggesting abandoning Injection-Date totally

I am.

Usenet does not change because of an RFC. Never has,
never will. It may well die, but it's not gonna change,
regardless of how many RFCs will be published.

A single line of code written by Russ most likely has
a deeper impact on Usenet than anything this WG could
ever come up with.

It's dead, Jim.

And it doesn't even smell funny any more.

> It's just that I think Injection-Date
> will catch on more rapidly if [...]

You're naive.

Even Google abandoned their spaces-in-"Newsgroups:"
experiment just a few months ago, and that was a
relatively minor change proposed _fifteen_ years ago.

To: [email-masked]
Subject: Re: #1416
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 08 Aug 2008 14:22:56 -0700
Sender: [email-masked]

[email-masked] writes:

> I find it unlikley that any further RFC will be produced for news. Hands
> up from everyone here who wants to be in the next IETF USEFOR WG.

Given that I fully intended to pursue standards advancement of NNTP and
then, er, didn't, and have since significantly reduced the amount of time
I spend on netnews-related things and probably will be doing more of that
in the future... I'm afraid my hand's staying down.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
Subject: Re: #1416
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 08 Aug 2008 14:21:32 -0700
Sender: [email-masked]

[email-masked] writes:

> I'm with Frank in being confused by this statement. According to USEFOR:
>
>    A "message identifier" (Section 3.1.3) is a unique identifier for an
>    article, usually supplied by the "user agent" that posted it or,
>    failing that, by the "news server".  It distinguishes the article
>    from every other article ever posted anywhere.  Articles with the
>    same message identifier are treated as if they are the same article
>    regardless of any differences in the body or header fields.
>
> This clearly states that two articles with the same message id, even
> with different dates, are the same article. "Every article ... ever". If
> you say that "message id plus date" is the unique identifier, then you
> are saying that two messages with different dates but identical message
> ids are two different messages -- a violation of the definition of
> message id.

I really think the confusion here is just my unfortunate summary; there is
probably no simple way to talk about this.  From the rest of your message,
you understand what's going on, so we're not really disagreeing.  I think
the best thing to do at this point is to just ignore anything I write in
e-mail that's confusing in favor of paying attention to the language in
the current I-D, since that's all that really matters anyway.

If the language in the current I-D is still confusing, please someone
write something better and I'll incorporate it.  Or, for that matter, pick
up as editor; I'm happy to step down in favor of someone else who has more
time and energy to try to explain all of this.  I'm pretty burned out.

>> It requires, if I remember correctly, badly-timed multiple injection of
>> the message such that it expires on the basis of its original Date and
>> is then reaccepted on the basis of the modified Injection-Date.

> If a message is expired based on the Date header, why would the server
> then accept the same article with the same Date header later?

Becuase of the added Injection-Date header with a newer Date, which
overrides the article's Date header for this purpose.

> If it does, it's just going to expire it again.

Most servers expire based on arrival time, not based on the Date header.
There are various internal data structure reasons why this is often
easier, plus it's arguably a more correct user experience.  See the third
bullet point in the history section in the current draft for the technical
details.

> So the benefit you see is that articles with really old dates will still
> be propogated, even though the date indicates that the message predates
> the start of a local history file and thus the article is assumed to
> have been seen already?

Correct.  That's the intended benefit of Injection-Date.

>> Our protocol, as currently documented in -10, contains nothing about
>> old article prevention except at the injecting agent point (see below).

> By assuming "old date" (defined as "prior to oldest local history file
> entry") means "has already been seen", section 3.3 does implicitly deal
> with old article rejection. Not an important point, but still true.

Right.  But it only does so in a context of not accepting articles that
were already accepted, not in a broader sense of rejecting articles with
old dates only because they have old dates.

I agree that in practice with most servers using the typical history
mechanism, the distinction is purely academic.

>> Date or Injection-Date is a way to avoid having to keep perpetual
>> history to satisfy this requirement.

> Yes, by assuming that "old" means "already seen". Only then can you say
> that not finding a message id in the history means that you've already
> seen the article, when normally not finding the message id means you
> haven't seen it yet.

Exactly.  I don't think we're disagreeing here.

>> The only protocol function (from a USEPRO, not a USEFOR, perspective)
>> of the Date and Injection-Date header is to enable the normal history
>> mechanism to function.
>
> Some headers, sometimes, serve more than just a "protocol function from
> a USEPRO perspective". I see value in Injection-Date as a record of when
> the article was injected versus when it was written.

Right, that's why I worded it that way.  The header serves other purposes
outside of the USEPRO context.

> For that reason, I think I have to object to the exclusion statement and
> require injecting agents to put in an Injection-Date header unless that
> header is already there, without regard to existing message id or date
> headers.

Okay.

You, Harald, and Charles feel that way by my current count, with Forrest,
myself, and Dan on the other side.  I'm not sure which way Frank is
leaning.

Given that the people who want Injection-Date to always be added feel
strongly about it and given that the circumstances when this fails are
obscure, maybe the best approach is to just go forward with always adding
Injection-Date.  As discussed in other threads, I'd really like to
finalize this and get it out the door before we all burn out completely
and never publish anything, and either is a significant improvement over
RFC 1036.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: #1416
Date:  Fri, 8 Aug 2008 22:37:10 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 25
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

<[email-masked]> wrote:
=20
> Hands up from everyone here who wants to be in the
> next IETF USEFOR WG.

I'd look in Netnews drafts tackling topics related
to gateways, i18n (EAI for News), any updates of the
existing stuff, and maybe drafts in the direction of
"dkim" or "useage".

If somebody has the energy to try this.  As former
WG USEFOR qualifies as spectacular failure wrt i18n;
and for my own egocentric calculations I'd be annoyed
if GMaNe cannot support future EAI mailing lists.=20

But fortunately this does not require a new USEFOR WG.

 Frank
--=20
<http://article.gmane.org/gmane.ietf.usenet.format/28390>
| Date: 2005-05-18 13:22:07 GMT (3 years, 11 weeks,
| 5 days, 13 hours and 8 minutes ago)
[...]
| I don't expect to be WG chair for more than about
| 3 months

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: #1416
Date:  Fri, 8 Aug 2008 21:47:16 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 43
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Russ Allbery wrote:

> I'm fairly sure that I didn't say that

| Briefly: the Message-ID plus the Date is
| the unique identifier for the message

>From my POV you wrote this in a message
with a globally unique forever identifier
<[email-masked]>

That this message also had a
| Date: Thu, 07 Aug 2008 18:31:05 -0700
is important for a history of "seen unique
identifiers", but the identifier alone is
supposed to be unique. =20

This identifier cannot be used for another
message, no matter what its Date might be.

Similar the Message-ID=20
<[email-masked]>
is considered as "globally unique forever".

Using this identifier with a date different
from Date: 2 Dec 93 03:20:57 GMT for another
message is not allowed.

> I think you actually understand it

I'd hope that I understood this concept when
folks explained it to me about 13 years ago,
down to the s-o-1036 fine print of "forever".

> just needling me out of some sense of
> personal amusement.  (That amusement, for
> the record, isn't shared.)

The last time anything on this list amused me
was years ago when Bruce "forced" me to read
dozens of old RFCs I've never before heard of.

 Frank

Date: Fri, 8 Aug 2008 12:45:16 -0700 (PDT)
From: [email-masked]
To: [email-masked]
Subject: Re: #1416
Sender: [email-masked]

Lisa Dusseault <lisa@xxxxxxxxxxxxxxxxx>:

> Recall that even if an incomplete revision is made RFC, it can itself be 
> replaced by another RFC.

I find it unlikley that any further RFC will be produced for news. Hands 
up from everyone here who wants to be in the next IETF USEFOR WG.

Date: Fri, 8 Aug 2008 12:36:57 -0700 (PDT)
From: [email-masked]
To: [email-masked]
Subject: Re: #1416
Sender: [email-masked]

Russ Allbery <rra@xxxxxxxxxxxx>:

> Briefly: the Message-ID plus the Date is the unique identifier for the
> message.

I'm with Frank in being confused by this statement. According to USEFOR:

    A "message identifier" (Section 3.1.3) is a unique identifier for an
    article, usually supplied by the "user agent" that posted it or,
    failing that, by the "news server".  It distinguishes the article
    from every other article ever posted anywhere.  Articles with the
    same message identifier are treated as if they are the same article
    regardless of any differences in the body or header fields.

This clearly states that two articles with the same message id, even with 
different dates, are the same article. "Every article ... ever". If you 
say that "message id plus date" is the unique identifier, then you are 
saying that two messages with different dates but identical message ids 
are two different messages -- a violation of the definition of message id.

> If you allow an injecting agent to change one or the other (and
> adding an Injection-Date is effectively changing the Date),

It does not change the Date header.

> It requires, if I remember correctly, badly-timed multiple
> injection of the message such that it expires on the basis of its original
> Date and is then reaccepted on the basis of the modified Injection-Date.

If a message is expired based on the Date header, why would the server 
then accept the same article with the same Date header later? If it does,
it's just going to expire it again. This just doesn't make sense. It 
sounds too much like a bug and not a feature.

> Assuming that the message ever gets to you, yes.  However, unless enough
> other servers implement Injection-Date to allow flood-fill to reach you, a
> message that benefits from this feature won't get to you for you to be
> able to benefit.

So the benefit you see is that articles with really old dates will still 
be propogated, even though the date indicates that the message predates 
the start of a local history file and thus the article is assumed to have 
been seen already?

> Our protocol, as currently documented in -10, contains nothing about old
> article prevention except at the injecting agent point (see below).

By assuming "old date" (defined as "prior to oldest local history file 
entry") means "has already been seen", section 3.3 does implicitly deal
with old article rejection. Not an important point, but still true.

> Date or Injection-Date is a way to avoid having to keep perpetual history
> to satisfy this requirement.

Yes, by assuming that "old" means "already seen". Only then can you say 
that not finding a message id in the history means that you've already 
seen the article, when normally not finding the message id means you 
haven't seen it yet.

> The
> only protocol function (from a USEPRO, not a USEFOR, perspective) of the
> Date and Injection-Date header is to enable the normal history mechanism
> to function.

Some headers, sometimes, serve more than just a "protocol function from a 
USEPRO perspective". I see value in Injection-Date as a record of when the 
article was injected versus when it was written.

For that reason, I think I have to object to the exclusion statement and 
require injecting agents to put in an Injection-Date header unless that 
header is already there, without regard to existing message id or date 
headers.

To: [email-masked]
Subject: Re: #1416
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Fri, 08 Aug 2008 11:19:33 -0700
Sender: [email-masked]

"Frank Ellermann" <[email-masked]> writes:
> Charles Lindsey wrote:

>> in this caase I think both Russ and I would be willing
>> to accept a majority view if it was clear we had one.

> Good, if there is no clear trend, but most still agree
> with "anything is better than nothing" we could flip a
> coin.  

I agree that anything is better than nothing.

> Fortunately there is no time to revisit old rat-holes,
> otherwise I'd wonder why Russ wrote that a Message-ID
> is "more unique" when extended by a timestamp.

I'm fairly sure that I didn't say that, and a quick grep of my e-mail
reveals that I have not used the phrase "more unique" in any e-mail I have
sent for the past month and a half.  I would explain (again) why the Date
is part of article uniqueness in the USEPRO protocol, but I've done so
multiple times before, I think the draft covers it, and I think you
actually understand it and are just needling me out of some sense of
personal amusement.  (That amusement, for the record, isn't shared.)

We've discussed how article uniqueness works in netnews repeatedly; please
don't take cheap shots at a simplified summary.  If you don't think we
have time for sort of nitpicking, then actually don't do it.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Fri, 08 Aug 2008 14:16:36 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To:  [email-masked]
Subject: Re: #1416
Sender: [email-masked]

> Fortunately there is no time to revisit old rat-holes,
> otherwise I'd wonder why Russ wrote that a Message-ID
> is "more unique" when extended by a timestamp.

That is because the server's record of message IDs is a
finite list in practice.

No one should offer an opinion on how to resolve #1416
unless and until they understand this point.

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: #1416
Date:  Fri, 8 Aug 2008 19:40:41 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 17
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Charles Lindsey wrote:

> in this caase I think both Russ and I would be willing
> to accept a majority view if it was clear we had one.

Good, if there is no clear trend, but most still agree
with "anything is better than nothing" we could flip a
coin. =20

With two "candidates" (1 =3D Russ', 2 =3D your version) and
the NomCom 2008 entropy sources you'd get either 1 or 2.

Fortunately there is no time to revisit old rat-holes,
otherwise I'd wonder why Russ wrote that a Message-ID
is "more unique" when extended by a timestamp.

 Frank

To: [email-masked]
Xref: clerew local.usefor:24797
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416
Date: Fri, 8 Aug 2008 12:25:24 GMT
Lines: 51
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>[email-masked] writes:

>> I don't think I understand the issue, because I don't see the reason for
>> the exemption of "has date and message id" from insertion of Injection
>> Date.

>Briefly: the Message-ID plus the Date is the unique identifier for the
>message.  If you allow an injecting agent to change one or the other (and
>adding an Injection-Date is effectively changing the Date), you break
>uniqueness.  This creates the possibility of duplicates.

Which is, of course, why we REQUIRE people doing multiple injections to
insert all these things themselves before injecting any of them.

>In this case, the scenario under which duplicates arise is possible but
>unlikely.  It requires, if I remember correctly, badly-timed multiple
>injection of the message such that it expires on the basis of its original
>Date and is then reaccepted on the basis of the modified Injection-Date.

It requires the badly timed events to cause arrival at some site within the
critical 24 hours mentioned in the text, having failed to flood around the
site that introduced the delay. Moreover, the duplicate copies that a user
at that site sees will only differ in the presence of an Injection-Date in
the one, but not in the other. The full scenario is explained in
http://www.imc.org/ietf-usefor/drafts/issue-1416.

>> Isn't the benefit of the Injection-Date header available to any server
>> that is updated to read it, regardless of any other transport server?

>Assuming that the message ever gets to you, yes.  However, unless enough
>other servers implement Injection-Date to allow flood-fill to reach you, a
>message that benefits from this feature won't get to you for you to be
>able to benefit.

The benefit of Injection-Date is that a badly delayed article is more
likely to reach you, assuming you and your upstreams have taken advantage
of it, of course. I think you are more likely to get that benefit if the
disputed sentence is omitted.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24796
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416
Date: Fri, 8 Aug 2008 12:00:25 GMT
Lines: 40
Sender: [email-masked]

In <[email-masked]> "Frank Ellermann" <[email-masked]> writes:

>Charles Lindsey wrote:

>> Russ's vote for IR is cancelled out by my vote for IC

>My enthusiasm for voting in technical questions is limited.

>Harald noted elsewhere that the idea of humming is to find
>rough consensus or its absence without counting.  Here with
>the very limited participation it is anyway not compelling.

Voting, or rather "counting heads" will give an indication of rough
consensus if there is a substantial majority one way or the other. If it
is evenly balanced, then you are still stuck of course.

But in this caase I think both Russ and I would be willing to accept a
majority view if it was clear we had one.

>But I do think that the Injection-Date in the approved RFC
>represents a *possible* and plausible concept.

Sure, but nobody is suggesting abandoning Injection-Date totally - it is
just a matter of who is supposed to add it and when. As I just explained
to Lisa, even Russ's version of #1416 is better than no version at all.

It's just that I think Injection-Date will catch on more rapidly if we
allow Injecting Agents to add it more-or-less always, in spite of the very
rare risk that an occasional user might see the same article twice.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.usefor:24795
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416
Date: Fri, 8 Aug 2008 11:52:09 GMT
Lines: 27
Sender: [email-masked]

In <[email-masked]> Lisa Dusseault <[email-masked]> writes:

>On Aug 7, 2008, at 4:14 PM, [email-masked] wrote:

>The WG came to consensus around most of the latest I-D with the  
>exception of issue #1416, and partial consensus is not the same as no  
>consensus when we look at the whole document.  Recall that even if an  
>incomplete revision is made RFC, it can itself be replaced by another  
>RFC.  Perfection is never attainable, mere improvement is often  
>helpful, and the process is iterative.

I think you need the various changes that Russ has recently put before us,
including the resolution to #1416. I.e., it is better with that text in
whether the sentence that I want removed is removed or not, though
obviously I prefer it with the sentence removed.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Cc: [email-masked]
From: Lisa Dusseault <[email-masked]>
To: [email-masked]
Subject: Re: #1416
Date: Thu, 7 Aug 2008 18:53:32 -0700
Sender: [email-masked]

On Aug 7, 2008, at 4:14 PM, [email-masked] wrote:

>
> As a side note, I'm a bit dissappointed that IETF will grab an  
> unfinished work and publish it as the product of this group, as if  
> it were the completed, consensus product. If a group cannot reach  
> consensus on a draft, and simply stops working on it (as this group  
> appears to have), then the product is not ready for publication.

I would be happy to hear arguments that the current draft is worse  
than the RFC it would replace.  If there is consensus around that  
point then I'd retract my promise to process the document by the end  
of the month.  However, I'd still close the WG as there has been no  
progress for so long.

The WG came to consensus around most of the latest I-D with the  
exception of issue #1416, and partial consensus is not the same as no  
consensus when we look at the whole document.  Recall that even if an  
incomplete revision is made RFC, it can itself be replaced by another  
RFC.  Perfection is never attainable, mere improvement is often  
helpful, and the process is iterative.

Lisa

To: [email-masked]
Subject: Re: #1416
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Thu, 07 Aug 2008 18:31:05 -0700
Sender: [email-masked]

[email-masked] writes:

> I don't think I understand the issue, because I don't see the reason for
> the exemption of "has date and message id" from insertion of Injection
> Date.

Briefly: the Message-ID plus the Date is the unique identifier for the
message.  If you allow an injecting agent to change one or the other (and
adding an Injection-Date is effectively changing the Date), you break
uniqueness.  This creates the possibility of duplicates.

In this case, the scenario under which duplicates arise is possible but
unlikely.  It requires, if I remember correctly, badly-timed multiple
injection of the message such that it expires on the basis of its original
Date and is then reaccepted on the basis of the modified Injection-Date.

> Russ writes:
>
> * The Injection-Date feature will not realistically be able to be used to
>   allow for articles with very old Date headers for the forseeable future,
>   possibly ever.  It will require upgrading all servers to use
>   Injection-Date rather than Date, which I think is very unlikely to
>   happen.
>
> Isn't the benefit of the Injection-Date header available to any server
> that is updated to read it, regardless of any other transport server?

Assuming that the message ever gets to you, yes.  However, unless enough
other servers implement Injection-Date to allow flood-fill to reach you, a
message that benefits from this feature won't get to you for you to be
able to benefit.

> The comment about servers detecting duplicates by Injection-Date is
> confusing. Injection-Date isn't how duplicates are supposed to be
> detected. Yes, I see section 3.3 in the draft, but that section is
> incorrectly combining duplicate suppression and old-article prevention.
> An injection date of 20 days ago doesn't mean the article is a
> duplicate; it means the article is simply old. The assumption is that
> "older than history" means "appeared in history", which is not a bad
> assumption, but mixes two different issues.

Our protocol, as currently documented in -10, contains nothing about old
article prevention except at the injecting agent point (see below).  There
are no requirements that relaying agents or serving agents reject old
articles, only a requirement that servers reject articles that they've
already seen.

Date or Injection-Date is a way to avoid having to keep perpetual history
to satisfy this requirement.

Individual sites can of course reject old articles, as well as anything
else they don't want to carry, but it's not a protocol requirement.  The
only protocol function (from a USEPRO, not a USEFOR, perspective) of the
Date and Injection-Date header is to enable the normal history mechanism
to function.

> An article with a Date header 20 days in the past and a current
> Injection Date is just as old, and 3.3 allows either or both to be used
> in rejecting "duplicates". Un-updated servers will use Date. Updated
> servers should be using both. Problem solved.

If updated relaying or serving agents use both when both are present, it
completely defeats the point of having Injection-Date and we should just
remove it.  If that's the implication that you got from 3.3, the wording
is buggy.  That was supposed to be covered by this:

   (In the following discussion, the "date" of an article is defined to be
   the date represented by its Injection-Date header field if present,
   otherwise its Date header field.)

implying that if Injection-Date is present, Date is ignored.

Injection agents are encouraged to check for stale Dates due to the
problem of silent discards of those messages from un-updated agents, but
the threshold they use is expected to be long enough that articles with
less stale Dates may still potentially benefit from Injection-Date at
later points in their propagation.

> Now, if a posting agent is handing an injection agent an article with a
> CURRENT date header and message id and is truly a delayed (by twenty
> days?) multiple injection attempt, the problem is a broken posting
> agent, and leaving out the Injection Date header is not going to assist
> anyone in rejecting the article.

Yes.  This isn't the scenario.

> As a side note, I'm a bit dissappointed that IETF will grab an
> unfinished work and publish it as the product of this group, as if it
> were the completed, consensus product. If a group cannot reach consensus
> on a draft, and simply stops working on it (as this group appears to
> have), then the product is not ready for publication.

I generally agree with this.  I'm not willing to see -10 published; the
current text is not a particularly useful direction to go, and I think we
reached a consensus on that a long time ago.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Thu, 7 Aug 2008 16:14:26 -0700 (PDT)
From: [email-masked]
To: [email-masked]
Subject: Re: #1416
Sender: [email-masked]

I don't think I understand the issue, because I don't see the reason for 
the exemption of "has date and message id" from insertion of Injection 
Date.

Russ writes:

* The Injection-Date feature will not realistically be able to be used to
   allow for articles with very old Date headers for the forseeable future,
   possibly ever.  It will require upgrading all servers to use
   Injection-Date rather than Date, which I think is very unlikely to
   happen.

Isn't the benefit of the Injection-Date header available to any server 
that is updated to read it, regardless of any other transport server? 
Those that do not read it will do nothing different; those that do can use 
it. It would be nice if all servers updated, but the benefits don't 
require it.

Also Russ:

* Given the very slow development pace of netnews software, I think
   backward compatibility should be our highest goal.

I don't see a backward compatibility issue. It appears that the issue 
involves "injection of an article with message id and date header", and 
the assumption that such an article is part of a multiple-injection set. 
What harm is there if an Injection-Date is added to such an article?

The comment about servers detecting duplicates by Injection-Date is 
confusing. Injection-Date isn't how duplicates are supposed to be 
detected. Yes, I see section 3.3 in the draft, but that section is 
incorrectly combining duplicate suppression and old-article prevention. An 
injection date of 20 days ago doesn't mean the article is a duplicate; it 
means the article is simply old. The assumption is that "older than 
history" means "appeared in history", which is not a bad assumption, but 
mixes two different issues.

An article with a Date header 20 days in the past and a current Injection 
Date is just as old, and 3.3 allows either or both to be used in rejecting
"duplicates". Un-updated servers will use Date. Updated servers should be 
using both. Problem solved.

Now, if a posting agent is handing an injection agent an article with a 
CURRENT date header and message id and is truly a delayed (by twenty 
days?) multiple injection attempt, the problem is a broken posting agent, 
and leaving out the Injection Date header is not going to assist anyone in 
rejecting the article. If it has an old date header, then that date can be 
used under 3.3 just as the Injection Date can. A current date and current 
injection-date would still require a history of the message id to be 
branded a duplicate.

As a side note, I'm a bit dissappointed that IETF will grab an unfinished 
work and publish it as the product of this group, as if it were the 
completed, consensus product. If a group cannot reach consensus on a 
draft, and simply stops working on it (as this group appears to have), 
then the product is not ready for publication.

To: [email-masked]
From:  "Frank Ellermann" <[email-masked]>
Subject:  Re: #1416
Date:  Thu, 7 Aug 2008 18:18:17 +0200
Organization:  <http://purl.net/xyzzy>
Lines: 40
Reply-To:  "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Charles Lindsey wrote:

> Russ's vote for IR is cancelled out by my vote for IC

My enthusiasm for voting in technical questions is limited.

Harald noted elsewhere that the idea of humming is to find
rough consensus or its absence without counting.  Here with
the very limited participation it is anyway not compelling.

But I do think that the Injection-Date in the approved RFC
represents a *possible* and plausible concept.  Overruling
that older IETF consensus because the concept is actually
not as possible as the WG hoped after years of discussions
is IMO a bad idea, and nobody is going to modify this older
decision anytime soon, let alone in the next nineteen days.

Even if Injection-Date as intended turns out to be wishful
thinking in practice.  There is an "official" procedure to
remove features from standards:  This can be done when the
future implementation and interop reports for a promotion
on "standards track" show that this feature didn't get the
traction justifying to keep it as intended.

IMO usefor-usepro should reflect the usefor-usefor design,
and not try a roll-back.  After some time - not less than
six months, but likely years, if at all - folks with the
energy to promote these two documents to "draft standard"
can fix whatever needs fixing (*1). =20

What is most important now is to get a "consistent story"
out (as RFCs with numbers, not rotting in an editor queue)
before the WG is terminated in about 19*24 hours.

 Frank
--=20
*1: There will be fixes, if anybody tries to promote the
    documents, e.g., a simplified <top-label> TBD "soon",
    and the simplified <msg-id-core> of 2822upd as "soon"
    as 2822upd can be approved and published.

Date: Thu, 7 Aug 2008 09:48:30 -0400
From: Dan Schlitt <[email-masked]>
cc: [email-masked]
Subject: Re: #1416
Sender: [email-masked]

I also think that Russ has the stronger position.

/dan

-- 

Dan Schlitt
[email-masked]

On Thu, 7 Aug 2008, Forrest J. Cavalier III wrote:

>
> Charles Lindsey wrote:
> > Agreed. Clearly Russ's vote for IR is cancelled out by my vote for IC, so
> > it is the rest of the WG that gets to make the choice. But we need to get
> > that choice made so that we can move on.
> >
>
> I'm with Russ, if that matters to the chair.
>
> I have no idea why every vote counts equally here, for two reasons.
>
> 1. Russ has experience writing and maintaining news server software for a
> production environment.  I do too.  Others here do too.
>
> 2. Technical merit matters.  I agree completely with Russ's statements
> about how rapidly software will get changed in production.  There is
> lots of evidence for that in the last decade.  I believe that
> the choices in what you call "IR" are appropriate for that.
>

Date: Thu, 07 Aug 2008 08:20:57 -0400
From: "Forrest J. Cavalier III" <[email-masked]>
To:  [email-masked]
Subject: Re: #1416
Sender: [email-masked]

Charles Lindsey wrote:
> Agreed. Clearly Russ's vote for IR is cancelled out by my vote for IC, so
> it is the rest of the WG that gets to make the choice. But we need to get
> that choice made so that we can move on.
> 

I'm with Russ, if that matters to the chair.

I have no idea why every vote counts equally here, for two reasons.

1. Russ has experience writing and maintaining news server software for a 
production environment.  I do too.  Others here do too.

2. Technical merit matters.  I agree completely with Russ's statements
about how rapidly software will get changed in production.  There is
lots of evidence for that in the last decade.  I believe that
the choices in what you call "IR" are appropriate for that.

To: [email-masked]
Xref: clerew local.usefor:24789
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416
Date: Thu, 7 Aug 2008 10:44:17 GMT
Lines: 45
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>This is the topic that we've gone back and forth on at some length.  To
>summarize my opinion:

>* I'm not sure that having all messages have Injection-Date header fields
>  should be a general goal of this work.  The header is primarily useful
>  when Date is well in the past and not that useful otherwise.  It's
>  certainly fine if we get there, but I don't think getting there is a
>  high priority.

>* The Injection-Date feature will not realistically be able to be used to
>  allow for articles with very old Date headers for the forseeable future,
>  possibly ever.  It will require upgrading all servers to use
>  Injection-Date rather than Date, which I think is very unlikely to
>  happen.  Accordingly, the benefit of servers adding Injection-Date is
>  fairly minor for the forseeable future; the only effect will be somewhat
>  more correct behavior with articles with borderline Dates.

>* Given the very slow development pace of netnews software, I think
>  backward compatibility should be our highest goal.

Not surprisingly, I take an opposite view on all of the above.

>I'm unlikely to change my mind in this area, but I'm happy to be
>overridden by the working group if other people just don't agree with me.
>If that happens, I'll gladly prepare a document that implements Charles's
>approach.

>I think a working group chair needs to decide if that's happened.

Agreed. Clearly Russ's vote for IR is cancelled out by my vote for IC, so
it is the rest of the WG that gets to make the choice. But we need to get
that choice made so that we can move on.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Subject: Re: #1416
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Wed, 06 Aug 2008 14:54:55 -0700
Sender: [email-masked]

Harald Alvestrand <[email-masked]> writes:

>> -            <t>The injecting agent MUST then add an Injection-Date header
>> -            field containing the current date and time.</t>
>> +            <t>If the proto-article already had an Injection-Date header
>> +            field, it MUST NOT be modified or replaced.  If the
>> +            proto-article had both a Message-ID header field and a Date
>> +            header field, an Injection-Date header field MUST NOT be
>> +            added, since the proto-article may have been multiply injected
>> +            by a posting agent that predates this standard.  Otherwise,
>> +            the injecting agent MUST add an Injection-Date header field
>> +            containing the current date and time.</t>

> This ("if the prot-article had both a Message-ID header field and a Date
> header field") is the place that bothers me most, since it will mean
> that singly injected articles produced by "modern" UAs that generate
> their own message-IDs will never get an Injection-Date header field.

Yes, this is the point of disagreement.

> It seems to sacrifice a general goal (that messages should have
> Injection-Date) for a narrow benefit (that multiple copies of multiply
> injected articles that are injected over a large span of time won't be
> detected as duplicates by servers that check Injection-Date and ignore
> Date).

> I'm not sure that's the right tradeoff.

This is the topic that we've gone back and forth on at some length.  To
summarize my opinion:

* I'm not sure that having all messages have Injection-Date header fields
  should be a general goal of this work.  The header is primarily useful
  when Date is well in the past and not that useful otherwise.  It's
  certainly fine if we get there, but I don't think getting there is a
  high priority.

* The Injection-Date feature will not realistically be able to be used to
  allow for articles with very old Date headers for the forseeable future,
  possibly ever.  It will require upgrading all servers to use
  Injection-Date rather than Date, which I think is very unlikely to
  happen.  Accordingly, the benefit of servers adding Injection-Date is
  fairly minor for the forseeable future; the only effect will be somewhat
  more correct behavior with articles with borderline Dates.

* Given the very slow development pace of netnews software, I think
  backward compatibility should be our highest goal.

I'm unlikely to change my mind in this area, but I'm happy to be
overridden by the working group if other people just don't agree with me.
If that happens, I'll gladly prepare a document that implements Charles's
approach.

I think a working group chair needs to decide if that's happened.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

Date: Tue, 05 Aug 2008 12:00:53 +0100
From: Harald Alvestrand <[email-masked]>
To: Russ Allbery <[email-masked]>
CC:  [email-masked]
Subject: Re: #1416
Sender: [email-masked]

>
> -            <t>The injecting agent MUST then add an Injection-Date header
> -            field containing the current date and time.</t>
> +            <t>If the proto-article already had an Injection-Date header
> +            field, it MUST NOT be modified or replaced.  If the
> +            proto-article had both a Message-ID header field and a Date
> +            header field, an Injection-Date header field MUST NOT be
> +            added, since the proto-article may have been multiply injected
> +            by a posting agent that predates this standard.  Otherwise,
> +            the injecting agent MUST add an Injection-Date header field
> +            containing the current date and time.</t>
This ("if the prot-article had both a Message-ID header field and a Date 
header field") is the place that bothers me most, since it will mean 
that singly injected articles produced by "modern" UAs that generate 
their own message-IDs will never get an Injection-Date header field.

It seems to sacrifice a general goal (that messages should have 
Injection-Date) for a narrow benefit (that multiple copies of multiply 
injected articles that are injected over a large span of time won't be 
detected as duplicates by servers that check Injection-Date and ignore 
Date).

I'm not sure that's the right tradeoff.

                        Harald

To: [email-masked]
Xref: clerew local.usefor:24786
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: #1416
Date: Mon, 4 Aug 2008 14:08:49 GMT
Lines: 72
Sender: [email-masked]

In <[email-masked]> Russ Allbery <[email-masked]> writes:

>"Charles Lindsey" <[email-masked]> writes:

>> That left just one small worry. I wanted (version IC) Injection Agents
>> to _always_ insert an Injection-Date (except when it was already
>> present, of course). Russ argued (version IR) that this would sometimes
>> cause existing implementations to behave oddly, and proposed a
>> less-intuitive rule for when Injection Agents should insert it.

>I think there's also a remaining point of disagreement over whether
>injection agents SHOULD reject articles with a stale Date.

>I just submitted draft-ietf-usefor-usepro-10 including the (I believe
>non-controversial) new history section and corresponding rewordings in the
>duties of a relaying and serving agent.  Below is the remaining diff for
>my solution for #1416.  Please note if any of the below is uncontroversial
>so that I can commit those sections and reduce the diff to only the
>disputed portion.

90% of what follows is agreed, and it might be helpful to get it into an
ID so that we can see it all in context, and then argue over the small
remaining difference. Such concerns as I have with the rest of it are all
in the "niggles" category.

>             <t>It SHOULD reject any proto-article whose Newsgroups header
>             field does not contain at least one &lt;newsgroup-name> for a
>@@ -770,8 +829,14 @@
>             the source of the article and possibly other trace information
>             as described in Section 3.2.8 of <xref target="USEFOR" />.</t>
> 
>-            <t>The injecting agent MUST then add an Injection-Date header
>-            field containing the current date and time.</t>
>+            <t>If the proto-article already had an Injection-Date header
>+            field, it MUST NOT be modified or replaced.  If the
>+            proto-article had both a Message-ID header field and a Date
>+            header field, an Injection-Date header field MUST NOT be
>+            added, since the proto-article may have been multiply injected
>+            by a posting agent that predates this standard.  Otherwise,
>+            the injecting agent MUST add an Injection-Date header field
>+            containing the current date and time.</t>

That is the principle point of difference between IR and IC. I would
prefer it to say

>+            <t>If the proto-article already had an Injection-Date header
>+            field, it MUST NOT be modified or replaced.  Otherwise,
>+            the injecting agent MUST add an Injection-Date header field
>+            containing the current date and time.</t>

If that change is made, there are probably a few consequential changes
and niggles elsewhere, but that is the essence of it.

In that case, there is a small possibility that, until all agents have
been upgraded, a few users may occasionally see an article they have seen
before. See <http://www.imc.org/ietf-usefor/drafts/issue-1416> for the
somewhat complicated chain of events that can give rise to this.

OTOH, without my change, most articles will never receive an Injection-Date
at all, and that will continue to be the case undefinitely (unless Posting
agents routinely start to do it, which I consider unlikely).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Subject: Re: #1416
From: Russ Allbery <[email-masked]>
Organization: The Eyrie
Date: Sun, 03 Aug 2008 16:37:43 -0700
Sender: [email-masked]

"Charles Lindsey" <[email-masked]> writes:

> That left just one small worry. I wanted (version IC) Injection Agents
> to _always_ insert an Injection-Date (except when it was already
> present, of course). Russ argued (version IR) that this would sometimes
> cause existing implementations to behave oddly, and proposed a
> less-intuitive rule for when Injection Agents should insert it.

I think there's also a remaining point of disagreement over whether
injection agents SHOULD reject articles with a stale Date.

I just submitted draft-ietf-usefor-usepro-10 including the (I believe
non-controversial) new history section and corresponding rewordings in the
duties of a relaying and serving agent.  Below is the remaining diff for
my solution for #1416.  Please note if any of the below is uncontroversial
so that I can commit those sections and reduce the diff to only the
disputed portion.

--- usepro.xml	2008-08-03 16:12:51.000000000 -0700
+++ usepro-1416.xml	2008-08-03 16:34:50.000000000 -0700
@@ -18,6 +18,8 @@
     'http://xml.resource.org/public/rfc/bibxml/reference.RFC.2822.xml'>
   <!ENTITY rfc3629 PUBLIC '' 
     'http://xml.resource.org/public/rfc/bibxml/reference.RFC.3629.xml'>
+  <!ENTITY rfc3798 PUBLIC '' 
+    'http://xml.resource.org/public/rfc/bibxml/reference.RFC.3798.xml'>
   <!ENTITY rfc3977 PUBLIC '' 
     'http://xml.resource.org/public/rfc/bibxml/reference.RFC.3977.xml'>
   <!ENTITY rfc4234 PUBLIC '' 
@@ -165,8 +167,9 @@
 
         <t>"Injecting" an article is the processing of a proto-article by
         an injecting agent.  Normally this action is done once and only
-        once for a given article.  "Reinjecting" an article is passing an
-        already-injected article to an injection agent.</t>
+        once for a given article.  "Multiple injection" is passing the
+        same article to multiple injecting agents, either serially or in
+        parallel, by one or several posting agents.</t>
 
         <t>A "gateway" is software which receives news articles and
         converts them to messages of some other kind (such as <xref
@@ -519,9 +522,34 @@
 
         <t>Posting agents SHOULD ensure that proto-articles they create
         are valid according to <xref target="USEFOR" /> and any other
-        applicable policies.  They MUST NOT create any Injection-Date or
-        Injection-Info header fields; these headers will be added by the
-        injecting agent.</t>
+        applicable policies.  They MUST NOT create any Injection-Info
+        header field; this header field may only be added by the injecting
+        agent.</t>
+
+        <t>If the proto-article already contains both Message-ID and Date
+        header fields, posting agents MAY add an Injection-Date header
+        field to that proto-article immediately before passing that
+        proto-article to an injection agent.  They SHOULD do so if the
+        Date header field (representing the composition time of the
+        proto-article) is more than a day in the past at the time of
+        injection.  They MUST do so if the proto-article is being
+        submitted to more than one injecting agent; see <xref
+        target="multi-injection" />.</t>
+
+        <t>The Injection-Date header field is new in this revision of the
+        Netnews protocol and is designed to allow the Date header field to
+        hold the composition date (as recommended in section 3.6.1 of
+        <xref target="RFC2822" />), even if the proto-article is not to be
+        injected for some time after its composition.  However, note that
+        all implementations predating this specification ignore the
+        Injection-Date header field and use the Date header field in its
+        stead for rejecting articles older than their cutoff (see <xref
+        target="history" />), and injecting agents predating this
+        specification do not add an Injection-Date header.  Articles with
+        a Date header field substantially in the past will still be
+        rejected by implementations predating this specification,
+        regardless of the Injection-Date header field, and hence may
+        suffer poorer propagation.</t>
 
         <t>Contrary to <xref target="RFC2822" />, which implies that the
         mailbox or mailboxes in the From header field should be that of
@@ -544,48 +572,75 @@
           agent.</t>
 
           <t>A proto-article has the same format as a normal article
-          except that the Injection-Date, Injection-Info, and Xref header
-          fields MUST NOT be present; the Path header field MUST NOT
-          contain a "POSTED" &lt;diag-keyword>; and any of the following
-          mandatory header fields MAY be omitted: Message-ID, Date, and
-          Path.  In all other respects, a proto-article MUST be a valid
-          Netnews article.  In particular, the header fields which may be
-          omitted MUST NOT be present with invalid content.</t>
+          except that the Injection-Info and Xref header fields MUST NOT
+          be present; the Path header field MUST NOT contain a "POSTED"
+          &lt;diag-keyword>; and any of the following mandatory header
+          fields MAY be omitted: Message-ID, Date, and Path.  In all other
+          respects, a proto-article MUST be a valid Netnews article.  In
+          particular, the header fields which may be omitted MUST NOT be
+          present with invalid content.</t>
 
           <t>If a posting agent intends to offer the same proto-article to
-          multiple injecting agents, the header fields Message-ID and Date
-          MUST be present and identical in all copies of the
-          proto-article.</t>
+          multiple injecting agents, the header fields Message-ID, Date,
+          and Injection-Date MUST be present and identical in all copies
+          of the proto-article.  See <xref target="multi-injection" />.</t>
         </section>
 
-        <section anchor="reinjection" title="Reinjection of Articles">
-          <t>A given article SHOULD be processed by an injecting agent
-          once and only once.  The Injection-Date or Injection-Info
-          header fields are added by an injecting agent and are not
-          permitted in a proto-article.  Their presence (or the presence
-          of other unstandardized or obsolete trace headers such as
-          NNTP-Posting-Host, NNTP-Posting-Date, or X-Trace) indicates
-          that the proto-article is instead an article and has already
-          been processed by an injecting agent.  A posting agent SHOULD
-          normally reject such articles.</t>
-
-          <t>In the exceptional case that an article needs to be
-          reinjected for some reason (such as transferring an article from
-          one Netnews to another where those networks have no relaying
-          agreement), the posting agent doing the reinjection MUST convert
-          the article back into a proto-article before passing it to an
-          injecting agent (such as by renaming the Injection-Info and
-          Injection-Date header fields and removing any Xref header field)
-          and MUST perform the date checks on the existing Injection-Date
-          or Date header fields that would otherwise be done by the
-          injecting agent.</t>
-
-          <t>Reinjecting articles may cause loops, loss of trace
-          information, and other problems and should only be done with
-          care and when there is no available alternative.  A posting
-          agent that does reinjection is a limited type of gateway and as
-          such is subject to all of the requirements of an incoming
-          gateway in addition to the requirements of a posting agent.</t>
+        <section anchor="multi-injection"
+                 title="Multiple Injection of Articles">
+          <t>Under some circumstances (posting to multiple disjoint
+          networks, injecting agents with spotty connectivity, or for
+          redundancy, for example), a posting agent may wish to offer the
+          same article to multiple injecting agents.  In this unusual
+          case, the goal is to not create multiple independent articles
+          but rather to inject the same article at multiple points and let
+          the normal duplicate suppression facility of Netnews (see <xref
+          target="history" />) ensure that any given agent accepts the
+          article only once, even if supposedly disjoint networks have
+          unexpected links.</t>
+
+          <t>Whenever possible, multiple injection SHOULD be done by
+          offering the same proto-article to multiple injecting agents.
+          The posting agent MUST supply the Message-ID, Date, and
+          Injection-Date header fields, and the proto-article as offered
+          to each injecting agent MUST be identical.</t>
+
+          <t>In some cases, offering the same proto-article to all
+          injecting agents may not be possible (such as when gatewaying,
+          after the fact, articles found on one Netnews network to
+          another, supposedly unconnected one).  In this case, the posting
+          agent MUST convert the article back into a proto-article before
+          passing it to another injecting agent, but it MUST retain
+          unmodified the Message-ID, Date, and Injection-Date header
+          fields.  It MUST NOT add an Injection-Date header field if it is
+          missing from the existing article.  It MUST remove any Xref
+          header field and either rename or remove any Injection-Info
+          header field and other trace fields.
+            <list style="empty">
+              <t>NOTE: Multiple injection inherently risks duplicating
+              articles.  Multiple injection after the fact, by converting
+              an article back to a proto-article and injecting it again,
+              additionally risks loops, loss of trace information,
+              unintended repeat injection into the same network, and other
+              problems.  It should be done with care and only when there
+              is no alternative.  The requirement to retain Message-ID,
+              Date, and Injection-Date header fields minimizes the
+              possibility of a loop and ensures that the newly injected
+              article is not treated as a new, separate article.</t>
+            </list>
+          </t>
+
+          <t>Multiple injection of an article listing one or more
+          moderated newsgroups in its Newsgroups header field SHOULD only
+          be done by a moderator and MUST only be done after the
+          proto-article has been approved for all moderated groups to
+          which it is to be posted and has an Approved header field (see
+          <xref target="moderator" />).  Multiple injection of an
+          unapproved article intended for moderated newsgroups will
+          normally only result in the moderator receiving multiple copies,
+          and if the newsgroup status is not consistent across all
+          injecting agents, may result in duplication of the article or
+          other problems.</t>
         </section>
 
         <section anchor="followups" title="Followups">
@@ -710,23 +765,27 @@
 
             <t>It MUST reject any proto-article that does not have the
             proper mandatory header fields for a proto-article; that has
-            Injection-Date, Injection-Info, or Xref header fields; that
-            has a Path header field containing the "POSTED"
-            &lt;diag-keyword>; or that is not syntactically valid as
-            defined by <xref target="USEFOR" />.  It SHOULD reject any
-            proto-article which contains a header field deprecated for
-            Netnews.  It MAY reject any proto-article that contains trace
-            header fields indicating that it was already injected by an
-            injecting agent that did not add Injection-Info or
-            Injection-Date.</t>
-
-            <t>It SHOULD reject any article whose Date header field is
-            more than 24 hours into the future (and MAY use a margin less
-            than 24 hours).  It SHOULD reject any article whose Date
-            header appears to be stale (more than 72 hours into the past,
-            for example, or too old to still be recorded in the database
-            of a relaying agent the injecting agent will be using) since
-            not all news servers support Injection-Date.</t>
+            Injection-Info or Xref header fields; that has a Path header
+            field containing the "POSTED" &lt;diag-keyword>; or that is
+            not syntactically valid as defined by <xref target="USEFOR"
+            />.  It SHOULD reject any proto-article which contains a
+            header field deprecated for Netnews (see, for example, <xref
+            target="RFC3798" />).  It MAY reject any proto-article that
+            contains trace header fields (e.g., NNTP-Posting-Host)
+            indicating that it was already injected by an injecting agent
+            that did not add Injection-Info or Injection-Date.</t>
+
+            <t>It SHOULD reject any article whose Injection-Date or Date
+            header field is more than 24 hours into the future (and MAY
+            use a margin less than 24 hours).  It SHOULD reject any
+            article whose Injection-Date header field is too far in the
+            past (older than the cutoff interval of a relaying agent the
+            injecting agent is using, for example).  It SHOULD similarly
+            reject any article whose Date header field is too far in the
+            past, since not all news servers support Injection-Date and
+            only the injecting agent can provide a useful error message to
+            the posting agent.  In either case, this interval SHOULD NOT
+            be any shorter than 72 hours into the past.</t>
 
             <t>It SHOULD reject any proto-article whose Newsgroups header
             field does not contain at least one &lt;newsgroup-name> for a
@@ -770,8 +829,14 @@
             the source of the article and possibly other trace information
             as described in Section 3.2.8 of <xref target="USEFOR" />.</t>
 
-            <t>The injecting agent MUST then add an Injection-Date header
-            field containing the current date and time.</t>
+            <t>If the proto-article already had an Injection-Date header
+            field, it MUST NOT be modified or replaced.  If the
+            proto-article had both a Message-ID header field and a Date
+            header field, an Injection-Date header field MUST NOT be
+            added, since the proto-article may have been multiply injected
+            by a posting agent that predates this standard.  Otherwise,
+            the injecting agent MUST add an Injection-Date header field
+            containing the current date and time.</t>
 
             <t>Finally, the injecting agent forwards the article to one or
             more relaying agents, and the injection process is
@@ -1068,8 +1133,7 @@
             for reasons understood by the moderator (such as delays in the
             moderation process) in which case they MAY substitute the
             current date.  Any Injection-Date, Injection-Info, or Xref
-            header fields already present (though there should be none)
-            MUST be removed.</t>
+            header fields already present MUST be removed.</t>
 
             <t>Any Path header field MUST either be removed or truncated
             to only those entries following its "POSTED"
@@ -2109,6 +2173,7 @@
       &rfc1036;
       &rfc2045;
       &rfc2606;
+      &rfc3798;
       &rfc3977;
       <reference anchor="USEAGE">
         <front>

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

From: [email-masked]
To: [email-masked]
Cc: [email-masked]
Subject: I-D Action:draft-ietf-usefor-usepro-10.txt 
Date: Sun,  3 Aug 2008 16:30:02 -0700 (PDT)
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Usenet Article Standard Update Working Group of the IETF.

	Title           : Netnews Architecture and Protocols
	Author(s)       : R. Allbery, C. Lindsey
	Filename        : draft-ietf-usefor-usepro-10.txt
	Pages           : 50
	Date            : 2008-08-03

This document defines the architecture of Netnews systems and
specifies the correct manipulation and interpretation of Netnews
articles by software which originates, distributes, stores, and
displays them.  It also specifies the requirements that must be met
by any protocol used to transport and serve Netnews articles.Internet
Draft Comments

Comments are solicited and should be addressed to the Usenet Format
Working Group at [email-masked].

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-usefor-usepro-10.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Message/External-body;
	name="draft-ietf-usefor-usepro-10.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:     <[email-masked]>

--NextPart--

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sun,  3 Aug 2008 16:15:36 -0700 (PDT)
Sender: [email-masked]

    Date: Sunday, August 3, 2008 @ 16:15:34
  Author: eagle
Revision: 4229

Purely syntactical fixes to eliminate warnings from xml2rfc.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-03 23:04:12 UTC (rev 4228)
+++ docs/usefor/usepro.xml	2008-08-03 23:15:34 UTC (rev 4229)
@@ -1369,18 +1369,18 @@
    Required parameters:      none
    Optional parameters:      One and only one of "usage=moderate",
                              "usage=inject", or "usage=relay".
-   Encoding considerations:  A transfer-encoding different from that of
-                             the article transmitted MAY be supplied to
-                             ensure correct transmission over some 7bit
-                             transport medium.
+   Encoding considerations:  A transfer-encoding different from that
+                             of the article transmitted MAY be
+                             supplied to ensure correct transmission
+                             over some 7bit transport medium.
    Security considerations:  A news article may be a control message,
                              which if processed could have effects on
                              the recipient host's system beyond just
                              storage of the article.
    Published specification:  This specification.
    Body part:                A complete proto-article ready for
-                             injection into Netnews or an article being
-                             relayed to another agent
+                             injection into Netnews or an article
+                             being relayed to another agent.
           </artwork>
         </figure>
 
@@ -1411,12 +1411,13 @@
    MIME type name:           application
    MIME subtype name:        news-groupinfo
    Required parameters:      none
-   Optional parameters:      charset, which MUST be a charset registered
-                             for use with MIME text types and has the
-                             same syntax as the parameter defined for
-                             text/plain [RFC2046].  Specifies the
-                             charset of the body part.  If not given,
-                             the charset defaults to US-ASCII [ASCII].
+   Optional parameters:      charset, which MUST be a charset
+                             registered for use with MIME text types
+                             and has the same syntax as the parameter
+                             defined for text/plain [RFC2046].
+                             Specifies the charset of the body part.
+                             If not given, the charset defaults to
+                             US-ASCII [ASCII].
    Disposition:              by default, inline
    Encoding considerations:  7bit or 8bit MUST be used to maintain
                              compatibility. 
@@ -1485,21 +1486,23 @@
    MIME type name:           application
    MIME subtype name:        news-checkgroups
    Required parameters:      none
-   Optional parameters:      charset, which MUST be a charset registered
-                             for use with MIME text types and has the
-                             same syntax as the parameter defined for
-                             text/plain [RFC2046].  Specifies the
-                             charset of the body part.  If not given,
-                             the charset defaults to US-ASCII [ASCII].
+   Optional parameters:      charset, which MUST be a charset
+                             registered for use with MIME text types
+                             and has the same syntax as the parameter
+                             defined for text/plain [RFC2046].
+                             Specifies the charset of the body part.
+                             If not given, the charset defaults to
+                             US-ASCII [ASCII].
    Disposition:              by default, inline
    Encoding considerations:  7bit or 8bit MUST be used to maintain
                              compatibility. 
-   Security considerations:  This media type provides only a means for
-                             conveying a list of newsgroups and does not
-                             provide any information indicating whether
-                             the sender is authorized to state which
-                             newsgroups should exist within a hierarchy.
-                             Such authorization must be accomplished by
+   Security considerations:  This media type provides only a means
+                             for conveying a list of newsgroups and
+                             does not provide any information
+                             indicating whether the sender is
+                             authorized to state which newsgroups
+                             should exist within a hierarchy.  Such
+                             authorization must be accomplished by
                              other means.
    Published specification:  This specification.
           </artwork>
@@ -1632,7 +1635,8 @@
             <artwork>
       control-command     =/ Newgroup-command
       Newgroup-command    = "newgroup" Newgroup-arguments
-      Newgroup-arguments  = 1*WSP newsgroup-name [ 1*WSP newgroup-flag ]
+      Newgroup-arguments  = 1*WSP newsgroup-name
+                               [ 1*WSP newgroup-flag ]
       newgroup-flag       = "moderated"
             </artwork>
           </figure>
@@ -2066,9 +2070,12 @@
     <references title="Normative References">
       <reference anchor="ASCII">
         <front>
-          <title>American National Standard for Information Systems -
-          Coded Character Sets - 7-Bit American National Standard Code for
-          Information Interchange (7-Bit ASCII)</title>
+          <title>Coded Character Sets - 7-Bit American National Standard
+          Code for Information Interchange (7-Bit ASCII)</title>
+          <author>
+            <organization>American National Standard for Information
+            Systems</organization>
+          </author>
           <date year="1986" />
         </front>
         <seriesInfo name="ANSI" value="X3.4" />

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sun,  3 Aug 2008 16:03:52 -0700 (PDT)
Sender: [email-masked]

    Date: Sunday, August 3, 2008 @ 16:03:51
  Author: eagle
Revision: 4227

Make usepro-10.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-03 23:02:50 UTC (rev 4226)
+++ docs/usefor/usepro.xml	2008-08-03 23:03:51 UTC (rev 4227)
@@ -26,7 +26,7 @@
     'http://xml.resource.org/public/rfc/bibxml/reference.RFC.4288.xml'>
 ]>
 
-<rfc ipr="full3978" docName="draft-ietf-usefor-usepro-09" obsoletes="1036"
+<rfc ipr="full3978" docName="draft-ietf-usefor-usepro-10" obsoletes="1036"
      category="std">
   <front>
     <title>Netnews Architecture and Protocols</title>
@@ -62,7 +62,7 @@
       </address>
     </author>
 
-    <date month="November" year="2007" />
+    <date month="August" year="2008" />
 
     <area>Applications</area>
     <workgroup>Usenet Format Working Group</workgroup>

To: [email-masked]
From: [email-masked]
Subject: Commit in docs/usefor (usepro.xml)
Date: Sun,  3 Aug 2008 16:02:51 -0700 (PDT)
Sender: [email-masked]

    Date: Sunday, August 3, 2008 @ 16:02:50
  Author: eagle
Revision: 4226

Add a section describing the history mechanism and algorithm.  Update
references in Duties of an Injecting Agent and Duties of a Serving
Agent accordingly, moving the specifics of date-based rejection from
the Duties sections to the history section and making clearer the
distinction between the protocol requirement and the implications of
specific implementations satisfying that requirement.

Modified:
  docs/usefor/usepro.xml

Modified: docs/usefor/usepro.xml
===================================================================
--- docs/usefor/usepro.xml	2008-08-03 21:07:34 UTC (rev 4225)
+++ docs/usefor/usepro.xml	2008-08-03 23:02:50 UTC (rev 4226)
@@ -452,6 +452,66 @@
         </section>
       </section>
 
+      <section anchor="history"
+               title="Article History and Duplicate Suppression">
+        <t>Netnews normally uses a flood-fill algorithm for propagation of
+        articles in which each news server offers the articles it accepts
+        to multiple peers and each news server may be offered the same
+        article from multiple other news servers.  Accordingly, duplicate
+        suppression is key; if a news server accepted every article it was
+        offered, it may needlessly accept (and then potentially
+        retransmit) dozens of copies of every article.</t>
+
+        <t>Relaying and serving agents therefore MUST keep a record of
+        articles they have already seen and use that record to reject
+        additional offers of the same article.  This record is called the
+        "history" file or database.</t>
+
+        <t>Each article is uniquely identified by its message identifier,
+        so a relaying or serving agent could satisfy this requirement by
+        storing a record of every message identifier that agent has ever
+        seen.  Such a history database would grow without bound, however,
+        so it is common and permitted to optimize based on the
+        Injection-Date or Date header field of an article as follows.  (In
+        the following discussion, the "date" of an article is defined to
+        be the date represented by its Injection-Date header field if
+        present, otherwise its Date header field.)
+          <list style="symbols">
+            <t>Agents MAY select a cutoff interval and reject any article
+            with a date farther in the past than that cutoff interval.  If
+            this interval is shorter than the time it takes for an article
+            to propagate through the network, the agent might reject an
+            article it had not yet seen, so it ought not be aggressively
+            short.  For Usenet, for example, a cutoff interval of no less
+            than seven days is conventional.</t>
+
+            <t>Agents that enforce such a cutoff MAY then drop records of
+            articles that had dates older than the cutoff from their
+            history databases.  If such an article were offered to the
+            agent again, it would be rejected due to the cutoff date, so
+            the history record is no longer required to suppress the
+            duplicate.</t>
+
+            <t>Alternatively, agents MAY drop history records according to
+            the date when the article was first seen by that agent rather
+            than the date of the article.  In this case, the history
+            retention interval MUST be at least 24 hours longer than the
+            cutoff interval to allow for articles dated in the future.
+            This interval matches the allowable error in the date of the
+            article (see <xref target="injecting" />).</t>
+          </list>
+        </t>
+
+        <t>These are just two implementation strategies for article
+        history, albeit the most common ones.  Relaying and serving agents
+        are not required to use these strategies, only to meet the
+        requirement of not accepting an article more than once.  However,
+        these strategies are safe and widely deployed and implementors are
+        encouraged to use one of them, especially if they do not have
+        extensive experience with Netnews and the subtle effects of its
+        flood-fill algorithm.</t>
+      </section>
+
       <section anchor="posting" title="Duties of a Posting Agent">
         <t>A posting agent is the component of a user agent that assists a
         poster in creating a valid proto-article and forwarding it to an
@@ -806,19 +866,19 @@
             field or Message-ID header field, or without either an
             Injection-Date or Date header field.</t>
 
-            <t>It MUST reject any article that has already been
-            successfully sent to it, based on the Message-ID header field
-            of the article.  To satisfy this requirement, a relaying agent
-            normally keeps a database of message identifiers it has
-            already accepted.</t>
-
             <t>It MUST examine the Injection-Date header field or, if
             absent, the Date header field, and reject the article if that
-            date predates the earliest articles of which it keeps record
-            or if that date is more than 24 hours into the future.  It MAY
-            reject articles with dates in the future with a smaller margin
-            than 24 hours.</t>
+            date is more than 24 hours into the future.  It MAY reject
+            articles with dates in the future with a smaller margin than
+            24 hours.</t>
 
+            <t>It MUST reject any article that has already been accepted.
+            If it implements one of the mechanisms described in <xref
+            target="history" />, this means that it MUST reject any
+            article whose date falls outside the cutoff interval since it
+            won't know whether such articles had been accepted previously
+            or not.</t>
+
             <t>It SHOULD reject any article that does not include all the
             mandatory header fields.  It MAY reject any article that
             contains header fields that do not have valid contents.</t>
@@ -891,16 +951,16 @@
 
             <t>It MUST examine the Injection-Date header field or, if
             absent, the Date header field, and reject the article if that
-            date predates the earliest articles of which it keeps record
-            or if that date is more than 24 hours into the future.  It MAY
-            reject articles with dates in the future with a smaller margin
-            than 24 hours.</t>
+            date is more than 24 hours into the future.  It MAY reject
+            articles with dates in the future with a smaller margin than
+            24 hours.</t>
 
-            <t>It MUST reject any article that has already been
-            successfully sent to it, based on the Message-ID header field
-            of the article.  To satisfy this requirement, a relaying agent
-            normally keeps a database of message identifiers it has
-            already accepted.</t>
+            <t>It MUST reject any article that has already been accepted.
+            If it implements one of the mechanisms described in <xref
+            target="history" />, this means that it MUST reject any
+            article whose date falls outside the cutoff interval since it
+            won't know whether such articles had been accepted previously
+            or not.</t>
 
             <t>It SHOULD reject any article that matches an
             already-received and honored cancel message or Supersedes