Subject: Re: Syntax validation of articles by injecting agents

From: Russ Allbery <[email-masked]>
Date: Mon, 04 Jan 2010 18:12:39 -0800

"Charles Lindsey" <[email-masked]> writes:

> Eh? How does it imply that? If there are no Content-* headers, then
> there is no requirement for MIME-Version.

See Julien's post:

| NN for instance does not generate MIME-Version: header fields
| although "user agents MUST meet the definition of MIME conformance"
| ("a mail user agent that is MIME-conformant MUST always generate
| a "MIME-Version: 1.0" header field in any message it creates").
| I believe this sentence applies to news user agents too, otherwise
| a reference to MIME is useless.

The first quoted statement is from RFC 5536 section 2.3, with an
accompanying reference to RFC 2049.  The second quoted statement is from
RFC 2049 section 2.

Now that I think about it, though, this only places a requirement on the
user agent.  It doesn't require that the server reject the message, so I
think the original problem isn't actually that significant of a problem.
MIME-compliant agents, such as injecting agents, are allowed to accept
non-MIME messages.

> But aren't injecting agents allowed to remove a Path header that is
> received?

See point 2 in Duties of an Injecting Agent in RFC 5537, which requires a
syntax check on the Path header.

> For sure, many of them routinely do. In any case, fixing (or removing) a
> broken header is always a way out of the problem.

Injecting agents are specifically discouraged (SHOULD NOT) from fixing or
remove any header fieldss other than Path.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 5 Jan 2010 11:56:36 GMT

>accompanying reference to RFC 2049.  The second quoted statement is from

But I don't think RFC 2049 ever intended to imply that MIME-Version was
needed for a message that did not actually use any of the MIME features.
As evidence of that, I can cite RFC 2047, which states:

   (4) A MIME-Version header field is NOT required to be present for
       'encoded-word's to be interpreted according to this
       specification.  One reason for this is that the mail reader is
       not expected to parse the entire message header before displaying
       lines that may contain 'encoded-word's.

>MIME-compliant agents, such as injecting agents, are allowed to accept

Exactly.

>remove any header fieldss other than Path.

Which is fair enough, but gives leeway for Paths to be removed, which
copes with the particular problem that was raised. For sure, many
injecting agents DO remove it regardless, because they regard it as an
"infringement of their rights" :-) .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131            Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Richard Clayton <[email-masked]>
Date: Tue, 5 Jan 2010 12:33:41 +0000

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes

>needed for a message that did not actually use any of the MIME features.

Hardly on topic for this list, but yes that's exactly what it means, and
what it was intended to mean.  It's no big deal for a client to generate
the header field, so it's hardly onerous!  (see this message for an
example of a compliant client)

What it does for you is to ensure that if you are corresponding with
someone whose client is MIME compatible, you learn that is so "for free"

You might note that adding "MIME-Version: 1.0" to a message that is
otherwise "not MIME" (but correctly eschews the use of "non-ASCII"
characters etc), will make no difference to its interpretation.

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 06 Jan 2010 15:56:52 +0100

Julien ÉLIE wrote:
>
> Hi,
>
> RFC 5537 mentions that an injecting agent MUST reject any proto-article
> that is not syntactically valid as defined by RFC 5536.
RFC 5536 says about articles:

   An article is said to be conformant to this specification if it
   conforms to the format specified in Section 3 of [RFC5322] and to the
   additional requirements of this specification.

This doesn't require MIME.
It says about agents:

   User agents MUST meet the definition of MIME conformance in [RFC2049]
   and MUST also support [RFC2231].  This level of MIME conformance
   provides support for internationalization and multimedia in message
   bodies [RFC2045], [RFC2046], and [RFC2231], and support for
   internationalization of header fields [RFC2047] and [RFC2231].  Note
   that [Errata] currently exist for [RFC2045], [RFC2046], [RFC2047] and
   [RFC2231].

Yes, that requires that user agents conforming to RFC 5536 must add 
MIME-version headers.

I think the *important* part of all that conformance is that Just-Send-8 
messages are strictly illegal, and that if you use MIME, 2047 or 2231, 
you have to use it correctly.

                    Harald

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 6 Jan 2010 16:37:05 GMT

>example of a compliant client)

Then how do you account for the option of not including in in RFC 2047?

---

From: Julien ÉLIE <[email-masked]>
Date: Wed, 6 Jan 2010 18:52:14 +0100

Hi Charles,

> Then how do you account for the option of not including in in RFC 2047?

The Section 6.1 you quote in RFC 2047 only specifies how a MIME-compliant
reader MUST parse any messages.
The fact that a MIME-Version: header field is not required to be present
for <encoded-word>s to be interpreted according to the specification
only means, to my mind, that a MIME-compliant reader should properly decode
<encoded-word>s of broken agents.

This way, a MIME-compliant agent will not write
    =?iso-8859-1?q?this=20is=20some=20text?=
to the user if the MIME-Version: header field is missing!  It will try
to decode it and properly display "this is some text".

Another reason is given by RFC 2047:

       One reason for this is that the mail reader is
       not expected to parse the entire message header before displaying
       lines that may contain 'encoded-word's.

It then allows a MIME-compliant reader that decodes headers as soon as
it receives it, not to wait for the MIME-Version: header field in order
to properly decode headers.  It will not have to decode them twice
for instance.

Aren't these two reasons enough to say that RFC 2047 does not stipulate
that a MIME-Version: header field is optional?

If not, see Section 4 of RFC 2045:

   Therefore, this document defines a new header field, "MIME-Version",
   which is to be used to declare the version of the Internet message
   body format standard in use.

   Messages composed in accordance with this document MUST include such
   a header field, with the following verbatim text:

     MIME-Version: 1.0

   The presence of this header field is an assertion that the message
   has been composed in compliance with this document.

Regarding RFC 2045:

   This initial document specifies the various headers used to describe
   the structure of MIME messages.  The second document, RFC 2046,
   defines the general structure of the MIME media typing system and
   defines an initial set of media types.  The third document, RFC 2047,
   describes extensions to RFC 822 to allow non-US-ASCII text data in
   Internet mail header fields.  The fourth document, RFC 2048, specifies
   various IANA registration procedures for MIME-related facilities.  The
   fifth and final document, RFC 2049, describes MIME conformance
   criteria as well as providing some illustrative examples of MIME
   message formats, acknowledgements, and the bibliography.

-- 
Julien ÉLIE

« Medicus dedit qui temporis morbo curam,
  Is plus remedii quam cutis sector dedit. »

---

From: Julien ÉLIE <[email-masked]>
Date: Wed, 6 Jan 2010 19:20:18 +0100

Hi Harald,

> I think the *important* part of all that conformance is that Just-Send-8 messages are strictly illegal, and that if you use MIME, 
> 2047 or 2231, you have to use it correctly.

Yes, I agree with that.

And unfortunately, there are still posts in the French hierarchy fr.*
that either have no MIME headers or that have Content-Type: without
any MIME-Version: header field.
They contain 8-bit chars.

Examples of such readers:  Xnews, MacSOUP, lots of malconfigured Outlook
Express...

So I believe it would be currently too harmful to reject posts with illegal
8-bit chars...  Such articles are not valid.

-- 
Julien ÉLIE

---

From: Richard Clayton <[email-masked]>
Date: Wed, 6 Jan 2010 18:35:35 +0000

In message <[email-masked]>, Charles Lindsey
<[email-masked]> writes
>Then how do you account for the option of not including in in RFC 2047?

I think it's saying that you must expect MUAs to parse encoded words
without checking if the Mime-Version: header field is or is not present
(and giving permission for MUAs to be optimised that way)

- -- 
richard                                                   Richard Clayton

Those who would give up essential Liberty, to purchase a little temporary 
Safety, deserve neither Liberty nor Safety. Benjamin Franklin 11 Nov 1755

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 7 Jan 2010 11:30:44 GMT

>   has been composed in compliance with this document.

Well that implies, to me, that if the message contains nothing that
purports to be "composed in compliance with this document", then there is
no obligation to inlcude the MIME-Version.

---

From: Seth <[email-masked]>
Date: Thu, 7 Jan 2010 12:59:26 -0500 (EST)

"Charles Lindsey" <[email-masked]> wrote:
> In <FD8718C15E1440E993AEFBF849934D82@Iulius>
> =?Windows-1252?Q?Julien_=C9LIE?= <[email-masked]> writes:
> (quotes)

> no obligation to inlcude the MIME-Version.

Even if it is composed in compliance with the document, there is no
obligation to *assert* that fact.

Seth

---

From: Russ Allbery <[email-masked]>
Date: Thu, 07 Jan 2010 11:10:12 -0800

>> purports to be "composed in compliance with this document", then there
>> is no obligation to inlcude the MIME-Version.

> obligation to *assert* that fact.

Actually, yes, there is.  I really don't see how RFC 2049 could possibly
be any clearer.

|    A mail user agent that is MIME-conformant MUST:
| 
|     (1)   Always generate a "MIME-Version: 1.0" header field in
|           any message it creates.

I don't think there's much need to go farther afield into other MIME
documents when RFC 2049 is as clear as crystal and when we explicitly
stated in RFC 5536 that all user agents complying with RFC 5536 MUST meet
the definition of MIME conformance in RFC 2049.

(This does not, as pointed out elsewhere in the thread, imply that servers
have to reject messages generated by non-conformant user agents.)