Subject: [EAI] Comments on draft-ietf-eai-downgrade-02

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 04 Sep 2006 20:21:38 +0100

s/EAI/UTF8SMTP/ throughout the document, including in the title.

1.  Introduction

    Traditional mail systems which are defined by [RFC2821] and [RFC2822]
    allow US-ASCII characters in SMTP envelope and mail headers in body
    part.  The EAI proposal [I-D.ietf-eai-framework],
    [I-D.ietf-eai-utf8headers] and [I-D.ietf-eai-smtpext] allows UTF-8
    characters in SMTP envelope and mail headers in body part.

s/mail headers in body part/mail header field bodies/ (twice)

otherwise, people will think you are talking about headers inside the  
<body>s of mail messages. "field body" is the term RFC 2822 uses for a  
header field without its header name.

    ... Not only
    SMTP envelope, but also UTF-8 characters in mail headers MUST be
    converted to US-ASCII.

s/mail headers/mail header fields/ here and in lots of other places, to  
accord with the terminology of RFC 2822.

And then s/mail header fields/mail header fields and in the headers fields  
of MIME multipart and message types/.

Otherwise, an already downgraded message will get munged if it then  
encounters a non-8BITMIME server.

    In this document, requirements for downgrading is described in
    section Section 3, SMTP Downgrading is described in Section 4, and
    SMTP DATA/Header Downgrading is described in Section 5.

s/is described/are described/

I don't like the use of the term "SMTP DATA/header" to describe the mail  
messages that are transported. We are speaking of objects that conform to  
RFC 2822 as modified by our utf8headers document, but these objects are  
going to turn up in all sorts of places apart from within SMTP, such as  
within other transports (e.g. UUCP), in gateways from other media (e.g.  
News), within mailing list expanders, and within POP3, IMAP and other  
storage agents, and they are likely to have to be downgraded (or upgraded)  
according to your rules in Section 5 in all sorts of situations (e.g.  
various gateways). Of course, your Section 4 is SMTP-specific, but not  
your Section 5.

So a better term would be "Email header downgrading".

3.  Downgrade Requirements

3.1.  Timing and conditions of downgrading

    Note: If the UTF8SMTP header exists, downgrading will be performed.
    If UTF-8 characters exist in mail headers without the UTF8SMTP
    header, this is a protocol error, ...

No, only if that header exists with the parameter "UTF8", not when its  
parameter is "downgraded" or "ASCII".

Note also that I have suggested renaming this header as something like  
"Header-Content", and using "UTF8SMTP" as the parameter.

3.2.  Requirements

    Followings are requirements of downgrading.

I think that is more a list of hoped-for properties rather than a list of  
requirements. I doubt we are going to succeed in realizing them all (e.g.  
an upgrade may not always be an exact inverse of the original downgrade).

4.  SMTP Downgrading

    Downgrading in SMTP envelope uses ALT-ADDRESS parameter proposed in
    [I-D.ietf-eai-smtpext].  An address is downgradable if the address is
    all-ASCII address or the address has all-ASCII address specified by
    ALT-ADDRESS parameter.

But what about punycode downgraded (possibly under the control of an  
"atomic" or "downgradable" keyword)? It is clear that such a downgrade is  
achievable, given the offficial definition or punycode. We discussed that,  
but we never agreed to remove it from the draft AKAIR.

    One SMTP session may contain multiple recipients.  Downgrading SHOULD
    be performed for each SMTP recipient address.  ...

s/for each SMTP recipient address/for each SMTP recipient address  
individually/

    EAI-Downgraded-From:  <Non-ASCII {US-ASCII}> <US-ASCII>
    EAI-Downgraded-To:  <Non-ASCII {US-ASCII}> <US-ASCII>

You need to give a syntax for these new header fields. Also, they should  
be regarded as Trace fields, so that they come in the middle of the  
sequence of Received headers, to indicate whereabouts in the sequence of  
MTAs the downgrading was done.

Also, we need a name not involving "EAI-", and I would suggest something  
involving "Envelope-". So we would get something like:

    trace         =       [return]
                          1*( received / envelope-down )
    envelope-down = "Envelope-Downgraded-From:" downchange CRLF /
                    "Envelope-Downgraded-To:" downchange CRLF
    downchange    = [CFWS] "<" utf8-addr-spec [alt-address] ">" [CFWS]
                       "<" [CFWS] addr-spec [CFWS ">" [CFWS]

    ESMTP ORCPT parameter is used for Delivery Status Notifications
    (DSNs) [RFC3461].  ... ORCPT
    parameter downgrading should be defined here.

Then please let us have a specific proposal for the next draft.

5.  SMTP DATA/Header Downgrading

As I said earlier, this needs a title that does not confine it to things  
that appear in an SMTP DATA command.

    Originator address(es):  ...
    Destination address(es):  ...
    IDs:  IDs such as Message-ID:, Date:, In-Reply-To: and References: ...
    Trace headers:  ...

You need to add Content-Type, Content-Disposition, etc. to your list,
and also List-*.

Headers (Content-*) containing <parameter>s will be downgraded according  
to RFC 2231, and headers containing IRIs (List-*) according to RFC 3987.

5.1.  Header conversion

    To preserve all header information, define generic encapsulation
    header: "Downgraded: HeaderName: HeaderValue".  The header value is
    encoded by [RFC2047] with UTF-8 tag.

Again, some syntax is needed. I would suggest:

    fields     =/ downgraded
    downgraded = "Downgraded:" [CFWS] field-name ":" unstructured CRLF
    field-name = 1*ftext
    ftext      = %d33-57 / %d59-126 ; printable ASCII except ":"

    Downgrading:
       *  Check if the mail has 'UTF8SMTP' header and its value is
          "UTF8".  Otherwise, downgrading is not required.
       *  Check each header whether it contains UTF-8 characters or not.

You need to deal with the RFC 2047 stuff here:

|        If the header contains any <unstructured>, or any <commemt>s
|        or <phrase>es
|        + encode them according to RFC 2047
|        If the header contains UTF-8 characters,
|                     ^
|                     still
          +  If the header is an address header which is described in
             Section 5.1.1,
             -  Preserve the header in 'Downgraded' header.
No, that comes a little later
             -  Downgrade the header defined in Section 5.1.1.
|        + If the header field contains any <parameter> ... Section 5.1.2  
...
|        + If the header contains any IRI ... Section 5.1.3 ...
          +  Preserve the header in 'Downgraded' header.

          +  The other header case, encode the header by [RFC2047] with
             UTF-8 tag.
But NO!!! You cannot just apply RFC 2047 to any other header which just  
happens to contain UTF-8. So far, nobody has defined any UTF-8 headers not  
already covered above, but for sure they will appear in the future. The  
only thing you can do with them (though maybe the guy who invented such a  
header also invented a downgrade for it) is to create a Dowwgraded header  
for it (which of course can then contain RFC 2047).

       *  Change 'UTF8SMTP' header value to "Downgraded".

And point out that you don't need to create a Downgraded header field if  
the only thing you did was to apply RFC 2047 to <unstructured>s,  
<comment>s and <phrase>s.

    Upgrading:

          +  If the header is address header described in Section 5.1.1,
             -  Apply address header downgrading to the decoded header.
Add "using the Downgraded header field"
             -  Remove the header line which is the same with the
                downgraded line.

What do you mean by "same"? Suppose the folding if different. Suppose  
extra whitespace has appeared/disappeared? Suppose some RFC 2047 encoding  
has been done differently, or with a different character set?
And what if there isn't any matdhing header? Surely that is a protocol  
error? Do we want to define what happens in that case (e.g. leave the  
header in place, in which case you may have two From (or To) headers,  
which is a breach of RFC 2822)?

          +  Remove the 'Downgraded' header.
          +  Add decoded header to mail header.
       *  If each mail header has [RFC2047] encoded part and which
          encoding is "UTF-8", it is a downgraded header, so decode it.
And of course that is one possible cause why the upgraded header may not  
be identical to the pre-downgraded header.
       *  Change 'UTF8SMTP' header value to "UTF8".

    Pros:
       *  EAI incompliant MUA displays the downgraded mail body except

s/mail body/header field bodies/

    Cons:
          ... Also, restoration order of the
          downgraded headers is not guaranteed.  ...

I think you ought to define exactly where the Downgraded headers and  
modified headers are to be put (e.g. at the same place in the message as  
the old version). This should be a "SHOULD".

5.1.1.  Downgrading address headers

    The header value is composed of single or multiple mailbox/angle-addr
    fields defined in [I-D.ietf-eai-utf8headers].

s/header value/header field body/

The next bit would need to be adapted to the changes I have suggested for  
5.1 above.

    2.  By mailbox/angle-addr downgrading, if the field became empty, the
        field should be removed.
    3.  If all header field is removed, remove the header.

But suppose the header was
     From: "Joe Doe" <utf8-joe@utf8-domain>
You will get
     From: "Joe Doe"
which makes it Non-RFC 2822-compliant, but you cannot remove it because it  
is not empty.

    4.  If From header is removed, generate new From: header from
        envelope-from address.

Ugh! Though I cannot think of anything better.

    2.  <Non-ASCII>
           Non-ASCII mail address without ALT-ADDRESS parameter case,
           remove this angle-addr.

No, that is not right. If the <angle-addr> is
     <ascii-joe@utf8-domain>
then you should be able to translate the utf8-domain with punycode and  
carry on.

And, as I have already said, I would still like to be able to translate  
the <local-part> using punycode, possible restricted by an  
atomic/downgradable parameter.

8.  IANA Considerations

    IANA is requested to add the "EAI-Downgraded-From:",
    "EAI-Downgraded-To:" and "Downgraded:" new headers to the registry
    with the entries pointing to this specification for its definition.

No, you have to provide templates for all your new headers, as I explained  
when commenting on the utf8headers draft.

Appendix A.  Examples

Some nice esamples here, but it would be better to show them as a  
continuous series in one Appendix. As it is, each Appendix starts by  
repeating where the previous Appendix left off, which is wasteful as well  
as confusing.

    Message-Id: MESSAGE_ID

s/MESSAGE_ID/<MESSAGE_ID>/ in all the examples.

    From: <NON-ASCII-FROM {ASCII-FROM}>
    To: <NON-ASCII-TO {ASCII-TO}>
    CC: <NON-ASCII-CC>

Please can we include some <display-name>s and some cases of multiple  
<angle-addr>s in the same header in the examples?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima