Subject: Re: [EAI] popimap-downgrade-00 - group syntax

From: John C Klensin <[email-masked]>
Date: Wed, 03 Nov 2010 21:06:38 -0400

--On Wednesday, 27 October, 2010 12:13 -0500 Pete Resnick
<[email-masked]> wrote:

> I sent these comments to Fujiwara-san, but forgot to send them
> to the list. Issues we need to deal with:
> 
> 1. Now that we are allowing the downgrade of "From:" and
> "Resent-From:" without an alt-address, we must point out that
> RFC 5322 does *not* allow group syntax in "From:" and
> "Resent-From:" and that this document changes that syntax. I
> think that is fine to do that, but this document needs to
> explicitly say that. Perhaps something like this:
>...

Pete, I'm less confident about this.  Remember that the purpose
of any downgrade function is to make the message acceptable to
legacy systems, especially standards-conforming ones.  For
several reasons, I'd assume that there are competent (and
conforming) MUAs out there that would get very annoyed upon
seeing group syntax in those fields.

It may still be the best solution for us but, if it is, we need
more than the syntax change text that you suggest.  We need,
IMO, an explicit analysis and warning about what might occur,
either inline or in Security Considerations.

> 2. MAILBOX downgrading makes group syntax recursive for other
> address fields. This one I'm less confident about. But let's
> say you get a message like this:
> 
> To: My Friends: <[email-masked]>, <[email-masked]>;
> 
> According to the rules, you downgrade this to:
> 
> To: MyFriends: Internationalized Address <<Yao's 2047
> address>> Removed:;, Internationalized Address <<Xiaodong's
> 2047 address>> Removed:;
> 
> I think this is OK, but it changes the syntax of group to:
> 
> group = display-name ":" [group / group-list] ";" [CFWS]
> 
> If everyone is OK with this, it should go in the document
> somewhere.

Same comment as above, only more so.  Remember how many MUAs
(including one with which you are very familiar) went for years
without being able to handle group syntax properly and then
contemplate what might happen with this modification, especially
if it is misparsed by a legacy system and the MUA tries to "fix
things up" in the name of robustness.

     john

---

From: [email-masked]
Date: Mon, 08 Nov 2010 01:27:34 +0900 (JST)

> From: Pete Resnick <[email-masked]>
> I sent these comments to Fujiwara-san, but forgot to send them to the
> list. Issues we need to deal with:

I received both comment from Pete an John before submission. (Thanks)
Then, I added two author notes.

First:
   [[ Note: RFC 5322 does not allow group syntax in "From:", "Resent-
   From:", "Sender:", "Resent-Sender:", but proposed method uses group
   syntax.  This problem need to be fixed. ]]

> 1. Now that we are allowing the downgrade of "From:" and "Resent-From:" without an alt-address, we must point out that RFC 5322 does *not* allow group syntax in "From:" and "Resent-From:" and that this document changes that syntax. I think that is fine to do that, but this document needs to explicitly say that. Perhaps something like this:

snip

> The definition for these header fields in [RFC5322] only allow <mailbox> or <mailbox-list>, which means that they cannot contain the group syntax for MAILBOX downgrading described in section 4.1.7. Therefore, this specification changes the syntax of the above fields as follows to allow a group to appear:
> 
> from = "From:" address-list CRLF
> sender = "Sender:" address CRLF
> resent-from = "Resent-From:" address-list CRLF
> resent-sender = "Resent-Sender:" address CRLF
> 
> If the header field contains <mailbox> elements that contain non-ASCII addresses, perform COMMENT downgrading, DISPLAY-NAME downgrading, and MAILBOX downgrading.

This suggestion may break RFC 5322 compliant MUA.

I have another idea and it is not beautiful.

1. Recover Downgraded-{From,Sender,Resent-From,Resent-Sender} header fields.
2. for each {From,Sender,Resent-From,Resent-Sender} header fields,
   2.1 if the heaer field contains non-ASCII address,
         encapsulate it into Downgraded-* header field.
   2.2 if the header field is not a "From" header, remove the header field.
       (because it is not necessary in the point of the RFC5322 conformance.)
   2.3 if the header fiel is a "From" header, if it contains one or more 
       ASCII mailbox, remove non-ASCII mailbox element.
   2.4 if the heaer field is a "From" header and it does not contain ASCII
       mailbox, rewrite it as pre-configured pop-server's daemon E-mail address.

Then generated pop-imap downgraded message contains:
  From: <pop-server@pop-dom>
  Downgraded-From: ENCODED-WORD(From)
  Downgraded-Sender: ENCODED-WORD(Sender)
  Downgraded-Resent-From: ENCODED-WORD(Resent-From)
  Downgraded-Resent-Sender: ENCODED-WORD(Resent-Sender)

The recipient can read most of the received message except From: header field.

Or encode From: header field into Subject header value?

  Subject: popimap-downgraded: From=<From>: Subject=<Subject>

--------------------------------------------------------------------------
Second:

   [[ Note: If the original non-ASCII address is a part of a group
   address, this rewriting may conflict the original DISPLAY-NAME.  This
   problem need to be fixed. ]]

> 2. MAILBOX downgrading makes group syntax recursive for other address fields. This one I'm less confident about. But let's say you get a message like this:
> 

In many cases, both <mailbox>s are removed before pop/imap servers.
But need to consider.

> To: MyFriends: Internationalized Address <<Yao's 2047 address>> Removed:;, Internationalized Address <<Xiaodong's 2047 address>> Removed:;
> 
> If everyone is OK with this, it should go in the document somewhere.

But it may break RFC 5322 compliant MUAs.

Then, I have an idea.
If <group-list> contains non-ASCII <mailbox>s, encode whole <group-list> to 
"Internationalize Address" ENCODED-WORD(<group-list>) Removed:;"

> To: My Friends: <[email-masked]>, <[email-masked]>;

This example may be encoded as:

To: Internationalized Address =?UTF-8?Q?MyFriends:_<<Yao's 2047 address>>,_<<Xiaodong's 2047 address>>;?= Removed:;

# Encoded word may be splited (defined in RFC 2047)

Is it acceptable?

--
Kazunori Fujiwara, JPRS <[email-masked]>

---

From: Bill McQuillan <[email-masked]>
Date: Sun, 7 Nov 2010 09:57:18 -0800

> 1. Recover Downgraded-{From,Sender,Resent-From,Resent-Sender} header fields.

Please, if we need something like the Downgraded-* fields, do NOT call them that. The names are very misleading.

Instead, how about Encoded-From, Encoded-To, etc.?

Or even, Incompatible-From, etc.?
 
-- 
Bill McQuillan <[email-masked]>

---

From: "Daniel Taharlev" <[email-masked]>
Date: Mon, 8 Nov 2010 10:37:48 +0200

What about "Compatible-From" or "Compliant-From"?
----- Original Message ----- 
From: "Bill McQuillan" <[email-masked]>
To: "IMA Discussion" <[email-masked]>
Sent: Sunday, November 07, 2010 7:57 PM
Subject: Re: [EAI] popimap-downgrade-00 - group syntax

> Please, if we need something like the Downgraded-* fields, do NOT call 
> them that. The names are very misleading.
>
> _______________________________________________
> IMA mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ima
>

---

From: Bill McQuillan <[email-masked]>
Date: Mon, 8 Nov 2010 01:27:22 -0800

> What about "Compatible-From" or "Compliant-From"?

I would be worried that I'm not sure what the field is supposed to be
compatible with or compliant to.

Consider the consumer of this header field: Someone behind a legacy MTA who
can get his/her (probably legacy) MUA to display unknown header fields.

That consumer would wonder why the From: field didn't seem to correspond
with what might be expected, but the "XXXXXX-From:" field might have its
encoded words decoded into something that he/she recognizes. What would be
an appropriate name for that field?

From-Actually-Was: ?
From-Untranslated: ?
Untranslated-From: ?
Replaced-From: ?

I am more interested that the wordage mean something to the user rather
than the implementers.

---

From: "Daniel Taharlev" <[email-masked]>
Date: Mon, 8 Nov 2010 11:29:50 +0200

For the user, a term -- even if not completely accurate -- like 
"Secondary-From" would make the most sense, although it's not really a 
secondary.

---

From: [email-masked]
Date: Mon, 08 Nov 2010 18:39:43 +0900 (JST)

There are multiple proposals for new preserving header fields.
The discussion is OK, but it is another issue.

But "Downgraded-*" header field name was not rejected at the
experimental stage.  Why did you comment it last year ?

I will use "Downgraded-*" header field name until naming discussion will fix.

Regards,

---

From: Jason Nelson <[email-masked]>
Date: Tue, 9 Nov 2010 05:57:37 +0000

The real problem I have with appending anything to the header name when changing encoding is that it makes header consumption by clients dependent on knowledge of the fact that the message has been downgraded.  

Let us say that we add information in a normal x-header fashion, and it is a text header.  The client knows to look for x-fubar and consume it.
Now it needs to know to look for...downgraded-x-fubar?  X-downgraded-fubar?  If I'm going to update the clients to consume the downgraded content, I'd just update them to consume utf8 directly.  The downgraded content isn't as compatible as it seems.

Are there any UTF8-isms that don't downgrade to 2047 encoding cleanly outside of the local part of addresses?

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Monday, November 08, 2010 1:40 AM
To: [email-masked]
Cc: [email-masked]
Subject: [EAI] Preserving header field name issue (Re: popimap-downgrade-00 - group syntax)

But "Downgraded-*" header field name was not rejected at the experimental stage.  Why did you comment it last year ?

Regards,

--
Kazunori Fujiwara, JPRS <[email-masked]> _______________________________________________
IMA mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ima