Subject: [EAI] Comments on Downgrading

From: Marcos Sanz/Denic <[email-masked]>
Date: Fri, 7 Apr 2006 16:59:11 +0200

All,

these are my comments on draft-yoneya-ima-downgrade-01. I haven't been 
following the last discussions in the list. If some of these issues (or 
any of those in my other mails) have already been dealt with, then just 
give me a pointer.

* Section 3.1: "SMTP client detects UTF-8 is included in SMTP envelope or 
mail headers". Haven't we introduced the i18n-header to avoid making this 
realtime, unreliable checks? And is it really meant "detects UTF-8" or it 
is rather meant "detects 8th bit set"?
* Section 3.2: s/Recent- headers/Resent- headers/
* Section 3.2: s/Referenece/References/
* Section 3.3: s/receipient/recipient/g (this appears a couple of times 
more)
* Section 4: "MUA of mail sender MUST append ALT-ADDR or ATOMIC". Aren't 
both parameters optional? Then why MUST?
* Section 4: s/next session/next section/
* Section 4: "local-part: Punycode without normalization". I am sure this 
has been discussed before (I saw a Normalizing-thread in the list), but 
fwiw here it comes my vote against non-normalized identifiers.
* Section 4: The IMA-Downgraded-From and -To headers should be defined in 
the Internationalized Email Headers draft, too, since that is the 
equivalent of updating RFC2822.
* Section 4: s/delived/delivered/
* Section 5.1: s/new body contains one MIME part/new body contains one new 
MIME part/
* Section 5.2: s/headers which contains/headers which contain/
* Section 6.2: "if MDA knows MUA is IMA compliant". How should that 
knowledge be conveyed?
* Section 7: I am missing a comment in the Security Considerations about 
whose responsibility is that the owners of the internationalized mailboxes 
match the owners of the alternative ASCII ones.
* Section 8: s/MUST be differ/MUST differ/

Best regards,
Marcos Sanz

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 25 Apr 2006 11:12:43 +0200

This seems to have generated no response.... I'll try to provoke....

>is rather meant "detects 8th bit set"?
There are just-send-8859-1 mailers out there in the wild.
UTF-8 can be detected with fair reliability.

>fwiw here it comes my vote against non-normalized identifiers.
the question is more what normalization, I think....

>* Section 6.2: "if MDA knows MUA is IMA compliant". How should that 
Is this a matter that can safely be left to local consideration?

>match the owners of the alternative ASCII ones.
I think the transport system should not care - there may be use cases 
where there's no entity apart from the end user who can tell that they 
have the same owner. (Think of someone running ASCII mail on yahoo! and 
internationalized mail on Gmail, for instance).

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 25 Apr 2006 17:53:45 +0100

> Is this a matter that can safely be left to local consideration?

Not entirely. General purpose MTAs (Exim, Sendmail and the like) often  
serve communities some of whose members use POP3/IMAP/Good ol'  
mbox/whatever-else, and will presently be faced with the problem that some  
members of their communities will be IMA-compliant and some not.

So such an MTA has two possibilities:
    1. Decide, when it sees the RCPT header and after consulting its tables  
of which of its recipients can do IMA, whether or not to bounce (or  
downgrade) the message at that stage.
    2. Accept the message anyway, with the possibility that it may have to  
bounce (or otherwise deal with) it later on when it tries to do its  
internal delivery.

Our problem is that the IETF has always considered the interface between  
the MTA and the storage agents to be an internal matter, since no "wire"  
is involved, and has therefore never tried to standardize it in any way. I  
have a feeling that this was a mistake, and that we really need some  
understanding of that process before we can complete our task.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: [email-masked]
Date: Wed, 26 Apr 2006 22:36:56 +0900 (JST)

Sorry too late my response and thanks a lot for comments.

I will update it according to your comments.

> From: Harald Alvestrand <[email-masked]>

Considering implementation costs, I want to use detecting 8th bit set
instead of UTF-8 check.

> UTF-8 can be detected with fair reliability.

Are ISO-8859-1 characters (not in US-ASCII) allowed in mail headers?

> >both parameters optional? Then why MUST?

I will rewrite this part such as:

  "If downgrade is expected, mail sender MUA MUST append ALT-ADDR or ATOMIC"

> the question is more what normalization, I think....

# I think endpoint MDA only knows normalization method.

> >equivalent of updating RFC2822.

agree (or maybe.)

> Is this a matter that can safely be left to local consideration?

Yes, I think so.
In some cases, downgrading is necessary while spooling.

# I'm happy if pop server support EAI.

> internationalized mail on Gmail, for instance).

I agree.
But SPF and DKIM support needs careful considerations.

--
Kazunori Fujiwara, JPRS

---

From: Chris Newman <[email-masked]>
Date: Wed, 26 Apr 2006 11:31:00 -0700

Charles Lindsey wrote on 4/25/06 17:53 +0100:

> On Tue, 25 Apr 2006 10:12:43 +0100, Harald Alvestrand  <[email-masked]>
> wrote:
>
>
>> Marcos Sanz/Denic wrote:
>
> Not entirely. General purpose MTAs (Exim, Sendmail and the like) often  serve
> communities some of whose members use POP3/IMAP/Good ol'  mbox/whatever-else,
> and will presently be faced with the problem that some  members of their
> communities will be IMA-compliant and some not.
>
> bounce (or otherwise deal with) it later on when it tries to do its  internal
> delivery.

I'm not fond of the statement "if MDA knows MUA is IMA compliant" simply 
because the MDA can not have that knowledge in the general case.  The choice of 
MUA can occur (or change) after the MDA has completed its task.

I'd say there are three scenarios:

1. The mailbox storage supports native UTF-8 headers and access is limited to
   mechanisms (extended POP/IMAP) that support down-conversion for legacy MUAs.
   In this case the MDA should not down-convert and may up-convert.
2. The mailbox storage supports native UTF-8 headers, but some MUAs that 
directly
   access the storage do not.  In this case the MDA must provide a mechanism
   for the user to specify a preference for down-conversion (e.g. a Sieve
   extension for down-conversion) and if the user does so than the MDA must
   down-convert; otherwise it should not down-convert.  The type of people who
   use MUAs that directly access network mail stores tend to be quite capable
   of configuration twiddling to make their stuff work.
3. The mailbox storage does not support native UTF-8 headers.  In this case,
   the MDA must down-convert.

Regardless, this is in the realm of advice and not wire standards (except that 
defining a Sieve down-convert extension is helpful for scenario 2).

> Our problem is that the IETF has always considered the interface between  the
> MTA and the storage agents to be an internal matter, since no "wire"  is
> involved, and has therefore never tried to standardize it in any way.

Incorrect.  See RFC 2033.

> I  have a feeling that this was a mistake, and that we really need some

We have people with that understanding reviewing our specifications.  That's 
more than sufficient for an experimental specification.

                - Chris

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 27 Apr 2006 09:58:28 +0200

>Considering implementation costs, I want to use detecting 8th bit set
The implementation cost for a proper UTF-8 detector is rather small (you 
check for the non-presence of the 13 forbidden bytes, and that sequences 
are properly formed - all that can be done in a fairly small finite 
state machine in one pass, and you still don't have to look at any byte 
more than oce), and the cost of accepting ISO 8859-1 into processing 
stages that expect properly formatted UTF-8 is that you have to do the 
error checking for that condition *everywhere*.

If we want to have heuristic UTF-8 detection, I think we should 
recommend UTF-8 detection, not the 8th bit check.

But if we go with the flag (i18n-header or otherwise), we have moved 
heuristic UTF-8 detection into the realm of "handling nonconformant 
data", which our standards leave out a lot of the time anyway.

>Are ISO-8859-1 characters (not in US-ASCII) allowed in mail headers?
No, they're forbidden. There are mailers that send them anyway.

---

From: Tony Hansen <[email-masked]>
Date: Thu, 27 Apr 2006 10:09:40 -0400

Harald Alvestrand wrote:
>>>> * Section 3.1: "SMTP client detects UTF-8 is included in SMTP
>>>> envelope or mail headers". Haven't we introduced the i18n-header to
>>>> avoid making this realtime, unreliable checks? And is it really
>>>> meant "detects UTF-8" or it is rather meant "detects 8th bit set"?
>>

Agreed: if we want UTF-8 detection, we should do a utf-8 check, and not
a "just 8-bit check".

	Tony Hansen
	[email-masked]

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 27 Apr 2006 12:38:28 +0100

On Wed, 26 Apr 2006 19:31:00 +0100, Chris Newman <[email-masked]>  
wrote:

> Regardless, this is in the realm of advice and not wire standards  
> (except that defining a Sieve down-convert extension is helpful for  
> scenario 2).

Maybe. We need to be sure that viable implementations are possible in all  
the scenarios that might arise, and writing down the "advice" somewhere is  
a good discipline to make sure all cases have been covered. Whether  
"somehere" turns out to be one of our standards, or the overview document,  
or some other informational document is a separate issue.
>> involved, and has therefore never tried to standardize it in any way.

Not so. LMTP is still describing a protocol to be used on a "wire", even  
though that wire is expected to be a part of a LAN.

But that does raise the interesting question as to whether our SMTP draft  
needs to make mention that it also defines an extension to RFC 2033, and  
whether there are any special considerations that it needs to mention  
which apply to LMTP.

For example, if a message has arrived at its final destination site (for  
some definition of "site"), and the site includes various IMAP and/or POP  
storage agents (each of which might or might not be IMA-capable), it is  
usual to use LMTP to distribute the message to those various storage  
agents? If so, then I would expect the LMTP machinery to be handling the  
bouncing/downgrading for each agent. Does out draft already cover that  
situation (I expect it does, but someone needs to check).

---

From: Chris Newman <[email-masked]>
Date: Thu, 27 Apr 2006 14:07:56 -0700

Charles Lindsey wrote on 4/27/06 12:38 +0100:
> whether there are any special considerations that it needs to mention  which
> apply to LMTP.
>
> usual to use LMTP to distribute the message to those various storage  agents?
> If so, then I would expect the LMTP machinery to be handling the

That is a good point.  Our extension document should mention that (a) the UTF-8 
headers extension may be used with LMTP and (b) We need an error code that 
applies to both SMTP and LMTP which means "this recipient doesn't accept UTF-8 
header material".  That error code is important because the final MTA may be 
capable of down-conversion but should only do it if the specific recipient 
requires it.  Regardless, LMTP (RFC 2033) should be an informative reference; I 
see no need to make it a normative reference.

Note that (a) is important because a revision of the LMTP specification should 
explicitly state that SMTP extensions do not apply to LMTP unless otherwise 
stated.  Some SMTP extensions (e.g. DSNs) make no sense in LMTP.

                - Chris