Subject: [EAI] ISSUE utf8 meaage/global

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 19 Sep 2007 23:20:53 +0100

4.2.  Changes on MIME headers

    This specification updates section 6.4 of RFC 2045.  RFC 2045
    prohibits applying a content-transfer-encoding to all subtypes of
    message/.  This specification relaxes the rule, allowing newly
    defined MIME types to permit content-transfer-encoding. (see
    Section 4.6).

I reiterate my objection to this breach of the "EXPRESSLY FORBIDDEN" rule  
in RFC 2045, unless it can be established that a significant part of the  
installed base of MTAs will cope with it when downgrading to Non-8BITMIME  
(it is already established that a significant part of that base will _not_  
cope with it).

It is claimed in 4.6 that existing MTAs will cope with this unknown MIME  
type by treating it as application/octet stream (although it is unclear  
whether that default pvovision was ever intended to apply to other than  
MUAs, and whether MUAs handle Q-P and Base64 in message types is  
irrelevant to my argument). The issue is what the current installed base  
of MTAs actually _does_ (rightly or wrongly).

Note that we are now proposing to apply this relaxation to four new  
message types:
    message/global
    message/global-delivery-status
    message/global-headers
    message/global-disposition-notification
of which problems with the third could be avoided by using  
text/global-headers instead.

I understand that the consensus is against me on this issue, and I am  
therefore not asking that it be reopened at this stage. However, I repeat  
what I have already stated before, that I intend to raise it at IESG Last  
Call.

Note that I have written separate EDITORIAL comments on the utf8 and dsn  
drafts.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 25 Sep 2007 00:14:59 +0200

Charles Lindsey wrote:
 
> I reiterate my objection to this breach of the "EXPRESSLY FORBIDDEN"
> rule in RFC 2045

+1

---

From: Yuri Inglikov <[email-masked]>
Date: Mon, 24 Sep 2007 21:37:30 -0700

+1

I consider such restriction for compound types as quite fundamental, handling messages without such restriction can be extremely expensive.

Yuri Inglikov

-----Original Message-----
From: Frank Ellermann [mailto:[email-masked]]
Sent: Monday, September 24, 2007 3:15 PM
To: [email-masked]
Subject: [EAI] Re: ISSUE utf8 meaage/global

+1

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 27 Sep 2007 14:50:39 +0200

This message has been recorded as a comment on closed issue #1485.

Charles Lindsey wrote:
> rule in RFC 2045, unless it can be established that a significant part 
> of the installed base of MTAs will cope with it when downgrading to 
> Non-8BITMIME (it is already established that a significant part of 
> that base will _not_ cope with it).
>
> It is claimed in 4.6 that existing MTAs will cope with this unknown 
> MIME type by treating it as application/octet stream (although it is 
> unclear whether that default pvovision was ever intended to apply to 
> other than MUAs, and whether MUAs handle Q-P and Base64 in message 
> types is irrelevant to my argument). The issue is what the current 
> installed base of MTAs actually _does_ (rightly or wrongly).
>
> therefore not asking that it be reopened at this stage. However, I 
> repeat what I have already stated before, that I intend to raise it at 
> IESG Last Call.
>
> Note that I have written separate EDITORIAL comments on the utf8 and 
> dsn drafts.
>
> --Charles H. Lindsey ---------At Home, doing my own thing------------------------ 
>
> Tel: +44 161 436 6131

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 27 Sep 2007 21:46:55 +0200

> This message has been recorded as a comment on closed issue #1485.

Your records of #1485 mainly document how you arrived at the names
message/global etc. in 8BITMIME and UTF8SMTP environments.  These
names are fine, the issue is the violation of a rule in RFC 2045.

I've checked the relevant RFCs (4288, 2046, 2045, and 2049) again.

1 - RFC 4288 says "message/* subtypes MUST follow RFC 2046 rules".
2 - RFC 2046 allows only 7bit/8bit/binary for multipart/* (sic!),
    it has no overall CTE restriction for any message/* (ignoring
    a lower case "should ASCII").  It has individual restrictions
    for the discussed subtypes rfc822, partial, and external-body.
3 - RFC 2046 and 2049 both say that an unknown message/* has to be
    handled like application/octet-stream.
4 - RFC 2045 has the famous "EXPRESSLY FORBIDDEN" about any CTEs
    different from 7bit, 8bit, and binary for all composite media
    types, especially for multipart/* (confirmed in RFC 2046) and
    for message/* (NOT confirmed in RFC 2046, where restrictions
    are obviously and intentionally listed per message/* subtype).
5 - By definition a 7bit MTA cannot "know" (= support) any subtype
    defined only for 8bit and binary.  Therefore it must handle
    message/global and its variants like application/octet-stream.
6 - CTE B64 and QP are perfectly okay for application/octet-stream.

Maybe the I-D should explain why the apparent violation of RFC 2045
is okay.  Actually either RFC 2045 or 2046 must be incorrect, and
picking RFC 2046 as correct makes sense.  Maybe we should submit
this issue as an erratum for RFC 2045.

 Frank

---

From: Yuri Inglikov <[email-masked]>
Date: Thu, 27 Sep 2007 13:50:51 -0700

Could we at least limit the maximum nesting of encoded message/global to one level? Implementations will likely need to put a cap on it anyway and leaving it open-ended can create interoperability issues. On the other hand, one-level encoding should be sufficient to address all UTF8SMTP scenarios?

Yuri

-----Original Message-----
From: Frank Ellermann [mailto:[email-masked]]
Sent: Thursday, September 27, 2007 12:47 PM
To: [email-masked]
Subject: [EAI] Re: #1485 (closed) Re: ISSUE utf8 meaage/global

 Frank

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 27 Sep 2007 23:32:26 +0200

Yuri Inglikov wrote:

> Could we at least limit the maximum nesting of encoded message/global
> to one level?

Indeed, doing it bottom-up would be unwise, top-down B64 once per
message/global part (including its complete body) is good enough.

Maybe the I-D could RECOMMEND this recipe, or is this an obvious
implementation detail ?

 Frank

---

From: Yuri Inglikov <[email-masked]>
Date: Fri, 28 Sep 2007 01:00:12 -0700

Ah, if this is an "obvious implementation detail", or even if this is just "a recommendation", then anybody needs to be prepared to handle arbitrary nesting. It should be completely prohibited to have any effect, like in that famous "EXPRESSLY FORBIDDEN" statement. Trivial (and often the only feasible) implementation of decoding nested encoded messages has quadratic complexity. It is unwise to define such "feature" in e-mail standard.

-----Original Message-----
From: Frank Ellermann [mailto:[email-masked]]
Sent: Thursday, September 27, 2007 2:32 PM
To: [email-masked]
Subject: [EAI] Re: Re: #1485 (closed) Re: ISSUE utf8 meaage/global

 Frank

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 28 Sep 2007 11:57:19 +0200

Yuri Inglikov wrote:
> Ah, if this is an "obvious implementation detail", or even if this is just "a recommendation", then anybody needs to be prepared to handle arbitrary nesting. It should be completely prohibited to have any effect, like in that famous "EXPRESSLY FORBIDDEN" statement.
Agreed on that. If we accept the current proposal, I don't expect double 
encoding to happen much in practice - the main case I can think of 
offhand would be someone in UTF8 land who gets a message/rfc822 with 
parts using Q-P or base64, forwards that message to another UTF8 user 
(which doesn't impose any encoding), and the second user then forwards 
the whole message as a message/global to a non-UTF8 user. Pass the 
message around the same circle once more to get a triple encoding.

(If the UTF8 user just forwards the first 822 message to an ASCII user, 
the outer message gets downgraded to message/rfc822, and no double 
encoding of the inner message happens; you need 2 levels of forwarding 
to get a double encoding.)
> Trivial (and often the only feasible) implementation of decoding nested encoded messages has quadratic complexity. It is unwise to define such "feature" in e-mail standard.
>   
quadratic in what?

I believe that if your language-of-choice supports queues and either 
cheap threads or coroutines, there's a fairly trivial implementation 
that's linear in the number of levels of encoding (a message part that 
is doubly encoded needs twice the processing power of a message that's 
singly encoded).

---

From: "Frank Ellermann" <[email-masked]>
Date: Fri, 28 Sep 2007 17:24:14 +0200

> is doubly encoded needs twice the processing power of a message that's 

We could have a situation if the MUA is smarter in decoding than the AV
software before this MUA.  A new point for the security considerations
in the utfheaders I-D, the bad guys will try to abuse this feature of
message/global.
 
 Frank

---

From: Yuri Inglikov <[email-masked]>
Date: Fri, 28 Sep 2007 08:53:41 -0700

>> there's a fairly trivial implementation that's linear in the number of levels of encoding

Complexity of decoding each level is proportional to its length, not constant. The overall complexity is something like O(n*m) where n is size of outer MIME in bytes and m is the number of nested levels. Intuitively it is obvious as you need to touch every byte of most nested encoding m times to decode it. In extreme the m can be pretty close to n, hence the quadratic complexity. I can imagine a linear complexity implementation if I speculate that there is a decode<m>(b) operation which can be implemented in a constant time, but I am not sure how feasible such operation can be, considering artifacts like arbitrary folding of nested encoded content and mixing 2 encoding schemes. I would be very curious to see a trivial O(n) implementation of arbitrarily nested decoding.

Please don't allow this.

-----Original Message-----
From: Harald Alvestrand [mailto:[email-masked]]
Sent: Friday, September 28, 2007 2:57 AM
To: Yuri Inglikov
Cc: Frank Ellermann; [email-masked]
Subject: Re: [EAI] Re: Re: #1485 (closed) Re: ISSUE utf8 meaage/global

quadratic in what?