Subject: Re: [EAI] UTF-8/MIME

From: "Jiankang YAO" <[email-masked]>
Date: Thu, 19 Aug 2010 09:38:06 +0800

----- Original Message ----- 
From: "Shawn Steele" <[email-masked]>
To: <[email-masked]>
Sent: Thursday, August 19, 2010 2:31 AM
Subject: Re: [EAI] UTF-8/MIME

>> my understanding is that headers are in utf-8,
>> the body part might still use mime(gb2312).
> 
> I'm obviously really confused.  My understanding was that EAI required UTF-8 headers and MIME encoded bodies, which should also be UTF-8, but an appropriate MIME part.  Presumably the body could also be GB2312 MIME, though I'd discourage that as much as possible.
> 
> So if the sentence in the RFC ever applied, it would have to be in the absence of MIME, which sort of means it's already not an EAI message.  So a non-MIME message with UTF-8 headers is an anomaly, and I'd sort of expect the body to have the same encoding.  Indeed, if I find non-MIME messages now with GB2312 bodies, then likely the header is also GB2312, not UTF-8.  John will be quick to point out that's not permitted by the standards, but I've heard anecdotal stories of it working, at least in controlled environments.  Clearly "older" mail clients often sent non-MIME mail with 8 bit bodies using whatever the local code page was.  Hopefully any mail client updated to handle EAI addresses would be smart enough to at least use UTF-8 and tag it properly.
> 
> So: If the intent was to say "a non-MIME message with EAI headers should...", then I think the language isn't needed.  Non-MIME is clearly non-EAI so out of scope for this document.  If the intent was "a non-EAI message that happens to not be ASCII should....", then it's probably also not needed.  The only reason I could see to keep it would be if we intended to explicitly add support for UTF-8 mail in absence of any EAI/MIME markup, which I understand the WG doesn't want, and would still preclude your GB2312 case.
> 
> Is your intent to enable a GB2312 mail client to use non-ASCII addresses without the rest of EAI/MIME?  
>

no, I have not such intension.

>
>Although (John will shudder) I can see why a client/server might want to privately recognize that convention, I don't see >how you can standardize it since code pages vary depending on the environment.  The only possible >mechanism that would interoperate outside of a limited environment would be to use something like UTF-8 that is recognizable >everywhere.
> 

my intension is to avoid the possible confusion:

we may open a hole for raw gb2312 encoding if we think the default encoding in body is 8bit utf8 when there are no parameters in the body.

specifying the ascii when  there are no parameters in the body is much clearer.

>
> I think I'm still confused.
> 
> - Shawn
> 
>  
> http://blogs.msdn.com/shawnste
> (Selfhost 7799)
> 
> 
> 
> ----- Original Message ----- 
> From: "Shawn Steele" <[email-masked]>
> To: "Jiankang YAO" <[email-masked]>; "John C Klensin" <[email-masked]>; <[email-masked]>
> Sent: Wednesday, August 18, 2010 4:41 PM
> Subject: RE: [EAI] rfc5336bis
> 
> 
>> The current text would forbid GB2312 or any other local code page.  (It says it has to be ASCII).
>> 
>> For those other clients the headers would also likely be a local code page & not UTF-8.
>> 
>> -Shawn
>> 
>> ???? ?????
>> http://blogs.msdn.com/shawnste
>> 
>> 
>> ________________________________________
>> From: Jiankang YAO [[email-masked]]
>> Sent: Tuesday, August 17, 2010 8:55 PM
>> To: Shawn Steele; John C Klensin; [email-masked]
>> Subject: Re: [EAI] rfc5336bis
>> 
>> I prefer that we still keep the current text.
>> 
>> besides the reason mentioned by John,  if we allow that the body can be
>> non-ascii data(utf-8) if there are no body parameters,  it is possible that the body may be other 8bit encoding standard such as
>> chinese code standard GB2312.
>> many MUAs sends the message with the local encoding.
>> 
>> 
>> 
>> ----- Original Message -----
>> From: "Shawn Steele" <[email-masked]>
>> To: "John C Klensin" <[email-masked]>; "Jiankang YAO" <[email-masked]>; <[email-masked]>
>> Sent: Wednesday, August 18, 2010 7:41 AM
>> Subject: RE: [EAI] rfc5336bis
>> 
>> 
>> That's sort of what I was concluding as well.  Either the message is MIME, so the headers are MIME and the BODY then should also be MIME.  (So the case can't ever happen), or the MIME requirement was ignored, so raw UTF-8 headers shouldn't be happening, so the rest is also irrelevant.  (I don't mind assuming UTF-8 for non-MIME and crossing my fingers, but I realize that isn't a popular opinion).
>> 
>> Can we just strike that reference then?  Since it sounds like it's just saying "use MIME like we said earlier?"
>> 
>> -Shawn
>> 
>> -----Original Message-----
>> From: John C Klensin [mailto:[email-masked]]
>> Sent: Tuesday, August 17, 2010 4:38 PM
>> To: Shawn Steele; Jiankang YAO; [email-masked]
>> Subject: RE: [EAI] rfc5336bis
>> 
>> 
>> 
>> --On Tuesday, August 17, 2010 22:43 +0000 Shawn Steele <[email-masked]> wrote:
>> 
>>> That's sort of my point.  What I read is that "if headers are UTF-8,
>>> but there's no MIME, then BODY is ASCII".  If there's no MIME, headers
>>> shouldn't be in UTF-8, so we shouldn't even get here.  If there's no
>>> MIME, but headers are UTF-8 and we want to recognize that, then I'd
>>> think the only logical conclusion would be that the body is also
>>> UTF-8.
>>>
>>> Maybe I completely misread what the original text meant.
>> 
>> I think we can't have UTF-8 headers (or any other piece of
>> UTF8SMTPbis) without MIME:
>> 
>> Reasons:
>> 
>> (1) There is no reason for not requiring MIME in this
>> day and age.
>> 
>> (2) UTF8SMTP[bis] requires 8BITMIME in order to make the
>> addresses and headers work.  And 8BITMIME absolutely
>> requires MIME.
>> 
>> (3) Absent MIME, we are back to RFC 822 (or 5322 if you
>> prefer) and, for it, the only valid headers are ASCII.
>> Absent MIME, there are no encoded words either.
>> 
>> Put differently, UTF-8 headers without MIME don't imply some sort of default UTF-8 body.  That case implies "seriously broken message" with no other options.
>> 
>> So I don't know what the original text meant either, but I'm
>> pretty sure it leads into a dead end.   Or worse.
>> 
>>   john
> 
> ------------------------------
> 
> Message: 5
> Date: Wed, 18 Aug 2010 09:59:44 -0400
> From: Andrew Sullivan <[email-masked]>
> Subject: Re: [EAI] proposal - can we reach consensus? - text for
> ascii@non-ascii issues
> To: [email-masked]
> Message-ID: <[email-masked]>
> Content-Type: text/plain; charset=us-ascii
> 
> On Tue, Aug 17, 2010 at 11:25:19PM +0000, Shawn Steele wrote:
>> John suggested:
>> > "this specification applies only when a UTF8SMTPbis-aware client is trying to send a message that requires the UTF8SMTPbis extensions to a server that advertises the ability to handle those extensions as specified here.  For all other cases, and for addresses and messages that do not require the extension, SMTP clients and servers are expected to behave exactly as specified in RFC 5321 and its successors." and then drop all of this material.
>> 
>> Can we reach a consensus that something like the above in the introductory material would address people's concerns with UTF8SMTP and non-UTF8SMTP interoperability and then drop the text that is being hard to pin down?  Does anyone strongly object to this idea?  (I won't hold it against you?)
> 
> I prefer it to attempting to cover all the cases that someone might do
> if they're not conforming to the RFCs we're working on.  To me,
> attempting to list all the ways in which someone can do something
> _other_ than the standard behaviour will just lead people to interpret
> those descriptions as the right thing to do sometimes. 
> 
> A
> 
> 
> -- 
> Andrew Sullivan
> [email-masked]
> 
> 
> ------------------------------
> 
> Message: 6
> Date: Wed, 18 Aug 2010 08:56:26 -0600
> From: Peter Saint-Andre <[email-masked]>
> Subject: Re: [EAI] proposal - can we reach consensus? - text for
> ascii@non-ascii issues
> To: [email-masked]
> Message-ID: <[email-masked]>
> Content-Type: text/plain; charset=UTF-8
> 
> On 8/18/10 7:59 AM, Andrew Sullivan wrote:
>> On Tue, Aug 17, 2010 at 11:25:19PM +0000, Shawn Steele wrote:
>>> John suggested:
>>>> "this specification applies only when a UTF8SMTPbis-aware client
>>>> is trying to send a message that requires the UTF8SMTPbis
>>>> extensions to a server that advertises the ability to handle
>>>> those extensions as specified here.  For all other cases, and for
>>>> addresses and messages that do not require the extension, SMTP
>>>> clients and servers are expected to behave exactly as specified
>>>> in RFC 5321 and its successors." and then drop all of this
>>>> material.
>>> 
>>> Can we reach a consensus that something like the above in the
>>> introductory material would address people's concerns with UTF8SMTP
>>> and non-UTF8SMTP interoperability and then drop the text that is
>>> being hard to pin down?  Does anyone strongly object to this idea?
>>> (I won't hold it against you?)
>> 
>> I prefer it to attempting to cover all the cases that someone might
>> do if they're not conforming to the RFCs we're working on.  To me, 
>> attempting to list all the ways in which someone can do something 
>> _other_ than the standard behaviour will just lead people to
>> interpret those descriptions as the right thing to do sometimes.
> 
> Agreed.
> 
> Peter
> 
> -- 
> Peter Saint-Andre
> https://stpeter.im/
> 
> 
> 
> 
> ------------------------------
> 
> _______________________________________________
> IMA mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ima
> 
> 
> End of IMA Digest, Vol 61, Issue 16
> ***********************************
> 
> _______________________________________________
> IMA mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ima
>

---

From: Shawn Steele <[email-masked]>
Date: Thu, 19 Aug 2010 01:39:46 +0000

I think that if there's no parameters in the body, then you're not EAI email, so you're in some odd broken case.  To me this sounds like "not EAI, so use legacy SMTP", which would then prohibit it.  So the general words John proposed would work?

-Shawn

---

From: "Jiankang YAO" <[email-masked]>
Date: Thu, 19 Aug 2010 10:00:12 +0800

so we add something like "UTF-8 headers without MIME don't imply some sort of default UTF-8 body.  "?

---

From: "Jiankang YAO" <[email-masked]>
Date: Thu, 19 Aug 2010 16:50:30 +0800

----- Original Message ----- 
From: "Shawn Steele" <[email-masked]>
To: "Jiankang YAO" <[email-masked]>; <[email-masked]>
Sent: Thursday, August 19, 2010 9:39 AM
Subject: RE: [EAI] UTF-8/MIME

>I think that if there's no parameters in the body, then you're not EAI email, so you're in some odd broken case.  
>
another alternate is:

if it is broken, we may simply remove this text,

 "            If there is no BODY parameter, the header contains UTF-8
            characters, but all the body parts are in ASCII (possibly as the
            result of a content-transfer-encoding)."

Jiankang Yao
>
>To me this sounds like "not EAI, so use legacy SMTP", which would then prohibit it.  So the general words John proposed would work?
> 
> -Shawn
>

---

From: Shawn Steele <[email-masked]>
Date: Thu, 19 Aug 2010 15:31:56 +0000

That would be my suggestion.  Then leave John's text in the intro that says 

-Shawn

Sent: Thursday, August 19, 2010 1:50 AM

----- Original Message -----
From: "Shawn Steele" <[email-masked]>
To: "Jiankang YAO" <[email-masked]>; <[email-masked]>
Sent: Thursday, August 19, 2010 9:39 AM
Subject: RE: [EAI] UTF-8/MIME

another alternate is:

Jiankang Yao