Subject: Re: [EAI] UTF-8/MIME

From: Shawn Steele <[email-masked]>
Date: Thu, 19 Aug 2010 17:43:13 +0000

> Now you've got me confused. An EAI message (i.e. one coming from an agent asserting a need for UTF8SMTP) might contain headers using UTF-8, but the body entirely in ASCII. So no MIME stuff anywhere in it. So you cannot say that EAI REQUIRES MIME, though for sure it would be stupid to implement it without MIME.

John was asserting that EAI messages have to be MIME.  IMO I think that's overkill, people use "simple" submission tools, telnet even, but I don't really want to block the RFC over that discussion.

> In fact, I suspect that default practice is going to happen anyway within EAI-only communities, so we might as well make it official.

> Another downside is that you can't represent a legacy non-MIME message with an 8bit body. While incorrect, these message do exist and many user agents will handle them in a reasonable way.

As John points out, that's violating the standards, so I don't think we need to address that case.  People can still do what they want.  "handle them in a reasonable way" is subjective.  If your machines happen to use the same code page, you're "lucky".  Most of the support calls I get about that kind of thing is because the people weren't lucky.

> You need to keep the UTF-8 flag intact. Otherwise (and without 8bit headers), you can't know whether the body defaults to UTF-8 or whether you need to use the old handling (probably NOT as UTF-8).

Actually, the utf-8 or old handling part may not be that hard.  If you can decode it in UTF-8 w/o error, then it's almost certainly UTF-8.  There's a few really rare cases (Nestlé® or something like that), but you could try UTF-8 & if it didn't work, then you could do your "old handling".  That's because the UTF-8 lead/trail bytes would make really strange words in any other code pages.

If we did want to allow a non-MIME behavior, I'd say it needs to be UTF-8; the header's already 8-bit, so it'd be pointless to restrict the body to 7 bit.  If UTF-8 decoding failed for whatever legacy "old handling" reason, the client could still fall back to that.  However I don't think we should delay the RFC over this discussion.

-Shawn

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 20 Aug 2010 20:24:50 +0100

On Thu, 19 Aug 2010 18:43:13 +0100, Shawn Steele
<[email-masked]> wrote:

> John was asserting that EAI messages have to be MIME.  IMO I think  
> that's overkill, people use "simple" submission tools, telnet even, but  
> I don't really want to block the RFC over that discussion.

Interesting point. From time to time I use telnet to send email (usually
when testing that something is working). I try to use as few header as
possible, and if I were doing it in some EAI environment, it would be
tedious to type in 3 extra headers (MINE, CT and CTE) which appeared to
serve no useful purpose.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: John C Klensin <[email-masked]>
Date: Sun, 22 Aug 2010 23:32:13 -0400

--On Friday, August 20, 2010 20:24 +0100 Charles Lindsey
<[email-masked]> wrote:

>> John was asserting that EAI messages have to be MIME.  IMO I
>> think   that's overkill, people use "simple" submission
>> tools, telnet even, but   I don't really want to block the
>> RFC over that discussion.
> 

Please remember that 

	-- sending UTF-8 in headers is a serious violation of
	the mail specs (including 822/ 2822/ 5322) without 
   8BITMIME and MIME
	
	-- encoded words don't work (aren't defined) without
	MIME.

	-- Telnet itself is defined only for NVT (US-ASCII)
	unless options are negotiated that get to some 8bit form.
	Option negotiation is defined only for Telnet over
	port 23.  So one really cannot use standard Telnet to 
   send UTF-8 to port 25. 

So, while I would like  to do lots of things, including testing
basic email by using Telnet to port 25 but omitting, e.g., the
Date header field, I recognize that doing so is outside the spec
and that some systems will reject the mail if I don't specify a
Date and at least one of a To, Cc, or Bcc header field.   It
seems to me that the EAI situation is either no different or
that what you want to do is even more outside the spec.  

As with other things, some things that are outside the specs
will work, either in the name of robustness or because of sloppy
implementations.    But I really, really, do not see why we are
having this conversation unless the intent is to waste the WG's
time and delay the documents.  I'm in favor of these sorts of
inquiries and excursions into what people would like to do in a
different world for general clarification, but please take/ keep
them off the WG list.

     john

---

From: Nick Teint <[email-masked]>
Date: Mon, 23 Aug 2010 10:10:53 +0200

On 2010-08-20, at 21:24, Charles Lindsey wrote:
> Interesting point. From time to time I use telnet to send email (usually when testing that something is working). I try to use as few header as possible, and if I were doing it in some EAI environment, it would be tedious to type in 3 extra headers (MINE, CT and CTE) which appeared to serve no useful purpose.

You won't need the headers unless the body contains anything other than text/plain; charset=US-ASCII. You will usually get away with undeclared 8bit chars, too.

The best practice for mail transfer agents is to look at neither the MIME headers nor the body. (You can't avoid it if you need to downgrade from 8BITMIME to ASCII, though.)

NT