Subject: [EAI] EDITORIAL: dsn

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 19 Sep 2007 23:19:49 +0100

The DSN draft is still using the term "UTF8SMTP message" (in at least two  
places). I think it shold now be "global message".

4.  UTF-8 Delivery Status Notifications

    ....  As the present DSN format does not permit returning of
    undeliverable UTF8SMTP message, three new media types are needed.
                           ^^^^^^^
                           messages

    The third type, used for header return, is message/global-headers and
    contains only the UTF-8 headers of a message (all lines prior to the
    first blank line in a UTF8SMTP message).  Unlike message/global, this
    body part provides no difficulties for present infrastructure.

On the contrary, it provides precisely the same difficulties :-( .

4.1.  Additional requirements on SMTP servers

    If an SMTP server which advertises both UTF8SMTP and DSN needs to
    return an undeliverable UTF8SMTP message, then it MUST NOT downgrade
    [I-D.ietf-eai-downgrade] the UTF8SMTP message when generating the
    corresponding multipart/report.  If the return path SMTP server does
    not support UTF8SMTP, then the undeliverable body part and headers
    MUST be encoded using a 7 bit Content-Transfer-Encoding such as
    base64 or quoted-printable [RFC2045], as detailed in Section 4.
    Otherwise 8bit Content-Transfer-Encoding can be used.

Why do you want to forbid downgrade here? The UTF-8 draft explicitly  
allows downgrade as an alternative to re-encoding in Q-P or Base64, so why  
be different?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 20 Sep 2007 18:22:18 +0100

Charles Lindsey wrote:

> The DSN draft is still using the term "UTF8SMTP message" (in at least 
> two  places). I think it shold now be "global message".

UTF8SMTP is referencing the SMTP extension name, so I don't think this 
is particularly confusing.
I personally find "global message" to be more confusing (unlike 
message/global).

>    undeliverable UTF8SMTP message, three new media types are needed.

Changed, thanks.

> On the contrary, it provides precisely the same difficulties :-( .

I think we need an interop event to prove one way or another.

I can delete the second sentence, if people prefer.

> allows downgrade as an alternative to re-encoding in Q-P or Base64, so 
> why  be different?

I've already commented on UTF-8 headers draft, saying that it is in 
conflict with EAI DSN.

The issue was discussed in Chicago: basically use of C-T-E is safer then 
downgrade, because downgrade is not always possible. It is better to 
change C-T-E, then to lose the bounce message because it can't be converted.
Another argument for using C-T-E conversion is that servers can already 
do that (or can be made capable of doing that using some minor code 
changes) and they don't need to implement downgrade just for processing 
of DSN reports.

Regards,
Alexey

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 21 Sep 2007 11:23:24 +0100

> do that .....

I have repeatedly asked for a single example of a server that can already  
do that, and no example has been forthcoming (whereas we know of several  
servers that currently cannot do it). That is the whole basis of the ISSUE  
that I have raised.

> ...(or can be made capable of doing that using some minor code changes)  
> ...

The whole basis of our endeavour is that only those agents that want to  
take advantage of our new "global"/"UTF8SNTP" message need to have their  
implementations changed. Now you are suggesting that all existing MTAs  
will need to be upgraded (which just ain't going to happen, however minor  
the code changes involved).

> ... and they don't need to implement downgrade just for processing of  
> DSN reports.

The downgrade option would be used by a UTF8SMTP-capable agent (such as  
the one generating the DSN) when it saw that it was being asked to send  
the DSN to a Non-UTF8SMTP agent. If it decided not to do that (it is  
currently just an option in utf8headers), then a subsequent Non-UTF8SMTP  
would (hopefully) encode it in Q-P or Base64 before giving it to a  
Non-8BITMIME agent. Evidently, your degree of Hopefulness is greater than  
mine :-) .

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 21 Sep 2007 12:10:21 +0100

Charles Lindsey wrote:

>>> Why do you want to forbid downgrade here? The UTF-8 draft 
>>> explicitly   allows downgrade as an alternative to re-encoding in 
>>> Q-P or Base64, so  why  be different?
>>
>> The issue was discussed in Chicago: basically use of C-T-E is safer 
>> then  downgrade, because downgrade is not always possible. It is 
>> better to  change C-T-E, then to lose the bounce message because it 
>> can't be  converted.
>> Another argument for using C-T-E conversion is that servers can 
>> already  do that .....
>
> I have repeatedly asked for a single example of a server that can 
> already  do that, and no example has been forthcoming (whereas we know 
> of several  servers that currently cannot do it). That is the whole 
> basis of the ISSUE  that I have raised.

I believe Isode pp can do that. But I haven't tested this yet ;-).

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 24 Sep 2007 14:13:16 +0100

> I believe Isode pp can do that. But I haven't tested this yet ;-).

OK, can anyone else confirm this? I just Googled for ISODE PP, and all the  
mentions of it that I found seemed increadibly ancient.

Is it widely used in pure SMTP circles (as opposed to situations where  
X.400 is being gatewayed to/from)?

---

From: John C Klensin <[email-masked]>
Date: Mon, 24 Sep 2007 11:42:09 -0400

Charles,

I'm confused about what you are asking for, but 

	* Every server that can do 8BITMIME downgrading (and
	that includes such popular ones as Sun and Sendmail),
	does C-T-E remapping.  This is also true of servers that
	do BINARYMIME downgrading, but there may be fewer of
	those.
	
	* Every delivery server (or IMAP/POP server/processor)
	that converts content from C-T-E q-p or base64 to the
	corresponding 8bit form (and there are many of those),
	does C-T-E remapping.

I, at least, haven't responded to your request because I have
believed that I either didn't understand it (which is still
possible) or that the feature was so wideely deployed as to not
be worth discussing further.

    john

--On Monday, 24 September, 2007 14:13 +0100 Charles Lindsey
<[email-masked]> wrote:

>>>> Another argument for using C-T-E conversion is that servers
>>>> can   already  do that .....
>>> 
>>> I have repeatedly asked for a single example of a server
>>> that can   already  do that, and no example has been
>>> forthcoming (whereas we know   of several  servers that
>>> currently cannot do it). That is the whole   basis of the
>>> ISSUE  that I have raised.
>> 
> OK, can anyone else confirm this? I just Googled for ISODE PP,
> and all the mentions of it that I found seemed increadibly
> ancient.
> 
> Is it widely used in pure SMTP circles (as opposed to
> situations where X.400 is being gatewayed to/from)?

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 24 Sep 2007 17:02:24 +0100

Charles Lindsey wrote:

>>> already  do that, and no example has been forthcoming (whereas we 
>>> know  of several  servers that currently cannot do it). That is the 
>>> whole  basis of the ISSUE  that I have raised.
>>
> OK, can anyone else confirm this? I just Googled for ISODE PP, and all 
> the  mentions of it that I found seemed increadibly ancient.

http://www.isode.com/products/m-switch.html

(pp is an internal name, the product is now known as M-Switch)

> Is it widely used in pure SMTP circles (as opposed to situations 
> where  X.400 is being gatewayed to/from)?

It can be run as a pure Internet MTA. I am certainly not communicating 
to you using X.400/MIXER ;-)

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 25 Sep 2007 14:11:30 +0100

> 	do BINARYMIME downgrading, but there may be fewer of

NO! That is not true, which is the whole point of my argument.

Let us consider sendmail, with which I have some familiarity. I have  
examined the code, and what it does appears to be as follows:

Essentially, it a two-pass process. As it receives stuff from the SMTP  
DATA command, it looks to see if a bit8 is set anywhere in the body, and  
sets a flag accordingly (later, if this flag is not present, it can bypass  
a lot of stuff).

Then it places the message in its queue.

On taking it out of the queue to send to some particular site, it observes  
whether that site supports 8BITMIME (and eventually will presumably detect  
UTF8SMTP likewise).

So it the flag is set and 8BITMIME is not supported it enters a  
downgrading routine which first examines the Content-Type and  
distinguishes three cases:

1. multipart types;
2. message types;
3. all others.

For "all others", obviously, it sets about changing the C-T-E to Q-P or  
Bas64 (after deciding which off these seems likely to be best).

For "multipart types" is takes note of the 'boundary' string and goes  
looking for the various parts one-by-one, and calls itself recursively for  
each one.

For "message types" it first checks for the special case message/rfc822,  
and proceeds to call itself recursively on the encapsulated message (which  
may cause individual parts of that encapsulated message to be re-encoded  
in Q-P of Base64).

But in the case of message types other than message/rfc822 it simply  
leaves the content of the message alone; so if there is any bit8 present  
it simply gets sent as-is (that situation is just not supposed to happen  
with any message type, but as a strategy which will "probably work most of  
the time" it is probably a sensible thing to do).

BUT, since it NEVER tries to change the encoding of such a message type,  
it never infringes that "EXPRESSLY FORBIDDEN" in RFC 2045. So you can't  
accuse iot of non-compliance in that regard.

It is not easy to set up an experiment to verify this is what actually  
happens (you need to find or create a Non-8BITMIME MTA, and there are not  
many of them around). However, sendmail can also be configured to use LMTP  
for final delivery, so I did set up a Non-8BITMIME storage agent and  
configured sendmail to deliver to it by LMTP, and it behaved just as I  
have described above (well it is some months now since I did the  
experiment, so I do not remember the exact details of what I did).

As for exim and qmail, it has been reported on this list that neither of  
them will downgrade anything at all, though both transmit the full 8bits  
onwards. Exim out of the box does not announce 8BITMIME, though it may be  
(and often is) configured to do so. Qmail always announces 8BITMIME, which  
is naughty but which DJB will stoutly defend.

Between them, sendmail, exim and qmail cover quite a substantial slice of  
deployed MTAs.

> 	corresponding 8bit form (and there are many of those),

Yes, it may be that delivery agents regularly convert to Q-P or Base64  
(and MUAs then upconvert it for display). But that is irrelevant to my  
argument, which is solely concerned as to whether such encodings will be  
applied by MTAs during transit.