Subject: Re: [EAI] clarify Received header format

From: John C Klensin <[email-masked]>
Date: Wed, 07 Jun 2006 14:32:43 -0400

Chris,

I went back and reread this note in the aftermath of the Beijing
interim meeting.   I think we disagree, but want to make the
point of disagreement very clear.  I hope that will contribute
to a useful discussion that will move us forward.

--On Friday, May 05, 2006 16:25 -0700 Chris Newman
<[email-masked]> wrote:

> I agree it's absolutely critical to have the UTF-8 address in
> the Received header for diagnostic purposes.

We have already lost "absolutely critical" in the 2821 space,
since 2821 strongly recommends against the use of "for" in the
Received field provided by any SMTP receiver implementation that
receives more than one RCPT command for the message.    That
effectively means "never" if there is more than one recipient on
the same host at the submission server and "not until
forwarding, relaying, and routing get things down to one
recipient" at intermediate or delivery servers. (I think the
2821 language is not as clear as it might be, but that was the
intent).

> But 7-bit
> downgrading of Received headers must be as non-destructive and
> mechanically simple as possible.  Tampering with diagnostic
> trace information is very risky (but so is providing
> incomplete information).

Ok, we agree so far, modulo the above discussion about
incomplete information.

> I suggest we require 7-bit in all syntactic parts of the
> Received headers except for comments where we permit UTF-8.

Ok so far.  This effectively prohibit "for utf8-address", which
is what the working copy of "framework" now says.

> On down-conversion, the comments with UTF-8 characters are
> 2047 encoded; no other changes to the Received header are
> permitted.  This permits the UTF-8 address to be carried in
> canonical form as a comment in a pure UTF-8 environment, and
> the only permitted down-conversion is one that's
> non-destructive.
>
> All domain names in the Received header will be IDN encoded
> (but the 8-bit domain should be included in a comment), and
> the for clause will contain either the 7-bit alternate address
> or the 7-bit down-conversion (with the true UTF-8 address in
> the comments).

Here we have our first problem:

The semantics of the ACE-converted address (under ATOMIC) or the
ALT-ADDRESS are "an address to which the sender, presumably on
out of band instructions from the recipient, specified that the
message be sent if the base utf8-address turns out to be
unusable.  There is no guarantee that the base address and
alternate addresses are the same, or even refer to the same
users.  If we structure "for" in a way that assumes they are the
same user at the same mailbox, we are sending out misleading
information.   And, IMO, misleading information is even worse
than incomplete information.

Our second problem relates to where the Received field comes
from.  I know you don't need this tutorial, but there seems to
be enough confusion by others...  An MTA that is going to supply
a Received field always deals with two messages:

(i) The message it receives.   Except for the submission MTA,
that message always contains existing Received fields.   The
submission MTA may see a Received field to.  Let's call these
the "old received fields".

(ii) The message it transmits.  This message will always contain
one or more Received fields supplied by the MTA in question (the
"new Received fields") plus the previous, "old" ones.  As an
aside, if we follow the same principles for downgrading that we
do for gateways and used "for" or a comment equivalent (ignoring
how to represent it for the moment) there would normally be two
"new" Received fields from the downgrading MTA:

	Received: from previous-MTA-domain by current-MTA-domain
	at DateTime for UTF8-recipient...
	
	and
	
	Received: from current-MTA-domain by current-MTA-domain
	at DateTime for Alternate-recipient...

I see no problem at all in the MTA putting comments (with
encoded words or otherwise) in the new Received fields it
supplies.

I am very concerned about its changing any information at all in
old Received fields that it receives and is to forward on.  To
do so risks very serious distortions of what came before, which
is why we have prohibited it.  If we were to permit this, I
think it should be done only with specific annotation in one of
the new Received headers about what was changed and why, but it
would be better to avoid it.

But, if old Received lines contain UTF-8 information, even in
comments, your suggestion above requires converting that
information into encoded words.  That takes us far out on the
slipperly slope about changing old Received headers and seems to
be to be undesirable without far stronger justification.

As an aside, a suggestion was made during the design team
meeting in September.  It seems to have gotten lost, but it was
to document the relationship between a header UTF-8 address and
its ASCII replacement during or after downgrading by mapping the
UTF-8 header address into part of the display name, e.g., 

   Red-nosed clown <[email-masked]>
with ALT-ADDRESS equal to [email-masked]
would become something like (see "aside" below)
  "Red-nosed clown ( =?iso-8859-1?q?b=F3z0?=@clowns.example.com
)"

It still seems to me that would provide the right sort of
traceability, in the right place, without distorting any other
rules.

          john

Aside: RFC 2047 specifies a preference for character coding
based on ISO 8859-NN where possible and, as far as I can tell,
that suggestion has not been updated.  It is clearly not part of
the work of this WG, but it is probably getting near time to
update 2047 to express a preference for Unicode and to specify
whether that preference is for UTF-8 (i.e., a variable number of
octets per character) or, e.g., the equivalent of a U+NNNN
encoding.  It seems to me that anyone strongly advocating the
use of encoded words to solve downgrading problems needs to take
on the task of being sure that update to 2047 gets done somehow.

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 08 Jun 2006 10:35:58 +0100

> is what the working copy of "framework" now says.

No, it means they must be put into comments in some notation not too  
unlike the present "for" field. That is what Chris suggested, and is  
better than omitting them entirely (what then happens in the case of  
multiple RCPT TOs is then no different from now). Yes, it is unusual for  
standards to specify notations for what goes in comments, but that might  
be better than losing the feature entirely - but it would be a "SHOULD" at  
most.
>> the only permitted down-conversion is one that's

Alternatively, you REQUIRE that all comments in Received headers be  
generated in 2047 encoding in the first place, so further downgrading  
would ever be necessary. Might make it slightly harder for investigators  
of anomalies who examine the headers in their "raw" form, but we can live  
with that.

> 	at DateTime for Alternate-recipient...

Yes, I like the idea that an MTA MAY add two Received headers:
    "Here is what I received"
    "Hers is what I sent on",
  which would exhibit what it had done (use ALT-ADDRESS or ACEd the  
original one) in its "for" field.

The first one might contain one of Chris's "for field in a comment"s. The  
second could contain a genuine normal "for" field.

> update 2047 ...

RFC 2047 is a can of worms, and it will be a brave man who attempts to  
update it. But I would think the proper place to do it woud be in some  
2822-bis.

But in the meantime, wherever we ask in our documents for something to be  
generated in or downgraded to 2047, we can say that it SHOULD/MUST use  
utf-8.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Tony Finch <[email-masked]>
Date: Thu, 8 Jun 2006 13:10:39 +0100

>   "Hers is what I sent on",

Isn't the latter completely redundant with the Received: line added by the
next hop?

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BISCAY: EAST 4 OR 5, OCCASIONALLY 6 IN NORTH, BECOMING VARIABLE 3 OR 4 IN
SOUTHWEST. SHOWERS IN WEST. MODERATE OR GOOD.

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 08 Jun 2006 20:03:52 +0100

> Isn't the latter completely redundant with the Received: line added by  

Probably, but only if the next hop chooses to insert a "for" field, which  
not all do. But recording the change somewhere is important, and the agent  
that made the change is best motivated to do it.