Subject: TECH-ERROR: SenderID sets recomendation for forwarders that are not compatible with RFC 2822

From: "william(at)elan.net" <[email-masked]>
Date: Sun, 12 Sep 2004 22:47:32 -0700 (PDT)

In marid-core draft says in section 7.2 it is written:

7.2 E-Mail Forwarders
   A program that forwards received mail to other addresses MUST add an
   appropriate header that contains an email address that it is
   authorized to use.  Such programs SHOULD use the Resent-From header
   for this purpose.

But this recomendation for Forwarders to use Resent-From header directly
contradicts what is written in section 3.6.6 of the RFC2822:

   Note: Reintroducing a message into the transport system and using
   resent fields is a different operation from "forwarding".
   "Forwarding" has two meanings: One sense of forwarding is that a mail
   reading program can be told by a user to forward a copy of a message
   to another person, making the forwarded message the body of the new
   message.  A forwarded message in this sense does not appear to have
   come from the original sender, but is an entirely new message from
   the forwarder of the message.  On the other hand, forwarding is also
   used to mean when a mail transport program gets a message and
   forwards it on to a different destination for final delivery. Resent
   header fields are not intended for use with either type of forwarding"

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: David Woodhouse <[email-masked]>
Date: Mon, 13 Sep 2004 11:02:31 +0100

> contradicts what is written in section 3.6.6 of the RFC2822:

We should not abuse the Resent-From: header. Instead, we should either
introduce a new header or simply add a new field to the Received:
header. I favour the latter.

I have not seen any reason given for abusing the existing Resent-From:
header, nor do I understand why it is useful to do so instead of
introducing something new and unambiguous.

-- 
dwmw2

---

From: Tony Finch <[email-masked]>
Date: Mon, 13 Sep 2004 12:02:05 +0100

> header. I favour the latter.

The Received: field already includes a "for" clause which contains the
recipients of the message. It is usually omitted if there is more than one
recipient for privacy reasons (otherwise it could expose BCC recipients).

If Sender-ID were to use that clause, what should it contain in the
multiple recipients case? At least Resent-From: mailer-daemon makes some
kind of sense. Received: for unspecified-recipients@unspecified-domains?

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
ROCKALL: CYCLONIC, BECOMING NORTH 6 TO GALE 8, OCCASIONALLY SEVERE GALE 9.
SHOWERS. MAINLY GOOD.

---

From: Tony Finch <[email-masked]>
Date: Mon, 13 Sep 2004 12:33:19 +0100

On Mon, 13 Sep 2004, David Woodhouse wrote:
>
> The 'for' clause indicates recipients, while a new clause would indicate
> the purported _sender_ for the hop in question. It would be 'on behalf
> of' or something similar.

The Received: field is created at the wrong time for that information to
be included, unless each hop adds both an incoming Received: field (like
the current one) and an outgoing field for recording the submitter.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
COLWYN BAY TO THE MULL OF GALLOWAY INCLUDING THE ISLE OF MAN: SOUTHWEST 5 TO
7, INCREASING GALE 8 FOR A TIME, BECOMING CYCLONIC 5 LATER. SQUALLY SHOWERS.
MAINLY GOOD. MODERATE OR ROUGH.

---

From: David Woodhouse <[email-masked]>
Date: Mon, 13 Sep 2004 12:24:26 +0100

> kind of sense. Received: for unspecified-recipients@unspecified-domains?

For there to be multiple senders, I assume you speak of the case where
the original message was to multiple local recipients, each of whom has
a .forward file directed at a users on a the same third party host, and
your MTA wants to send those messages in the same SMTP transaction. 

The problem is the same as with the Resent-From: header. Yes, you
probably just use a mailer-dæmon address.

I wasn't proposing any change to the information you include -- only
that you should put it somewhere new instead of abusing the Resent-From:
header.

-- 
dwmw2

---

From: David Woodhouse <[email-masked]>
Date: Mon, 13 Sep 2004 12:37:55 +0100

> the current one) and an outgoing field for recording the submitter.

That's true. So perhaps the Received: header isn't the right place for
such information -- but certainly the Resent-From: header isn't the
right place for it either. Then we should add a new header for the
purpose.

An alternative would be to provide the selected address to the next hop
by means of an extension such as 'SUBMITTER-RFC2822' which gets logged
in the _next_ Received: header. I don't like that much -- defining a new
header makes more sense.

-- 
dwmw2

---

From: "william(at)elan.net" <[email-masked]>
Date: Mon, 13 Sep 2004 05:31:36 -0700 (PDT)

> header. I favour the latter.

The received header unfortunetly has not been standartized (ok to be 
more precise it standartized by RFC2821 in section 4.4 but many systems 
still don't do it in correct way).

Additionally Received header are specially designated to be trace fields,
so they are like a loggin info. Normally in programming, one would not think 
of using loggin data to implement program processing (other then database
logs where it is a journal that can be used to revert back to original
state or in case of filesystems, like ext3 JFS to fix the original 
state in case of known errors). So in my view Received header is a bad 
place for adding this data.

Plus to that Received header is normally added by systems when it received
the message but Submitter information is added when message is being sent out.

> introducing something new and unambiguous.

I completely agree. I'll be submitting proposal later this week on it.
In the mean time I'll present the main ideas in separate email to 
get a view if WG has any interest in it and to try to resolve some
of the issues I have not decided on.

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: Tony Finch <[email-masked]>
Date: Mon, 13 Sep 2004 13:43:02 +0100

> still don't do it in correct way).

It has been specified for over 20 years: the format was originally
specified in RFC 821 (STD 10) and hasn't changed significantly since.

Tony.

---

From: David Woodhouse <[email-masked]>
Date: Mon, 13 Sep 2004 13:51:25 +0100

On Mon, 2004-09-13 at 05:31 -0700, william(at)elan.net wrote:
> so they are like a loggin info.

That's all they are here, surely? In a world where SenderID was
ubiquitous you'd have mail servers automatically rewriting RFC2821 and
RFC2822 identities on outgoing mail, and the SenderID validates _only_
that one hop; it's not end-to-end validation such as PGP, DomainKeys or
Signed Envelope Senders would offer.

You end up using a domain-based blacklist instead of the IP-address-
based blacklists which are already common, but other than that the
problem hasn't changed much. It's just a way of determining which are
legitimate mail servers, and which are not.

The information as to the 'purported responsible' domain in each hop
really is just trace info which is of little importance after the fact.

A sending mail server can choose any 'domain' it likes when modifying
the mail it sends, as long as its IP address is designed as one of the
permitted addresses for the 'domain'. The domain used doesn't have to be
the same for all outgoing mail, and doesn't have to bear any relation to
any domain name you'll actually see or use in your interaction with your
email client. It's _purely_ a lookup key for your trust database or
blacklist, and could be considered just that -- an arbitrary key. Mail
forwarded by pobox.com uses 'bounce2.pobox.com' for example.

Incidentally, once we realise that it's _just_ a single hop we're
validating, and that the key is essentially arbitrary, the whole problem
we're trying to solve can be seen in a different light.

All we're doing is offering a way for sending mail hosts to offer an
arbitrary 'trust key' of their choice, which the recipient can then look
up in a database. 

As long as there's a way of checking that the host in question really is
an authorised user of that 'trust key' it doesn't matter _what_ it is. 

For example, we could use signatures on TLS certificates. A listing in
the database could be keyed on the _signatures_ on the cert which is
presented by the sending mail host -- so the database lookup could
result in a trust level (or a blacklist/whitelist entry) for some
certificate related to 'infradead.org' and any mail server which
presents a cert signed by that certificate would be treated accordingly.

A trust scheme based on TLS certificates (or perhaps some other database
key which is easy to verify) could be introduced without all the
breakage at forwarding sites which would be caused by the proposed
'mailfrom' or 'pra' scopes of SenderID, and would achieve much the same
benefit, since SenderID does _not_ offer end-to-end validation but only
per-hop trust assessments of individual hosts.

> the message but Submitter information is added when message is being sent out.

Tony already pointed that out, and it is true -- so it seems a new
header does make more sense for purely practical reasons.

> I completely agree. I'll be submitting proposal later this week on it.

OK, good.

-- 
dwmw2

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 13 Sep 2004 15:29:43 +0200

Tony Finch writes:
>>  more precise it standartized by RFC2821 in section 4.4 but many 
>>  systems still don't do it in correct way).
>

Further, the widespread noncompliance isn't a problem in this context: 
MTA authors who want to support sender-id can just as easily teach 
their software generate received fields as they can add a new field.

Arnt

---

From: Douglas Otis <[email-masked]>
Date: Mon, 13 Sep 2004 16:49:47 -0700

> legitimate mail servers, and which are not.

Neither Sender-ID nor SPF safely allow the implementation of a name
based blacklist.  I would agree there may be a means of excluding
messages where the SMTP client IP address was not authorized, but
nothing else of significance is possible from this association.  A
positive association does not indicate the mailbox domain holder to be
the originator.  A negative association does not indicate the mailbox
domain holder to be a spammer.

-Doug

---

From: [email-masked] (Claus Färber)
Date: 14 Sep 2004 14:36:00 +0200

David Woodhouse <[email-masked]> schrieb/wrote:
> arbitrary 'trust key' of their choice, which the recipient can then
> look up in a database.

This observation raises an important question: Why should building a
database based on a 'trust key' reduce spam?

The most obvious 'trust key' is the IP address of the sending MTA.
Databases based on the IP address have already been built and are in
production use. They do reduce spam but there is still a lot of spam
that gets through.

If Sender ID/SPF just changes the key from 'IP address' to something
else, then I don't see how it can reduce spam UNLESS spammers can't
change that key as easily as IP addresses. With domain-based keys, this
is not the case: It's actually easier to register a throw-away domain
name than to get a new block of IP addresses.

Well, IP address-based databases have some problems on their own, which
may or may not have hindered deployment (e.g. the server behind a DSL
`dialup'' line). But SPF has similar problems.

Claus
-- 
http://www.faerber.muc.de

---

From: David Woodhouse <[email-masked]>
Date: Wed, 15 Sep 2004 18:30:50 +0100

> `dialup'' line). But SPF has similar problems.

All this is true, but people still seem to rabidly want SPF/SenderID. 

Partly I think this stems from a lack of realisation that in the
presence of SRS or the Resent-From:^WForwarded-For: header addition, SPF
and SenderID are still only capable of providing a way of determining
who claims to be responsible for the individual mail server in question,
rather than being a true end-to-end validation that the mail in question
really did come from the user from whom it claims to come.

But partly it does make some sense. Databases based on IP addresses are
not entirely perfect, and some improvement could be obtained by using an
alternative 'trust key' to identify the party responsible for the server
in question. Not only can the problem of false positives and false
negatives due to dynamic IP be avoided, but by using a key which covers
_groups_ of servers, such as a domain name or the signer of a TLS
certificate, we get to reduce the size of the database dramatically.

What does _not_ make sense is to allow people to think that the use of a
'domain name' as the key actually gives any secure indication that the
mail really did come from the domain it claims to come from. There are
other groups working on such end-to-end authentication, and that is an
orthogonal issue. Or at least it _should_ be; but some people seem
confused.

-- 
dwmw2