Subject: Re: pros and cons of RMX (Re: [Asrg] Declaration to the world) (fwd)

From: [email-masked]
Date: Thu, 6 Mar 2003 20:52:45 -0800 (PST)

(Sorry last email got set before I finished...)

Ok, you're right - ietf are good with technology and set FROM in the mail 
transmission to mailing list (which actually causes some spam filters to 
think email is forged), but there are mail lists and forwarding systems 
that do not do this and set FROM to the original sender. 

My point is that the spammer would just need to use different FROM address 
during email delivery and different "From" in the header and majority of 
those sending anti-spam reports would use "From" in the header for their
report so they would still think somebody is forging email from your domain.

So to deal with all that I end up having to setup filter to block email 
that has different transmission from and header from, which happens for 
all mailing lists setup to properly forward emails from you, and I have to 
whitelist these mailing lists anyway to accept email and those that do not 
incert mailing list into from and keep yours, I'd have whitelist as well. 
Well, this works fine for private system, but for ISP it becomes harder 
to keep up with all its users' mailing lists.

On 7 Mar 2003, wayne wrote:

> In <[email-masked]> [email-masked] writes:
> 
> > How is this going to work for [email-masked] email?
> > 
> > Lets say I run this code to check your dnsbl. Now you send email to asrg, 
> > responding to this post, my MTA receives it and goes to your domain 
> > specific dnsbl, it sees that you support it and checks ip address of the 
> > ietf mail server and that ip is not in your dnsbl, so your email is rejected.
> 
> Ah, the email that you are replying to didn't come from my domain, it
> came from ietf.org and and a From_ address of [email-masked].  If you
> had implemented the check, you would have found that ietf.org doesn't
> have a domain specific blacklist, and therefore you should accept
> email from [email-masked].
> 
> My message to the ASRG list was received by my machine just fine.
> 
> 
> > (Damn, And I really wanted to read your reply! :)
> 
> Yeah, sure you did.  ;->
> 
> 
> -wayne
> 
> 
> _______________________________________________
> Asrg mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/asrg

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: wayne <[email-masked]>
Date: 07 Mar 2003 01:07:51 -0600

In <[email-masked]> [email-masked] writes:

> [ ...]

You raise several good points about the problems that authenticating
email claiming from being from a certain domain and coming from a
certain IP address causes.  I will have to think about them a little
bit, but it is too late tonight to do it right now.

I'm not really sure if this kind of thing is that useful.  However,
most of the reasons that have been given so far against RMX or domain
specific DNSBLs seem bogus to me.  (Such as the claims that it would
be hard to do, or being DNS based means it can't be done, or that I
should not be allowed to ask people to not accept email claiming to be
from me that aren't from my machines.)

This *IS* an idea that can be easily set up and tested and that seems
to me to be something that a research group should be interested in.

-wayne

---

From: "Gary Feldman" <[email-masked]>
Date: Fri, 7 Mar 2003 12:11:59 -0500

> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of [email-masked]
> Sent: Thursday, March 06, 2003 11:53 PM
... 
> that do not do this and set FROM to the original sender. 

I think this is backwards.  The mail headers I see from this
mailing list keep From: set to the original sender, and add
a Sender: field identifying asrg-admin as the true sender.
MS Outlook 2002, for what it's worth, reasonably interprets
these fields to combine them with the "on behalf of" 
verbiage illustrated above.

Gary

---

From: [email-masked]
Date: Fri, 7 Mar 2003 07:59:08 -0800 (PST)

That is because you do not see in the header FROM as was set during mail 
transmission (when one mail server connects to another before initiating 
data transmission it sends out FROM (sender) and RCPT (recepient) commands.

Some email servers will keep this around and consider spam any email where 
during transmission FROM is not equivalent to "From" header. Such software 
is hotmail for example. Because many mailing lists (quite appropriatly) 
set FROM during transmission to their own, these get classified as SPAM 
and mailing lists have to be whitelisted. If RMX or RTDNSBL are implemented, 
thse mail filters (checking FROM in transmission and "From" in header) 
have to be implemented as much as RTDNSBL because otherwise spammers would 
just begin to set absolutly different FROM during transmission to domain 
that does not have RMX and still use their preferred forgery domain in "From"
header which will make you think email came from that domain and send spam 
report to them. Now as I mention those mailing lists that actually set 
FROM during transmission to actual sender (taking it from "From" header)
are all also have to be whitelisted bacause otherwise RMX software would 
think email is being forged because sender mailing list is not authorized
to send email for this domain. In the end no matter what we do ALL mailing 
lists have to be whitelisted for RMX or RTDNSBL.

This matter has been discussed before and everybody agreed that all of 
the various RMX proposals break mailinglists and that they would have to 
be whitelisted. Some pointed out that those using hotmail already 
whitelist mailing lists on regular basis and do it on per-user level - but 
that is because email is classified by HOTMAIL as either spam or not and 
their email servers do not answer with 500 but just based either send 
email to regular folder or spam folder for each user, allowing filters to 
be implimented on per-user level. Current DNSBL and RMX proposals actually
do send 500 answer during email transmission, which means whitelisting of 
mailing lists have to be done on per-mailserver basis and this is a 
problem for ISPs.

I hope I explained this all correctly and fully so everybody can understand,
this has been discussed before and no solution has been given yet that would
not allow for spammer to also use the same solution to get around the filters.
In my opinion until somebody tells me how to deal with mailing list problem 
RMX should not be implemented, if we do have a solution, then sure - RMX 
or RTDNSBL can be done - it'll not stop spam but just limit forgeries to 
different domains, but nothing bad would happen from it then and if there 
is some good, then why not do it.

---

From: wayne <[email-masked]>
Date: 07 Mar 2003 11:51:55 -0600

> a Sender: field identifying asrg-admin as the true sender.

There is a difference between the From_ header and the From: header.

As William points out, this raises the point of "which headers should
be authenticated?"

Because I cut and pasted code, my 20 line patch to SpamAssassin
currently checks the From_ header and the Reply-to: header.  I'm not
sure if that is the right thing to do.  The Sender: header should
certainly be considered, but as others have pointed out, you will get
false positives.

One thing I like about SpamAssassin is that a false positive on one
test will not automatically cause a false positive as a result.  So,
now that I've implemented a domain specific DNSBL for midwestcs.com,
if someone sends me email forged to say that it came from
midwestcs.com, it will likely still make it into my inbox.  Unless, of
course, there are several other indications that it is spam.

Personally, the biggest problem I've had with "people" forging my
domain name is from KLEZ worms and such.  I've had several people
nastily accuse me of sending them the worm, just because the worm
forged my name.

-wayne

---

From: "Gary Feldman" <[email-masked]>
Date: Fri, 7 Mar 2003 13:02:06 -0500

> From: [email-masked] [mailto:[email-masked]] 
> Sent: Friday, March 07, 2003 10:59 AM

> transmission (when one mail server connects to another before 

Actually, it's more because I'm used to seeing it labelled the
"MAIL FROM:" command, and not simply FROM (to more clearly
disambiguate it from the FROM header).  But I suppose different
subgroups fall into different habits in terminology.

Gary

---

From: "Gary Feldman" <[email-masked]>
Date: Fri, 7 Mar 2003 13:10:46 -0500

> Behalf Of wayne
> Sent: Friday, March 07, 2003 12:52 PM
 
> There is a difference between the From_ header and the From: header.

As suggested in my previous email, I find the use of the term "From_ 
header" confusing.  I have a very simple view of the world: headers are
defined in RFC 2822 and related RFCs, and refers to the stuff at the
top of the message data (as message data is defined in 2821).  Commands
are 
defined in RFC 2821 and related RFCs.  The use of "header" to describe
stuff sent in the message envelope triggers my "imprecise language"
alarm. 

I also have no idea where the underscore originates.  Or perhaps are you

talking about something entirely separate from these two RFC sets?

I suppose it's possible my personal choice of terminology is 
non-standard, and I should adapt.  

Gary

---

From: Keith Moore <[email-masked]>
Date: Fri, 7 Mar 2003 13:57:52 -0500

> during transmission FROM is not equivalent to "From" header.

Those email servers are broken, plain and simple.  It's entirely reasonable
for MAIL FROM to be different than From.  This says nothing at all about
the legitimacy of the mail.  Stupid measures like this are causing as much
harm as the spam itself.

I'm all for having ways of making senders more accountable.  But tying
it to the From header, or to the MAIL FROM field, is just too broken to
consider.

Keith

---

From: Keith Moore <[email-masked]>
Date: Fri, 7 Mar 2003 14:13:12 -0500

> As William points out, this raises the point of "which headers should

None of the existing header fields contains an identifier which is suitable
as a principal name.  From: is, by design, settable to one or more addresses,
none of which need to match that of the person who sends the mail.  Reply-to
doesn't have to have anything to do with who sends it - it's perfectly
reasonable to ask that replies be sent to a secretary, a list, whatever. 
The SMTP MAIL FROM field is where (non)delivery reports are sent - again,
not the same as the message sender.  Sender is the closest match, but it is so
widely misused as to be useless.

If you want messages to contain some sort of sender identity that works, you
need a new header field for that purpose.  And it's easier to implement
that way anyway.

Keith

---

From: Vernon Schryver <[email-masked]>
Date: Fri, 7 Mar 2003 13:19:37 -0700 (MST)

> From: "Gary Feldman" <[email-masked]>

> stuff sent in the message envelope triggers my "imprecise language"

Such alarms are valuable because imprecise language often indicates
imprecise or simply wrong thinking.

> talking about something entirely separate from these two RFC sets?

I can't say what the other person meant, but:

 - "_" has historically been used to indicate a single ASCII blank.

 - "From_" tends to refer to the non-RFC 822 header historically and
    optionally added to the start of messages by various mail systems.
    This header tends (or tended) to contain the RFC 821 sender or 
    SMTP Mail_From value and the data.  When UUCP is used, it tends to
    contain the UUCP path, except when rmail has been hacked to recognize
    when the last host is an FQDN and use that instead.
 
    I think the sendmail "Ops" document calls this the "UNIX From" header.
    Look for options controlling it in the "mailer" rules or control
    lines in sendmail.cf.

Vernon Schryver    [email-masked]
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: wayne <[email-masked]>
Date: 07 Mar 2003 16:12:02 -0600

In <[email-masked]> Vernon Schryver <[email-masked]> writes:

> > header" confusing.  
>  [ correct explaination deleted ]

Thanks for giving Gary the explanation Vernon, I'm sorry for any
confusion my archaic terminology caused and for exposing my UUCP
roots.  I'll try to use more modern (and accurate) terminology in the
future.  For what is worth, there are still thousands of references to
"from_ header" out there on the net.

The point remains that domain specific DNSBLs are incredibly easy to
implement and to provide some measure of authentication of where mail
claims to be from.  I still have doubts about whether this
authentication will reduce the spam problem.  I have much stronger
doubts about whether adding a new DNS RMX record will be worth the
effort.

For what it is worth, Spamassassin already does some of these kinds of
checks.  There appears to be some better spots to do this kind of
checking in spamassassin than my original 20 line patch used.  I may
try hacking on SA a little more to see if I can get a good, general
patch that others can test.

The nice thing about spamassassin is that no single check will cause a
false positive.  You can check all sorts of headers via a DSDNSBL and
as long as there is some correspondence between "unauthorized" use of
a domain and spam, it can be a useful check.

-wayne