Subject: Re: [dmarc-ietf] A policy for direct mail flows only, was ARC questions

From: "Douglas E. Foster" <[email-masked]>
Date: Wed, 25 Nov 2020 07:57:38 -0500

Indirect mail flows are difficult to detect.   SMTP address rewrite is already common practice for forwarding.More to the point, John's interest is finding ways to increase the trust level for forwarded mail, while your idea says that direct mail is more trusted than indirect maill, which is the problem he is trying to overcome.We need to be able to evaluate indirect mail based on both the submitter MTA. and the originator MTA.   ARC gets us started in that direction.   I think more filtering data is needed and am working on a proposal to that effect.Doug<div>
</div><div>
</div><!-- originalMessage --><div>-------- Original message --------</div><div>From: Alessandro Vesely <[email-masked]> </div><div>Date: 11/25/20  6:28 AM  (GMT-05:00) </div><div>To: dmarc-ietf <[email-masked]> </div><div>Subject: [dmarc-ietf] A policy for direct mail flows only, was ARC questions </div><div>
</div>On Mon 23/Nov/2020 22:27:41 +0100 John Levine wrote:
> ARC deals with the problem that most list software forwards everything
> with a subscriber's address on the From: line and does a lousy job of
> spam filtering. The question is if the entity sending the message to
> the list was who it purported to be. 
> 
> For example, if a message from a list fails DMARC alignment, but ARC
> says it was aligned on the way in, it's likely a real message from a
> subscriber. If it was unaligned on the way in, it's likely spam.

I publish p=none in order to avoid spurious rejections due to casual message 
modifications that happen in transit.  However, I'm quite confident that SPF or 
DKIM verify, since users submit messages through the right mail server.

Couldn't I address direct flows only?  Doing so would prevent a casual spammer 
from abusing mailing lists I'm subscribed to by simply faking From:.

A direct flow is one were SPF credentials (helo name or return address) are 
aligned with From:.  That includes some simple forwarding, but not mailing list 
traffic.  Direct policy could be expressed as dp=.  Authenticate as usual, 
either SPF or DKIM.  On failure, discard only if direct flow.  For example:
   v=DMARC1; p=none; dp=reject;

Makes sense?

Best
Ale
-- 

_______________________________________________
dmarc mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dmarc

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 25 Nov 2020 18:30:22 +0100

Hi,

On 25/11/2020 13:57, Douglas E. Foster wrote:
> Indirect mail flows are difficult to detect.   SMTP address rewrite is already 
> common practice for forwarding.

Return address rewriting is a Good Thing™, unlike From: rewriting.  I'd welcome 
forwarding my email, even if modified (I'm not a bank).  At the same time, I'd 
stop spam pretending to be coming directly from my server.  SPF -all yields 
such possibility, except that it sometimes stops internal forwarding at 
uncoordinated admds.  My proposed dp=reject would ameliorate spf-all while 
still being more permissive than p=reject.  A midway policy, which tackles the 
"mailing list problem" from the other side.

> More to the point, John's interest is finding ways to increase the trust level 
> for forwarded mail, while your idea says that direct mail is more trusted than 
> indirect mail, which is the problem he is trying to overcome.

Yes, they are two different problems.

> We need to be able to evaluate indirect mail based on both the submitter MTA. 
> and the originator MTA.   ARC gets us started in that direction.   I think more 
> filtering data is needed and am working on a proposal to that effect.

Without resorting to ARC, it is still possible to validate author domain's 
signatures directly if the MLM just adds a subject tag and a footer, like, for 
example, this list does.   While ARC solves "deep" forwarding problems, which 
may arise in the context of email address portability, MLM transformation 
reversion solves the simpler mailing list problem, including reverting munged 
From:'s.

Best
Ale
--

---

From: Jesse Thompson <[email-masked]>
Date: Wed, 25 Nov 2020 12:24:02 -0600

On 11/25/20 11:30 AM, Alessandro Vesely wrote:
> Without resorting to ARC, it is still possible to validate author domain's signatures directly if the MLM just adds a subject tag and a footer, like, for example, this list does.   While ARC solves "deep" forwarding problems, which may arise in the context of email address portability, MLM transformation reversion solves the simpler mailing list problem, including reverting munged From:'s.

I agree that ARC isn't really needed to do this (trust the last hop from the MLM and determine the original authenticity from the MLM's perspective), although I think that ARC somewhat standardizes what headers/value to look for (I may be wrong).  Plus, if it eventually solves the "deep" forwarding issue, then ARC is certainly better than trying to follow received header chains, etc.

Anecdotally, after much debate, our team is leaning more towards *not* reverting munged From:'s from our own MLM

1. Until ARC has a reputation model that is commonly adopted, header munging isn't going to subside.  I still find MLM operators who are just now realizing that they have to munge messages.  We need to tell users that this is the new, growing, reality.

2. If we only unmunge for our own domains' users' authoring messages to our own MLM, it has limited overall effect, and it distorts the user-reality story from point #1.  We would have to unmunge for all domains' authors sending to all "trusted" MLMs in order to give the users what they expect from their prior reality.

3. Since we can only unmunge for our own recipients, it just creates an inconsistent experience on top of the already inconsistent experience of the conditional munging most MLMs do based on the authors' DMARC policies.  

We would rather see:

3a) MLMs that munge for all messages regardless of the author's domain's DMARC policy 

3b) MLMs that allow operators to configure recipient destinations that they know are going to trust non-munged messages (via ARC or whatever)

Jesse

P.S. If anyone knows how to trick Google Groups into doing 3a and/or 3b, please ping me (off list is OK)

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 25 Nov 2020 20:11:26 +0100

Hi,

On 25/11/2020 19:24, Jesse Thompson wrote:
> I agree that ARC isn't really needed to do this (trust the last hop from the MLM and determine the original authenticity from the MLM's perspective)

I didn't mean to trust the MLM.  I meant remove the subject tag and the footer, 
then the original DKIM signature verifies.  See:
https://datatracker.ietf.org/doc/draft-vesely-dmarc-mlm-transform/

> Plus, if it eventually solves the "deep" forwarding issue, then ARC is certainly better than trying to follow received header chains, etc.

IMHO, that's where the real value or ARC lies.  Large mailbox providers forward 
lots of messages to one another, as set up by users, and they seem to prefer to 
forward messages anyway rather than filter before forwarding.  That's what John 
reported in:
https://mailarchive.ietf.org/arch/msg/dmarc/OmTzwzP9GuE1oF5m1TvUZVA799c

> 1. Until ARC has a reputation model that is commonly adopted, header munging isn't going to subside.  I still find MLM operators who are just now realizing that they have to munge messages.  We nee…

Yup.

> 3. Since we can only unmunge for our own recipients, it just creates an inconsistent experience on top of the already inconsistent experience of the conditional munging most MLMs do based on the au…

If the original signature verifies, each MDA can restore the unmunged From: 
right before committing to local storage.  That way, the rewritten From: 
becomes a transfer artifact, not seen by users.

Best
Ale
--

---

From: Michael Thomas <[email-masked]>
Date: Wed, 25 Nov 2020 11:16:34 -0800

On 11/25/20 11:11 AM, Alessandro Vesely wrote:
>>> Without resorting to ARC, it is still possible to validate author 
>>> domain's signatures directly if the MLM just adds a subject tag and 
>>> a footer, like, for example, this list does.   While ARC solves 
>>> "deep" forwarding problems, which may arise in the context of email 
>>> address portability, MLM transformation reversion solves the simpler 
>>> mailing list problem, including reverting munged From:'s.
>>
>> I agree that ARC isn't really needed to do this (trust the last hop 
>> from the MLM and determine the original authenticity from the MLM's 
>> perspective)
>
>
> I didn't mean to trust the MLM.  I meant remove the subject tag and 
> the footer, then the original DKIM signature verifies.  See:

When I was at Cisco, with l= and some subject line heuristics I could 
get probably like 90+% verification rate across the entire company, a 
company that uses external mailing lists a lot. Definitely not 100% though.

Mike

---

From: "John Levine" <[email-masked]>
Date: 25 Nov 2020 20:31:38 -0000

In article <[email-masked]>,
>company that uses external mailing lists a lot. Definitely not 100% though.

I think you will find that at very large mail systems like gmail and
Microsoft and Yahoo, 90% might as well be 0%. The volume of errors is
just too high and the number of complaints would be impossible.

While I almost never see the sort of spam leakage through mailing lists 
that Brandon reports, I believe him when he says it's enough of a problem
that Gmail can't just whitelist traffic from mailing lists.

R's,
John
-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Michael Thomas <[email-masked]>
Date: Wed, 25 Nov 2020 12:37:35 -0800

> that Gmail can't just whitelist traffic from mailing lists.

Not everything is service provider. We were investigating this from an 
enterprise standpoint.

And if you can't trust mailing traffic from providers what is the point 
of ARC?

Mike

---

From: "John Levine" <[email-masked]>
Date: 25 Nov 2020 15:55:52 -0500

>And if you can't trust mailing traffic from providers what is the point 

Um, please see the previous umpteen messages describing how ARC works
and how it addresses this particular problem.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 26 Nov 2020 09:59:00 +0100

On 25/11/2020 20:16, Michael Thomas wrote:
>>>> signatures directly if the MLM just adds a subject tag and a footer, like, 
>>>> for example, this list does.   While ARC solves "deep" forwarding problems, 
>>>> which may arise in the context of email address portability, MLM 
>>>> transformation reversion solves the simpler mailing list problem, including 
>>>> reverting munged From:'s.
>>>
>>> I agree that ARC isn't really needed to do this (trust the last hop from the 
>>> MLM and determine the original authenticity from the MLM's perspective)
>>
>> I didn't mean to trust the MLM.  I meant remove the subject tag and the 
>> footer, then the original DKIM signature verifies.  See:
> When I was at Cisco, with l= and some subject line heuristics I could get 
> probably like 90+% verification rate across the entire company, a company that 
> uses external mailing lists a lot. Definitely not 100% though.

DKIM itself is not 100%.  You always have lines beginning with "From " or 
occasional autoconversions.

l= doesn't cover multipart/alternative nor Content-Transfer-Encoding: base64. 
In addition, the DKIM spec discourages its usage and suggests that "Assessors 
might wish to ignore signatures that use the tag."

Best
Ale
--

---

From: Brandon Long <[email-masked]>
Date: Mon, 30 Nov 2020 20:56:46 -0800

On Thu, Nov 26, 2020 at 12:59 AM Alessandro Vesely <[email-masked]> wrote:

> domain's
> from the
> get

Right, some of the other dkim-light or diff concepts we discussed would be
better than using l=

We again got hung up on the 100% solution, though... something that handled
subject-prefix and
footer in a transport agnostic way might have worked.  The fact that DKIM
isn't transport agnostic
is an achilles heel to even that, though, since we'd have to come up with a
new canonicalization
and get it to widespread adoption before the simple diff could work.  Or
require mailing lists to
be a lot more strict in how they do their email rewriting, but I imagine
that's harder work than
even ARC.

Brandon