Subject: [dmarc-ietf] DMARC and ARC

From: Douglas Foster <[email-masked]>
Date: Fri, 4 Dec 2020 22:58:30 -0500

Michael Thomas has asked some questions which have stimulated my thinking
about how to describe the big-picture questions here.

I suggest that too much attention has been focused on solving the “mailing
list” problem, and too little on the “message receiver” problem.
Forwarded email creates big problems for message receivers.

First, lets begin with the obvious:   malicious messages come from
enterprises that are in the malicious message business.   They rarely send
just one message, and their content changes continually.   Therefore, my
priority is to block malicious sources.   Messages that are correctly
blocked on content, rather than source, are the canary-in-the-mine which
warns me that my sender blocks need to be tightened.

I currently use 5 identifiers to apply a reputation, and therefore a
disposition, to a message source:    Source IP, HELO name, ReverseDNS name,
SMTP Address, and From Address.   These identifiers are combined with the
available verification methods:   Forward-confirming the two DNS names to
the source IP, SPF PASS, and DMARC-compliant.    On any specific message,
some of this information is not required to assign a reputation and
disposition, but across a day’s messages, each of these identifiers and
attributes are needed at some point.

If a message is not forwarded, every organization involved in its delivery
is assumed to have a relationship to the sender and therefore a shared
responsibility for the final product.   DMARC, SPF, and many spam filters
assume that the adjacent MTA is the only source that needs to be evaluated.

Forwarding introduces an intermediary organization which presumably
operates on behalf of the recipient, rather than the sender.   It is not
involved in the creation of the message and has no economic relationship
with most of the message sources.   More importantly, because it will be
forwarding messages from sources with a variety of reputations, the
forwarder will be perceived as having a very unreliable reputation –
sending both very much unwanted content and very much wanted content from
the same or overlapping identifiers.   SPF and DMARC force the forwarder to
reliably identify itself, but in this process, they force the forwarder to
hide information that the receiving MTA needs for proper message
filtering.  This aggravates any effort to filter based on original-source
identity.

Consequently, both the recipient MTA and the forwarding MTA need for the
recipient MTA to evaluate a message based on the true source, the entity
(or entities) prior to the forwarder.   Unfortunately, we have no reliable
way to know if a message has been forwarded, although some guessing
strategies are available.   Even when forwarding is presumed,
reconstructing the 5-identifier / 4 verification dataset is nearly
impossible due to data loss.

Any attempt to evaluate prior sources requires evaluating the entire
Received chain, and the supporting data, including ARC, which goes with it.
ARC begins to give us a way to trust parts of the Received chain, and this
moves us toward the possibility of evaluating the Received chain
effectively.   But it does not meet my needs, because it does not ensure
that I can use any specific ARC Set to reconstruct the 5 identifier / 4
verifications dataset needed to evaluate the source correctly.

Additionally, evaluating the Received chain is a huge increase in
complexity, for both the code to implement an algorithm and the run-time
processing needed to execute the algorithm.    Processing power and coding
sophistication keeps getting better, so perhaps the main problem is getting
the data set right.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 6 Dec 2020 21:15:32 -0800

> warns me that my sender blocks need to be tightened.

I was under the impression from my work in the anti-spam world that sources
also change.  It's trivial to come from a new IP address or sign with a new
domain name when I think you're blocking me.  Thus, negative reputations
are generally not useful to accumulate long term.  On the contrary, the
thing that's mostly reliable is static sources that have good reputations,
because they tend to remain (mostly) static, and they work to preserve
their reputations.  I tend to give them preferential treatment.

> assume that the adjacent MTA is the only source that needs to be evaluated.
This seems overly general.  A message going from A to B to C to me here at
Gmail means Gmail has a relationship with A?

> filtering.  This aggravates any effort to filter based on original-source
I'm also confused here, because it's ambiguous what you mean by Forwarder.
Some forwarders simply replace the envelope and send the message on its way
with no body modifications and only trace header field changes.  They don't
meet the definition you're describing because no details are hidden.
Others, like MLMs, may mutate the message and re-post it, but I would argue
that's not forwarding, that's a new message; the list is the originator.

-MSK

---

From: Dotzero <[email-masked]>
Date: Mon, 7 Dec 2020 06:41:00 -0500

On Mon, Dec 7, 2020 at 12:15 AM Murray S. Kucherawy <[email-masked]>
wrote:

> I was under the impression from my work in the anti-spam world that
> sources also change.  It's trivial to come from a new IP address or sign
> with a new domain name when I think you're blocking me.  Thus, negative
> reputations are generally not useful to accumulate long term.  On the
> contrary, the thing that's mostly reliable is static sources that have good
> reputations, because they tend to remain (mostly) static, and they work to
> preserve their reputations.  I tend to give them preferential treatment.
>
> I'm also confused here, because it's ambiguous what you mean by
> Forwarder.  Some forwarders simply replace the envelope and send the
> message on its way with no body modifications and only trace header field
> changes.  They don't meet the definition you're describing because no
> details are hidden.  Others, like MLMs, may mutate the message and re-post
> it, but I would argue that's not forwarding, that's a new message; the list
> is the originator.
>

If there were truly a consensus on the list being the originator then the
>From would/should be from the list domain and a lot of the list as
intermediary issues would go away. As past discussions have shown, there is
not such a consensus.

Michael Hammer

---

From: Douglas Foster <[email-masked]>
Date: Mon, 7 Dec 2020 07:37:58 -0500

Thanks for engaging Murray.

On the usefulness of source blacklisting:

I have been using the model I described for about a year.    I have two
categories of mailers that account for nearly all of my spam:
 spamer-owned infrastructure which sends direct, and email service
providers that accept all customers, including criminals.   ESP messages
will arrive SPF-aligned with the ESP domain, the From address indicates the
client, and DKIM signing is used when necessary for DMARC compliance.

I don't think I have seen any problems with infected servers or even
infected accounts from trusted sources, which is surprising.   Compliments
to GMail that the worst that I get from them is unwanted-but-persistent
advertising.

Our employee base is measured in 4 digits, not 6, so our volume is smaller
than many, and my desired-sender list is correspondingly small.   That has
meant that I have not had false positives with blacklisting.

When a confirmed spam is received from spam infrastructure, the best
defense is to blacklist the IP, the host domain, and the address domains
(unless one of these has been spoofed.)    In practice, this is tedious to
do consistently.   Blocking on server domain has proved pretty effective.
 When blocking on IP, we often block a subnet.   Depending how quickly we
identify the spam source, the message history sometimes provides a good
outline of the boundaries of the spam infrastructure.   Bottom line:
 spammer domains do change frequently, and spammers often have a bunch of
servers.  But their servers do not actually change very much.   Obviously,
this means I have not yet been attacked by a thousand-member bot-net.

For spam received from ESPs, blocking on the From address works, because
the ESP does verify the client identity and does not allow clients to spoof
each other.   They just don't vet client integrity.    So I classify
messages arriving from the ESP into known-bad=block, known-good=allow, and
unknown=quarantine.   Forwarded messages will typically rewrite the SMTP
domain for SPF compliance.   This means that I no longer know that the
unclassified From address is from the untrusted ESP and therefore needs to
be quarantined.    DMARC is a great solution for authorizing ESPs to send
direct messages on behalf of a domain, because SPF did not work well for
those messages.   It is not a perfect solution for forwarded mail because
even unmodified messages are subjected to data loss during forwarding.

On Accountability:

A message from A to B to C to Gmail does not mean that Gmail has a
relationship with A.    Gmail is acting as your agent, and it does a pretty
good job of protecting you.    A to B to C do have a relationship with each
other, or one of them would have been bypassed.    For legitimate senders,
I expect that the organizational hops will be at most two:    the source
domain, and the outbound email filtering service used by the source domain.

Mailing lists should be the cleanest traffic on the internet, because it
should be a small matter for the list to verify sources and minimize risk
by rejecting even slightly-doubtful messages.    Yet we have had exchanges
about how mailing list do not want to sign messages because they think
it will make them accountable for what they forward.   They are accountable
for what they forward, with or without DKIM.

Auto-forwarding operations are much more problematic because they have
strong incentives to minimize false positives.   They are still
accountable, but they will be perceived by recipient as unreliable - the
same way that I handle my problem ESPs.

New vs Existing messages:

For these purposes, mailing list output is only a new message if the
mailing list is the only party responsible for its content.  I am focused
on identifying malicious and unwanted message sources, and the mailing list
is asserting that it is not the originating source.

DF

--