Subject: [dmarc-ietf] ARC questions

From: Michael Thomas <[email-masked]>
Date: Sat, 21 Nov 2020 13:32:54 -0800

Hi all, long time.

I finally read through the ARC spec after seeing it accidentally in mail 
headers wondering what it was, especially since it was so DKIM like. My 
barely informed take is that it allows intermediaries to say "this is 
what it looked like to me at this point [and before i messed it]". So 
far, so good. It seems that a receiver can then verify that the ARC 
signature especially if the "original" DKIM signature is broken. So far, 
so good again.

If I'm a receiver who is going to be making some filtering decisions 
based on ARC, I see that it passed by some authenticator along the way 
which is fine, but my question is why I should trust that intermediary 
in general? I mean, this is easy if it's gmail since I know google has 
an interest in good email practices out of band, but what if the ARC 
signer is actually an attacker that I have no idea who they are?

Which is to say, how do I go about trusting the ARC signer to not be 
doing something bad? I don't have a specific attack in mind (still too 
new to this), but say if spam.com ARC signs a message it adulters to its 
advantage how do I know that I should disregard its ARC results? Or 
maybe not so much disregard results per se, but not want to "accept" the 
changes to the original message?

Ok, maybe here is an attack. Suppose this message is scrapped by a 
spammer since this is a public email list. It has a broken original DKIM 
signature but a valid ARC signature from ietf.org. The attacker takes 
the message, adds the Viagra scams in the body to the ARC signed message 
and reinjects the new message toward the targets of their choice (? 
mailing list members only? not sure).

Or did I miss where ARC resigns the body? Or is there a tie in for ARC 
with the mailing list's resigned DKIM signature for the new message?

Sorry so many questions, and probably misunderstanding what's going on.

Mike

---

From: "John Levine" <[email-masked]>
Date: 21 Nov 2020 21:14:17 -0500

In article <[email-masked]> you write:
>in general?

The short answer is that you shouldn't, any more than you should trust
random DKIM signatures.

When people were designing ARC, it seemd overcomplicated to me. Large
mail systems know where all the mailing lists are so why not just
whitelist them and be done with it? The answer is that legit lists
leak a lot of spam and it is common for a formerly well-behaved list
to start spewing spam. Most lists do little filtering beyond verifying
that the From: address is a subscriber, so when a spambot steals an
address book that contains both the list address and some subscribers
to that list, a lot of spam leaks through.

ARC lets recipient systems do retroactive filtering that the
forwarding system didn't. For example, although the overall error rate
of rejecting mail due to SPF -all or DMARC p=reject can be high, on
incoming mail to mailing lists both are quite reliable since the kind
of forwarding that breaks them is rare in that context. If I ever get
around to adding ARC checks to my filters, that's the sort of thing
I'll be looking for.

This also means that ARC isn't useful if you don't have a reputation
system to tell you where the lists and other forwarders that might add
legit ARC signatures are. There's been some handwaving about how we
might come up with shared DNSWLs of mailing list hosts, but it hasn't
happened yet.

R's,
John

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Sun, 22 Nov 2020 11:41:56 -0700

As usual, John has pretty well nailed the response, but there was one other
part of your question (Mike) that I thought deserved explanation:

On Sat, Nov 21, 2020 at 7:14 PM John Levine <[email-masked]> wrote:

> legit ARC signatures are.
>

> with the mailing list's resigned DKIM signature for the new message?

The ARC-Message-Signature (referred to as the AMS) includes a signature
over the newly modified message (headers & body) in a way very similar to a
DKIM-Signature. But this does not solve the problem of a malicious
forwarder that does a wholesale replacement of the (presumably) good
content with spam. That's were your own reputation and content analysis has
to come in.

--Kurt

---

From: Michael Thomas <[email-masked]>
Date: Sun, 22 Nov 2020 10:50:55 -0800

On 11/22/20 10:41 AM, Kurt Andersen (b) wrote:
> As usual, John has pretty well nailed the response, but there was one 
> other part of your question (Mike) that I thought deserved explanation:
>
> On Sat, Nov 21, 2020 at 7:14 PM John Levine <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     In article <[email-masked]
>     <mailto:[email-masked]>> you write:
> On Sat, Nov 21, 2020 at 2:33 PM Michael Thomas <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
> The ARC-Message-Signature (referred to as the AMS) includes a 
> signature over the newly modified message (headers & body) in a way 
> very similar to a DKIM-Signature. But this does not solve the problem 
> of a malicious forwarder that does a wholesale replacement of the 
> (presumably) good content with spam. That's were your own reputation 
> and content analysis has to come in.
>
Is there a reason that there is a separate ARC-signature rather than 
just using the DKIM signature that is normally created for the new 
message? Since ARC is new, you'd not want the intermediary to stop DKIM 
signing the message so you end up with essentially two signatures doing 
essentially the same thing?

Mike

---

From: "John R Levine" <[email-masked]>
Date: 22 Nov 2020 14:14:04 -0500

> Is there a reason that there is a separate ARC-signature rather than just 
> using the DKIM signature that is normally created for the new message? Since 
> ARC is new, you'd not want the intermediary to stop DKIM signing the message 
> so you end up with essentially two signatures doing essentially the same 
> thing?

The ARC signature has a sequence number so you can track the chain of 
custody.  You are right that it is similar to the DKIM signature but the 
extra ovehead doesn't seem excessive.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Michael Thomas <[email-masked]>
Date: Sun, 22 Nov 2020 11:29:26 -0800

On 11/22/20 11:14 AM, John R Levine wrote:
>> message? Since ARC is new, you'd not want the intermediary to stop 
>> DKIM signing the message so you end up with essentially two 
>> signatures doing essentially the same thing?
>
> custody.  You are right that it is similar to the DKIM signature but 
> the extra ovehead doesn't seem excessive.
>
Did the wg consider just grafting that onto the DKIM signature itself 
instead of having essentially a duplicate signature? Receivers are 
already supposed to ignore any tags they don't understand so it 
shouldn't hurt backward compatibility.

Mike

---

From: "John R Levine" <[email-masked]>
Date: 22 Nov 2020 14:56:23 -0500

On Sun, 22 Nov 2020, Michael Thomas wrote:
> Did the wg consider just grafting that onto the DKIM signature itself instead 
> of having essentially a duplicate signature? Receivers are already supposed 
> to ignore any tags they don't understand so it shouldn't hurt backward 
> compatibility.

ARC is an experiment that came from the people who designed DMARC.  It's 
not a WG product.

Having adapted the perl DKIM module to handle ARC signing and 
verification, I can say that the extra signature is not a big deal.  If 
you look at mail coming from large mail systems, they're full of other 
junk headers and the extra overhead of AMS along with DKIM is not 
important.

Regards,

---

From: Joseph Brennan <[email-masked]>
Date: Mon, 23 Nov 2020 09:50:03 -0500

>> legit ARC signatures are.
And if you know which hosts are legit mailing lists or forwarders, you
already know what ARC would tell you.

-- 
Joseph Brennan
Lead, Email and Systems Applications
Columbia University Information Technology

---

From: Todd Herr <[email-masked]>
Date: Mon, 23 Nov 2020 10:38:48 -0500

> already know what ARC would tell you.
I believe, though, that the intent of ARC is that it be scalable in ways
that manual enumeration of known legit mailing lists and forwarders is not.

Standard authentication protocols (SPF, DKIM, DMARC) are fine for direct
mail flow. When the message arrives at its destination, the receiver can
check authentication protocols and apply handling rules for a message based
on its past history with the authenticated identities associated with that
message.

For indirect mail flows, messages might not pass such checks at their final
destination, but it's possible that they did pass those checks on an
intermediate hop. As Michael said in his original post,  ARC 'allows
intermediaries to say "this is
what it looked like to me at this point [and before i messed it]".'

The question with ARC then becomes, does the final destination trust the
results reported in the ARC header set(s), and it's kind of a complicated
question. If I as the final destination trust the ARC header set(s), I'm
saying that I believe the reported results with no real way to
independently verify the results, meaning that I can't do direct DKIM
validation or SPF checking using standard tools; if I could, there'd be no
need for ARC.

I suppose that an approach to building up an ARC reputation might be one
where over time a receiving site can compare indirect mail flow that is
purported to have X as an authenticated identifier with mail that comes
direct to the receiving site with X as an authenticated identifier. That
is, if direct mail with X as an authenticated identifier generally hits Y
on my internal reputation scale, does indirect mail that is ARC-signed by Z
to have X as an authenticated identifier also generally hit Y on my
internal reputation scale? If so, great; I can trust Z as an ARC signer.

The devil is in the details, though. If mail through Z doesn't generally
hit Y, then what explains the variance? If it's for most or all values of
X, then it's probably because Z is a bad actor. If it's hitting Y for most
values of X but not for all, is it because those particular mail streams
are ones that are still legitimate from an authentication perspective, but
aren't of the same quality due to reasons, or is Z engaging in some
elaborate scheme using valid mail as legit shields to get its nasty payload
through in the noise? If variances can be dealt with in a way that doesn't
require too much manual intervention, then perhaps there's hope here. Even
at that, such a system would need to be bootstrapped with some list of
manually enumerated known good intermediaries, but over time it could
theoretically grow organically based on data collected.

I don't believe the above challenges are insurmountable (says the guy who
hasn't had operational responsibility for a mail system in over a decade)
but I'm not in a position to assign the necessary resources to solve this
problem at each site that has an interest in doing so.

-- 

*Todd Herr* | Sr. Technical Program Manager
*e:* [email-masked]
*p:* 703.220.4153

This email and all data transmitted with it contains confidential and/or
proprietary information intended solely for the use of individual(s)
authorized to receive it. If you are not an intended and authorized
recipient you are hereby notified of any use, disclosure, copying or
distribution of the information included in this transmission is prohibited
and may be unlawful. Please immediately notify the sender by replying to
this email and then delete it from your system.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 23 Nov 2020 09:02:28 -0800

On 11/23/2020 7:38 AM, Todd Herr wrote:
> On Mon, Nov 23, 2020 at 9:50 AM Joseph Brennan <[email-masked] 
> <mailto:[email-masked]>> wrote:
>     And if you know which hosts are legit mailing lists or forwarders,
>     you already know what ARC would tell you.
>
>
> I believe, though, that the intent of ARC is that it be scalable in 
> ways that manual enumeration of known legit mailing lists and 
> forwarders is not.

"if you know which hosts are legit" buries an assumption that is 
problematic, namely that you know who handled the message.  The fack 
that a message purports to be handled by a mailing list you trust does 
not mean it actually was.

That's the issue that ARC resolves.

ARC (and DKIM) produce noise-free uses of identifiers.  If the 
authentication validates, the receiver knows is really was handled by 
who is saying it was handled by.  Without these, you don't.

d/

-- 

Dave Crocker
[email-masked]
408.329.0791

Volunteer, Silicon Valley Chapter
American Red Cross
[email-masked]

---

From: "Doug Foster" <[email-masked]>
Date: Mon, 23 Nov 2020 12:15:26 -0500

My wishlist for ARC:

ARC tells me that somebody changed some data, but it does not tell me which MTA performed the forwarding operation, added content, or performed address rewriting.  If we could get HELO names into the ARC data, then those names could be correlated with the Received header chain to make better filtering decisions.

DF   

From: dmarc [mailto:[email-masked]] On Behalf Of Dave Crocker
Sent: Monday, November 23, 2020 12:02 PM
To: Todd Herr; Joseph Brennan
Cc: [email-masked]
Subject: Re: [dmarc-ietf] ARC questions

And if you know which hosts are legit mailing lists or forwarders, you already know what ARC would tell you.

I believe, though, that the intent of ARC is that it be scalable in ways that manual enumeration of known legit mailing lists and forwarders is not. 

"if you know which hosts are legit" buries an assumption that is problematic, namely that you know who handled the message.  The fack that a message purports to be handled by a mailing list you trust does not mean it actually was.

ARC (and DKIM) produce noise-free uses of identifiers.  If the authentication validates, the receiver knows is really was handled by who is saying it was handled by.  Without these, you don't.

d/

-- 

Dave Crocker
[email-masked]
408.329.0791

---

From: Dave Crocker <[email-masked]>
Date: Mon, 23 Nov 2020 09:23:40 -0800

On 11/23/2020 9:15 AM, Doug Foster wrote:
> ARC tells me that somebody changed some data, but it does not tell me 
> which MTA performed the forwarding operation, added content, or 
> performed address rewriting.  If we could get HELO names into the ARC 
> data, then those names could be correlated with the Received header 
> chain to make better filtering decisions.

That's not ARC.  Feel free to recruit others to support a new 
initiative, for a different protocol.

d/

-- 
Dave Crocker
[email-masked]
408.329.0791

---

From: Todd Herr <[email-masked]>
Date: Mon, 23 Nov 2020 13:34:28 -0500

On Mon, Nov 23, 2020 at 12:02 PM Dave Crocker <[email-masked]> wrote:

> problematic, namely that you know who handled the message.  The fack that a
> message purports to be handled by a mailing list you trust does not mean it
> actually was.
>
> authentication validates, the receiver knows is really was handled by who
> is saying it was handled by.  Without these, you don't.
>
>
> Yes, but knowing it really was handled by who is saying it was handled by
isn't the entirety of the problem.

I can know from ARC headers that X handled the message and what email
authentication checks X purports to have done when handling the message and
what results X claims to have obtained. What I have to decide in that case
is "do I trust X to record correct and valid results?" because the answer
to that question will impact my disposition of the message when it reaches
me.

It's obviously not the place of the ARC protocol spec to proscribe how
trust in ARC results can be determined, but without some system in place
for assigning trust levels to ARC Sealers, ARC has limited utility for
sites that serve as the terminal destination for a message.

--

---

From: Dave Crocker <[email-masked]>
Date: Mon, 23 Nov 2020 10:40:23 -0800

On 11/23/2020 10:34 AM, Todd Herr wrote:
> Yes, but knowing it really was handled by who is saying it was handled 
> by isn't the entirety of the problem.

Of course.  But it helps (quite a lot) to be clear about what this 
specific mechanism does do.

d/

-- 
Dave Crocker
[email-masked]
408.329.0791

---

From: Michael Thomas <[email-masked]>
Date: Mon, 23 Nov 2020 10:49:31 -0800

On 11/22/20 11:56 AM, John R Levine wrote:
>>> The ARC signature has a sequence number so you can track the chain 
>>> of custody.  You are right that it is similar to the DKIM signature 
>>> but the extra ovehead doesn't seem excessive.
>>>
> ARC is an experiment that came from the people who designed DMARC.  
> It's not a WG product.
>
> verification, I can say that the extra signature is not a big deal.  
> If you look at mail coming from large mail systems, they're full of 
> other junk headers and the extra overhead of AMS along with DKIM is 
> not important.
>
 From what I can tell, the main thing that ARC is doing is binding an 
auth-res to a dkim signature-like thing. But as I recall -- it's been a 
long time -- there were ordering requirements ala received headers for 
where new dkim-signatures and auth-res go in the header. Assuming my 
memory is correct, that means you can reconstruct "what this looked like 
before i messed with it" already by signing the incoming auth-res as 
part of the new DKIM signature.

Is there something more going on here?

Mike

---

From: "John R Levine" <[email-masked]>
Date: 23 Nov 2020 14:28:18 -0500

> From what I can tell, the main thing that ARC is doing is binding an auth-res 
> to a dkim signature-like thing. But as I recall -- it's been a long time -- 
> there were ordering requirements ala received headers for where new 
> dkim-signatures and auth-res go in the header. Assuming my memory is correct, 
> that means you can reconstruct "what this looked like before i messed with 
> it" already by signing the incoming auth-res as part of the new DKIM 
> signature.
>

Not really.  There are ordering rules but mail systems do not follow them 
reliably, DKIM signatures in practice are not ordered.  Also, A-R can be 
deleted in some situations, so ARC makes copies of them to be more robust 
in transit.

Regards,

---

From: Brandon Long <[email-masked]>
Date: Mon, 23 Nov 2020 11:29:32 -0800

> shouldn't hurt backward compatibility.

The DKIM-Signature is an "ownership" thing, it's a message originator that
is saying
"associate this message to me".

Intermediaries don't want to take ownership of the message in that sense,
though there
are some mailing lists that do.

So, if Google added a DKIM signature as google.com to all of the messages
going through
it, that would not be good.  Even if the new tag indicated "I don't own
this", it doesn't change
the existing receivers who haven't been updated.

Brandon

---

From: Dave Crocker <[email-masked]>
Date: Mon, 23 Nov 2020 11:33:53 -0800

On 11/23/2020 11:29 AM, Brandon Long wrote:
> The DKIM-Signature is an "ownership" thing, it's a message originator 
> that is saying

That is not DKIM's semantics:

    "DomainKeys Identified Mail (DKIM) permits a person, role, or
    organization to claim some responsibility for a message by
    associating a domain name"

This says nothing about whether the organization has anything to do with 
origination.

There is nothing to prohibit or preclude handling agents other than the 
originator from signing.

> 
> Intermediaries don't want to take ownership of the message in that 
> sense, though there

Signing with DKIM does not take 'ownership'.

d/
-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Brandon Long <[email-masked]>
Date: Mon, 23 Nov 2020 11:34:25 -0800

On Mon, Nov 23, 2020 at 11:28 AM John R Levine <[email-masked]> wrote:

> auth-res

Right, A-R is explicitly called out to be deleted in the RFC on inbound if
the admd
matches (so someone can't fake an A-R to you).  Even if it's not fake, a
message
can go through multiple hops through the same admd and each hop have a
different
A-R result, but there is no mechanism for multiple A-Rs for the same admd
having
any ordering or such.

>From the other direction, one could say that ARC is a superset of A-R and
DKIM with
different purpose, and you might be able to subsume them into ARC, but you
couldn't
build ARC out of the originals.

Brandon

---

From: Brandon Long <[email-masked]>
Date: Mon, 23 Nov 2020 11:42:19 -0800

> There is nothing to prohibit or preclude handling agents other than the

Yes, of course, a handling agent can do it, but there are plenty of reasons
why they shouldn't.

> Signing with DKIM does not take 'ownership'.

Yes, responsibility is the proper word.  My point survives the word change.

DKIM says the domain takes responsibility for the message, while ARC says
the domain takes responsibility for evaluating the status of the message
when
they received and forwarded it.

Brandon

---

From: Michael Thomas <[email-masked]>
Date: Mon, 23 Nov 2020 11:45:56 -0800

On 11/23/20 11:28 AM, John R Levine wrote:
>> auth-res to a dkim signature-like thing. But as I recall -- it's been 
>> a long time -- there were ordering requirements ala received headers 
>> for where new dkim-signatures and auth-res go in the header. Assuming 
>> my memory is correct, that means you can reconstruct "what this 
>> looked like before i messed with it" already by signing the incoming 
>> auth-res as part of the new DKIM signature.
>>
> Not really.  There are ordering rules but mail systems do not follow 
> them reliably, DKIM signatures in practice are not ordered.  Also, A-R 
> can be deleted in some situations, so ARC makes copies of them to be 
> more robust in transit.
>
If auth-res is sometimes deleted, why wouldn't we expect the arc 
auth-res to not be deleted too?

I imagine that the vast majority of intermediaries that break signatures 
number exactly one extra domain, so it's not very hard to reconstruct 
the chain of custody from origin to destination. Assuming the 
intermediary resigns with the incoming auth-res, the destination can 
choose to believe that auth-res or not, right? Since this is an 
experiment, do we have an idea of what the rest of the problem is after 
the typical mailing list-like signature breakers are excluded?

Mike

---

From: Michael Thomas <[email-masked]>
Date: Mon, 23 Nov 2020 11:48:11 -0800

On 11/23/20 11:42 AM, Brandon Long wrote:
> the domain takes responsibility for evaluating the status of the 
> message when
But it can already do that be re-signing its auth-res.

Mike

---

From: Brandon Long <[email-masked]>
Date: Mon, 23 Nov 2020 11:49:46 -0800

> the typical mailing list-like signature breakers are excluded?

No, as in the RFC says to remove them, so it's a standard part of
implementation.

RFC 7601 4.1:

> instances of the header field that appear to originate within the ADMD but

are actually added by foreign MTAs will be removed before delivery.

That's very different than "just maybe it might be removed"

Brandon

---

From: Dave Crocker <[email-masked]>
Date: Mon, 23 Nov 2020 11:53:12 -0800

On 11/23/2020 11:42 AM, Brandon Long wrote:
> 
> 
> On Mon, Nov 23, 2020 at 11:34 AM Dave Crocker <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 

Please enumerate and explain.  If it's that dangerous, we should 
document it, especially I don't recall that constraint being in any of 
the design or standardization discussions.

> Yes, responsibility is the proper word.  My point survives the word change.

I disagree.

> they received and forwarded it.

This implies that the word 'some' is irrelevant.  It isn't.  And it was 
included intentionally.

d/
-- 
Dave Crocker
[email-masked]
408.329.0791

---

From: Michael Thomas <[email-masked]>
Date: Mon, 23 Nov 2020 11:55:24 -0800

> That's very different than "just maybe it might be removed"
The receiving MTA in the next domain doesn't have to discard the 
information before removing it. The act of removing it is so there isn't 
confusion about the ultimate auth-res, especially with MUA's. The 
incoming MTA is free to consider the previous auth-res just like it's 
free to consider the previous arc auth-res.

Mike

---

From: Michael Thomas <[email-masked]>
Date: Mon, 23 Nov 2020 11:58:12 -0800

On 11/23/20 11:34 AM, Brandon Long wrote:
>
> From the other direction, one could say that ARC is a superset of A-R 
> and DKIM with
> different purpose, and you might be able to subsume them into ARC, but 
> you couldn't

It's seems to me that the superset involves explicitly binding an 
auth-res to a dkim-like signature. Is there more beyond that?

Mike

---

From: Brandon Long <[email-masked]>
Date: Mon, 23 Nov 2020 11:59:02 -0800

On Mon, Nov 23, 2020 at 11:55 AM Michael Thomas <[email-masked]> wrote:

> confusion about the ultimate auth-res, especially with MUA's. The incoming
> MTA is free to consider the previous auth-res just like it's free to
> consider the previous arc auth-res.
>
Which works for one hop.

Brandon

---

From: Brandon Long <[email-masked]>
Date: Mon, 23 Nov 2020 12:01:11 -0800

On Mon, Nov 23, 2020 at 11:58 AM Michael Thomas <[email-masked]> wrote:

> It's seems to me that the superset involves explicitly binding an auth-res
> to a dkim-like signature. Is there more beyond that?
>
Right, explicitly binding auth-res to a dkim-like signature that survives
through multiple hops.

The dkim signature acts like an AMS i=0, and the auth-res is like the AAR
max(i)

Brandon

---

From: "John R Levine" <[email-masked]>
Date: 23 Nov 2020 15:09:25 -0500

> If auth-res is sometimes deleted, why wouldn't we expect the arc auth-res to 
> not be deleted too?

Please see RFC 7001, section 5.

> Since this is an experiment, do we have an idea 
> of what the rest of the problem is after the typical mailing list-like 
> signature breakers are excluded?

Sorry, this question makes no sense. The point of ARC is to deal with
the kind of breakage that mailing lists cause.

Regards,

---

From: Brandon Long <[email-masked]>
Date: Mon, 23 Nov 2020 12:15:15 -0800

> the design or standardization discussions.

DKIM often ties a domain to reputation and other anti-spam features.  If you
forward spam to another host and sign it while forwarding, then the other
host
will think you send spam.

DMARC ties DKIM to the From header and at least is interpreted as being an
anti-phishing feature.  DKIM signing mail that you forward could mean
upgrading
a phishing message to passing DMARC.

This recent article also goes into things that DKIM signatures imply:
https://blog.cryptographyengineering.com/2020/11/16/ok-google-please-publish-your-dkim-secret-keys/

Perhaps this all means that DKIM has been used for more than it was
intended for.

> This implies that the word 'some' is irrelevant.  It isn't.  And it was

Automated systems can't really tell how much responsibility an intermediary
was
intending to take for the message.  OTOH, they typically are using it only
for a certain
purpose, so they assume that the intermediary took responsibility in the
sense that they
want... or rather, the people who wrote the code do.  Or the journalist
writing the story.

ARC was specified to have a more specific responsibility, and as different
from DKIM so
that it wasn't mistaken for the uses that people were already using DKIM
for.

Brandon

---

From: Michael Thomas <[email-masked]>
Date: Mon, 23 Nov 2020 12:18:12 -0800

On 11/23/20 12:09 PM, John R Levine wrote:
>
>> Since this is an experiment, do we have an idea of what the rest of 
>> the problem is after the typical mailing list-like signature breakers 
>> are excluded?
>

1) A mailing list creates an auth-res on the incoming mail to the list

2) It modified the message

3) It resigns the message with DKIM

4) It is then delivered to the subscriber's mail server

5) The destination mail server can look at the incoming message 
including the mailing list's auth-res and decide whether to trust it or 
not just like ARC.

It seems to me this covers the vast majority of cases. What are the 
other cases where this is not sufficient and how significant are they in 
reality?

Mike

---

From: "John R Levine" <[email-masked]>
Date: 23 Nov 2020 15:29:51 -0500

> 5) The destination mail server can look at the incoming message including the 
> mailing list's auth-res and decide whether to trust it or not just like ARC.
>
> It seems to me this covers the vast majority of cases. What are the other 
> cases where this is not sufficient and how significant are they in reality?

Two or more levels of forward are quite common, particularly in large mail 
systems.  Look at mail coming out of Google and Microsoft's hosted mail 
and you'll see a lot of ARC headers.

Considering that the ARC RFC was published over a year ago, and it is 
implemented all over the place, could you explain what the point of this 
discussion is?  The people who designed ARC are not idiots.  If we could 
have fixed the mailing list problem with existing DKIM signatures, we 
would have.

Regards,

---

From: Michael Thomas <[email-masked]>
Date: Mon, 23 Nov 2020 12:45:48 -0800

On 11/23/20 12:29 PM, John R Levine wrote:
>> including the mailing list's auth-res and decide whether to trust it 
>> or not just like ARC.
>>
>> other cases where this is not sufficient and how significant are they 
>> in reality?
>
> Two or more levels of forward are quite common, particularly in large 
> mail systems.  Look at mail coming out of Google and Microsoft's 
> hosted mail and you'll see a lot of ARC headers.
>
> implemented all over the place, could you explain what the point of 
> this discussion is?  The people who designed ARC are not idiots.  If 
> we could have fixed the mailing list problem with existing DKIM 
> signatures, we would have.
>
Then why is it not standards track? And am I to understand that I'm not 
allowed to comment on an experiment? Perhaps the working group chairs or 
AD can clarify that.

In any case, if this all boils down to whether I trust the intermediary 
who resigned the message, then that is a previously solved problem: you 
can base the reputation check based on the resigned signature. I'm not 
entirely sure what the value of  the previous auth-res is. If I recall 
correctly, the document was sort of short on what the specific utility 
is, but I may have missed it.

Mike

---

From: Dave Crocker <[email-masked]>
Date: Mon, 23 Nov 2020 12:48:44 -0800

On 11/23/2020 12:15 PM, Brandon Long wrote:
> On Mon, Nov 23, 2020 at 11:53 AM Dave Crocker <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
> forward spam to another host and sign it while forwarding, then the 
> other host

Well, ummm... errrr... yes.  That's because, in such circumstances, you do.

More significantly, the signature makes sure that such as an assessment 
will only be made accurately, rather than penalizing you for problematic 
mail that is attributed to you but that you did not handle.

> a phishing message to passing DMARC.

I don't understand.  The first sentence makes sense to me, but the 
second doesn't.

"Upgrading...to passing DMARC only applies if a) the signature matches 
the From: field domain, and b) that domain has an associated DMARC 
record.  But if you don't watch DMARC to apply in that case, what is the 
DMARC record there fore?

> <https://blog.cryptographyengineering.com/2020/11/16/ok-google-please-publish-your-dkim-secret-keys/>

The level of condescension, ignorance, and error throughout that article 
is impressive.  Given that it was written by someone whose profession 
requires extreme care about complex matters, the level of carelessness 
in the article is especially unfortunate.

Conveniently, he put his biggest error in bold font:

      "*DKIM provides a life-long guarantee of email authenticity that 
anyone can use to cryptographically verify the authenticity of stolen 
emails, even years after they were sent."*

DKIM does no such thing.

and this was quickly followed by the normal-font:

    "The key design goal of DKIM was to prevent spammers from forging 
emails /while in transit/. "

This, too, is not what DKIM does.  DKIM provides a noise-free channel 
for assessing signers, not for detecting spammers.  The difference is 
important; and I claim fundamental.

ps. making sure that DKIM signature become invalid  relatively soon -- I 
think that removing the keys is simpler and just as effective as 
publishing the private keys -- seems like a reasonable suggestion.

**

> Perhaps this all means that DKIM has been used for more than it was 

"More than" suggests that the use has legitimacy.  It doesn't.

>     >      > Intermediaries don't want to take ownership of the
>     message in that
>     >      > sense, though there
>     >      > are some mailing lists that do.
>     >
> Automated systems can't really tell how much responsibility an 
> intermediary was
> intending to take for the message.

People who write them can.

> OTOH, they typically are using it only for a certain
> purpose, so they assume that the intermediary took responsibility in 
> the sense that they
> want... or rather, the people who wrote the code do.  Or the 
> journalist writing the story.
>
> ARC was specified to have a more specific responsibility,

Forgive me but I think that:

>     Authenticated Received Chain (ARC) creates a mechanism for individual
>     Internet Mail Handlers to add their authentication assessment to a
>     message's ordered set of handling results.

specifies a nature and responsibility pretty much identical to what DKIM 
claims.  The enhancements are a) chaining, and b) carriage of earlier 
assessments.  But in terms of 'responsibility', this reads the same as DKIM.

> and as different from DKIM so
> that it wasn't mistaken for the uses that people were already using 
> DKIM for.
Oh?

d/

-- 
Dave Crocker
[email-masked]
408.329.0791

---

From: Michael Thomas <[email-masked]>
Date: Mon, 23 Nov 2020 12:54:35 -0800

> Perhaps this all means that DKIM has been used for more than it was 

It is a quirk that we didn't consider at the time. You can't count on 
that property because providers can change their selectors at any time. 
That said, there is an awful lot of hand wringing for not much gain. 
It's not like you need cryptographic non-repudiation to be pretty sure 
something wasn't forged. That and non-repudiation has its benefits as well.

Mike

---

From: "John Levine" <[email-masked]>
Date: 23 Nov 2020 21:27:41 -0000

In article <[email-masked]>,
Dave Crocker  <[email-masked]> wrote:
>problematic, namely that you know who handled the message.  The fact

Pretty close, but not quite.

You know that a message came from a mailing list because you have your
list of IPs or DKIM signatures of lists you trust.

ARC deals with the problem that most list software forwards everything
with a subscriber's address on the From: line and does a lousy job of
spam filtering. The question is if the entity sending the message to
the list was who it purported to be. 

For example, if a message from a list fails DMARC alignment, but ARC
says it was aligned on the way in, it's likely a real message from a
subscriber. If it was unaligned on the way in, it's likely wpam.

R's,
John
-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

---

From: "John Levine" <[email-masked]>
Date: 23 Nov 2020 16:53:39 -0500

In article <[email-masked]> you write:
>direct to the receiving site with X as an authenticated identifier. ...

I think it's more likely to be ad-hoc, does this source send mail that
looks like mailing lists and the recipients don't flag or complain
about it.

R's,
John

>this email and then delete it from your system.

Too late.

---

From: Michael Thomas <[email-masked]>
Date: Mon, 23 Nov 2020 14:04:24 -0800

On 11/23/20 12:48 PM, Dave Crocker wrote:
> The level of condescension, ignorance, and error throughout that 
> article is impressive.  Given that it was written by someone whose 
> profession requires extreme care about complex matters, the level of 
> carelessness in the article is especially unfortunate.
>

Yeah, that was pretty bad. "DKIM can be used to verify a piece of mail 
due to operator practices, but there are absolutely no guarantee that a 
signature will verify in the future due to those same practices."

>
> ps. making sure that DKIM signature become invalid  relatively soon -- 
> I think that removing the keys is simpler and just as effective as 
Stephen Farrell is threatening to write an ID on the subject of 
publishing private keys. Frankly the stakeholders -- providers and users 
-- are not very well aligned on when where and why a provider would do 
such a thing. And writing an ID to say how to invalidate key when just 
unpublishing old selectors when you rotate keys is an easy second best 
shows that inertia is the actual issue, not the technical shortcoming.

Mike

---

From: Dave Crocker <[email-masked]>
Date: Mon, 23 Nov 2020 14:30:30 -0800

> list of IPs or DKIM signatures of lists you trust.

Except that was not stated or, really, even implied in the text of the 
message I was replying to.  Rather, something like that seemed to be 
taken as an assumption, but without any clear foundation.

For these kinds of discussions, which are mostly about understanding 
these capabilities clearly, accurately, and precisely, the core 
requirement is to separate the essential bits of information and the 
basis for knowing each bit.

> spam filtering.

Forgive me but I believe misbehavior by mailing lists has never been 
meaningfully documented for this work.  Quite the contrary.

List mail has been collateral damage, not because lists have misbehaved 
but because they got caught by a spontaneous change in the email service 
by some providers.

d/

-- 
Dave Crocker
[email-masked]
408.329.0791

---

From: "John R Levine" <[email-masked]>
Date: 23 Nov 2020 17:42:23 -0500

> message I was replying to.  Rather, something like that seemed to be taken as 
> an assumption, but without any clear foundation.

Well, OK, but it's still true.

> meaningfully documented for this work.  Quite the contrary.

I believe that Brandon has specifically said that Gmail sees this problem 
and that is why whitelisting mail from mailing lists isn't adequate.

Regards,

---

From: Dave Crocker <[email-masked]>
Date: Mon, 23 Nov 2020 14:51:59 -0800

On 11/23/2020 2:42 PM, John R Levine wrote:
> I believe that Brandon has specifically said that Gmail sees this 
> problem and that is why whitelisting mail from mailing lists isn't 
> adequate. 

And that constitute "meaningfully document[ing]"?

My point wasn't that it isn't a valid concern, but that it was not 

> with a subscriber's address on the From: line and does a lousy job of
offers a characterization of mailing lists that hasn't been documented.

And, again, when ARC work was pursued, I don't recall anyone claiming 
that mailing lists were (significant) sources of misbehavior.

d/

-- 
Dave Crocker
[email-masked]
408.329.0791

---

From: "John R Levine" <[email-masked]>
Date: 23 Nov 2020 17:58:37 -0500

> And that constitute "meaningfully document[ing]"?

Works for me.  I doubt I was the only person wondering why it needed all 
that mechanism when whitelisting mailing lists got you most of the way 
there, particularly since you need that whitelist anyway to know when to 
start ARC analysis.

> And, again, when ARC work was pursued, I don't recall anyone claiming that 
> mailing lists were (significant) sources of misbehavior.

Well, OK.  Please feel free to provide footnoted documentation of what 
the actual motivation for ARC was if you believe it was something else.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Mon, 23 Nov 2020 15:00:47 -0800

> the actual motivation for ARC was if you believe it was something else.

Typically, the burden of substantiating a claim falls on the person 
making the affirmative claim.

d/

-- 
Dave Crocker
[email-masked]
408.329.0791

---

From: Michael Thomas <[email-masked]>
Date: Mon, 23 Nov 2020 15:51:12 -0800

On 11/23/20 3:00 PM, Dave Crocker wrote:
>>> And, again, when ARC work was pursued, I don't recall anyone 
>>> claiming that mailing lists were (significant) sources of misbehavior.
>> Well, OK.  Please feel free to provide footnoted documentation of 
>> what the actual motivation for ARC was if you believe it was 
>> something else.
>
>
What I'm struggling to understand is what having authenticated auth-res 
from a previous hop helps. this is what i found:

"With this information, Internet Mail Handlers MAY inform local policy 
decisions regarding disposition of messages that experience 
authentication failure due to intermediate processing."

that and:

"When an Authenticated Received Chain is used to determine message 
disposition, the DMARC processor can communicate this local policy 
decision to Domain Owners as described in Section 7.2.2."

seems to be the only motivation I can find. without ARC, a receiver 
could always check the new DKIM signature from, say, the mailing list 
and look up its reputation to decide whether to pass it along or not 
overriding the originating domain's policy. my recollection was that 
this was the "you break it, you own it" policy which i recall being the 
consensus. and indeed, there is nothing to stop a filter to look at the 
mailing list's auth-res and take it into account even if it's not part 
of the headers in the signature. maybe there is some attack there i'm 
not seeing off the top of my head, but it seems like this really hinges 
on reputation as was pointed out to me earlier.

It would be kind of nice to understand what gap ARC actually plugs and 
why it's important if you ask me. Also: there seem to be a lot of ways 
to achieve this, but this one is probably the most complicated one that 
I can envision.

Mike

---

From: Brandon Long <[email-masked]>
Date: Mon, 23 Nov 2020 16:13:31 -0800

> mail that is attributed to you but that you did not handle.
If the result is marking all of the mail from that mailing list as spam,
then you've likely done your users a disservice.  Being able to
differentiate is useful.  Also, forwarders often don't have all of
the signals that the user's mailbox does.. not the least of which is that
different recipients have different judgements on what is spam, and the
"this is spam" signal rarely makes it back to the forwarder.

> I don't understand.  The first sentence makes sense to me, but the second
> doesn't.
>
> "Upgrading...to passing DMARC only applies if a) the signature matches the
> From: field domain, and b) that domain has an associated DMARC record.  But
> if you don't watch DMARC to apply in that case, what is the DMARC record
> there fore?
>
I send a phishing message to a mailing list or alias at a domain with a
>From header of that domain, and the list blindly
re-signs all mail sent to the list, I've now "authenticated" the spoofed
message, and it will now "pass" DMARC.  Perhaps it
upgraded from a quarantine to none, since the mailing list doesn't have the
concept of a spam folder, or perhaps the sales@ team
has decided they want all of the forwarded messages, even if probably spam,
so that they can go through them to make sure...
but it lost the quarantine disposition on the forward when it gained
authentication.

This recent article also goes into things that DKIM signatures imply:
> requires extreme care about complex matters, the level of carelessness in
> the article is especially unfortunate.
>
> emails *while in transit*. "
>
> This, too, is not what DKIM does.  DKIM provides a noise-free channel for
> assessing signers, not for detecting spammers.  The difference is
> think that removing the keys is simpler and just as effective as publishing
> the private keys -- seems like a reasonable suggestion.
>
>
We don't always have control over how our work is used.  If I proposed
extending a standard in a new direction that would
be perfectly fine with the original intent of the standard, and that
clashed with how the standard had come to be used in
practice, my extension is DOA.

> assessments.  But in terms of 'responsibility', this reads the same as DKIM.

I don't see how "claim some responsibility" is the same as "add their
authentication assessment".  I guess they are claiming
responsibility for the assessment, but that's a very specific thing, and
not the "some [unknown] responsibility"

> that it wasn't mistaken for the uses that people were already using DKIM

This was definitely a topic of discussion during the initial meetings where
we went from XOAR to ARC.

Brandon

---

From: Dave Crocker <[email-masked]>
Date: Mon, 23 Nov 2020 17:09:43 -0800

On 11/23/2020 4:13 PM, Brandon Long wrote:
>
>
> On Mon, Nov 23, 2020 at 12:48 PM Dave Crocker <[email-masked] 
>>     forward spam to another host and sign it while forwarding, then
>>     the other host
>     More significantly, the signature makes sure that such as an
>     assessment will only be made accurately, rather than penalizing
>     you for problematic mail that is attributed to you but that you
>     did not handle.
>
> If the result is marking all of the mail from that mailing list as 
> spam, then you've likely done your users a disservice.

Why would you put forward a hypothetical that might reasonably be 
characterized as unreasonable, especially given that you also make clear 
why it is unreasonable? (*)

> Being able to differentiate is useful.  Also, forwarders often don't 
> have all of
> the signals that the user's mailbox does.. not the least of which is 
> that different recipients have different judgements on what is spam, 
> and the "this is spam" signal rarely makes it back to the forwarder.

Except that mailing lists are also recipients, notably including likely 
history from authors, and possibly more history than a final recipient?

There are, of course, possible signals a final recipient might have 
about an author, that the mailing list won't have. Equally there might 
be others the list has that the final recipient doesn't, such as knowing 
about other mail from the author, to other lists the mailing list system 
operates...

>>     anti-phishing feature.  DKIM signing mail that you forward could
>>     mean upgrading
>     "Upgrading...to passing DMARC only applies if a) the signature
>     matches the From: field domain, and b) that domain has an
>     associated DMARC record.  But if you don't watch DMARC to apply in
>     that case, what is the DMARC record there fore?
>
> I send a phishing message to a mailing list or alias at a domain with 
> a From header of that domain, and the list blindly
> re-signs all mail sent to the list, I've now "authenticated" the 
> spoofed message, and it will now "pass" DMARC.

There are so many different ways this represents really poor mailing 
list setup, operation and possibly design, I again wonder at your 
offering it as an example of any point relevant to this exchange.

It's not that what you suggest hasn't happened, it's that the fact that 
it represents multiple problems also suggests it can -- and probably 
should -- have multiple solutions.

> Perhaps it
> upgraded from a quarantine to none, since the mailing list doesn't 
> have the concept of a spam folder, or perhaps the sales@ team
> has decided they want all of the forwarded messages, even if probably 
> spam, so that they can go through them to make sure...

More problematic hypotheticals, all of which arguably represent poor 
services, just as originators can be poorly run services.

>>     Perhaps this all means that DKIM has been used for more than it
>>     was intended for.
>
> We don't always have control over how our work is used.

No, but we do have control over a) how we write about it, b) how we talk 
about it, and c) what we do about misuses of the work.

You appear to be taking the view that however others choose to interpret 
a specification is what the specification is for and how it operates.  
Except that that is only one -- and I'd argue highly problematic -- 
approach to misuses of a specification.

> If I proposed extending a standard in a new direction that would

Possibly.  But not automatically.

For reference, note that 90+% of email is spam.  Does that mean that a 
proposal to counter that (inappropriate) use of email is DOA?  That 
seems to be the logic you've applied.

>     specifies a nature and responsibility pretty much identical to
>     what DKIM claims.  The enhancements are a) chaining, and b)
>     carriage of earlier assessments.  But in terms of
>     'responsibility', this reads the same as DKIM.
>
>
> responsibility for the assessment, but that's a very specific thing, 
> and not the "some [unknown] responsibility"

I cited assessment as a difference.  And yes, that's the only difference 
in responsibility.  Otherwise, it's just an authentication of having 
processed the message.

>>     that it wasn't mistaken for the uses that people were already
>>     using DKIM for.
>     Oh?
>
>
> This was definitely a topic of discussion during the initial meetings 
> where we went from XOAR to ARC.

Sorry, I don't recall that.

d/

(*) https://en.wikipedia.org/wiki/Straw_man

-- 
Dave Crocker
[email-masked]
408.329.0791

---

From: "John Levine" <[email-masked]>
Date: 23 Nov 2020 21:04:53 -0500

>from a previous hop helps. this is what i found:

See some of the previous messages. My usual example is a mailing list
message that fails DMARC at the final recipient but passed DMARC (as
recorded in AAR) when it arrived at the list. This lets the final
recipient distinguish between real messages from subscribers and mail
from spambots that happened to scrape both the list address and some
subscribers' address and sends mail to one pretending to be from the
other. (That definitely happens, I've seen it on lists I'm on.)

I agree that the ARC document does not do a great job of explaining that.

>to achieve this, but this one is probably the most complicated one that 

If you want to pass the A-R results through multiple rounds of
forwarding, you can't do much less.

R's,
John

---

From: Michael Thomas <[email-masked]>
Date: Tue, 24 Nov 2020 14:49:16 -0800

> forwarding, you can't do much less.

Sorry, changing the auth-res to old-auth-res and dkim signing the 
message would be completely sufficient, and far easier to understand 
with a lot less bloat. All of this hand wringing about dozens of message 
manglers in the path before it get to the destination and not be able to 
figure out which auth-res was which seems wildly out of proportion for 
what the normal case is: 1 message mangler in the path before it arrives 
at the receiver's domain. Just like this message right here. That's why 
I asked how common that was, which was dismissed, but not answered.

Mike

---

From: Michael Thomas <[email-masked]>
Date: Tue, 24 Nov 2020 14:57:42 -0800

You'd be wrong. The only ad hominem was yours from yesterday and it was 
I think where *you* dismissed the very question I raised:

"Two or more levels of forward are quite common, particularly in large 
mail systems.  Look at mail coming out of Google and Microsoft's hosted 
mail and you'll see a lot of ARC headers.

would have."

And as I've said repeatedly, do not contact me in private.

Mike

On 11/24/20 2:53 PM, John R Levine wrote:
> This appears to me to be an ad-hominem attack on the people who 
> designed ARC, so I think we're done.
>
> On Tue, 24 Nov 2020, Michael Thomas wrote:
>
>>
>> with a lot less bloat. All of this hand wringing about dozens of 
>> message manglers in the path before it get to the destination and not 
>> be able to figure out which auth-res was which seems wildly out of 
>> proportion for what the normal case is: 1 message mangler in the path 
>> before it arrives at the receiver's domain. Just like this message 
>> right here. That's why I asked how common that was, which was 
>> dismissed, but not answered.
>>
>> Mike
>>
>>
>
> Regards,

---

From: Seth Blank <[email-masked]>
Date: Tue, 24 Nov 2020 15:02:37 -0800

As Chair-

This thread is quickly becoming unproductive and veering to personal
attacks, which will not be tolerated.

Please engage productively and on the merits, take the conversation
elsewhere, or disengage.

Seth

-- 

*Seth Blank* | VP, Standards and New Technologies
*e:* [email-masked]
*p:* 415.273.8818

---

From: Brandon Long <[email-masked]>
Date: Tue, 24 Nov 2020 15:24:52 -0800

> I asked how common that was, which was dismissed, but not answered.

Note that this was exactly what we started with,
X-Original-Authentication-Results
and with Google's implementation signing that with X-Google-DKIM-Signature.

Our version didn't just sign with DKIM-Signature because of the reasons
already stated in this
thread, that our understanding of how DKIM-Signature was being used made it
a poor choice.

Our experience also showed that more than one hop is quite common in
enterprise deployments,
and those are also the places where the most complexity arises.  Others
shared our experience
as well.

You say that your message had 1 mangler in the path, but really you don't
know that.  It is
likely that some people on this list are on enterprise accounts who are
behind mangling inbound
gateways (rewriting urls to go through safety checking hops is common, for
example).  Or maybe
they are on with University accounts using exchange but forward their mail
to a personal or department
gmail account.

It may be that ARC is too complicated, that other, simpler, solutions would
have worked for 90% of cases
and would have been better than what we've got (From rewriting), and that
the real pain for receivers having
lots of hops should be borne by those receivers.

Brandon

---

From: Michael Thomas <[email-masked]>
Date: Tue, 24 Nov 2020 15:57:19 -0800

On 11/24/20 3:24 PM, Brandon Long wrote:
>
> On Tue, Nov 24, 2020 at 2:49 PM Michael Thomas <[email-masked] 
> Our version didn't just sign with DKIM-Signature because of the 
> reasons already stated in this
> thread, that our understanding of how DKIM-Signature was being used 
> made it a poor choice.

Sorry, I didn't see that. Pointer?

> and those are also the places where the most complexity arises.  
> Others shared our experience
> as well.

That's more than one modifying intermediary in *separate* administrative 
domains? Within your own administrative domain there shouldn't be an 
issue of trust since you can trust your own servers auth-res and that 
they are not maliciously trying to forge an auth-res for better 
treatment. and course it's best to stop a bad message as soon as 
possible in the mail pipeline if for no other reason than why waste the 
resources.

>
> You say that your message had 1 mangler in the path, but really you 
> don't know that.  It is
> likely that some people on this list are on enterprise accounts who 
> are behind mangling inbound
> gateways (rewriting urls to go through safety checking hops is common, 
> for example).  Or maybe
> they are on with University accounts using exchange but forward their 
> mail to a personal or department
> gmail account.

Well sure, I'm sure it can happen but is it common enough to worry 
about? And I'm still not convinced that figuring out who signed what and 
which auth-res it belonged to is in insurmountable problem. for one, 
there are received headers which better not be getting out of order to 
help correlate with the messages' path through intermediary verifiers too.

Mike

---

From: Brandon Long <[email-masked]>
Date: Tue, 24 Nov 2020 16:56:46 -0800

On Tue, Nov 24, 2020 at 3:57 PM Michael Thomas <[email-masked]> wrote:

> thread, that our understanding of how DKIM-Signature was being used made
> it a poor choice.
>
>
> domains? Within your own administrative domain there shouldn't be an issue
> of trust since you can trust your own servers auth-res and that they are
> not maliciously trying to forge an auth-res for better treatment. and
> course it's best to stop a bad message as soon as possible in the mail
> pipeline if for no other reason than why waste the resources.
>
The administrative domain in practice tends to be per-vendor and
multi-tenant.  Ie, gsuite/gmail is on google.com, various third party
anti-spam gateways also different, on-premise being also separate.  Passing
that trust between domains is one of the things that ARC can handle.

O365 does a better job than Gmail at this, since there you can set up third
parties to be called from within a particular customer more specifically
(like an API), you can do this with GSuite or ad-hoc MTAs, but it tends to
be more manual by chaining MTAs and there isn't as much of an auth
mechanism to use.

As for stopping bad messages early, there tends to be strong majority in
favor of that for malware, less so for spam where people want access from
their mailboxes to false positives.  Some products do have separate
administrative quarantine areas that can be checked by admins, but that's
also an extra burden for IT... some of which want that burden, and some
definitely don't.  Some quarantines can be self-administered by users, but
that's obviously a separate thing they have to check, whether that's worse
than a spam folder, *shrug*.

> Well sure, I'm sure it can happen but is it common enough to worry about?
> And I'm still not convinced that figuring out who signed what and which
> auth-res it belonged to is in insurmountable problem. for one, there are
> received headers which better not be getting out of order to help correlate
> with the messages' path through intermediary verifiers too.
>
So, we do have some Received header walking code, and allow admins to set
up the list of IPs that are considered "internal" (beyond private
addresses)... but O365 uses public IPs internally and
has a huge and unpublished ranges of them, making it nearly impossible for
admins to maintain a list.  Obviously we can all add code to handle
Microsoft specially... and whichever other ones come up.

Brandon

---

From: Michael Thomas <[email-masked]>
Date: Tue, 24 Nov 2020 17:27:30 -0800

On 11/24/20 4:56 PM, Brandon Long wrote:
>
>
> On Tue, Nov 24, 2020 at 3:57 PM Michael Thomas <[email-masked] 
>>     Our experience also showed that more than one hop is quite common
>>     in enterprise deployments,
>     That's more than one modifying intermediary in *separate*
>     administrative domains? Within your own administrative domain
>     there shouldn't be an issue of trust since you can trust your own
>     servers auth-res and that they are not maliciously trying to forge
>     an auth-res for better treatment. and course it's best to stop a
>     bad message as soon as possible in the mail pipeline if for no
>     other reason than why waste the resources.
>
> multi-tenant.  Ie, gsuite/gmail is on google.com <http://google.com>, 
> various third party anti-spam gateways also different, on-premise 
> being also separate.  Passing that trust between domains is one of the 
> things that ARC can handle.

One of the failings of many, many ietf documents is to not tell the 
story of what exactly the problem is and why the protocol is needed. 
It's really frustrating to the reader to not have the context of endless 
wg list discussions distilled so that an outsider can understand the 
design tradeoffs. I still don't understand why a DKIM signature was a 
"poor choice". That's especially true when something is an experiment 
that assumedly has yes/no endpoint.

So mtcc.com is now hosted by gsuite (::sigh::), so you're saying that it 
would run into problems? I haven't put up a DMARC policy, but if I did I 
might run into issues with false positives? Like I said, I'm trying to 
get my head around what the actual problems are, and this is coming from 
a person who stressed mightily from the very beginning about the mailing 
list problem.

>     about? And I'm still not convinced that figuring out who signed
>     what and which auth-res it belonged to is in insurmountable
>     problem. for one, there are received headers which better not be
>     getting out of order to help correlate with the messages' path
>     through intermediary verifiers too.
>
> So, we do have some Received header walking code, and allow admins to 
> set up the list of IPs that are considered "internal" (beyond private 
> has a huge and unpublished ranges of them, making it nearly impossible 
> for admins to maintain a list.  Obviously we can all add code to 
> handle Microsoft specially... and whichever other ones come up.
>

But can't you trust the received headers once you have possession of the 
message within your domain? I'm sorry if I'm being dense here but 
received headers are definitely ordered and if you loan out a message to 
be processed by a different domain and they aren't trustworthy, that's 
the least of your problems. That's part of the overall problem though: 
it's really hard to understand what the problem here actually is. 
Explain it like I'm five :)

Mike

---

From: "John R Levine" <[email-masked]>
Date: 24 Nov 2020 20:52:41 -0500

On Tue, 24 Nov 2020, Michael Thomas wrote:
>> enterprise deployments, and those are also the places where the most 
>> complexity arises. Others shared our experience as well.
>
> domains?

Microsoft currently appears to add an ARC seal for everything that
transits their hosted mail system, no mailing lists or whatever
involved. I think I may have seen stuff from them with two seals.

Here's the headers of something from MS that I picked out of the
spamtrap. It's a fake Sharepoint message, and all of the SPF and DMARC
checks failed, which of course makes one wonder why they sent it to me
at all, but there's plenty of ARC-age there in case I want to do the
filtering they didn't.

Regards,

================================================================
Date: Tue, 24 Nov 2020 23:04:25 +0000
Subject: Ashley Barker shared "I'M 28 Years Horny & Sexy" with you.
Sender: Ashley Barker <[email-masked]>
To: [email-masked],
  [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked],
  [email-masked], [email-masked], [email-masked],
  [email-masked]
Reply-To: [email-masked]
Cc: Ashley Barker <[email-masked]>
From: Ashley Barker <[email-masked]>
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3c412e91-e450-440a-8d6a-3245b726f1e3;Ip=[13.75.93.237];Helo=[[10.218.0.17]]
X-MS-Exchange-CrossTenant-AuthSource:
 	HK2APC01FT007.eop-APC01.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: KL1PR03MB4837

---

From: Douglas Foster <[email-masked]>
Date: Tue, 24 Nov 2020 22:27:00 -0500

Michael, I think the purpose is stated well enough:   Mailing lists want to
keep adding their content to messages, without being blocked by
recipients.   This means that they have to provide recipients with enough
information for them to accept the forwarded content.   Google and
Microsoft seem to be on-board with this project, so it seems pretty
successful already.   This train is not easily stopped.

As to the technical pieces:
- Unvalidated data, which includes the entire Received header chain, is
only useful for blacklisting.   A spammer will fabricate any header content
that he thinks will get his garbage through your spam filter.

- The recipient filter needs to be convinced that the submitting MTA is not
lying and that the submitting MTA has not been duped.   Digital signatures
are the only tool we have for validating forwarded content.   Since the
goal is favorable treatment, digital signatures are necessary.

- Interlocking signatures are needed to get around the problem of DKIM
signatures being damaged by mid-stream content changes.

As John said early on, this all works in conjunction with private knowledge
that decides who you believe is not lying.   Google and Microsoft have that
kind of data.

In my opinion, ARC does leave a lot of unanswered questions about how you
use the data that ARC provides.   Again, the big organizations have the
brain power at their disposal to figure that out for themselves, later.

It seems like a lot of software logic to create an ARC set, even more code
to parse it, and even more code to use it intelligently.   This is a big
problem if you are trying to write the code yourself, but a small problem
if you have a big programming organization.

The email market is moving to a few large organizations that provide
hosting for most everyone, a few other large cloud services that provide
filtering for much of the remainder, and then everybody else.   Smaller
organizations are stuck with lousy vendor options or custom development,
neither of which is entirely satisfactory.   ARC may not be the right
solution for smaller organizations with fewer development resources, but
80% of the world needs the big organizations to be successful, and the
mailing lists need the big organizations to be accepting of their product.
 ARC seems to meet those important goals.

Doug Foster

On Tue, Nov 24, 2020 at 6:57 PM Michael Thomas <[email-masked]> wrote:

> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 24 Nov 2020 20:19:08 -0800

On Tue, Nov 24, 2020 at 7:27 PM Douglas Foster <
[email-masked]> wrote:

> Michael, I think the purpose is stated well enough:   Mailing lists want
> to keep adding their content to messages, without being blocked by

That sounds about right.  Put another way: DMARC's success is at least in
part stymied by what MLMs do that invalidates DKIM; ARC is an attempt to
carry forward from the MLM, in a credible way, an indication of what the
MLM saw in terms of DKIM results when the message got to the MLM.  So then,
although an author signature will fail post-MLM, the MLM signature will
pass, and the ARC data will tell you that the author signature was good
when the MLM got it.  If you trust the chain, then that can be an
alternative to the DKIM input when the final recipient performs a DMARC
evaluation.

Sections 1 and 7.2.1 of RFC 8617 do say all of this, though perhaps not as
concretely as one might like.

> brain power at their disposal to figure that out for themselves, later.

This is why it was published as Experimental.  Its efficacy is not (yet)
known, nor are any side effects.  Although, now that you have me thinking
about it: It's been a year; have we any meaningful data about this yet?

> if you have a big programming organization.

When I implemented it, there was a great deal of processing logic that was
recycled from DKIM.  That was advantageous.  But I agree, someone
implementing ARC with no context at all could easily find it a challenge to
get it to interoperate.

-MSK

---

From: Michael Thomas <[email-masked]>
Date: Wed, 25 Nov 2020 11:03:37 -0800

On 11/24/20 8:19 PM, Murray S. Kucherawy wrote:
> On Tue, Nov 24, 2020 at 7:27 PM Douglas Foster 
> <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Michael, I think the purpose is stated well enough:   Mailing
>     lists want to keep adding their content to messages, without being
>     blocked by recipients.   This means that they have to provide
>     recipients with enough information for them to accept the
>     forwarded content.  Google and Microsoft seem to be on-board with
>     this project, so it seems pretty successful already.   This train
>     is not easily stopped.
>
>
> That sounds about right.  Put another way: DMARC's success is at least 
> in part stymied by what MLMs do that invalidates DKIM; ARC is an 
> attempt to carry forward from the MLM, in a credible way, an 
> indication of what the MLM saw in terms of DKIM results when the 
> message got to the MLM.  So then, although an author signature will 
> fail post-MLM, the MLM signature will pass, and the ARC data will tell 
> you that the author signature was good when the MLM got it.  If you 
> trust the chain, then that can be an alternative to the DKIM input 
> when the final recipient performs a DMARC evaluation.

That's been known for over 15 years. I'm still trying to understand the 
assertion that DKIM signatures are a "bad fit". I just looked at a 
random message off of this thread and they look identical except for the 
i= field. another field could trivially be added to DKIM and auth-res -- 
since unknown fields are supposed to be ignored -- if this binding 
property is absolutely needed, which I remain unconvinced by as well.

>
> Sections 1 and 7.2.1 of RFC 8617 do say all of this, though perhaps 
> not as concretely as one might like.
>
>     In my opinion, ARC does leave a lot of unanswered questions about
>     how you use the data that ARC provides.   Again, the big
>     organizations have the brain power at their disposal to figure
>     that out for themselves, later.
>
>
> This is why it was published as Experimental.  Its efficacy is not 
> (yet) known, nor are any side effects. Although, now that you have me 
> thinking about it: It's been a year; have we any meaningful data about 
> this yet?

If you ask me, part of the experiment should be able to show use cases 
that DKIM + auth-res (or maybe old-auth-res if needed) CANNOT address 
that ARC can, and most especially what percentage of email traffic we're 
actually talking here. As far as I can see ARC doesn't bring anything 
especially new for the mailing list kind of use cases since it really 
easy to see who the intermediary was that broke the signature. I have 
been told there are some vague other kinds of use cases but is unclear 
whether those use cases actually happen before or after the message 
arrives at the front door of the final receiving domain. Yes, i read 
section 7, but it doesn't tell me why this can't be handled with current 
technology.

It seems to me that the lynchpin is whether I trust the domain that 
broke the signature and resigned. That's either a previously solved or 
unsolved problem depending on your perspective. If I trust the 
intermediary domain to not send me spam via reputation, I forward it 
along to be delivered and ignore the DMARC record. Why do I care whether 
it originally validated from the sending domain or not? If you ask me, 
that's the intermediary domain's problem since they have an incentive to 
keep their reputation and not send spam through.

Mike

---

From: Michael Thomas <[email-masked]>
Date: Wed, 25 Nov 2020 11:07:40 -0800

On 11/24/20 7:27 PM, Douglas Foster wrote:
>
>
> In my opinion, ARC does leave a lot of unanswered questions about how 
> you use the data that ARC provides.   Again, the big organizations 
> have the brain power at their disposal to figure that out for 
> themselves, later.
>

They've had that data for over a decade from what I can tell. The exact 
same data that ARC provides, but perhaps a little hard to tease apart in 
some edge cases.

Mike

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 25 Nov 2020 16:14:27 -0800

On Wed, Nov 25, 2020 at 11:03 AM Michael Thomas <[email-masked]> wrote:

> assertion that DKIM signatures are a "bad fit". I just looked at a random
> message off of this thread and they look identical except for the i= field.
> another field could trivially be added to DKIM and auth-res -- since
> unknown fields are supposed to be ignored -- if this binding property is
> absolutely needed, which I remain unconvinced by as well.
>
I'm not sure about "bad fit".  The original plan was to have an
Authentication-Results (AR) clone called Original-Authentication-Results
(OAR) which was specifically the AR content generated at the first
non-submission MTA.  There was some friction (mostly from me, as I recall)
about having two header fields with identical syntax and nearly identical
semantics.  There was further complication in the fact that one ADMD could
apply multiple ARs on a single message (for multiple methods, or multiple
instances of the same method, or a combination of those), or they could be
reordered, etc.  To make it more resilient, things like instance numbers
were added.  The work was eventually generalized to be a "chain of custody"
sort of model, and in doing that I think the decision was made to make a
new name to ensure ARC and DKIM would be processed differently.

I imagine it could've been done as an Applicability Statement over DKIM and
AR, but I'm not sure now whether that would've been simpler.

> that DKIM + auth-res (or maybe old-auth-res if needed) CANNOT address that
> ARC can, and most especially what percentage of email traffic we're
> especially new for the mailing list kind of use cases since it really easy
> to see who the intermediary was that broke the signature. I have been told
> there are some vague other kinds of use cases but is unclear whether those
> use cases actually happen before or after the message arrives at the front
> door of the final receiving domain. Yes, i read section 7, but it doesn't
> tell me why this can't be handled with current technology.
>
> Obviously it's too late to include this in RFC 8617, but those would
indeed be interesting data to have.

When you say "easy to see", do you mean for software or for humans?

> It seems to me that the lynchpin is whether I trust the domain that broke
> the signature and resigned. That's either a previously solved or unsolved
> problem depending on your perspective. If I trust the intermediary domain
> to not send me spam via reputation, I forward it along to be delivered and
> ignore the DMARC record. Why do I care whether it originally validated from
> the sending domain or not? If you ask me, that's the intermediary domain's
> problem since they have an incentive to keep their reputation and not send
> spam through.
>
Suppose you're sending to a list that I'm on, I enforce DMARC, I trust that
MLM not to send spam via reputation, you have a good reputation, and you
publish a DMARC "reject" policy.  That means if a bad actor sends a message
to the list as you but doesn't sign it at all, I'll still accept it because
I trust that MLM even though according to your policy request, I should
bounce it.  ARC provides the missing data I need to enforce your request.

If the MLM enforces DMARC, that's slightly better, but I think ARC was
meant to be an incremental drop-in for MLM operators that don't (or won't)
do DMARC.

-MSK

---

From: Michael Thomas <[email-masked]>
Date: Wed, 25 Nov 2020 16:52:30 -0800

On 11/25/20 4:14 PM, Murray S. Kucherawy wrote:
> On Wed, Nov 25, 2020 at 11:03 AM Michael Thomas <[email-masked] 
>     That's been known for over 15 years. I'm still trying to
>     understand the assertion that DKIM signatures are a "bad fit". I
>     just looked at a random message off of this thread and they look
>     identical except for the i= field. another field could trivially
>     be added to DKIM and auth-res -- since unknown fields are supposed
>     to be ignored -- if this binding property is absolutely needed,
>     which I remain unconvinced by as well.
>
> Authentication-Results (AR) clone called 
> Original-Authentication-Results (OAR) which was specifically the AR 
> content generated at the first non-submission MTA. There was some 
> friction (mostly from me, as I recall) about having two header fields 
> with identical syntax and nearly identical semantics.  There was 
> further complication in the fact that one ADMD could apply multiple 
> ARs on a single message (for multiple methods, or multiple instances 
> of the same method, or a combination of those), or they could be 
> reordered, etc.  To make it more resilient, things like instance 
> numbers were added.  The work was eventually generalized to be a 
> "chain of custody" sort of model, and in doing that I think the 
> decision was made to make a new name to ensure ARC and DKIM would be 
> processed differently.

But what about DKIM? And why do they need to be processed differently? 
When I first saw this, I looked at the ARC-Signature which looks 
identical to a DKIM signature (i didn't notice the i= at the time), and 
am like what is this? The i= counter could be trivially be grafted onto 
DKIM if it were needed, which I'm still sort of dubious (though it is a 
nice to have feature, I admit). That way you only need a single DKIM 
signature with the OAR's signed. As far as I can tell, that would fall 
back gracefully for non-implementing DKIM verifiers. Do you disagree?

>
>>     If you ask me, part of the experiment should be able to show use
>>     cases that DKIM + auth-res (or maybe old-auth-res if needed)
>>     CANNOT address that ARC can, and most especially what percentage
>>     of email traffic we're actually talking here. As far as I can see
>>     ARC doesn't bring anything especially new for the mailing list
>>     kind of use cases since it really easy to see who the
>>     intermediary was that broke the signature. I have been told there
>>     are some vague other kinds of use cases but is unclear whether
>>     those use cases actually happen before or after the message
>>     arrives at the front door of the final receiving domain. Yes, i
>>     read section 7, but it doesn't tell me why this can't be handled
>>     with current technology.
>

Yeah, quantifying the problems kinda seems like the first order of 
business if you ask me. I mean, how many of these scenarios are in 
reality goofy self-inflicted wounds? I can't say but there seems to be a 
lot more "this can happen" than "this is how often this happens" from 
what I've see thus far on this thread.

> When you say "easy to see", do you mean for software or for humans?

Software. Only software can pry apart that ball of header spaghetti. But 
I think with the simple a mailing list it is pretty easy to determine, 
which now that I think about it I actually did back in the day when I 
was experimenting with recovering mailing list modifications. It didn't 
occur to me that that was supposed to be hard.

>     It seems to me that the lynchpin is whether I trust the domain
>     that broke the signature and resigned. That's either a previously
>     solved or unsolved problem depending on your perspective. If I
>     trust the intermediary domain to not send me spam via reputation,
>     I forward it along to be delivered and ignore the DMARC record.
>     Why do I care whether it originally validated from the sending
>     domain or not? If you ask me, that's the intermediary domain's
>     problem since they have an incentive to keep their reputation and
>     not send spam through.
>
> Suppose you're sending to a list that I'm on, I enforce DMARC, I trust 
> that MLM not to send spam via reputation, you have a good reputation, 
> and you publish a DMARC "reject" policy.  That means if a bad actor 
> sends a message to the list as you but doesn't sign it at all, I'll 
> still accept it because I trust that MLM even though according to your 
> policy request, I should bounce it.  ARC provides the missing data I 
> need to enforce your request.

Sure, but the easier answer: to use my mailing list, you must have 
either DKIM, SPF or both. Sounds like a good way to not be essentially 
an open relay. Don't mailing lists have those sorts of policies these 
days? I would say that ones that don't probably shouldn't have good 
reputations since they are, in effect... open relays.

Mike

PS: it's been, what, 15 years since our interop, partner? :)

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 26 Nov 2020 01:56:32 -0800

On Wed, Nov 25, 2020 at 4:52 PM Michael Thomas <[email-masked]> wrote:

> When I first saw this, I looked at the ARC-Signature which looks identical
> to a DKIM signature (i didn't notice the i= at the time), and am like what
> is this? The i= counter could be trivially be grafted onto DKIM if it were
> needed, which I'm still sort of dubious (though it is a nice to have
> feature, I admit). That way you only need a single DKIM signature with the
> OAR's signed. As far as I can tell, that would fall back gracefully for
> non-implementing DKIM verifiers. Do you disagree?
>

ARC was developed over months, even before this WG started, and I remember
all of these conversations happening involving the questions you're now
asking.  We landed at what became ARC.  I suppose an appendix might've been
nice enumerating everything that was considered and discarded, but that
record doesn't really exist.

What that means is that I couldn't tell you now why we rejected a pure
DKIM/OAR solution to this specific problem, but I know it was considered.
Maybe others who helped with RFC 8617 remember.

> business if you ask me. I mean, how many of these scenarios are in reality
> goofy self-inflicted wounds? I can't say but there seems to be a lot more
> "this can happen" than "this is how often this happens" from what I've see
> thus far on this thread.
>

Where do you suggest we go from here?

> Software. Only software can pry apart that ball of header spaghetti. But I
> think with the simple a mailing list it is pretty easy to determine, which
> now that I think about it I actually did back in the day when I was
> experimenting with recovering mailing list modifications. It didn't occur
> to me that that was supposed to be hard.
>
I haven't put hand to coding keyboard on this problem yet, but I'm trying
to imagine how it would be easy to determine (a) that Subject had been
modified (for example), (b) what the specific modification was, and (c)
which hop did it.  You could say a message failing to validate an author
signature with "[...]" at the front of Subject was likely tagged by an MLM,
or that everything after "--" should be ignored, or that those probably
happened at non-submission hop #1, but those are heuristics, and I think
we're hoping for something more deterministic.  The 80/20 rule isn't
sufficient.

But it's late and maybe I'm missing something.  What did you have in mind?

> Sure, but the easier answer: to use my mailing list, you must have either
> DKIM, SPF or both. Sounds like a good way to not be essentially an open
> relay. Don't mailing lists have those sorts of policies these days? I would
> say that ones that don't probably shouldn't have good reputations since
> they are, in effect... open relays.
>

That requires MLMs to change their behavior, which I believe is something
all of these protocols are hoping to avoid (or, at least, MLMs should
decide that on their own, not because the IETF forced them to by wielding
DMARC and ARC at them).

> PS: it's been, what, 15 years since our interop, partner? :)
2007.  Still got the t-shirt.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 26 Nov 2020 13:38:55 +0100

On 26/11/2020 10:56, Murray S. Kucherawy wrote:
>> business if you ask me.

Quantifications will differ depending on what you count.  Total number of 
messages versus total number of mail operators who find ARC useful.

Small operators had better not forward spam, whether ARC sealed or not.

> we're hoping for something more deterministic.  The 80/20 rule isn't

Again, you cannot get 100% lists.  For example, anonymizing lists will never 
let you recover an author domain's signature.  MLM has to comply.

On a compliant list like this one, you cannot get 100% users.  For example, 
those who sign a Content-Type: multipart/alternative without giving the 
original value, or a quoted-printable body that the MLM will encode differently 
will never verify.  Author domains have to comply.

On a compliant list, you can verify 99.99% compliant author domains' 
signatures.  (~0.01% due to cosmic rays and similar accidents.)

Best
Ale
--

---

From: Michael Thomas <[email-masked]>
Date: Thu, 26 Nov 2020 10:28:28 -0800

On 11/26/20 1:56 AM, Murray S. Kucherawy wrote:
>
> ARC was developed over months, even before this WG started, and I 
> remember all of these conversations happening involving the questions 
> you're now asking.  We landed at what became ARC.  I suppose an 
> appendix might've been nice enumerating everything that was considered 
> and discarded, but that record doesn't really exist.

Yeah, like I said that's sort of a common failing of IETF documents in 
general. I mean, I understand at a very low level what's going on here, 
but given the document I am completely clueless of the *why* of things 
are going on. It's very frustrating, doubly so when it's a self-labeled 
experiment.

> Where do you suggest we go from here?

I would think that an informational RFC might come in really handy both 
to answer the questions in ARC itself, and any other subsequent 
questions the wg deems needed since then. Leaving ARC in an experimental 
state ad infinitum doesn't seem optimal. Basically: 1) was it needed at 
all 2) did it help. 3) if it helped, how much did it help. (1) in 
particular is what interests me because adding two new signatures seems 
*really* heavy handed. That would go a long way toward answering the 
questions of whether it's should go standards track.

> I haven't put hand to coding keyboard on this problem yet, but I'm 
> trying to imagine how it would be easy to determine (a) that Subject 
> had been modified (for example), (b) what the specific modification 
> was, and (c) which hop did it.  You could say a message failing to 
> validate an author signature with "[...]" at the front of Subject was 
> likely tagged by an MLM, or that everything after "--" should be 
> ignored, or that those probably happened at non-submission hop #1, but 
> those are heuristics, and I think we're hoping for something more 
> deterministic.  The 80/20 rule isn't sufficient.
>

Our motivation at the time was one in particular: spear phishing. From 
an enterprise situation spear phishing is scary af, and not one that 
providers have much care about. That's what John gets wrong when he says 
that 90% pass rate is useless: for enterprise not wanting to get spear 
phished, a 10% false positive rate issuing warnings might be acceptable, 
and since we were grappling the intractable mailing list reputation (at 
least from our standpoint), maybe a better option.

But it was a combination of the l= value in the original signature to 
clip off what the mailing list added, and some heuristics on the subject 
line to remove added text. I had quite a few of them, and they were 
certainly pretty dodgy, but by and large they worked. The object was not 
to get to 100%, but just an acceptable false positive rate. At the time 
I mentioned that maybe it would be good to create a DKIM-friendly 
mailing list BCP, but that was scoffed at by the usual suspects because 
there was supposedly a massive long tail of transformations that can 
happen that nobody would quantify as to how common or important they 
were. Hence my desire of this being driven at least in part by numbers.

>     either DKIM, SPF or both. Sounds like a good way t
>
>     o not be essentially an open relay. Don't mailing lists have those
>     sorts of policies these days? I would say that ones that don't
>     probably shouldn't have good reputations since they are, in
>     effect... open relays.
>
>
> That requires MLMs to change their behavior, which I believe is 
> something all of these protocols are hoping to avoid (or, at least, 
> MLMs should decide that on their own, not because the IETF forced them 
> to by wielding DMARC and ARC at them).

I don't see making mailing lists coerced into better behavior as a 
non-starter. Everybody has had to change over the years to get mail 
delivered, and mailing lists, etc, shouldn't be viewed as sacred cows. 
If they want to forward their mail, they need to play by the evolving 
rules so as to make for a less spammy and phishy world. Back when DKIM 
was the new kid on the block, that might have been a valid argument. 
Today, not so much.

> 2007.  Still got the t-shirt.
But we interoped the combined DK and IIM spec a couple of years earlier, 
right? What a great day that was... we dunn'em proud, Murray :)

Mike

---

From: "John Levine" <[email-masked]>
Date: 26 Nov 2020 16:55:44 -0500

In article <[email-masked]> you write:
>all 2) did it help. 3) if it helped, how much did it help.

I agree that at some point we need to declare the experiment over and
see if it's worked, but it's way too early for that. At this point the
only widely used list software that can apply ARC seals is Sympa.
(Mailman 3 may, but most mailman users including the IETF are still
using mailman 2.)

> (1) in 

I don't get why a few extra signatures are a problem. Nearly all of my
mail goes out with two added DKIM signatures, one that matches the
>From domain or the list domain if it's a list, and one for my system.
It's just not a big deal.

>phished, a 10% false positive rate ...

Sorry, I meant 90% the other way, catching 90% of the bad stuff and
letting the other 10% through is not good enough. I agree that for
spear phishing the tolerance for false positives is likely to be
fairly high.

R's,
JOhn

---

From: Brandon Long <[email-masked]>
Date: Mon, 30 Nov 2020 21:43:30 -0800

On Tue, Nov 24, 2020 at 5:27 PM Michael Thomas <[email-masked]> wrote:

> One of the failings of many, many ietf documents is to not tell the story
> of what exactly the problem is and why the protocol is needed. It's really
> frustrating to the reader to not have the context of endless wg list
> discussions distilled so that an outsider can understand the design
> tradeoffs. I still don't understand why a DKIM signature was a "poor
> choice". That's especially true when something is an experiment that
> assumedly has yes/no endpoint.
>
To summarize what I said already in this thread, DKIM is taken by many
receivers as the responsible party for a message, in both spam and phishing
classifiers, with the latter being perhaps more relevant.
DMARC is the one example of that.  DKIM signing a message that transits
your system allows for reputation washing of the original message.  The ARC
chain is specified as a relay responsibility, which is a lesser burden.

Ignoring the existing usage of DKIM, DKIM+A-R would only work for a single
hop, and lead to some complication compared to the other DKIM signatures
already on the message.

DKIM is not the only thing that can be reputation washed, SPF can as well.
821.FROM rewriting is more common, so there are work-arounds.  OTOH, SPF
doesn't survive forwarding, but with ARC you can forward it.  DKIM+A-R
would only work for a single hop, and lead to some complication compared to
the other DKIM signatures already on the message.

> might run into issues with false positives? Like I said, I'm trying to get
> my head around what the actual problems are, and this is coming from a
> person who stressed mightily from the very beginning about the mailing list
> problem.
>
It's unlikely you have a complicated mail flow involving multiple vendors,
but certainly there are plenty of configuration options alllowing folks to
shoot themselves in the foot.

> message within your domain? I'm sorry if I'm being dense here but received
> headers are definitely ordered and if you loan out a message to be
> processed by a different domain and they aren't trustworthy, that's the
> least of your problems. That's part of the overall problem though: it's
> really hard to understand what the problem here actually is. Explain it
> like I'm five :)
>
For now, our received header walking code is for handling inbound
gateways.  If you use both O365 and GSuite, and use O365 as your MX before
forwarding to Google, there are many cases where they will break all DKIM
signatures (and obviously SPF) before reaching Google's MX.  Our workaround
is to specify your internal IPs to us, so we can walk past those received
headers to the first one added by your external MX.

Should these things use IP addresses for these things?  Probably not.  Some
sort of SMTP-AUTH could potentially be used to gateway more directly
between them, but we'd still need to disable most of our
spam/phishing pipelines which are dependent on reputation.

With ARC, we can instead say "trust this arc admd" and have the data passed
directly, and also work around the DKIM breakage.

As for another comment in the thread about adding two signatures per hop,
there was a suggestion that it could be replaced with a single signature
per hop near last call with some loss of information (maybe), but it was
decided to move forward with the existing solution that already had several
implementations and was experimental anyways.

As for how complicated it is to implement, I think starting from a DKIM
implementation it's fairly straight-forward... though obviously that
depends a bit on how your DKIM implementation was composed.  I think my
initial changes to dkimpy to add ARC support only took a couple days.  It's
not really an indication of how well written (or not) the RFC is, given
familiarity with the design prior to draft.

Brandon