Subject: [dmarc-ietf] Figuring out the tree walk

From: "John Levine" <[email-masked]>
Date: 11 Feb 2022 13:25:15 -0500

It appears that Alessandro Vesely  <[email-masked]> said:
>I think it is already clear to the WG that the tree walk is screwed up. 

Yes, we know we have to rewrite sections 4.5 and 4.6.

I think there are 2 1/2 situations we need to look at.

The first is finding the policy record for a domain that does not have its own DMARC
record.  That seems easy, walk up five levels and stop at the first DMARC record,
which is your policy record.  For this purpose it doesn't matter whether the domain
says psd=y.  I suppose that it if doesn't have psd=y, you can call the domain with the
record the org domain.

The second is deciding whether two diffent domains are in relaxed alignment.  This has
an easier case and a harder one.

The easier case is when one domain is a subdomain of the other.  You look at the domains
between the two and they are in relaxed alignment if one of these is true but I'm not
sure which one:

 a. there are no DMARC records at all
 b. there are no DMARC records witn psd=y

I don't think there is a lot of practical difference between the two.  If you don't find
a record, there is no policy and no org domain.

The harder case when the domains are siblings, or maybe a great aunt.  Possibilities:

 a. they are never aligned.  This would be the easiest, but would in principle be a significant
change from the current spec.  Does anyone know in practice how often mail uses sibling alignment?

 b. walk up from both, stop at the first DMARC record. If they're at
the same name and it's not a PSD, they are aligned and that's the org domain

 b+. walk up from both, if the DMARC records are at the same name, it has psd=y, and they have the
same name below the PSD, they are aligned and the name below the PSD is the org domain

 c and c+. like b or b+ but allow other non-PSD DMARC records below the org domain.

I lean toward "a" if we believe that sibling alignment is rarely used, otherwise b.  I don't like b+ or c+ because
I think it is reasonable to expect mailers who depend on an org domain to publish a record there, although I do not
think an org=y flag would be useful, since it would break existing org domain records that don't have the flag.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 13 Feb 2022 11:59:50 +0100

> says psd=y.  I suppose that it if doesn't have psd=y, you can call the domain with the 

More or less agreed.

> The second is deciding whether two different domains are in relaxed alignment.  This has 
>   b. there are no DMARC records with psd=y
>

Yes, I think we can relay on PSD domains not publishing DKIM or SPF stuff.

>   c and c+. like b or b+ but allow other non-PSD DMARC records below the org domain.

I'd avoid walking up from both, because there are likely multiple identifiers, 
typically SPF and DKIM.  Walking up from each becomes burdensome.  This leaves 
pretty much only a.  However, why not continue the walk upward?  The topmost 
domain having a DMARC record is the Organizational Domain.

In a few cases, the domain thus found can happen to be a PSD.  If there is 
psd=y, the plan in Section 4.6. works fine.  Until that flag is not reliable, 
we need to apply specific knowledge.  To this end, note that the list of 
domains at psddmarc.org is much much shorter and more easily maintainable than 
the PSL.

If flags are not reliable, a tree walk without extra knowledge or heuristics is 
not going to work in any case.  OTOH, if one day flags become reliable, we will 
have put a mark where DBOUND failed.

> think an org=y flag would be useful, since it would break existing org domain records that don't have the flag.

Finding org=y saves one or more extra lookups.  The org domain is determined 
even without that flag, albeit at an extra cost.  Perhaps more importantly, 
org= allows a subdomain to claim independence from its parent domain.

To put it more analytically, psd= and org= correspond to different ways to 
apply DMARC, with respect to both determining the alignment and feedback 
reporting.  Mail filters must ascertain which role a domain plays, whether 
declared or not.  Then, it is false to assume that psd=n ==> org=y, as org's 
subdomains may exist.  It could even make sense to define sub=y.

Best
Ale
--

---

From: "John R Levine" <[email-masked]>
Date: 13 Feb 2022 10:30:05 -0500

When I proposed the tree walk, I saw three self-evident conditions that 
any tree walk has to meet:

1.  The result of the tree walk is the same as the current scheme in 
nearly every case.

2.  The tree walk uses only the results of the DNS queries, not any 
external reference maintained by third parties.  If we're going to use a 
third party, we might as well stay with the PSD.

3.  The tree walk has to work with existing DMARC records, since we know 
that it takes a very long time for systems to change their software.  One 
exception to this is that we can ask PSD records to change, since there 
are less than a dozen PSDs deliberately publishing DMARC records, they 
know it's currently an experiment, and Scott knows them all.

So:

On Sun, 13 Feb 2022, Alessandro Vesely wrote:
> Yes, I think we can rely on PSD domains not publishing DKIM or SPF stuff.

Why not check and see?  I have.

>> The harder case when the domains are siblings, or maybe a great aunt.  ...

> I'd avoid walking up from both, because there are likely multiple 
> identifiers, typically SPF and DKIM.  Walking up from each becomes 
> burdensome.

I don't see how this follows.  In any DMARC check, there is one SPF 
identifier and some number of DKIM signatures, rarely more than two.  In 
most cases one name is an ancestor of the other, so sibling alignment is 
rare.  That means we have to make it work (see rule 1) but it doesn't have 
to be very fast.

   This leaves pretty much only a.  However, why not continue the 
> walk upward?  The topmost domain having a DMARC record is the Organizational 
> Domain.

I don't see how this helps.  In practice I think it is rare to do relaxed 
alignment and have a DMARC record between the original domain and the org 
domain.

> domains at psddmarc.org is much much shorter and more easily maintainable 
> than the PSL.

No.  That fails rule 2.

>> although I do not think an org=y flag would be useful, since it would break 
>> existing org domain records that don't have the flag.
>
> Finding org=y saves one or more extra lookups.

No.  That fails rule 3.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Douglas Foster <[email-masked]>
Date: Sun, 13 Feb 2022 12:52:53 -0500

I have been proceeding on the assumption that we can and should implement
support for independent subdomains.  This is a difference with RFC 7489
which recognizes the additional complexities in the deployed ecosystem.

The tree walk must find an organizational domain to act as an upper bound
on relaxed alignment.   I do not think we can assume that
DMARC-participating subdomains will always have an organization-level DMARC
policy, so the problem is more complex than finding the top-most policy.
Organization-level policies are probably the norm, but there is nothing in
RFC 7489 that makes it necessary, and nothing in DMARCbis which makes it
certain.   If we allow for this exception, we have no choice other than to
consult a list sometimes.

Ale's comment helped me to simplify my understanding of the tree walks:

1) Walk up from the From domain to the policy domain, then continue upward
until the organization domain is determined.  Consult a list if necessary.

2) If the Authorizing domain is between the From domain and the
Organization domain, the names are aligned.

3) If the MatchPoint domain is above the Organization Domain or empty, the
names are not aligned.

4) Otherwise, the Authorizing domain should be checked for an independent
subdomain boundary by walking up from it until any point is reached between
the From domain and the Organization domain.   If no boundary is found, the
names are aligned.   (If independent subdomain boundaries are intentionally
ignored, this second tree walk can be skipped.)

Doug

On Sun, Feb 13, 2022 at 6:00 AM Alessandro Vesely <[email-masked]> wrote:

> identifiers,
> than
> > I lean toward "a" if we believe that sibling alignment is rarely used,
> otherwise b.  I don't like b+ or c+ because
> > I think it is reasonable to expect mailers who depend on an org domain
> to publish a record there, although I do not
> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 13 Feb 2022 16:30:03 -0500

On Friday, February 11, 2022 1:25:15 PM EST John Levine wrote:
> The first is finding the policy record for a domain that does not have its
> own DMARC record.  That seems easy, walk up five levels and stop at the
> first DMARC record, which is your policy record.  For this purpose it
> doesn't matter whether the domain says psd=y.  I suppose that it if doesn't
> have psd=y, you can call the domain with the record the org domain.
> 
> The second is deciding whether two diffent domains are in relaxed alignment.
>  This has an easier case and a harder one.
> 
> The easier case is when one domain is a subdomain of the other.  You look at
> the domains between the two and they are in relaxed alignment if one of
> these is true but I'm not sure which one:
> 
> I don't think there is a lot of practical difference between the two.  If
> you don't find a record, there is no policy and no org domain.
> 
>  a. they are never aligned.  This would be the easiest, but would in
> principle be a significant change from the current spec.  Does anyone know
> in practice how often mail uses sibling alignment?
> 
>  b+. walk up from both, if the DMARC records are at the same name, it has
> psd=y, and they have the same name below the PSD, they are aligned and the
> name below the PSD is the org domain
> 
> otherwise b.  I don't like b+ or c+ because I think it is reasonable to
> expect mailers who depend on an org domain to publish a record there,

I think "a" would be cleanest, but I think it would cause too much backward 
compatibility trouble and should not be further considered.  In previous 
working group discussions on this I recall specific suggestions that this would 
be problematic and this is supported by the short survey that Elizabeth Zwicky 
reported on.

I think a challenge with your definition for "b" and "c" is that there aren't 
necessarily two domains at issue.  It could be three.  If you take the DKIM 
signing domain and the 5322.From domains as an input that might yield a 
different "org domain' than is yielded by the 5321.Mailfrom domain and the 
5322.From domain (for SPF).  I'm not sure it matters in practice for relaxed 
alignment since if either DKIM or SPF passes and is aligned it's a DMARC pass.  
>From a protocol specification perspective it's different from RFC 7489.  In RFC 
7489, organization domain is determined from a single input and the results 
(do they have the same organizational domain) are compared.

In RFC 7489, you can determine the org domain from any single domain.  I think 
it would be useful to preserve this property.  I think we can do this either 
by walking up to the last domain with a non-PSD DMARC record or by walking 
down to the first domain with a non-PSD DMARC record (I made an admittedly poor 
first attempt at this that's in the current draft).

I don't think it matters much which direction the lookup goes, so I'm fine with 
whatever the group prefers.  I do think though that we should preserve the 
concept of organizational domain as something that can be determined from a 
single input.

If we preserve that property of organizational domain, I think the risk of 
backward compatibility issues is lower.  Also, we could, if the group supports 
it, outsource this definition to a separate draft as Dave Crocker has 
suggested.  In that case, DMARCbis would only have to say to determine the 
organizational domain and the rest is external to that document.

I support the idea of a separate draft, but I don't think it's essential to 
making progress on DMARCbis.

Scott K

---

From: Dotzero <[email-masked]>
Date: Sun, 13 Feb 2022 17:15:22 -0500

> be problematic and this is supported by the short survey that Elizabeth

The problem with what Elizabeth shared, and I do appreciate her sharing it,
is that what she shared doesn't show the extent of the variety of behaviors
in those "sibling" relationships. Because those are undocumented and not
understood AND it isn't even clear that those relationships and behaviors
are what is intended by the "standard", serious questions are raised.  Can
a "standard" that allows a variety of undocumented non-standard behaviors
(and outcomes) be considered a standard? The details behind some of the
examples given by Elizabeth indicate potential abuse vectors beyond my
original concerns. Unfortunately, the receiving organizations most able to
provide detailed examples are unlikely to provide them publicly due to
privacy concerns, embarrassing customers, etc. It might be useful if an
organization like M3AAWG could process detailed examples from
members,obfuscate the organizational information and share it publicly as
an aggregation of examples to inform a better standards creation process.

Michael Hammer

---

From: "John Levine" <[email-masked]>
Date: 13 Feb 2022 18:04:27 -0500

>5322.From domain (for SPF). ...

I was thinking that you only compare them pairwise, 5322.From vs 5221.Mailfrom
for SPF, or 5322.From vs DKIM signature domain.

You'd start with the 5322.From to see what the org domain is, then check the other
two if they get to the same place.

>first attempt at this that's in the current draft).

That could work too, so long as we believe that domains that use relaxed alignment
consistently publish DMARC records.  Any stats on that?

>making progress on DMARCbis.

I don't see the point of a separate draft but I don't think it'll hurt anything.
After the discussion of PSDs I think it is unlikely that this alignment will be useful for
anything other than DMARC.

R's,
John

---

From: Todd Herr <[email-masked]>
Date: Mon, 14 Feb 2022 10:45:39 -0500

On Sun, Feb 13, 2022 at 5:16 PM Dotzero <[email-masked]> wrote:

> The problem with what Elizabeth shared, and I do appreciate her sharing
> it, is that what she shared doesn't show the extent of the variety of
> behaviors in those "sibling" relationships. Because those are undocumented
> and not understood AND it isn't even clear that those relationships and
> behaviors are what is intended by the "standard", serious questions are
> raised.  Can a "standard" that allows a variety of undocumented
> non-standard behaviors (and outcomes) be considered a standard? The details
> behind some of the examples given by Elizabeth indicate potential abuse
> vectors beyond my original concerns. Unfortunately, the receiving
> organizations most able to provide detailed examples are unlikely to
> provide them publicly due to privacy concerns, embarrassing customers, etc.
> It might be useful if an organization like M3AAWG could process detailed
> examples from members,obfuscate the organizational information and share it
> publicly as an aggregation of examples to inform a better standards
> creation process.
>
>
>
I'm not understanding what you mean when you state that "it isn't even
clear that those relationships and behaviors are what is intended by the
'standard'"

RFC 7489 seems to be clear in its definition of alignment.

In section 3.1.1 DKIM-Authenticated Identifiers, the text reads, in part:

   In relaxed mode, the Organizational Domains of both the [DKIM
<https://datatracker.ietf.org/doc/html/rfc7489#ref-DKIM>]-
   authenticated signing domain (taken from the value of the "d=" tag in
   the signature) and that of the RFC5322
<https://datatracker.ietf.org/doc/html/rfc5322>.From domain must be
equal if
   the identifiers are to be considered aligned.  In strict mode, only
   an exact match between both of the Fully Qualified Domain Names
   (FQDNs) is considered to produce Identifier Alignment.

and in section 3.1.2, SPF-Authenticated Identifiers, the text reads, in part:

   In relaxed mode, the [SPF
<https://datatracker.ietf.org/doc/html/rfc7489#ref-SPF>]-authenticated
domain and RFC5322
<https://datatracker.ietf.org/doc/html/rfc5322>.From
   domain must have the same Organizational Domain.  In strict mode,
   only an exact DNS domain match is considered to produce Identifier
   Alignment.

Both define relaxed alignment as "same Organizational Domain", with no
caveats on any other relationship between the two domains being
compared.

How do we determine intent, especially if the claim is that intent is
different from what was written in the RFC?

We're fortunate, I guess, that we know how to contact most if not all
of the people named in the

Acknowledgements section of the RFC, along with the two named authors,
but there is literature to suggest

that eyewitness testimony is not 100% reliable.

Are there notes from the time of writing RFC 7489 to confirm that the
intent of the group was to not allow alignment

among domains without a direct hierarchical relationship?

Whether or not there's evidence regarding intent, that doesn't mean
that the case for altering the definition of alignment

can't be discussed, but it does determine the argument to be made for
the change:

   - If there's evidence of intent, then the argument is "The
definition of alignment needs to change, because the current
definition isn't what the authors intended and it exposes domains to
security/abuse risks."
   - If not, then the argument is "The definition of alignment needs
to change, because the current definition exposes domains to
security/abuse risks."

My personal opinion is that arguing this case on just the merits of "The
current definition of alignment exposes domains to security/abuse risks,
here are examples of how they can be exploited, and here's why those
exploits can't be overcome through use of other features of DMARC and/or
the underlying protocols" is a pretty powerful argument to make, without
worrying about intent, but that's just my opinion.

-- 

*Todd Herr * | Technical Director, Standards and Ecosystem
*e:* [email-masked]
*m:* 703.220.4153

This email and all data transmitted with it contains confidential and/or
proprietary information intended solely for the use of individual(s)
authorized to receive it. If you are not an intended and authorized
recipient you are hereby notified of any use, disclosure, copying or
distribution of the information included in this transmission is prohibited
and may be unlawful. Please immediately notify the sender by replying to
this email and then delete it from your system.

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 14 Feb 2022 11:14:36 -0500

On Monday, February 14, 2022 10:45:39 AM EST Todd Herr wrote:
> <https://datatracker.ietf.org/doc/html/rfc7489#ref-DKIM>]->    authenticated signing domain (taken from the value of the "d=" tag in

I agree, but generic arm waving is not useful.  We need specifics to understand 
the concern.  I have a suspicion that it will amount to a third-party 
delegation concern that is best addressed by explaining potential risks and 
mitigations in security or privacy considerations, but without knowing 
anything about what the actual issue is there is no way to know.

Scott K

---

From: Dotzero <[email-masked]>
Date: Mon, 14 Feb 2022 12:27:39 -0500

> clear that those relationships and behaviors are what is intended by the

The heart and core of DMARC policy is about intent. That is, an
organization is expressing which messages are authorized are sent on behalf
of that organization through the use of either aligned DKIM signing or
aligned SPF records. Elizabeth provided more details of the high level
examples provided to this group but in a private conversation among parties
that are contractually bound, including a confidentiality clause. Among
those examples, there were cases of what can only be called
misconfigurations (flip flopping of from, mail from, DKIM signing, etc.
within the same organizational domain) and other irregularities.

>    In relaxed mode, the Organizational Domains of both the [DKIM <https://datatracker.ietf.org/doc/html/rfc7489#ref-DKIM>]-
>    the signature) and that of the RFC5322 <https://datatracker.ietf.org/doc/html/rfc5322>.From domain must be equal if
>    In relaxed mode, the [SPF <https://datatracker.ietf.org/doc/html/rfc7489#ref-SPF>]-authenticated domain and RFC5322 <https://datatracker.ietf.org/doc/html/rfc5322>.From
> Both define relaxed alignment as "same Organizational Domain", with no caveats on any other relationship between the two domains being compared.
>
>
> How do we determine intent, especially if the claim is that intent is different from what was written in the RFC?
>
>
> We're fortunate, I guess, that we know how to contact most if not all of the people named in the
>
> Acknowledgements section of the RFC, along with the two named authors, but there is literature to suggest
>
> Are there notes from the time of writing RFC 7489 to confirm that the intent of the group was to not allow alignment
>
Todd, you are well aware that meeting notes of DMARC.ORG are kept and in
fact, those notes have been kept going back well before the date on  RFC
7489. Unfortunately, those notes are subject to contractual obligations,
including a confidentiality clause, on the part of the participating
individuals and the organizations they represent. In fact, there are
meeting notes that go back to STAMP, which was the predecessor organization
to DMARC.ORG and which created the private implementation of what
subsequently became DMARC. Unfortunately STAMP is defunct and I know of no
mechanism by which notes and documentation from that group could be made
public. Presumably, the participating organizations in DMARC could agree to
release any pertinent notes related to alignment from the DMARC.ORG
archives.

I'll also point out that the authors of RFC7489 were selected at a DMARC.ORG
meeting by the participants who all were responsible for the creation and
submission of tht RFC.

>
> Whether or not there's evidence regarding intent, that doesn't mean that the case for altering the definition of alignment
>
> can't be discussed, but it does determine the argument to be made for the change:
>
>
>    - If there's evidence of intent, then the argument is "The definition of alignment needs to change, because the current definition isn't what the authors intended and it exposes domains to security/abuse risks."
>    - If not, then the argument is "The definition of alignment needs to change, because the current definition exposes domains to security/abuse risks."
>
I made the suggestion that either receivers with access to large data sets,
intermediaries or an organization like M3AAWG publicly share to this group
(domains obfuscated) sets of examples specifically so that we are not left
to depend on hand waving. I may have access to data but it is not my data
to share. While I personally believe the security/abuse risks are
compelling as to why sibling domains are problematic, I also believe that
having the working group see and understand current Sender implementations
involving DMARC "sibling domains" and the subsequent operational behavior
is important in avoiding similar situations in the future.

Michael Hammer

---

From: "John Levine" <[email-masked]>
Date: 14 Feb 2022 13:18:02 -0500

>> to change, because the current definition exposes domains to

On the one hand, Mike points out that there are what used to be called domain parks, and their operators
and customers don't appreciate the security risks of one customer impersonating another.  On the other
hand, Laura points out that there are real mailers that authenticate with sibling domains, and there is
currently no easy technical way to tell the difference.  In principle domain parks can list themselves
in the PSL, in practice most don't.

My opinion is that while the risk of impersonation via DMARC is real, other kinds of impersonation are
worse, notably web cookie stealing, and if they can live with the web problems, they can live with the
DMARC problems.

Also, if we go to a tree walk with a psd=y flag, the cost to the domain park of identifying itself
with a DMARC record is about as low as possible, way easier than petitioning the PSL which they still
have to do if they have cookies.

R"s,
John