Subject: [dmarc-ietf] Tree walk is screwed up

From: "John Levine" <[email-masked]>
Date: 19 Jan 2022 13:38:15 -0500

I took a look at sections 4.5 and 4.6 of the draft, the part that describes the tree walk
and PSD, and unfortunately what it currently says is seriously wrong.  Apologies for
not catching this sooner.

It currently says you do a tree walk to find a PSD record, and the org domain is the one
under the PSD.  That's fine if there's a PSD record, but in the vast majority of cases
there will not.  There is no chance that .COM or .ORG or .NET or any of the other public
gTLDs will ever publish a DMARC record because they are not in the business of managing
their customers' mail.  (There are also acronym-studded ICANN reasons they couldn't even
if they wanted to, ask me if you want the details.)

What I always intended with the tree walk is that you walk up the tree and if you find
a DMARC record that isn't a PSD, that's your org domain.  To see if two names are in relaxed
alignment, do a tree walk for both and if they end at the same place, they're aligned.  As
a special case albeit a very common one, if one name is a descendant of the other, and there
are no DMARC records in between, they're aligned.

This means that if there's no PSD and no higher level DMARC records, there's no org domain, but
I believe that we discussed this before and in practice it doesn't matter.

R's,
John

---

From: Todd Herr <[email-masked]>
Date: Wed, 19 Jan 2022 13:47:56 -0500

> not catching this sooner.
No worries; that's what draft revisions are for...

I'll work to rewrite these sections for the next version.

Scott Kitterman, do you mind if I just incorporate your suggested text
re:PSD into that same rev, rather than your going through the machinations
of a pull request?

-- 

*Todd Herr * | Technical Director, Standards and Ecosystem
*e:* [email-masked]
*m:* 703.220.4153

This email and all data transmitted with it contains confidential and/or
proprietary information intended solely for the use of individual(s)
authorized to receive it. If you are not an intended and authorized
recipient you are hereby notified of any use, disclosure, copying or
distribution of the information included in this transmission is prohibited
and may be unlawful. Please immediately notify the sender by replying to
this email and then delete it from your system.

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 20 Jan 2022 12:36:28 +0000

>re:PSD into that same rev, rather than your going through the machinations

That's fine.  I only did the PR to make it easier for you.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 20 Jan 2022 14:00:03 +0100

> are no DMARC records in between, they're aligned.

Why would a DMARC record in between invalidate the alignment?

DKIM-Signature: d=a.b.example.com [...]
From: [email-masked]

_dmarc.example.com    IN TXT "v=DMARC1; p=reject;"
_dmarc.b.example.com. IN TXT "v=DMARC1; p=none; inbetween=y"

Is that aligned?

Best
Ale
--

---

From: Douglas Foster <[email-masked]>
Date: Thu, 20 Jan 2022 09:57:48 -0500

I think this refinement of the concept will work better:

1) Check to see if the From domain participates in DMARC
Starting with the FROM domain, perform the previously specified tree walk
(start with the exact-match FROM domain, jumping to level 5 if necessary,
then continuing to walk up the tree.)
-
- If no policy is found, the domain does not participate in DMARC and
processing terminates.  The DMARC result is "NO POLICY".

-- If a policy is found with PSD=y, the domain does not participate in
DMARC but may need to be tested for non-existence.  If the policy also
specifies NP=reject, query the next-lower domain name for a resource
record.   If the DNS query result is NXDOMAIN, processing stops and the
DMARC policy is also "NXDOMAIN".  (I recommend using NXDOMAIN as a separate
result code from REJECT, as it seems to be a stronger repudiation.)

-- if a policy is found without PSD=y, this is the policy that will be
used.   This policy domain will be used for alignment tests, since it may
be shorter than the From domain.

2) If the domain participates in DMARC, check for alignment of an SMTP
domain or DKIM domain

Find the longest subdomain that is shared between the policy domain and the
domain being tested.
If the result is the policy domain, the names are in alignment.
If the result is shorter than the policy domain, start at that point and
walk up the tree until a DMARC policy is found.
If a policy is found, the names are in alignment.  One the match occurs
higher than the policy domain, the policy values are ignored, because the
policy domain is always used.
If no policy is found, the names are not in alignment.  Proceed to the next
name to be tested until a match is found or all candidate names have failed.

Design goals satisfied:
- no PSL
- no dependence on PSD=y
- support for PSD=y when it exists
- evaluators cannot use DMARC authentication unless a DMARC policy actually
exists.

Doug Foster

On Thu, Jan 20, 2022 at 8:00 AM Alessandro Vesely <[email-masked]> wrote:

> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 20 Jan 2022 11:57:39 -0500

On Thursday, January 20, 2022 9:57:48 AM EST Douglas Foster wrote:
...
> result code from REJECT, as it seems to be a stronger repudiation.)
...

No.  In this case policy discovery is complete and that's the policy that 
should be applied.  Additionally, that's not how the np= tag works.

All psd=y means is don't use this domain for determining alignment.  For 
policy discovery it's like any other.

Scott K

---

From: Douglas Foster <[email-masked]>
Date: Thu, 20 Jan 2022 12:18:03 -0500

So if the first tree walk stops at a a PSD=y policy, then the match string
used for alignment is the organizational domain, one segment down from the
PSD policy.  Any SPF or DKIM domain must match or be a child of the
organizational domain, so there is no secondary tree walk,

Does that correct the problem?

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 20 Jan 2022 12:27:52 -0500

That sounds closer.  I think Todd knows what he needs to change, so I'd prefer 
to see what goes into the document than to continue to tweak it.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 20 Jan 2022 23:29:05 -0500

>Why would a DMARC record in between invalidate the alignment?

For the same reason the PSL has a lot of two- and three-label domains.

>Is that aligned?

No, the org domain for c.example.com is example.com, while
the org domain for a.b.example.com is b.example.com.

I suppose we could try and invent rules that say only records with
the PSD flag prevent alignment, but then the description of the tree
walk becomes much more complicated and the tree walk gets longer and
slower and I don't see what actual problem that would solve.

In your model, when doing a tree walk from a.b.example.com, how do you
know not to stop at b.example.com?  Do you always do five levels and use
the one highest up the tree or what?

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 21 Jan 2022 13:54:48 +0100

> For the same reason the PSL has a lot of two- and three-label domains.

Except that the PSL is somehow vetted; that is, there are no self-appointed 
PSDs.  Anyway, b.example.com below exhibits no psd=y tag...

> the org domain for a.b.example.com is b.example.com.

If we specify it that way, we need to warn users about the effects that 
publishing a DMARC record involves.  Users of b.example.com need to know that 
they cannot override the DMARC policy of their org domain unless they want to 
become the org domain of the whole subtree.  That is, doing their own DKIM.

> slower and I don't see what actual problem that would solve.

Or we could use a multi-valued tag instead of psd=.  For example role=X, where 
X can be any of psd, org or sub.

> the one highest up the tree or what?

Personally, I'd consult the PSL if I find no role= tag.  Or I could do one more 
query for _dmarc.com to understand the role of the last record found.

Let's consider that we have a number of settled DMARC filters based on PSL, 
along with an installed base of DMARC records, which we shouldn't throw away. 
The specification of the tree walk has to strive for compatibility.

Best
Ale
--

---

From: Douglas Foster <[email-masked]>
Date: Fri, 21 Jan 2022 09:20:17 -0500

Your idea is brilliant - search for a policy that applies to both the FROM
domain and the other domain name.  If it is not a PSD policy, then the two
names are aligned.  It eliminates the PSL while also creating no need for
PSD policies.

But your organization assumption does not work.  In the RFC 7489 algorithm,
we skip over intermediate policies, but they can still exist.   We cannot
assume them away.   Redefining the organization to mean "the first policy
record over the current subdomain" is a significant redesign and
unnecessary.   There are also performance considerations.

Before testing for alignment, we need to test for DMARC participation.   It
would be inefficient to test the SMTP address and three DKIM signatures for
alignment, only to find that they all fail because the organization does
not have any DMARC policy records published.  Consequently, we start with
the tree walk on the FROM domain, and the lowest-level policy found
applies.   That walk serves to prune the tree, providing a lower limit on
any possible policy record that could apply to both the FROM address and
the aligned name.  Now we are ready to test candidate names against the
domain name that returned a policy record.

If the match is strict, alignment is now determined with a simple string
compare.

If the match is relaxed, we start from the domain name that returned a
policy record and keep walking up the tree.   Since we are looking for a
policy that applies to both names, we need to start at the point where the
two names merge - the longest common substring.

For DKIM signatures, signatures that do not align are ignored for alignment
testing.

An implementation might choose to build a list of domain names and compute
the common-substring matches first.  Matches that produce no match or a
match on TLD-only can be discarded.   With this approach, only the longest
match needs to be tested for alignment, although the draft asks the
evaluator to fully evaluate and report about each name.

DF

---

From: "John R Levine" <[email-masked]>
Date: 21 Jan 2022 09:28:07 -0500

> PSDs.

Sorry, but that is simply false.  The entire "private domains" part of the 
PSD is self-appointed PSDs.

> become the org domain of the whole subtree.

Yes, that is the plan.  Please go back and look at the discussion when we 
talked about the tree walk in the first place.

>  That is, doing their own DKIM.

Sorry, that makes no sense.  Everyone always does their own DKIM.

> Personally, I'd consult the PSL if I find no role= tag.

Um, the whole point of the tree walk is so we do not use the PSL, which 
has all sorts of well known problems we've already discussed at length.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 21 Jan 2022 09:31:22 -0500

On Friday, January 21, 2022 9:28:07 AM EST John R Levine wrote:
> > publishing a DMARC record involves.  Users of b.example.com need to know
> > that they cannot override the DMARC policy of their org domain unless
> > they want to become the org domain of the whole subtree.
> 

Yes.  Please.  We've already been through all this, let's not do it again.

Scott K

---

From: Barry Leiba <[email-masked]>
Date: Fri, 21 Jan 2022 09:38:11 -0500

> Yes.  Please.  We've already been through all this, let's not do it again.

Indeed: the purpose of this thread is to correct a problem with how
the tree walk is specified.  Revisiting the discussion we had and
resolved before is out of scope unless someone truly has NEW
information that we did not consider before.

Barry

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 21 Jan 2022 18:51:41 +0100

On Fri 21/Jan/2022 15:28:07 +0100 John R Levine wrote:
> Sorry, but that is simply false.  The entire "private domains" part of the PSD 
> is self-appointed PSDs.

Not all of them have write access to https://github.com/publicsuffix/list

> talked about the tree walk in the first place.

I don't remember that we ever convene that subdomains cannot override the org 
domain record.

> Sorry, that makes no sense.  Everyone always does their own DKIM.

Nope.  Lots of senders outsource DKIM signing.  In the above example, it is 
clear that c.example.com has its message signed by a.b.example.com.  Let me 
recall it:

Now, since b.example.com needs to override the DMARC policy of example.com, 
they also have to reconsider their mailout arrangements in order to DKIM sign 
their messages by themselves.

> Um, the whole point of the tree walk is so we do not use the PSL, which has all 
> sorts of well known problems we've already discussed at length.

That's why I said "personally".  I haven't yet thought how to modify my mail 
filter (also because tree walk hasn't been specified yet).  I don't think I'll 
change policy discovery on the day the RFC gets published, and even if I did, I 
don't think both users will immediately upgrade, and I guess several DMARC 
filters are in the same condition.  I hope the tree walk will be specified in a 
compatible way.

Best
Ale
--

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 21 Jan 2022 18:51:47 +0100

On Fri 21/Jan/2022 15:20:17 +0100 Douglas Foster wrote:
> names are aligned.  It eliminates the PSL while also creating no need for PSD 
> policies.

I think you misunderstood both my idea and the role of PSD policies.

Let me try and explain it algorithmically:

0. Set Domain = the From: domain
    Set policy = not found.

1. Lookup _dmarc.Domain.
    If it has a DMARC record:
       If Domain is the From: domain or Domain has role=org:
          Set policy found.
       If Domain has role=psd, they're not aligned.  Stop.
       If Domain matches d=, they're aligned.
       If one of d= and Domain is a subdomain of the other (by string comparison), they're aligned.
       If policy was set:
          Stop.

2. Set Domain = Domain's parent or 5-label grandparent.
    If not empty string,
       Go to 1.

If you find no role=org tags, then, after walking the tree, you can assume role=org for the last policy found.  A role=sub only serves to avoid that assumption.

Note that you need an extra lookup for _dmarc.com or whatever.  In exchange, you never have to walk up the d= or the spf trees.

Best
Ale
--

---

From: "John R Levine" <[email-masked]>
Date: 21 Jan 2022 12:59:16 -0500

> I don't remember that we ever convene that subdomains cannot override the org 

The "org domain record" is now whatever you find in the tree walk.  There 
is nothing to override.  The PSL is gone.

Regards,

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 22 Jan 2022 12:52:26 +0100

On Fri 21/Jan/2022 18:59:16 +0100 John R Levine wrote:
> The "org domain record" is now whatever you find in the tree walk.  There is 
> nothing to override.

No, the concept of Organizational Domain is foundational to DMARC.  We cannot 
overthrow it to spare an extra lookup.  When we talked about tree walk we knew 
that additional lookups might well have come out.

To specify that a.b.example.com and c.example.com are not aligned is wrong.

Subdomains do override their org domain policy, in some cases just to add an 
extra rua address.  This has to still be possible, without disrupting the 
alignment.

Todd managed to specify the tree walk under the assumption that every PSD would 
fulfill our wish to discover a psd=y tag at the end of the walk.  Since that 
won't happen, then perhaps the psd= tag is not enough to specify a working tree 
walk.

I propose role=, as it is more flexible than psd=.  In the example, when 
b.example.com realize (from feedback) what damage their record causes, they can 
set role=sub in order to allow tree walkers to discover the real org domain. 
If Todd devises a better way, he's welcome.

> The PSL is gone.

Long live the king.

Best
Ale
--

---

From: Dotzero <[email-masked]>
Date: Sat, 22 Jan 2022 09:15:02 -0500

On Sat, Jan 22, 2022 at 6:52 AM Alessandro Vesely <[email-masked]> wrote:

<SNIP>

> To specify that a.b.example.com and c.example.com are not aligned is

While this may be your personal opinion, it is not a fact. I have
previously pointed out that there are organizations that lease/rent or
otherwise provide subdomains as part of their commercial offerings. Your
assertion is akin to claiming that tenants in an apartment building are
family relatives simply because they have the same landlord. We can
identify a relationship between a.b.example.com and example.com. We can
also identify a relationship between c.example.com and example.com. In both
of these cases someone with control over example.com created the DNS
records that brought the subdomains into existence. We cannot identify a
meaningful relationship between a.b.example.com and c.example.com because
we have no way of knowing whether such a relationship exists or if they are
independent actors.

If we were to accept your argument in favor of this approach it opens up a
very large attack surface.Attacker finds sites (think blog hosting websites
for example) where multiple independent parties use subdomains. This would
be a great approach for targeted attacks against NGOs, Journalists, etc.

Michael Hammer

---

From: "John R Levine" <[email-masked]>
Date: 22 Jan 2022 10:29:34 -0500

Ale said:
>> No, the concept of Organizational Domain is foundational to DMARC. ...

We went though this whole discussion quite a while ago when we agreed to 
switch to the tree walk.  Unless you have a compelling argument about why 
our previous analysis and decision was wrong, I think we're done.  Keep in 
mind this paragraph from 7489:

    The process of determining a suffix is currently a heuristic one.  No
    list is guaranteed to be accurate or current.

R's,
John

PS:

Mike said:
> I have previously pointed out that there are organizations that 
> lease/rent or otherwise provide subdomains as part of their commercial 
> offerings. Your assertion is akin to claiming that tenants in an 
> apartment building are family relatives simply because they have the 
> same landlord. ...

This misses the point.  It would be a good idea for a multi-tenant domain 
to publish a PSD record to keep the tenants apart, just as it would be a 
good idea to send a PSL pull request to keep them from spoofing browser 
cookies, but I don't think it is a good idea to depend on that.  We know 
that at the TLD level, most TLDs won't ever publish a PSD.

---

From: Douglas Foster <[email-masked]>
Date: Sat, 22 Jan 2022 11:01:57 -0500

If a.b.example.com is considered aligned with c.example.com under RFC7489,
but will be considered unaligned under DMARCbis, then we have a pretty
significant incompatibility and need to move to DMARCv2.

In the current example of comparing a.b.example.com and c.example.com to
each other:  if there is a shared DMARC policy at example.com, why would we
not consider the two names in alignment?

In the case of organizations that lease subdomains, do both the subdomains
and the parent domain send email and publish DMARC policies?   If so, then
we are back to needing a PSL.

On Sat, Jan 22, 2022 at 9:15 AM Dotzero <[email-masked]> wrote:

> also identify a relationship between c.example.com and example.com. In
> both of these cases someone with control over example.com created the DNS

---

From: Dotzero <[email-masked]>
Date: Sat, 22 Jan 2022 11:32:06 -0500

On Sat, Jan 22, 2022 at 11:02 AM Douglas Foster <
[email-masked]> wrote:

> If a.b.example.com is considered aligned with c.example.com under
> RFC7489, but will be considered unaligned under DMARCbis, then we have a
> pretty significant incompatibility and need to move to DMARCv2.
>
> each other:  if there is a shared DMARC policy at example.com, why would
> we not consider the two names in alignment?
>

The alignment is because of the parent domain, not because of the sibling
domains. A distinctly different situation.

> we are back to needing a PSL.

The practice can/will vary. There isn't a need for a PSL if alignment is
not allowed based on a sibling domain's published policy. There would still
be potential problems with alignment based on the parent domain but nowhere
near as ugly.

Michael Hammer

---

From: Douglas Foster <[email-masked]>
Date: Sat, 22 Jan 2022 12:08:21 -0500

To correct my last post, we are not reduced to using a list, as we have
already anticipated this situation with the PSD flag.

The solution to the sub-lease situation is for the parent domain to publish
psd=y.

We may need two PSD flag, one for
"I am a real PSD and never send mail and should not be used for alignment"
and a second one for
"I am a lease-granter but I send my own mail, but I should not be used for
alignment".
I think we can get by with one flag, but am open to either possibility.

About sister domains, we have already litigated the question of sister
domains.   Under RFC 7489, they are considered aligned, although perhaps
not in your implementation.  If we change that interpretation, we change
DMARC versions, but I heard no enthusiasm for doing so.

The broken alignment calculation is an artifact of one proposed revision of
the tree walk.   It is neither the necessary algorithm nor the best one.

DF

On Sat, Jan 22, 2022 at 11:32 AM Dotzero <[email-masked]> wrote:

>> In the case of organizations that lease subdomains, do both the
>> subdomains and the parent domain send email and publish DMARC policies?
>>  If so, then we are back to needing a PSL.
>>
>

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 24 Jan 2022 10:14:13 +0100

On Sat 22/Jan/2022 16:29:34 +0100 John Levine wrote:
>> lease/rent or otherwise provide subdomains as part of their 
>> commercial offerings. Your assertion is akin to claiming that 
>> tenants in an apartment building are family relatives simply because 
>> they have the same landlord. ...
> 
> This misses the point.  It would be a good idea for a multi-tenant 
> domain to publish a PSD record to keep the tenants apart, just as it 
> would be a good idea to send a PSL pull request to keep them from 
> spoofing browser cookies, but I don't think it is a good idea to 
> depend on that.  We know that at the TLD level, most TLDs won't ever 
> publish a PSD.

If we fear that some TLDs won't qualify their role, then it is wrong 
to discourage setting psd=n for organizational domains.  Setting 
role=org is even more expressive, because it provides for role=sub as 
well.

I agree that determining the role of each domain in the tree is an 
heuristic process.  The more info we can gather during the walk, the 
more accurate the heuristic.

Best
Ale
--

---

From: "John R Levine" <[email-masked]>
Date: 24 Jan 2022 09:40:01 -0500

On Mon, 24 Jan 2022, Alessandro Vesely wrote:
> If we fear that some TLDs won't qualify their role, then it is wrong to 
> discourage setting psd=n for organizational domains.  Setting role=org is 
> even more expressive, because it provides for role=sub as well.

I think you're also missing the point -- most TLDs will never publish any 
DMARC record at all.  In that case, how could it make any difference what 
tags they don't put in the records they don't publish?

Regards,

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 24 Jan 2022 17:23:27 -0500

On Wednesday, January 19, 2022 1:38:15 PM EST John Levine wrote:
> I took a look at sections 4.5 and 4.6 of the draft, the part that describes
> the tree walk and PSD, and unfortunately what it currently says is
> seriously wrong.  Apologies for not catching this sooner.
> 
> It currently says you do a tree walk to find a PSD record, and the org
> domain is the one under the PSD.  That's fine if there's a PSD record, but
> in the vast majority of cases there will not.  There is no chance that .COM
> or .ORG or .NET or any of the other public gTLDs will ever publish a DMARC
> record because they are not in the business of managing their customers'
> mail.  (There are also acronym-studded ICANN reasons they couldn't even if
> they wanted to, ask me if you want the details.)
> 
> What I always intended with the tree walk is that you walk up the tree and
> if you find a DMARC record that isn't a PSD, that's your org domain.  To
> see if two names are in relaxed alignment, do a tree walk for both and if
> they end at the same place, they're aligned.  As a special case albeit a
> very common one, if one name is a descendant of the other, and there are no
> DMARC records in between, they're aligned.
> 
> This means that if there's no PSD and no higher level DMARC records, there's
> no org domain, but I believe that we discussed this before and in practice
> it doesn't matter.

Apologies for jumping back to the top of the thread, but I've lost track of 
where we are on this.

I took some time to try and code this up (it's not pretty, no judgment 
please), so the git version of authheaders has grown a new script called 
dmarc-policy-find[1].  If you know how to install python modules from source, 
you can try it yourself.

I tried to allow one to query via this script using any of the RFC 7489 
(DMARC), RFC 9091 (PSD), or DMARCbis methods for policy and organizational 
domain discovery.  In attempting to implement the dmarcbis-04 methods I did 
run into the problem that John describes and it's not clear what to do for 
cases where the PSD record is not present.

What I implemented is roughly:

For policy determination, first check the From domain, if that has a DMARC 
record, then that's the policy domain.  Otherwise, tree walk up to the apex 
looking for DMARC records.  First domain you find with a record is policy 
domain, use the policy (p=, sp=, np=) from that domain's DMARC record.  This 
matches my interpretation of dmarcbis-04.

For org domain determination (for alignment), if any of the records retrieved 
during the policy search have psd=y, then add one more label and that's the 
org domain (as written).  From there it's anyone's guess.  Unlike John, I 
continued down the tree and made the first match the org domain.  

I agree it doesn't matter much if you go down the tree or up the tree (for the 
number of steps specified in the tree walk).  There may be a corner case where 
down the tree is slightly more compatible with RFC 7489 results, but I haven't 
found a real life example of it.

Scott K

[1] https://github.com/ValiMail/authentication-headers

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 24 Jan 2022 17:27:14 -0500

On Monday, January 24, 2022 5:23:27 PM EST Scott Kitterman wrote:
> > This means that if there's no PSD and no higher level DMARC records,
> > there's no org domain, but I believe that we discussed this before and in
> > practice it doesn't matter.
> 
> dmarc-policy-find[1].  If you know how to install python modules from
> source, you can try it yourself.
> 
> For org domain determination (for alignment), if any of the records
> retrieved during the policy search have psd=y, then add one more label and
> that's the org domain (as written).  From there it's anyone's guess. 
> Unlike John, I continued down the tree and made the first match the org
> domain.
> 
> I agree it doesn't matter much if you go down the tree or up the tree (for
> the number of steps specified in the tree walk).  There may be a corner
> case where down the tree is slightly more compatible with RFC 7489 results,
> but I haven't found a real life example of it.
> 
> Scott K
> 

I forgot to mention:

Since no one publishes psd=y yet, I cheated and internally pretend that any 
domains on the psddmarc registry have psd=y, so I could test it.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 24 Jan 2022 19:46:48 -0500

>continued down the tree and made the first match the org domain.  

Seems reasonable.  What's the point of going more than one level below the PSD?
Make it look more like a pure tree walk?

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 25 Jan 2022 03:15:49 +0000

>Make it look more like a pure tree walk?

Yes.  For consistency.  You'd walk down until you hit a non-psd record or the limit.  Stopping at one more after the psd=y record is an optimization for a relatively rare case of a PSD record.  Other than that case you have to keep going until you find a DMARC record or hit the limit, since there's no knowing what's a PSD otherwise.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 25 Jan 2022 00:56:26 -0500

On Monday, January 24, 2022 10:15:49 PM EST Scott Kitterman wrote:
> >>domain, use the policy (p=, sp=, np=) from that domain's DMARC record. 
> >>This matches my interpretation of dmarcbis-04.
> >>
> >>retrieved during the policy search have psd=y, then add one more label
> >>and that's the org domain (as written).  From there it's anyone's guess. 
> >Seems reasonable.  What's the point of going more than one level below the
> >PSD? Make it look more like a pure tree walk?
> 
> Yes.  For consistency.  You'd walk down until you hit a non-psd record or
> the limit.  Stopping at one more after the psd=y record is an optimization
> for a relatively rare case of a PSD record.  Other than that case you have
> to keep going until you find a DMARC record or hit the limit, since there's
> no knowing what's a PSD otherwise.

The attached change would solve the problem, at least to a first approximation.  
The wording could be tightened up, but this is at least a complete 
description.

Scott K
diff --git a/draft-ietf-dmarc-dmarcbis-05.xml b/draft-ietf-dmarc-dmarcbis-05.xml
index 448a156..9dd68a3 100644
--- a/draft-ietf-dmarc-dmarcbis-05.xml
+++ b/draft-ietf-dmarc-dmarcbis-05.xml
@@ -509,12 +509,16 @@ itself as such using a tag in its published DMARC policy record. An Organization
 Domain is any subdomain of a PSD that includes exactly one more label than
 the PSD in its name.</t>
 <t>For any email message, the Organizational Domain of the RFC5322.From domain
-is determined by performing a DNS Tree Walk as described in <xref target="dns-tree-walk"></xref>.
+is determined by performing a DNS Tree Walk in the reverse order described in <xref target="dns-tree-walk"></xref>.
 The target of the search is a valid DMARC record that contains a psd tag with
 a value of 'y'. Once such a record has been found, the Organizational Domain
 for the DNS domain matching the one found in the RFC5322.From domain can be
-declared to be the target domain queried for in the step just prior to the
-query that found the PSD domain.</t>
+declared to be the target domain queried for in the next step just after the
+query that found the PSD domain.  If the first DMARC record returned does not
+contain a psd tag with a value of 'y', that domain is the Organizational Domain.
+If the maximum depth of the <xref target="dns-tree-walk"></xref> is reached
+without a DMARC record being located, the RFC5322.From domain is the
+Organizational Domain.</t>
 <t>For example, given the RFC5322.From domain &quot;a.mail.example.com&quot;, a series
 of DNS queries for DMARC records would be executed starting with
 &quot;_dmarc.a.mail.example.com&quot; and finishing with &quot;_dmarc.com&quot;. The &quot;_dmarc.com&quot;

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 25 Jan 2022 12:14:11 +0100

On Mon 24/Jan/2022 15:40:01 +0100 John Levine wrote:
>>> domain to publish a PSD record to keep the tenants apart, just as 
>>> it would be a good idea to send a PSL pull request to keep them 
>>> from spoofing browser cookies, but I don't think it is a good idea 
>>> to depend on that.  We know that at the TLD level, most TLDs won't 
>>> ever publish a PSD.
>>
>> role=org is even more expressive, because it provides for role=sub 
>> as well.
> 
> I think you're also missing the point -- most TLDs will never publish 
> any DMARC record at all.  In that case, how could it make any 
> difference what tags they don't put in the records they don't publish?

For the time being, TLDs with a DMARC record can be counted on the 
fingers of one hand, so they could be checked on a list.  Still, 
finding role=psd (or psd=y) adds more confidence to the heuristic.

The last domain with a DMARC record should be the org domain.  Still, 
finding role=org (or psd=n) adds more confidence to the heuristic.

I'm not opposed to the decision to switch from a PSL based heuristic 
to a tree walk based one.  I'm opposed to specifying an heuristic 
which is worse than the previous one.

Best
Ale
--

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 25 Jan 2022 12:30:51 +0100

> The wording could be tightened up, but this is at least a complete

I don't think "in the reverse order" makes much sense.  Usually, I 
don't walk downward on a DNS, so the meaning of "reverse" is not clear 
to me.  Phrases like "toward the root" or similar might be clearer.

Then, may I insist on having role=psd rather than psd=y?  It can 
better the heuristic.

Thanks
Ale
--

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 25 Jan 2022 11:47:21 +0000

>Then, may I insist on having role=psd rather than psd=y?  It can 

I agree it needs work.  I think the tree walk section needs to be updated to be bidirectional.  Walk up to find policy and walk down to find org.  I'm only suggesting that this be added for now as an interim step to fix a clear gap in -04.

My impression is that the group is generally okay with PSD=y.  I prefer it over your suggestion.  My strongest preference is that we pick something, stick with it, and move on.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 25 Jan 2022 18:36:23 +0100

> I agree it needs work.  I think the tree walk section needs to be updated to be bidirectional.  Walk up to find policy and walk down to find org.  I'm only suggesting that this be added for now as…

As John said, the gap is that PSD domains are not going to publish 
psd=y.  Maybe, eventually, the ones interested in DMARC will. 
However, this is going to be a problem during rollout.

> My impression is that the group is generally okay with PSD=y.  I prefer it over your suggestion.  My strongest preference is that we pick something, stick with it, and move on.

Would you mind elaborating on why you prefer psd= over role=, apart 
from the latter arriving late?

Besides, do you agree that finding psd=n can save a lookup in some cases?

Best
Ale
--

---

From: "John Levine" <[email-masked]>
Date: 25 Jan 2022 12:40:09 -0500

>My impression is that the group is generally okay with PSD=y.  I prefer it over your suggestion.  My strongest preference is that we pick something, stick with it, and move on.

I think I see where Ale's confusion is coming from. If we switch to a
tree walk, we will have an algorithm rather than a heuristic, so
anyone looking at the same domains and the same set of DMARC records
will get the same result. It also occurs to me that in the absence of
a PSL-like thing, the idea of an organizational domain is no longer
useful.

There's two questions to answer: what is the policy for a domain, and are two domains in relaxed alignment.

The answer to the first one is straightforward: start at the domain, walk up the tree, and the first DMARC record
you find is the policy record.  If you don't find one, there's no policy.

The answer to the second has two cases:

If one domain is a subdomain of the other, and there is no policy record (or maybe no PSD policy record) between
them, they're in relaxed alignment.

If they are cousin domains, walk up the tree from each until you find a policy record.  If you find the same policy
record and it's not a PSD and it allows relaxed alignment, they're in relaxed alignment.  If you find different
records, or only one record, or no records, they aren't.

As a special case, a domain with a PSD record is never aligned with anything but itself.
(I realize .bank will never send mail, but us.com might.)

The cousin domain rule doesn't exactly reproduce what the PSL is intended to do, but I think it covers
the useful cases and is unlikely to allow accidental cousin alignment which Mike keeps reminding us about.

Suggestions and tweaks (with an explanation of what problem they fix) welcome.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 25 Jan 2022 17:58:11 +0000

>Besides, do you agree that finding psd=n can save a lookup in some cases?

I think psd is more precisely constrained to the need.  Role is rather open ended.  I have no idea what it covers.

I think arriving late and not having a clear technical advantage is enough.  If this WG is ever going to achieve its goals, we need to decide stuff and not keep revisiting things.

Ideally, I'd like to get this nailed down so we can get an early (RFC 7120) assignment of the tag.  This will help us with getting things moving forward.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 25 Jan 2022 18:03:12 +0000

>Suggestions and tweaks (with an explanation of what problem they fix) welcome.

I think this is generally correct.  Can be used for relaxed alignment was always the important thing for organizational domain anyway.  I don't know that I'd bother to create the term now, but we already have it and people sort of know what it means, so I think we might as well keep it.

Scott K

---

From: Dotzero <[email-masked]>
Date: Tue, 25 Jan 2022 13:28:51 -0500

On Tue, Jan 25, 2022 at 12:40 PM John Levine <[email-masked]> wrote:

> >My impression is that the group is generally okay with PSD=y.  I prefer
> it over your suggestion.  My strongest preference is that we pick
> something, stick with it, and move on.
>

I have no problem with this. Those of us who originally created DMARC
considered this the use case for relaxed.

>
> If they are cousin domains, walk up the tree from each until you find a
> policy record.  If you find the same policy
> I think a better term is sibling domains. The phrase "cousin domains" has
typically been used for look alike domains rather than the subdomain issue.

> which Mike keeps reminding us about.

It actually does allow malicious, not accidental, alignment. I'm done
reminding. This allows an attack vector which can be useful for BEC
attacks, hostile governments targeting NGOs, journalists, etc. and other
targeted attacks. It is not a particularly useful attack vector for large
scale opportunistic abuse such as spam or widespread attempts to spread
malware. The group will address it or not as it chooses. I've been working
on developing real world attack example (defanged) and have just started
discussing the issue with various (red team) security folks I know and
reaching out to some .gov folks I know.  bSidesLV may be held this year and
I may present on this there or at other venues. If allowing alignment and a
pass based on a sibling domain is allowed in DMARC then the best defense is
for people to understand that there are potential real world risks in
relying on a DMARC pass in relaxed mode.

Michael Hammer

---

From: "John R Levine" <[email-masked]>
Date: 25 Jan 2022 13:35:21 -0500

> typically been used for look alike domains rather than the subdomain issue.

Agreed, sibling is better, although of course they could be great-aunts, 
too.

> targeted attacks.

I don't have strong opinions about whether to continue to allow great-aunt 
alignment other than to note this is such a well known problem that it is 
exactly the problem the PSL was invented to address, and we can argue 
about how well the PSL and other widely available mitigation techniques 
work and how reasonable it is to expect people to use them.

Do we have any stats on how often real mail depends on sibling alignment? 
If nobody actually uses it, the spec would be simpler if we could take it 
out.

Regards,

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 25 Jan 2022 11:21:16 -0800

> a PSL-like thing, the idea of an organizational domain is no longer

Aren't we basically trying to identify the same thing, just in a different
(and more robust) way?

I'd be careful about saying "no longer useful"; doing so implies a much
bigger change than we might be aiming to "sell" here.

-MSK

---

From: "John R Levine" <[email-masked]>
Date: 25 Jan 2022 14:26:15 -0500

> Aren't we basically trying to identify the same thing, just in a different

Yes, but that leads to the question of whether the org domain is useful on 
its own or it's just a way to figure out alignment.  I think it's the 
latter, dunno what other people think.

Regards,

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 25 Jan 2022 11:39:11 -0800

> latter, dunno what other people think.

It's an interesting thought exercise at least.  We should be sure to give a
decent treatment to this in the "differences since RFC 7489" part of the
document.

-MSK

---

From: "John Levine" <[email-masked]>
Date: 25 Jan 2022 16:24:36 -0500

>As John said, the gap is that PSD domains are not going to publish 

No, that is not at all what I said.

Most PSDs will publish no DMARC record at all.  Based on what Scott has said,
the handful that do publish a DMARC record will indeed include psd=y.

My point is that our tree walk has to have reasonable defaults in the likely
case that org and PSD domains won't have a DMARC record.

R's,
John

---

From: Douglas Foster <[email-masked]>
Date: Tue, 25 Jan 2022 18:30:40 -0500

>
>
> DMARC for PSD is based on the rule that the PSD is one segment above the
> organization domain, and the organization domain is assumed to be known
> with confidence from the PSL.

> When we switch directions, we cannot as easily assume that the
> organization domain is one segment below PSD=Y.    For this to be true,
> PSDs MUST populate their DMARC records from the lowest-level leaf nodes
> up.   For example, if "c.b.a" is a PSD, and "z.b.a" is an organization,
> then "c.b.a" must be flagged before "b.a" is flagged.    If "b.a" is
> flagged first, then "c.b.a" will be treated as an organization domain,
> incorrectly, because "c.b.a" is below the flagged "b.a" record.     The
> requirement to work bottom-up is contrary to the way that I expect people
> to address a problem like this.  In some cases tagging all of the leaf
> nodes may be especially problematic, such as when the PSL record says that
> "*.something" is a PSD, except for "www.something".

If we provide an org=y flag as Ale suggests, the problem can be mitigated
if organizations choose to document the top of their alignment structure
using the org flag.   But this mitigation cannot help the situation where
the PSD policy is being applied because an organization does not have a
policy of its own.

An overarching problem is that the PSD structure is not latent in the
current DNS, waiting for us to draw it out.   We have to make assumptions
that the DNS will contain data that can support our objectives, and that
organizations will choose to migrate their configuration to match
our assumptions.   For example, we want to assume that every DMARC-using
domain will have an existing policy record at the organization level.
 This is probably typical, but it is certainly not required by the design
of RFC 7489.    Overall, I doubt that we can replace the PSL without moving
to DMARCv2, and I don't think we have a standards-worthy document unless
the PSL is replaced.

Doug  (aka gloomy Eeyore today)

.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 25 Jan 2022 22:07:59 -0800

On Tue, Jan 25, 2022 at 3:31 PM Douglas Foster <
[email-masked]> wrote:

> Overall, I doubt that we can replace the PSL without moving to DMARCv2,
> and I don't think we have a standards-worthy document unless the PSL is
> replaced.
>

I don't think such an opinion is going to ruffle any feathers in here.  The
PSL has been non-ideal since the beginning, but it's all we had to work
with at the time.  RFC 7489 anticipated its replacement someday with
something better.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 26 Jan 2022 19:13:45 +0100

On Tue 25/Jan/2022 20:39:11 +0100 Murray S. Kucherawy wrote:
>>> Aren't we basically trying to identify the same thing, just in
>>> a different (and more robust) way? >>

The concept of Organizational Domain is a key concept, as it is easy 
to understand, based on intuitive set theory ideas.  With respect to 
earlier SPF experience, which required tagging each and every domain 
name in an organization, it is an invaluable progress.

Replacing it with a difficult graph-theoretical argument is going to 
make the notion of alignment much more difficult to explain.  The mere 
fact that adding a DMARC record can change alignment properties turns 
such a simple operation into something that requires an extensive 
analysis before it can be carried out.

If we drop Organizational Domain we're probably better off dropping 
alignment as well.  We're worsening DMARC in that case.

Best
Ale
--

---

From: Douglas Foster <[email-masked]>
Date: Sat, 29 Jan 2022 17:51:55 -0500

If we start from Dave Crocker's assumption that we will have at least two
algorithms, defined somewhere, then I find Michael Hammer's argument
compelling:   We should encourage migration to an environment that requires
a parent-child relationship for relaxed alignment.

The parent-child requirement virtually eliminates the risk of false
positives, based on two assumptions:
(1) that upward attacks are unlikely because registrants have strong
incentives to not attempt malicious impersonation of the entity that
registered them, whether the registrar is a true PSD or a lease grantor, as
this would likely lead to de-registration,
(2) that downward attacks are unlikely because PSDs do not send mail and
lease grantors are not likely to attack their lease-holders, as this
undermines the trust needed by their business model.

Additionally, the parent-child requirement seems essential to the success
of a simplified non-PSL algorithm.

I also suggest that removing siblings from relaxed alignment will not cause
many objections.  I have never seen a message that depends on sibling
authentication, and no one else has submitted any data suggesting that it
is commonly used.

Consequently, I see us moving to three documented algorithms:
- the original RFC 7489 algorithm based on PSL with siblings allowed,
- a modified algorithm based on PSL but with parent-child required, and
- a simplified algorithm that does not use the PSL and has parent-child
required.

To that end, it seems desirable to give domain owners a DNS flag to
indicate that they have embraced this new approach and do not want sibling
relationships to be used for relaxed authentication.

Similarly, the reporting specification should be tweaked to allow
evaluators to report which of these three algorithms were used for DMARC
evaluation.

If domain owners notice that many evaluators do not accept sibling
authentication, they will necessarily take measures to eliminate
sibling-dependent authentication from their mail stream.
Similarly, when evaluators see that some domain owners explicitly reject
sibling authentication, and others do not use it, they will become willing
to stop using sibling relationships in their evaluation.

Doug Foster