Subject: [dmarc-ietf] NXDOMAIN

From: Douglas Foster <[email-masked]>
Date: Mon, 5 Apr 2021 17:02:16 -0400

As a result of earlier discussions, I have been investigating NXDOMAIN as
an email filtering criteria.

One question from those discussions was the best way to detect NXDOMAIN.  I
realized that I needed a query which specifically returns NXDOMAIN as a
result, not simply the absence of a particular result.   Additionally, a
lookup on A/AAAA with results could represent either a domain name with no
host segment, or a host segment and a parent domain..   Consequently, the
best test seems to query for type=TXT, match=domainname.

I have applied this rule to incoming RFC5322.MailFrom addresses and found
the test to be useful.  For my mail stream, 20% of the messages with
SPF=NONE have this result because of NXDOMAIN.  The percentages were
roughly equal whether evaluating unique domain names or unique messages.

While both SPF=NONE and SPF=NXDOMAIN are indicators that the message is
probably unwanted, NXDOMAIN has a higher probability of being unwanted.

I have not yet begun evaluating NXDOMAIN on the RFC5322.From address, but
hope to get that done in the weeks ahead.

Is anyone else collecting data on NXDOMAIN, and able to share experience?

---

From: Todd Herr <[email-masked]>
Date: Mon, 5 Apr 2021 17:46:13 -0400

On Mon, Apr 5, 2021 at 5:02 PM Douglas Foster <
[email-masked]> wrote:

> One question from those discussions was the best way to detect NXDOMAIN.
> I realized that I needed a query which specifically returns NXDOMAIN as a

I can share experience...

Several jobs ago, when I was in position to set anti-spam policy for a
mid-sized US-based cable ISP, if the RFC5321.From or RFC5322.From domain
lacked both an MX record and an A/AAAA record, that was enough for me to
reject the mail on the grounds that I was unwilling to accept anything to
which the recipient could not reply. I don't recall any complaints from my
side of the transaction.

I'm not sure I'd make the same decision based on the lack of an SPF record,
especially for the RFC5322.From domain, since SPF is keyed to the
RFC5321.From domain (or the EHLO name), and so I have no expectations that
an SPF record will exist for the RFC5322.From domain. Even a lack of SPF
for the RFC5321.From domain doesn't, for me, automatically rule out a
message. I'd be more inclined to reject mail where the SPF record ended in
"+all" than I would where there was no SPF record, because "+all" lands
differently for me than does "no SPF record published". Both say that the
domain owner doesn't care about SPF, but "+all" strikes me as overt
disregard for SPF, while a lack of a record might just signal less
hostility.

Lack of an SPF record eliminates one possible path to a DMARC pass verdict,
and a DMARC pass verdict allows me to treat the mail as authentic, and
therefore worthy of treatment earned by previous authentic messages for the
DMARC domain, which could be good or bad, depending on the history.
Authenticated mail is not necessarily wanted mail; it's just mail for which
I can be reasonably sure of the identity of the responsible party, and if
that responsible party has a history of sending unwanted mail, then I can
treat its authenticated mail as unwanted. If I can't determine if the mail
is authentic, then I will treat it no better than, and perhaps worse than,
the treatment earned by previous authentic messages for the DMARC domain,
assuming any previous authentic messages exist.

-- 

*Todd Herr* | Sr. Technical Program Manager
*e:* [email-masked]
*m:* 703.220.4153

This email and all data transmitted with it contains confidential and/or
proprietary information intended solely for the use of individual(s)
authorized to receive it. If you are not an intended and authorized
recipient you are hereby notified of any use, disclosure, copying or
distribution of the information included in this transmission is prohibited
and may be unlawful. Please immediately notify the sender by replying to
this email and then delete it from your system.

---

From: Tim Wicinski <[email-masked]>
Date: Mon, 5 Apr 2021 17:46:45 -0400

Can you point me to some of the tests you are running? you can do that
offline.

you also want to do some testing with domains signed with DNSSEC and those
without.

This came up as an issue in opendmarc:

https://github.com/trusteddomainproject/OpenDMARC/issues/103#issuecomment-810036114

On Mon, Apr 5, 2021 at 5:02 PM Douglas Foster <
[email-masked]> wrote:

> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

---

From: Grant Taylor <[email-masked]>
Date: Mon, 5 Apr 2021 15:56:22 -0600

On 4/5/21 3:46 PM, Todd Herr wrote:
> reject the mail on the grounds that I was unwilling to accept anything 
> to which the recipient could not reply.

I'm surprised at requiring an /MX/ record.  I still see email routed to 
A / AAAA records for the domain when there isn't an MX record for said 
domain.

I wouldn't give your statement another thought if it was "an MX record 
or an A record or an AAAA record".  But /requiring/ an /MX/ record is a 
toe stubber for me.

-- 
Grant. . . .
unix || die

---

From: Todd Herr <[email-masked]>
Date: Mon, 5 Apr 2021 18:08:01 -0400

> or an A record or an AAAA record".  But /requiring/ an /MX/ record is a
It appears my phrasing was confusing, and for that I apologize.

I did not require an MX record, per se; the requirement was that there be
either an MX record or, failing that, an A/AAAA record.

Either one was sufficient, but if neither existed, then the mail was
rejected.

--

---

From: Douglas Foster <[email-masked]>
Date: Mon, 5 Apr 2021 18:32:40 -0400

Please forgive my typo.  I meant to say that I am currently testing
RFC5321.MailFrom (only).

(I am not doing evaluating any non-standard SPF algorithms.)

On Mon, Apr 5, 2021, 5:46 PM Todd Herr <[email-masked]> wrote:

> I'm not sure I'd make the same decision based on the lack of an SPF
> record, especially for the RFC5322.From domain, since SPF is keyed to the
> Lack of an SPF record eliminates one possible path to a DMARC pass
> verdict, and a DMARC pass verdict allows me to treat the mail as authentic,
> and therefore worthy of treatment earned by previous authentic messages for
> the DMARC domain, which could be good or bad, depending on the history.

---

From: "John Levine" <[email-masked]>
Date: 5 Apr 2021 20:45:42 -0400

>which the recipient could not reply. I don't recall any complaints from my

On my small system I have the same experience.  Nobody sends mail that you
want from nonexistent domains.

>I'm not sure I'd make the same decision based on the lack of an SPF record,

I certainly wouldn't. I get plenty of mail from addresses with no SPF
record. I think some of it is even DKIM signed.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 5 Apr 2021 18:00:43 -0700

> either an MX record or, failing that, an A/AAAA record.

Given that this is what RFC 5321 Section 5.1 prescribes in terms of
answering the question "Where should I try to connect to deliver this?",
that seems like a sane thing to use as a test.

-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 5 Apr 2021 18:07:48 -0700

> Is anyone else collecting data on NXDOMAIN, and able to share experience?

Due to terminology pedantry, I'm having trouble understanding this.
Someone please check my math here:

If I ask for MX records for example.com, NXDOMAIN comes back as the
response's error code not if there's no MX record for that name, but rather
if there are no records of any kind for that name.  On the other hand, if
there's no MX but there is an A or AAAA, I'm going to get a success error
code (NOERROR), but with an answer count of 0.

So I don't know what "detect NXDOMAIN" means: Your DNS reply either has
that error code, or it doesn't.

If instead you're trying to determine whether a name can receive mail, at
least according to DNS data, it seems to me you query one record type (MX)
and see if you get >0 answers, 0 answers, or NXDOMAIN.  If you got
NXDOMAIN, you're done; there's no way to route this message.  If you got 0
answers, you should query A and/or AAAA and send there.  If you got >0
answers, now you have to resolve the names you got to addresses; assuming
at least one of those resolves, you have someplace to send the message.
This is what RFC 5321 says to do.

I don't think querying TXT would tell you anything more.

What am I missing?

-MSK

---

From: Grant Taylor <[email-masked]>
Date: Mon, 5 Apr 2021 19:57:37 -0600

> It appears my phrasing was confusing, and for that I apologize.

Mistakes happen every day ending in "y".  Thank you for clarifying.

> I did not require an MX record, per se; the requirement was that there 
> be either an MX record or, failing that, an A/AAAA record.
> 

ACK

That's more like what I was thinking was the norm.  Thank you for 
clarification ~> confirmation.

-- 
Grant. . . .
unix || die

---

From: Laura Atkins <[email-masked]>
Date: Tue, 6 Apr 2021 09:40:59 +0100

> On 5 Apr 2021, at 22:46, Todd Herr <todd.herr=[email-masked]> wrote:
> 
> On Mon, Apr 5, 2021 at 5:02 PM Douglas Foster <[email-masked] <mailto:[email-masked]>> wrote:
> As a result of earlier discussions, I have been investigating NXDOMAIN as an email filtering criteria.
> 
> One question from those discussions was the best way to detect NXDOMAIN.  I realized that I needed a query which specifically returns NXDOMAIN as a result, not simply the absence of a particular result.   Additionally, a lookup on A/AAAA with results could represent either a domain name with no host segment, or a host segment and a parent domain..   Consequently, the best test seems to query for type=TXT, match=domainname.
> 
> I have applied this rule to incoming RFC5322.MailFrom addresses and found the test to be useful.  For my mail stream, 20% of the messages with SPF=NONE have this result because of NXDOMAIN.  The percentages were roughly equal whether evaluating unique domain names or unique messages.  
> 
> While both SPF=NONE and SPF=NXDOMAIN are indicators that the message is probably unwanted, NXDOMAIN has a higher probability of being unwanted.
> 
> I have not yet begun evaluating NXDOMAIN on the RFC5322.From address, but hope to get that done in the weeks ahead.   
> 
> Several jobs ago, when I was in position to set anti-spam policy for a mid-sized US-based cable ISP, if the RFC5321.From or RFC5322.From domain lacked both an MX record and an A/AAAA record, that was enough for me to reject the mail on the grounds that I was unwilling to accept anything to which the recipient could not reply. I don't recall any complaints from my side of the transaction.
> 
> I'm not sure I'd make the same decision based on the lack of an SPF record, especially for the RFC5322.From domain, since SPF is keyed to the RFC5321.From domain (or the EHLO name), and so I have no expectations that an SPF record will exist for the RFC5322.From domain.

There is a huge amount of dis- and mis-information about SPF for the 5322.from domain. Many, and I mean an embarrassingly high number of ‘many’, ESPs actually suggest that their customers publish include: _spf.espdomain.example in their 5322.from domain TXT record. This eventually causes operational issues due to more than 10 DNS lookups. 

Some of this was done because the ESP folks don't understand the SPF spec. But some of this was done because a large ISP decided to check SPF records on the 5322.from domain and delivery would suffer if there was no SPF. 

Misapplying SPF to the 5322.from is a bad idea and only compounds mistakes of the past. 

> Even a lack of SPF for the RFC5321.From domain doesn't, for me, automatically rule out a message. I'd be more inclined to reject mail where the SPF record ended in "+all" than I would where there was no SPF record, because "+all" lands differently for me than does "no SPF record published". Both say that the domain owner doesn't care about SPF, but "+all" strikes me as overt disregard for SPF, while a lack of a record might just signal less hostility. 

I agree with the sentiment here. Lack of any SPF record, particularly for small and non-bulk mail, simply means someone stood up a mailserver (or started using a service) without fully setting it up. That could be a mistake, or intentional, or malicious and experience tells us all of those are possible. Contrast that with a +all record, where experience tells us that is usually malicious. I actually found a +all in the wild in the past few weeks. It was in the process of investigating a client’s customer due to suspected spam. The +all was yet more confirmation that the customer was a spammer (as if the swedish hashbusting text and CAN SPAM violations weren’t enough). 

laura 

-- 
Having an Email Crisis?  We can help! 800 823-9674 

Laura Atkins
Word to the Wise
[email-masked]
(650) 437-0741		

Email Delivery Blog: https://wordtothewise.com/blog

---

From: Douglas Foster <[email-masked]>
Date: Tue, 6 Apr 2021 07:54:09 -0400

No typo.   I was looking to do better than SPF NONE, and NXDOMAIN was the
first thing that came to mind.

If the SPF Policy lookup returns NXDOMAIN, then we are at a full stop with
all the information needed to make a decision.   (The sender is violating
ICANN name registration policies).   Ignoring NXDOMAIN and continuing to
look for MX/A/AAAA is a waste of information and a waste of resources.

But clearly, the norm in this group is to check MX/A/AAA because it seems
likely to be a more powerful filter.   I wonder if that is actually true.
1)  A/AAAA is a pretty weak test, since many domains have A/AAAA records on
the domain name for web purposes.
2) Existing domain names that are being used for attacks are likely to have
an MX record, if for no other reason than updating a LetsEncrypt
certificate.
 Will someone collect data on frequency that a domain fails MX/A/AAA when
the domain exists?

Additionally, the connection between mail sending (SPF) and mail receiving
(MX/A/AAAA) seems tenuous to me.   Well-run domains do not send automatic
messages to unauthorized return paths, for fear of backscatter.   Many
domains seem to discard incoming automatic messages, either for fear of
attack or because the reply is too hard to integrate into automated systems.

I do respond to SPF NONE by applying a best-guess SPF policy which includes
MX and A, and sometimes produces SPF PASS.   But I no longer do that for
non-existent domains.

I can imagine collecting MX and A existence data while evaluating the
default SPF policy, but I worry about the complexity of doing so, and doubt
the benefit.   Doing it as a separate step is easy enough, but adds
unwanted overhead.

Overall, we have these result partitions for domains with SPF NONE:
1) Domain does not exist, NXDOMAIN
2) Domain exists but MX/A/AAAA do not exist
3) Domain exists and MX/A/AAAA exist but do not validate the source IP
4) Domain exists and source IP can be validated with MX/A/AAAA, producing
an inferred SPF PASS.

I want to be convinced that result #2 is a non-trivial volume of messages.
  If #2 is a trivial volume, then there is no loss from combining it with
#3, which makes the extra MX/A/AAAA test unnecessary

Convince me.

Doug.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 6 Apr 2021 10:58:26 -0700

> look for MX/A/AAAA is a waste of information and a waste of resources.

I agree.

> likely to be a more powerful filter.   I wonder if that is actually true.

In the context of the code doing the SPF evaluation, I don't think it is.
If TXT returns NXDOMAIN for a name, so will any other type.  That's the
definition of NXDOMAIN; there are no data of any type for this name.  See
also RFC 8020.  Fortunately, a properly functioning local nameserver will
cache the answer and just repeat it when subsequent MX, A, or AAAA queries
get issued, so the waste is relatively cheap.

I imagine some DNS APIs are ambiguous (i.e., lazy) about reporting "That
name does not exist (rcode=NXDOMAIN)" differently from "That name exists,
but there's no record of the type you requested (rcode=NOERROR,
ancount=0)", which can result in some wasted time, but I expect the end
result would be the same.

> the domain name for web purposes.

Independently of SPF, it's not a weak test since the standards allow
exactly this kind of setup for a domain that wants to receive mail.  Again,
Section 5.1 of RFC 5321.

> MX and A, and sometimes produces SPF PASS.   But I no longer do that for

"Best guess SPF" is discouraged.  See
http://www.open-spf.org/FAQ/Best_guess_record/.

-MSK

---

From: Douglas Foster <[email-masked]>
Date: Wed, 7 Apr 2021 07:04:20 -0500

I am hearing that we have a defacto standard to check return-path using
MX/A/AAAA.  It is implemented with sufficient breadth,that (unlike SPF)
senders should consider it mandatory.    Additionally, and more importantly
for this group, it is presumed to be equally applicable for validation of
the RFC5322.From, even though that address is unrelated to the
SMTP return-path.  But because the test is not explicitly documented, some
products (my vendors) do not implement it at all, leaving a protection
gap.

As best I can tell, return-path-check does not have a unique name,
specified result status codes, or a specific SMTP reject result code.    At
the simplest level, possible results are PASS, FAIL, TEMPERROR and
PERMERROR.   A more complex result set would probably be desirable for A-R,
to indicate which entity produced the pass, which address class was used
for the match, whether the match also resolved to the Source IP, and the
Source IP itself.    Other specification details include clarifying whether
the MX/A/AAA address result must match the Source IP address class
(presumably yes), and whether the result list should test for and reject
addresses that are in the Private, Multicast, or Reserved lists (probably
optional)

If DMARC is to be dependent on return-path-check and inter-dependent with
ARC, then I do not see how we can avoid producing a formal specification
for the test and integrating it into A-R, as a co-requisite to the DMARC
specification.

Doug Foster

On Tue, Apr 6, 2021 at 12:58 PM Murray S. Kucherawy <[email-masked]>
wrote:

>> If the SPF Policy lookup returns NXDOMAIN, then we are at a full stop
>> with all the information needed to make a decision.   (The sender is
>> violating ICANN name registration policies).   Ignoring NXDOMAIN and
>> continuing to look for MX/A/AAAA is a waste of information and a waste of
>> resources.
>>
>
> I agree.
>
> 1)  A/AAAA is a pretty weak test, since many domains have A/AAAA records
>> on the domain name for web purposes.
>>
>
> I do respond to SPF NONE by applying a best-guess SPF policy which
>> includes MX and A, and sometimes produces SPF PASS.   But I no longer do
>> that for non-existent domains.
>>
>

---

From: Todd Herr <[email-masked]>
Date: Wed, 7 Apr 2021 10:50:03 -0400

> for the test and integrating it into A-R, as a co-requisite to the DMARC
I'm of the opinion that you're overthinking things, and that you're trying
to bolt things onto DMARC that don't need to be bolted onto DMARC.

The purpose of DMARC is simple. It attempts to establish the authenticity
of the claimed identity of the RFC5322.From domain, and does this by
requiring either an aligned pass verdict for SPF or an aligned pass verdict
for DKIM. The results of DMARC validation checks usually lead to one of
three paths:

   1. If the domain is proven authentic through these checks, then I, as
   the receiver of the message, can reliably add this message to the set of
   data points that I maintain that make up the domain's reputation (for
   whatever that means) and I can reliably apply local policy rules to that
   message based on that domain's accumulated reputation.
   2. If the domain publishes a DMARC policy and the message fails DMARC
   checks, I can use the policy record, if I choose, as input into the message
   handling decision that I ultimately make.
   3. If the domain doesn't publish a DMARC policy record, then I've got
   less information to go on for the message, and so I'm likely to focus on
   the source IP's reputation, unless my own internal systems are so
   sophisticated and robust enough for me to track reputation data for
   unauthenticated traffic. We haven't yet reached a point in the ecosystem
   where "no auth, no entry" is in widespread adoption, and so lack of a DMARC
   record is not likely to be a reason to reject a message.

DMARC is just a data point to add to other data points that ultimately
determine what handling a given message will receive. At some sites, I'd
posit that the majority of inbound messages never make it to the DMARC
check stage, because the messages are rejected prior to the issuance of the
DATA command.

If I were still engaged in the operations side of email, especially at a
mailbox provider, I personally would see no need to log anything about the
results of the MX/A/AAAA check, whether it's done for RFC5321.From or
RFC5322.From. The mail system would be constructed such that the fact that
the message was accepted was in itself evidence that an MX and/or A/AAAA
record existed for the domain; note that this probably wouldn't be the only
acceptance criteria, but it would be one of them. While my decision to
include this as an acceptance criteria may be a common one, I don't think
it rises to the level of standard; I think it is better labeled a BCP. Also
worth noting is that I haven't worked in email operations since 2010, long
before RFC 7489 was published, but I was doing the MX and A/AAAA checks
back then, so if there's interest in documenting the practice, it might be
better documented in the latest updates to RFCs 5321 and 5322 and any
associated applicability statements that the emailcore working group is
doing.

I think that requiring that the MX/A/AAAA address match the Source IP
address class is perhaps a bridge too far; third-party ESPs employed by
many larger senders aren't typically in the business of handling inbound
mail for their customers, and so you're likely to see a lot of diversion,
especially for checks done on RFC5322.From addresses. For example:

washingtonpost.com. 43200 IN TXT "v=spf1 ip4:198.72.14.0/23 ip4:
192.72.255.0/24 ip4:54.156.98.51 ip4:54.210.51.17 include:
spf.protection.outlook.com include:madgexjb.com include:
spf-001a3c01.pphosted.com include:amazonses.com -all"

washingtonpost.com. 3600 IN MX 10 mxa-001a3c01.gslb.pphosted.com.

washingtonpost.com. 3600 IN MX 10 mxb-001a3c01.gslb.pphosted.com.

As for the overall validity of the MX or A/AAAA record, specifically does
it point to something that is reachable and willing to accept mail for the
domain in question, that's what periodic inspection of one's own queues and
bounce logs are for. If one notices that a significant amount of mail
destined for a remote domain isn't leaving your platform, and
troubleshooting doesn't rectify the problem, and it appears that the domain
in question is one that's sending unwanted mail, one can always block the
offending domain as a matter of local policy on the grounds that it's mail
is unwanted and can't be responded to. Accepting mail that can't be replied
to due to the remote site not answering isn't a problem that DMARC is
designed to diagnose or solve, though.

--

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 7 Apr 2021 22:06:33 -0700

Further to what Todd said:

On Wed, Apr 7, 2021 at 5:04 AM Douglas Foster <
[email-masked]> wrote:

> products (my vendors) do not implement it at all, leaving a protection gap.
>

I think I'm confused about how this ties to DMARC.  What you're describing
is, I believe, a common anti-spam tactic that was in use even before SPF or
DKIM were a thing.  It probably still is, irrespective of those protocols.

However, RFC 7489's Appendix A.4 points out that earlier versions of DMARC
had such a test, but it was removed because of too many false negatives.

> for the test and integrating it into A-R, as a co-requisite to the DMARC

How is DMARC dependent on a return path check?  SPF is, but DMARC isn't.
DMARC only cares what the SPF result was, and whether the domain it
evaluated is aligned with the RFC5322.From domain.  In fact, a DMARC filter
could operate correctly with no knowledge of what the return path was,
provided the A-R fields are correct and complete.

Another perspective: If the RFC5322.From domain doesn't exist, it's not
possible for a DMARC policy to be discovered for it, nor is it possible
that SPF or DKIM will pass for it or align to it.  Therefore, is such a
domain even in scope here, much less a need to describe a way to test it?

-MSK

---

From: Douglas Foster <[email-masked]>
Date: Thu, 8 Apr 2021 08:02:15 -0400

NXDOMAIN and Return-Path-Check are two different ways of identifying
non-existent domains.    I believe NXDOMAIN to be the better one, but since
the group seemed to be in love with return-path-check, I was willing to go
there.    Whichever one could solve the problem best, the issue for us is
that the problem has not been solved.

Prior to PSD for DMARC, the IETF role in sender authentication could be
summarized as follows:

"An attacker may seek to hide his own reputation and acquire a better one
by impersonating another domain.  This is of interest to IETF and has been
addressed by the specifications for SPF and DMARC.  An attacker may also
seek to hide his own reputation and acquire a neutral one by impersonating
a non-existent domain.   This is of no interest to IETF.;"

This dichotomy is hard to defend.   It has been made worse by embracing PSD
for DMARC.   The statement must now be revised to say:

"IETF is interested in attacks of the form
'otherdomain.nonexistentdomain.psd', but IETF is not interested in attacks
of the form 'nonexistentdomain.otherdomain.psd'.

Todd's comments suggest that the whole issue of non-existent domains is a
non-problem because everybody who matters is already well protected from
non-existent domains, because they have implemented non-IETF tests such as
return-path-check.    If this were true, attackers would move on to other
strategies and PSD for DMARC would have no advocates.

When we start using ARC to accept results which cannot be independently
tested, we need to know what tests were peformed.   Since address rewrite
can eliminate the ability to reproduce earlier tests for NXDOMAIN or
Return-Path, it must be part of the ARC results.  The success of ARC
depends on having this information included, and the success of "new" DMARC
has been tied to ARC.

Non-existent domain usage is not a minor issue and not an issue that is
already solved.   It is the elephant in the room, and it needs to be
addressed.

Doug

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Thu, 8 Apr 2021 08:44:33 -0700

> of the form 'nonexistentdomain.otherdomain.psd'.

I don't understand your assertion here. Non-existent domains under existing
org domains are already covered by the org-level DMARC policy assertion.
5322.From domain of nosuchdomain.example.com would be treated in accord
with the policy for example.com.

--Kurt

---

From: "John Levine" <[email-masked]>
Date: 8 Apr 2021 12:27:26 -0400

>with the policy for example.com.

I agree with Kurt here.  Non-existent domains are routinely handled in other ways
and are outside the scope of DMARC.

R's,
John

---

From: Douglas Foster <[email-masked]>
Date: Thu, 8 Apr 2021 12:50:26 -0400

They are covered by accident if the domain has an enforceable policy.   The
reality is that NX status means tharmt the subdomain is not under the
administrative control of any parent domain and therefore the presence or
absence of a domain policy is irrelevant.

Why is it problematic to document this risk, and indicate that when "No
Policy detected" occurs, it is recommended to check whether the domain
exists, and if it does not exist then local policy for nonexistent domains
should be applied?

On Thu, Apr 8, 2021, 11:44 AM Kurt Andersen (b) <[email-masked]> wrote:

> I don't understand your assertion here. Non-existent domains under
> existing org domains are already covered by the org-level DMARC policy
> assertion. 5322.From domain of nosuchdomain.example.com would be treated
> in accord with the policy for example.com.
>
> --Kurt
>

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 8 Apr 2021 09:53:52 -0700

> exists, and if it does not exist then local policy for nonexistent domains

Can you put together an example message exhibiting the properties you're
talking about, and what DNS records are in play in that scenario?

I still can't picture the problem you're trying to solve.

-MSK

---

From: "John Levine" <[email-masked]>
Date: 8 Apr 2021 14:29:47 -0400

It appears that Murray S. Kucherawy  <[email-masked]> said:
>-=-=-=-=-=-

My question would be what does it have to do with DMARC.

We already have policies for dealing with non-existent domains unrelated to DMARC.

R's,
John

---

From: "Jan Bouwhuis (DMARC)" <[email-masked]>
Date: Thu, 8 Apr 2021 22:20:21 +0200

You are right, it is a common practice to check on PTR records 
(https://tools.ietf.org/html/rfc2505).

Further for existing not mailing domains rfc7505 
https://tools.ietf.org/html/rfc7505 should be used.

Mail servers like postfix block connections by default when a null mx 
record is published.

Regards,

Jan Bouwhuis

---

From: Douglas Foster <[email-masked]>
Date: Thu, 8 Apr 2021 16:49:26 -0400

DNS Examples that Murray requested, which should also addresses John's
question about relevance to DMARC:

nslookup
> set type=txt

> _dmarc.junk.thisisjunk.com
*** <server> can't find _dmarc.junk.thisisjunk.com: Non-existent domain

Domain has no DMARC policy.
Is this because it chose not to deploy one, or because it does not exist?
 That answer requires a second query.

> junk.credcontrol.com
*** <server> can't find junk.credcontrol.com: Non-existent domain

The TXT query demonstrates that this is a non-existent domain, and
therefore not under the full administrative control of any parent domain.
The message is DMARC NOT_VERIFIED even if domain alignment occurs with a
DKIM signature or SPF PASS.  Since there is no domain-level policy record,
disposition depends on local policy related to non-existent domains and
this particular domain name.    The organizational policy record may be
useful if its requested action is more stringent than the local policy
default action for non-existent domains.

> junk.thisisjunk.com
*** <server> can't find junk.thisisjunk.com: Non-existent domain

Domain has no DMARC policy.
Is this because it chose not to deploy one, or because it does not exist?
 That answer requires a second query.

>thisisjunk.com
        primary name server = ns1.dreamhost.com
        responsible mail addr = hostmaster.dreamhost.com
        serial  = 2018071003
        refresh = 19193 (5 hours 19 mins 53 secs)
        retry   = 1800 (30 mins)
        expire  = 1814400 (21 days)
        default TTL = 14400 (4 hours)

The TXT query demonstrates that the domain exists.   This is true whether
the result returns data or NODATA, and in this case the result is NODATA.
The message can be DMARC-verified using domain alignment to a DKIM
Signature or SPF PASS.

Doug Foster

---

From: Todd Herr <[email-masked]>
Date: Fri, 9 Apr 2021 09:08:58 -0400

On Thu, Apr 8, 2021 at 4:51 PM Douglas Foster <
[email-masked]> wrote:

> I think you may have a cut and paste error here:

The above isn't a query for a DMARC policy record.

To your larger point though and your claim of a second query being required
to determine if a domain exists if there is no published DMARC policy, I
think you've got the order of the queries exactly backward.

My expectation for any policy rules that are based on a domain's existing
and/or a domain having published an MX or A/AAAA record is that such
queries will be done *before* any attempt to suss out a DMARC policy record
and perform DMARC validation checks, not after.

For example:

   1. Client connects to my server. At this point I know the source IP of
   the client, and so I can apply any policy rules I may have in place for the
   source IP, such as rejecting the connection due to the IP's being blocked,
   refusing the connection if the IP has no corresponding PTR record, refusing
   the connection if the PTR record matches a pattern of names from which I
   don't want to accept mail (such as the naming pattern being generic or
   indicating that it's dynamic space), perhaps throttling the connection
   based on a lack of FCrDNS for the IP or other rules based on local
   reputation data known about the IP.
   2. If the connection is still open, the client issues a MAIL FROM
   command. At this point I now have a domain (RFC5321.From) and I can run
   checks to make sure the domain exists in the DNS and publishes either an MX
   or A/AAAA record and do any DNSBL lookups for the domain. I can also, if I
   wish, do the SPF check here and reject if there's a failure and the record
   ends in "-all".
   3. If the connection is still open, the client issues one or more RCPT
   TO commands. I can answer "250 ok" or "550 5.1.1 user unknown" as
   appropriate.
   4. If the client has exhausted its RCPT TO commands and has gotten "250
   ok" in response to at least one of them, then it will issue a DATA command,
   and I can respond "ok" if I still potentially want to accept a message from
   this connection.
   5. Client transmits the message, ends with "."
   6. Before I reply with "ok", I will definitely do step 1 below, and
   might do either or both of steps 2 and 3:
      1. Test for existence of the RFC5322.From domain in DNS and do any
      DNSBL lookups on the domain.
      2. Run the message through an anti-spam filtering engine, if I'm the
      type to reject mail based on its content. If instead I'd just route it to
      the spam folder if the engine says it's spam, I may do this step
at a layer
      closer to my message delivery/storage layer.
      3. Do DMARC validation checks, if I'm the type to honor a domain's
      published policy and reject mail that fails DMARC checks if
that's what the
      policy asks me to do. If instead I'm the type to apply a quarantine
      disposition to anything with p=reject or p=quarantine, I might accept the
      message so that the connection can be closed and I can do DMARC
processing
      at a layer closer to my message delivery/storage layer.

Now, I could be wrong here, but I've always viewed DNS queries as
computationally cheaper than content scanning and DKIM hash validation
(note that I'm going to do DKIM hash validation regardless of the existence
of a DMARC policy record, but it's all a matter of where in my
infrastructure I do the validation). DNS queries that allow me to make an
immediate yes/no decision on message acceptance (IP or domain on block
list, domain exists and has an MX or A/AAAA record) get done before DNS
queries that might cause me to do more work. A DNS query for a DMARC record
can result in additional work for me if the DMARC policy record exists, not
just going through the validation steps but also recording the results in
the data store that supports my reporting engine, so I'm going to do that
one last, not first.

But that's just me.

--

---

From: Douglas Foster <[email-masked]>
Date: Fri, 9 Apr 2021 09:49:08 -0400

I am happy to have the concept optimized.  Your comments make sense to me.

  I just believe strongly that these documents teach implementers best
practices and that this particular document is the time and place to begin
teaching best practices around non-existent domains.

It is swell that postfix is widely used and responds appropriately to
non-existent domains  But clearly we have a lot of implementers who need
coaching.

DF

On Fri, Apr 9, 2021, 9:09 AM Todd Herr <todd.herr=
[email-masked]> wrote:

>> Is this because it chose not to deploy one, or because it does not
>> exist?   That answer requires a second query.
>>
>> > junk.credcontrol.com
>> Is this because it chose not to deploy one, or because it does not
>> exist?   That answer requires a second query.
>>
>> >thisisjunk.com
> To your larger point though and your claim of a second query being
> required to determine if a domain exists if there is no published DMARC
> policy, I think you've got the order of the queries exactly backward.
>
>    4. If the client has exhausted its RCPT TO commands and has gotten
>    "250 ok" in response to at least one of them, then it will issue a DATA
>    command, and I can respond "ok" if I still potentially want to accept a
>    message from this connection.
>       2. Run the message through an anti-spam filtering engine, if I'm
>       the type to reject mail based on its content. If instead I'd just route it
>       to the spam folder if the engine says it's spam, I may do this step at a
>       layer closer to my message delivery/storage layer.
>       published policy and reject mail that fails DMARC checks if that's what the
>       message so that the connection can be closed and I can do DMARC processing

---

From: Todd Herr <[email-masked]>
Date: Fri, 9 Apr 2021 11:12:28 -0400

> non-existent domains  But clearly we have a lot of implementers who need

I don't dispute that documenting best practices is a thing to do; I just
think that it's a thing for emailcore to do, not dmarc.

Consider this paragraph from the charter for emailcore
<https://datatracker.ietf.org/wg/emailcore/about/>:

The working group will also work on an Applicability Statement in
parallel with 5321bis and 5322bis, to capture relationships to other
documented and widely deployed work (recommended extensions, transport
security and other issues from the UTA working group's output, and such)
and current email practices. 5321bis and 5322bis will be submitted for
publication before this document is finalized, and the "bis" documents
will have priority for the working group's attention until they are
finished.

I don't see similar language in the charter for this group.

--

---

From: Douglas Foster <[email-masked]>
Date: Fri, 9 Apr 2021 18:05:50 -0400

I see no scope problem with language that states what alignments are not
acceptable, but that is the role of the chairs and the design team.

I shared what I thought was relevant and new information, and responded to
the questions that were asked.   I am going silent.