Subject: Re: [dmarc-ietf] 3.2.6 The meaning of non-existence (Sample Data)

From: Douglas Foster <[email-masked]>
Date: Sun, 19 Dec 2021 18:42:27 -0500

Here are some results based on 3025 messages, involving 1253 unique
RFC5322.From domains, collected over less than 24 hours.   These results
are collected AFTER excluding messages from blacklisted sources and sources
with SPF=NXDOMAIN, so a high percentage is not spam.

I detected 52 messages, from 10 unique domains, which failed the MX/A
test.  I do not test on AAAA because I do not accept mail using IPv6.

All 10 could produce DMARC PASS based on relaxed alignment, although I have
not evaluated whether they publish a DMARC policy.   I simply evaluate SPF
and DKIM based on relaxed alignment for all incoming messages.

All 10 domains had RFC5321.MailFrom and RFC5322.From domains that were
different.

7 of 10 had DMARC PASS based on both SPF and DKIM:
bc.qvcemail.com
doctors-digest.com
email.nutricia-na.com
mail.foodnetwork.com
mail.medscape.org
mktg.daily-harvest.com
email3.reachmd.com

1 of 10 had DMARC PASS based on SPF alignment only:
mg.homedepot.com

2 of 10 had DMARC PASS based on DKIM only:
info.extraspace.com
update.strava.com

1 of 10 had an SPF record on the RFC5322.From address.
email.nutricia-na.com

Overall, this suggests to me that ESP messages will have trouble complying
with any NP criteria, and this may force us to use a weaker one, such as
NXDOMAIN only, even though my preference is a strong one.

Doug Foster

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 20 Dec 2021 10:44:02 +0100

On Mon 20/Dec/2021 00:42:27 +0100 Douglas Foster wrote:
> 
> I detected 52 messages, from 10 unique domains, which failed the MX/A test.
> [...]
> 

What do you mean by "failed the MX/A test"?  Only doctors-digest.com seems to be non-existent under 3.2.6.

ale@pcale:~/tmp$ for d in $doms mg.homedepot.com info.extraspace.com update.strava.com; do r=$(get_root_domain $d|sed -rn 's/^ Root Domain: *(.*)$/\1/p'); echo "$d -> $r"; dig +short $r; dig +short $r mx; echo; done
bc.qvcemail.com -> qvcemail.com
167.140.19.203
100 smtp2.qvc.com.
100 smtp3.qvc.com.

doctors-digest.com -> doctors-digest.com

email.nutricia-na.com -> nutricia-na.com
52.36.54.191
20 mail3792.nutricianorthamerica.mkt4389.com.
5 bounce.email.nutricia-na.com.
10 reply.email.nutricia-na.com.

mail.foodnetwork.com -> foodnetwork.com
204.78.50.45
100 foodnetwork-com.mail.protection.outlook.com.
1 aspmx.l.google.com.
10 alt3.aspmx.l.google.com.
5 alt1.aspmx.l.google.com.
5 alt2.aspmx.l.google.com.
10 alt4.aspmx.l.google.com.

mail.medscape.org -> medscape.org
104.18.27.226
104.18.26.226
10 alt3.aspmx.l.google.com.
10 reply-mx.s6.exacttarget.com.
1 aspmx.l.google.com.
5 alt2.aspmx.l.google.com.
5 alt1.aspmx.l.google.com.
10 alt4.aspmx.l.google.com.

mktg.daily-harvest.com -> daily-harvest.com
104.18.1.9
104.18.0.9
10 alt4.aspmx.l.google.com.
1 aspmx.l.google.com.
10 alt3.aspmx.l.google.com.
5 alt1.aspmx.l.google.com.
5 alt2.aspmx.l.google.com.

email3.reachmd.com -> reachmd.com
34.195.222.240
34.233.81.108
10 mx1-us1.ppe-hosted.com.
10 mx2-us1.ppe-hosted.com.

mg.homedepot.com -> homedepot.com
35.201.95.83
20 mx0a-000e6601.pphosted.com.
10 mxb-000e6601.gslb.pphosted.com.
10 mxa-000e6601.gslb.pphosted.com.
20 mx0b-000e6601.pphosted.com.

info.extraspace.com -> extraspace.com
13.107.246.13
10 mxa-00257001.gslb.pphosted.com.
10 mxb-00257001.gslb.pphosted.com.

update.strava.com -> strava.com
3.227.103.50
44.195.56.39
52.0.47.160
3.217.33.77
3.237.58.53
34.197.5.198
54.209.232.157
52.72.119.210
30 alt2.aspmx.l.google.com.
50 aspmx3.googlemail.com.
20 alt1.aspmx.l.google.com.
10 aspmx.l.google.com.
40 aspmx2.googlemail.com.

---

From: Douglas Foster <[email-masked]>
Date: Mon, 20 Dec 2021 06:53:12 -0500

I am not doing any root domain lookups.   If that is part of the proposed
algorithm, somebody needs to document it.  I am simply looking for a
resource record matching the FROM domain name.

Because I am a Windows guy, I use the deprecated NSLOOKUP.   I have done
minimal work in DIG.   I retested one of the names and confirmed the same
results:

> set type=MX
Server:  G3100.myfiosgateway.com
Address:  192.168.1.1

*** G3100.myfiosgateway.com can't find info.extraspace.com: Non-existent
domain
> set type=A
Server:  G3100.myfiosgateway.com
Address:  192.168.1.1

*** G3100.myfiosgateway.com can't find info.extraspace.com: Non-existent
domain

Doug Foster

On Mon, Dec 20, 2021 at 4:44 AM Alessandro Vesely <[email-masked]> wrote:

> test.
> ale@pcale:~/tmp$ for d in $doms mg.homedepot.com info.extraspace.com
> update.strava.com; do r=$(get_root_domain $d|sed -rn 's/^ Root Domain:
> *(.*)$/\1/p'); echo "$d -> $r"; dig +short $r; dig +short $r mx; echo; done

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 20 Dec 2021 18:10:31 +0100

On Mon 20/Dec/2021 12:53:12 +0100 Douglas Foster wrote:
> algorithm, somebody needs to document it.  I am simply looking for a resource 
> record matching the FROM domain name.

Oops, yes, you're right.  Dunno why I looked up their org domain, probably lack 
of caffeine...

Those 10 domains are non-existing under 3.2.6.  Only 4 of them return NXDOMAIN.

One of them, mail.foodnetwork.com, has its own DMARC record with a policy 
different from that of its parent domain, which can be a reason to use a 
subdomain.  (Curiously, it is one of those returning NXDOMAIN.)

For the other 9, all what I can think of is some kind of misconfiguration.  I 
asked a few times why would one want to use a non-existing domain for the From: 
address, but got no answers.  Anyway, your numbers show that it's not a very 
frequent setup.

Best
Ale
--

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 20 Dec 2021 12:13:14 -0500

On Monday, December 20, 2021 12:10:31 PM EST Alessandro Vesely wrote:
> Oops, yes, you're right.  Dunno why I looked up their org domain, probably
> lack of caffeine...
> 
> For the other 9, all what I can think of is some kind of misconfiguration. 
> I asked a few times why would one want to use a non-existing domain for the
> From: address, but got no answers.  Anyway, your numbers show that it's not
> a very frequent setup.

... for legitimate mail.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 20 Dec 2021 14:59:45 -0500

>Those 10 domains are non-existing under 3.2.6.  Only 4 of them return NXDOMAIN.

If you prefix _domainkey to those names and do a lookup, several of them
return NOERROR which suggests they have DKIM keys.

For the umpteenth time, there is a DNS definition of non-existent which is the only
one you get to use in the IETF.  Can we stop wasting time with this fruitless
argument, please?

R's,
John

---

From: Douglas Foster <[email-masked]>
Date: Mon, 20 Dec 2021 16:57:12 -0500

It is not infrequent.   Here is some more detailed statistics:

  msgs domains description   Msg Fail  Domain Fail
3,025   1,253 All messages    1.72%   0.80%
1,098     296 Allowed msgs    4.74%   3.38%
  581     175 ESP messages    8.95%   5.71%
   52      10 MX/A Failures

The percentages are relative to the failure counts.
"ESP messages" counts any message where the From addresses are different.
So 5.7% of third-party mailings use a From address that is not found or not
easily found in DNS.  This is not insignificant.

John, I don't know what algorithm you are proposing.   Please clarify.

Doug

On Mon, Dec 20, 2021 at 12:10 PM Alessandro Vesely <[email-masked]> wrote:

> lack

---

From: Douglas Foster <[email-masked]>
Date: Mon, 20 Dec 2021 22:22:15 -0500

Using an NXDOMAIN test, I have these occurrences in the last 24 hours:
- 1 nxdomain that is from a legitimate sender,
- 1 that is NXDOMAIN because the ESP misspelled the client organization
name, and
- 1 that is a bogus NDR.

So the NXDOMAIN test will identify fewer problems, but will create fewer
problems also.

Caveat:  Consider that my email stream is modest.  Internet scale
multiplies everything by many orders of magnitude.   Does anyone have data
from a bigger message stream?

Doug

On Sun, Dec 19, 2021 at 6:42 PM Douglas Foster <
[email-masked]> wrote:

> All 10 could produce DMARC PASS based on relaxed alignment, although I
> have not evaluated whether they publish a DMARC policy.   I simply evaluate
> SPF and DKIM based on relaxed alignment for all incoming messages.
>

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 21 Dec 2021 13:05:49 +0100

> return NOERROR which suggests they have DKIM keys.

Hm...  one of them returns NXDOMAIN even though there is a DMARC record below.

ale@pcale:~/tmp$ dig mail.foodnetwork.com

; <<>> DiG 9.16.15-Debian <<>> mail.foodnetwork.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 64715
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: d56c07ed27e795d30100000061c1bc09cea5581e05ff08ab (good)
;; QUESTION SECTION:
;mail.foodnetwork.com.		IN	A

;; AUTHORITY SECTION:
foodnetwork.com.	875	IN	SOA	ns-298.awsdns-37.com. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400

ale@pcale:~/tmp$ dig _dmarc.mail.foodnetwork.com txt

; <<>> DiG 9.16.15-Debian <<>> _dmarc.mail.foodnetwork.com txt
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32999
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 00298aa760a081420100000061c1bc0e9fc192e7b9269cf7 (good)
;; QUESTION SECTION:
;_dmarc.mail.foodnetwork.com.	IN	TXT

;; ANSWER SECTION:
_dmarc.mail.foodnetwork.com. 300 IN	TXT	"v=DMARC1; p=reject; fo=1; ri=3600; rua=mailto:[email-masked]; ruf=mailto:[email-masked]"

> one you get to use in the IETF.

The definition in Section 3.2.6 is different.

> Can we stop wasting time with this fruitless argument, please?

Yes.

Best
Ale
--

---

From: "John R Levine" <[email-masked]>
Date: 21 Dec 2021 12:42:44 -0500

> ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 64715

Sigh.  That's just wrong.  I'll see if I can find someone who can fix it.

R's,
John

PS: pretty please can we not even think about changing the spec to work 
around other people's bugs