Subject: [dmarc-ietf] Nitpicky questions about DMARC record syntax

From: "John Levine" <[email-masked]>
Date: 15 Jan 2019 19:58:04 -0500

I am staring at RFC 7489, and at a bunch of purported DMARC records
(see previous message.)

The RFC says that all records must start with "v=DMARC1".  Is it OK
if they start with "v=dmarc1"?  It says that record is a DKIM tag-value
list, and the DKIM ABBF defines all the characters with hex escapes
rather than letters which tells me that it's specifically saying
that case matters.

How about if there's a space before the v=DMARC1?  The tag-value syntax
allows FWS before the first tag, but 7489 says in several places

   Records that do not start with a "v=" tag that identifies the
       current version of DMARC are discarded.

R's,
John

PS: I know it's not hard to write a parser that can accept mutant
forms.  The question is whether that's a good idea.  From a quick
look, people who get v=DMARC1 wrong often get other things wrong, too.

---

From: Dave Crocker <[email-masked]>
Date: Wed, 16 Jan 2019 07:05:10 -0800

> allows FWS before the first tag, but 7489 says in several places

The formal specification is quite clear on both of your questions:

    Section 6.4:

    dmarc-version   = "v" *WSP "=" *WSP %x44 %x4d %x41 %x52 %x43 %x31

which means that the white space is required to be allowed and the value 
in this tag-value is case sensitive.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: "John R Levine" <[email-masked]>
Date: 16 Jan 2019 10:30:17 -0500

> which means that the white space is required to be allowed and the value in 
> this tag-value is case sensitive.

Thanks, but the white space in question is before the "v".

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Dave Crocker <[email-masked]>
Date: Wed, 16 Jan 2019 07:38:29 -0800

On 1/16/2019 7:30 AM, John R Levine wrote:
>> which means that the white space is required to be allowed and the 
>> value in this tag-value is case sensitive.
> 

The ABNF rule I included, and the one that cites it (dmarc-record) do 
not show any white space permitted before the 'v', so no it's not legal.

So the formalities are quite clear on both your questions.

Whether there is benefit or detriment in making the parser more robust 
than the formal rules define goes to the heart of the last point in your 
original note.

d/

---

From: "John R Levine" <[email-masked]>
Date: 16 Jan 2019 10:52:36 -0500

> The ABNF rule I included, and the one that cites it (dmarc-record) do not 
> show any white space permitted before the 'v', so no it's not legal.

Ah, now that I look at it again, I see that the dmarc-record rule is the 
one that matters here, since it allows WSP after the version but not 
before.

OK, they're all broken.  That's what I'd sort of hoped.

Regards,

---

From: Kurt Andersen <[email-masked]>
Date: Wed, 16 Jan 2019 09:16:01 -0800

Is there really a benefit in filtering out people/organizations that are
not fastidious in the use of whitespace and character case?

Seems overly nitpicky and something to reconsider as we look forward
forward a standards track for DMARC.

On Wed, Jan 16, 2019, 07:52 John R Levine <[email-masked] wrote:

> not
> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

---

From: "John R Levine" <[email-masked]>
Date: 16 Jan 2019 12:26:10 -0500

> not fastidious in the use of whitespace and character case?

Maybe, but that's not what standards are about.  The point of a standard 
is to say here's what you do if you want to interoperate.  I have never 
found it productive to speculate about what you might or might not want to 
do when you run into people who didn't read the spec.

In the particlar case I ran into, they're all in .bank and I would expect 
that .bank's auditors would contact their clients and get them to fix 
things.  They have worse problems than wrong capitalizations, like banks 
publishing two different DMARC policies, or one that includes "pct:1" 
whatever that's supposed to mean.

Also, in this case keep in mind that the default is not to filter, so 
you're not going to lose any mail.  You might receive a few more phishes.

Regards,

---

From: Grant Taylor <[email-masked]>
Date: Wed, 16 Jan 2019 11:34:56 -0700

On 01/16/2019 10:26 AM, John R Levine wrote:
> found it productive to speculate about what you might or might not want 
> to do when you run into people who didn't read the spec.

I agree with you conceptually.

However I feel like rejecting things because of additional white space 
(in front of v=...) or the wrong case is being a little bit pedantic.

Rather, I think that if removing a spurious / leading space or folding 
case causes the DMARC record to be valid, it behooves us to tolerate 
such minor errors.

I don't want to be so pedantic that people push back on adopting what I 
(and I assume others) think is a good technology.

Is doing so against the letter of the specification, absolutely.  Is it 
within the spirit of the specification, I think so.

-- 
Grant. . . .
unix || die

---

From: Dave Crocker <[email-masked]>
Date: Wed, 16 Jan 2019 10:41:38 -0800

> within the spirit of the specification, I think so.

There is quite a bit of dynamic tension in this topic.

There is pressure to be tolerant, best captured in Jon Postel's 
robustness directive, and there is a slippery slope of having no firm, 
clear and precise standard.

If more flexibility is viewed by the community as desirable, then the 
community should enhance the specification to allow it.  This improves 
robustness while retaining a firm, clear and precise standard.

d/

---

From: "John Levine" <[email-masked]>
Date: 16 Jan 2019 14:18:23 -0500

>robustness while retaining a firm, clear and precise standard.

Do keep in mind that most of the DMARC records I've looked at follow
the spec.  They may not have the expected policy, but the syntax is
fine.  If a small minority get it wrong, I think it's better to
educate and fix them than to try to guess when someone misreads the
spec in a way that leads them to screw up the syntax of the record,
but not to screw up anything else.

Remember, that if your software rewrites an invalid record into a
correct one, you are trying to read the mind of the person who wrote
the misformed record.  I can guess what v=dmarc1 was supposed to say
but I have no clue what pct:1 is supposed to mean.  Let's not start.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Wed, 16 Jan 2019 12:00:13 -0800

On 1/16/2019 11:18 AM, John Levine wrote:
> the misformed record.

To emphasize a point you made earlier:  There are many, small 
adjustments that a receiver might make, with the intention of operating 
more robustly.  The current examples certainly quality as small and 
seemingly innocuous.  But the earlier point was that one deviation from 
the specification bodes ill for more important questions of conformance...

If they didn't read this part carefully, why believe they read other 
parts more carefully?

d/

---

From: Vladimir Dubrovin <[email-masked]>
Date: Thu, 17 Jan 2019 00:50:07 +0300

I believe in the situation where standard is absolutely clear like this,
any implementer must strictly follow the standard. Otherwise, it can
lead to unpredictable behavior and security issues.

Example: there are absolutely legal situations where non-trusted or less
privileged side can partially control the name and/or content of the DNS
record. I can provide users or customers with possibility to register a
hostname and TXT record in my zone but I want to prevent them from
corrupting or changing SPF/DMARC/etc policy. I can rely on filtering TXT
record content. Non-standard behavior of DMARC implementation can allow
to bypass this filtering.

While this example doesn't seem realistic, I can demonstrate quite
realistic ones. For example, a minor relaxation for ARC version check
can lead to DKIM signature spoofing, compromising DMARC as a result. For
DMARC DNS record itself, realistic scenario may appear in the future.

We already have a lot of problems in-the-wild because of standards
relaxations, e.g. it's usually possible to bypass DMARC which relies on
RFC5322 via malformed From: due to fact invalid From: header is
generally accepted and parsing From: header for DMARC validation and for
visual representation is usually made by different pieces of code with
different behavior.

16.01.2019 21:34, Grant Taylor пишет:
>> Maybe, but that's not what standards are about.  The point of a
>> standard is to say here's what you do if you want to interoperate.  I
>> have never found it productive to speculate about what you might or
>> might not want to do when you run into people who didn't read the spec.
>
> I don't want to be so pedantic that people push back on adopting what
> I (and I assume others) think is a good technology.
>
> Is doing so against the letter of the specification, absolutely.  Is
> it within the spirit of the specification, I think so.
>
>
>
>

-- 
Vladimir Dubrovin
@Mail.Ru

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 16 Jan 2019 18:09:46 -0500

Hi,

On Wed, Jan 16, 2019 at 11:34:56AM -0700, Grant Taylor wrote:
> 
> However I feel like rejecting things because of additional white space (in
> front of v=...) or the wrong case is being a little bit pedantic.

I want to point out, because it's making me extremely itchy, that the
DNS itself did this for years.  One result is that vendors are about
to have a flag day in which a whole bunch of things are deprecated at
once in an effort to get rid of a lot of cruft.

Vendors are going to have a difficult time rejecting any heuristic
improvements if some of them work.  Already it is hard for DNS
providers to process these records because they're all TXT and the
semantics of the RRTYPE say that anything is allowed.  So I think
stricter implementations overall are probably the better path to
interoperability here, even if that hurts in the immediate term.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Dotzero <[email-masked]>
Date: Wed, 16 Jan 2019 18:46:58 -0500

+1

Too many times we (collectively) have avoided the short term pain because
it is pain, but we have set ourselves up for greater pain at a later point.
Part of the problem with ignoring the requirements of a standard is that
while interoperability works in most cases, it sets up failure in corner
cases and opens up the potential for abuse in ways that are not easily
discerned.

Michael Hammer

On Wed, Jan 16, 2019 at 6:10 PM Andrew Sullivan <[email-masked]>
wrote:

> (in

---

From: "Peter M. Goldstein" <[email-masked]>
Date: Wed, 16 Jan 2019 15:55:53 -0800

+1

I concur with Mike and Andrew.  There's no no reason to ignore this element
of the standard because there's no real barrier (other than lack of
attention to the spec) preventing implementors from doing this correctly.
And all we'd be doing is pushing the burden of handling ambiguity to the
receivers.

Avoiding ambiguity is important for avoiding failure in interoperability.
And to Dave's point, this item (and others like it) essentially serves as a
"No Brown M&Ms" clause -
https://www.npr.org/sections/therecord/2012/02/14/146880432/the-truth-about-van-halen-and-those-brown-m-ms
.  If you're implementing a spec, it's important to pay attention to the
details.

Best,

Peter

---

From: [email-masked]
Date: Thu, 17 Jan 2019 04:42:26 -0800 (PST)

> If they didn't read this part carefully, why believe they read other

The seemingly innocuous nature of the accomodation is only one of several
factors that need to be considered when deciding whether or not to implement
these things. Others include, but are not limited to:

(0) What are the worst case security considerations?

(1) Whether or not the misbehavior is widespread.

(2) Is the misbehavior likely to be corrected if you don't accomodate it?

(3) What wiil the effect of telling customers experiencing difficulties that
    it's Someone Else's Problem be?

(4) What is the long term impact on your code going to be?

All that said, in the present case this appears to be a nobrainer: Since the
correct behavior is to ignore malformed records, the security implications may
be significant, it is not widespread behavior, it's very likely to be
corrected, telling people that banks should get their security right seems like
an easy argument to make, and it's a bit of a wart on the code.

I'll also note that transmitters as well as receivers can play the
accomodatation game, with similar effects: What should be common cases
get turned into corner cases, and interoperability suffers as a result.

				Ned

---

From: [email-masked]
Date: Thu, 17 Jan 2019 05:00:01 -0800 (PST)

> lead to unpredictable behavior and security issues.

Sorry, I meant to mention this in my previous response. It's one thing
when there's wiggle-room or lack of clarity in the standard. It's quite
another when things are clear, as is the case here.

> visual representation is usually made by different pieces of code with

Exactly. This is a security protocol and there are real security implications
of doing stuff like this. Another reason to Just Say No.

				Ned

---

From: Grant Taylor <[email-masked]>
Date: Thu, 17 Jan 2019 09:10:36 -0700

> within the spirit of the specification, I think so.

I've seen a number of intriguing, if not compelling, replies in this 
thread.  Some of which have changed my thoughts some.

I now concede accommodating a leading space is questionable.

However I still feel like /requiring/ exact case is contrary to the idea 
of "Be liberal in what you accept and conservative in what you send.".

I don't see any security implications in accepting the following:

dmarc-version = ("v" / "V") *WSP "=" *WSP ("D" / "d") ("M" / "m") ("A" / 
"a") ("R" / "r") ("C" / "c") "1"

I agree that this is contrary to the letter of the specification. 
However I think it is completely within the spirit.  Especially when 
dealing with DNS data which is inherently / invariable human entered.

I don't (yet) see any security implications of accepting improper case 
record data for the dmarc-version *IF* that is the /only/ TXT record at 
a given QName that is DMARC related.  -  If there are multiple DMARC 
records, especially if they are conflicting, strictly adhere to the 
standard.

I'm curious if anyone sees any security implications with the above 
dmarc-version.

This is me trying to learn and understand.  I'm not trying to argue one 
way or the other.

-- 
Grant. . . .
unix || die

---

From: "John Levine" <[email-masked]>
Date: 17 Jan 2019 11:53:17 -0500

>of "Be liberal in what you accept and conservative in what you send.".

Yup.  See

https://datatracker.ietf.org/doc/html/draft-iab-protocol-maintenance

a/k/a "Postel was wrong".

>"a") ("R" / "r") ("C" / "c") "1"

Please see the previous several dozen messages, particularly the one
about the brown M&M's.  If you know that someone didn't read the spec,
you can only guess what else they got wrong, and you're not doing
anyone a favor by doing that.

>dealing with DNS data which is inherently / invariable human entered.

Once again, please try not to assume that everyone's experience is the
same as yours.  On my DNS server, the DMARC records are generated
automatically when I add a new mail domain and their syntax is
correct.  (Or every DMARC record is wrong, but I would notice that
pretty soon.)  On the large systems which these days host most mail
it's hard to see how they could do it manually.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Thu, 17 Jan 2019 09:54:48 -0800

On 1/17/2019 8:53 AM, John Levine wrote:
> In article<[email-masked]
>  you write:
>> However I still feel like/requiring/  exact case is contrary to the idea

The common interpretation of Postel's dictum is to use it as an excuse 
to ignore specification details.

That was not what he intended.  Rather, he intended its use in the way 
that Ned invoked:

> On 1/17/2019 5:00 AM, [email-masked] wrote:

Be liberal when the specification leaves room for doubt.  Not when it 
doesn't.

d/

ps. I don't know why the spec, here, demands case sensitivity but it is 
quite clear that it does.