Subject: Re: [dmarc-ietf] [rfc-i] Question about changes introduced by erratum

From: Damian Lukowski <[email-masked]>
Date: Sun, 22 Mar 2020 00:24:11 +0100

Hello DMARC working group,

I am going through the changes between RFC7601 and RFC8601 and try to
understand the implication of the change introduced by erratum 5435.
The new resinfo definition uses 1*propspec, that is, by my understanding
of [1] and [2], potentially multiple consecutive propspecs without
obvious delimiters. The beginning of a ptype and the end of a pvalue
have no mandatory CFWS, so from my understanding, a verifier could
construct a valid authres-header-field that is cumbersome to parse.

I will refer to the "ptype.property" part of a propspec as its "key".

The spf method defines smtp.helo and smtp.mailfrom. RFC8601, RFC7601 and
RFC7001 have only examples of the form smtp.mailfrom=domain-name.
However, RFC7208 shows a local-part@domain-name form in [3], so I must
assume a parser for an RFC8601 resinfo needs to recognize both forms. So
consider

> spf=pass smtp.mailfrom=x.y.zsmtp.helo=a.b

This is "clearly"
- smtp.mailfrom = x.y.z
- smtp.helo = a.b

It seems that a 10 character lookahead is sufficient, as long as we are
in the domain-name case. However, with

> spf=pass smtp.mailfrom=x.y.zsmtp.helo=[email-masked]

the previous "cut" between x.y.z and smtp.helo is no longer valid, as we
have a single smtp.mailfrom with

- "x.y.zsmtp.helo=a.b" as the local-part, and
- "c.de" as the domain-name.

I see some issues:

1. The lookahead for the "@" is unbounded.
2. RFC8601 does not (that I am aware of) prohibit duplicate keys among
multiple propspecs. They may be either completely redundant, i.e. "k1=v1
k1=v1 ..." or contradicting/overwriting each other, i.e. "k1=v1 k1=v2
...". Without a delimiter between the propspecs, it seems even
impossible to parse them unambiguously.
3. I believe that a parser for potentially non-separated propspecs would
be ugly.

Did I overlook or misunderstand something? Would you consider a parser
RFC8601 compliant if it was only able to parse the "benevolent" forms of
an authres header?

Regards
 Damian

[1] https://www.rfc-editor.org/rfc/rfc5234
[2] https://www.rfc-editor.org/rfc/rfc5234#section-3.1 [no "implicit
     specification of linear white space"]
[3] https://www.rfc-editor.org/rfc/rfc7208#section-9.2

> But in answer to your question: [email-masked], since RFC8601 is from the dmarc WG.

[...]

>> I have a question about a change in an RFC introduced by a specific
>> erratum (5435). Is this the right list to ask such question?

---

From: "John Levine" <[email-masked]>
Date: 21 Mar 2020 21:04:04 -0400

In article <[email-masked]> you write:
>obvious delimiters. ...

It says:

resinfo = [CFWS] ";" methodspec [ CFWS reasonspec ]
            [ CFWS 1*propspec ]

I think both the erratum and RFC 8601 are wrong, and it should say:

resinfo = [CFWS] ";" methodspec [ CFWS reasonspec ]
            1*( CFWS propspec )

Every implementation I know puts space between multiple propspec's
which the current syntax wouldn't allow

>assume a parser for an RFC8601 resinfo needs to recognize both forms. So

That's a mistake in the examples in Appendix B.  The example at the
bottom of page 21 is correct -- the value for smtp.mailfrom is a
mailbox, not a domain name.

R's,
John

---

From: Damian Lukowski <[email-masked]>
Date: Sun, 22 Mar 2020 11:35:18 +0100

> my understanding was that RFCs decide whether an implementation is
> incorrect or in the case of a series, not up-to-date. If the authors
> decided to update the RFC instead, then I'd be happy of course.

Also please note that spaces between propspecs are not against RFC8601 -
they would use the optional CFWS at the end of a pvalue. It's just the
fact that they are not mandatory what bothered me.

---

From: "John Levine" <[email-masked]>
Date: 22 Mar 2020 10:50:37 -0400

>> decided to update the RFC instead, then I'd be happy of course.

In general you're right, but it sometimes happens that the RFC has
mistakes and what it says is not what the WG intended.  In this case
I'm quite sure that the intent is that spaces between propspecs are
required and the ABNF in 8601 is wrong.  Erratum 5435 misunderstood
how ABNF works and somehow we all missed that.

If you look through the archives of the WG you'll see that 8601 is
intended to be a compatible update to 7601, and as you noted the
ABNF change is definitely not compatible.

R's,
John

---

From: Damian Lukowski <[email-masked]>
Date: Sun, 22 Mar 2020 11:07:17 +0100

Hi,

> mailbox, not a domain name.

I assume you mean

> auth=pass smtp.auth=[email-masked] smtp.mailfrom=[email-masked]

I didn't consider this example relevant for the spf-case. For the
auth-case, the right-hand-side is defined in [AUTH] as mailbox and there
are no A-R examples that would state otherwise. [SPF] on the other hand
has an A-R example that is domain-name only, so I assumed that
smtp.mailfrom in spf context was more loosely defined via RFC7001's
pvalue (that is, with the optional local-part@).

Damian

---

From: "John Levine" <[email-masked]>
Date: 22 Mar 2020 11:44:37 -0400

In article <[email-masked]> you write:
>.... [SPF] on the other hand

I'm pretty sure the example is wrong, and was copied verbatim from
wrong examples going back to RFC5451.

RFC8601 sec 2.7.2 refers to RFC 7208 for "mailfrom".  RFC 7208 sec 1.1.3
says that the MAIL FROM, which I whink we can assume is the same thing,
is the RFC5321.MailFrom from RFC 5598.  RFC 5598 says that's a mailbox,
not a domain name.

gain, if you look at the existing implementations, they put the
address in spf smtp.mailfrom, not just the domain name.

R's,
John

PS: I have always found it frustrating that I have to chase through a
dozen documents to find the syntax of A-R.  I understand the desire
not to try to keep two copies of definitions in sync, but as we've
seen, that doesn't always lead to the result you want.

---

From: Damian Lukowski <[email-masked]>
Date: Sun, 22 Mar 2020 20:34:11 +0100

> is the RFC5321.MailFrom from RFC 5598.  RFC 5598 says that's a mailbox,

Those definitions are too confusing for me. RFC5598 says

> RFC5321.MailFrom is the address in the SMTP "Mail From" command

So is RFC5321.MailFrom the Reverse-Path from RFC5321, or only the
Mailbox part of a Reverse-Path? I would expect it to be the full
Reverse-path, because

> RFC5321.Return-Path:  Set by - Originator
>  The MDA records the RFC5321.MailFrom address into the
>  RFC5321.Return-Path field.

and a Return-Path field usually contains "<" ... ">", so it cannot be a
plain Mailbox.

But then the term "MAIL FROM" in
>    As such, throughout this document the term "MAIL FROM" will be used,
>    which is defined as the RFC5321.MailFrom (reverse-path) identity
>    described in [RFC5598].

cannot be identical to RFC5321.MailFrom, because if it were, we would
have smtp.mailfrom = "MAIL FROM" = RFC5321.MailFrom = "<" ... ">",
which is not given by the examples.

Another possibility to interpret the quote is that one obtains the "MAIL
FROM" by picking "the" identity out of the RFC5321.MailFrom. But there
are two sensible identities defined in RFC5598 that one could pick: The
mailbox or the domain name.

> The forms of identity used by Internet Mail are: mailbox, domain
> name, [...]

By the way, an RFC5321 mailbox allows address-literals which are not
covered by the right hand side of RFC8601's pvalue, and one cannot use
the left hand side because of

> If the value portion of a "pvalue" construction identifies something
> intended to be an email identity, then it MUST use the right-hand
> portion of that ABNF definition.

---

From: "John R Levine" <[email-masked]>
Date: 22 Mar 2020 15:41:30 -0400

> Mailbox part of a Reverse-Path? I would expect it to be the full

RFCs 2821 and 5321 say:

    Historically, the <reverse-path> was permitted to contain more than
    just a mailbox; however, contemporary systems SHOULD NOT use source
    routing (see Appendix C).

Source routes have been deprecated for 20 years.  We can forget about 
them.

> and a Return-Path field usually contains "<" ... ">", so it cannot be a

No, see RFC 2821 or 5321 sec 3.3.  The reverse-path is what's between the 
brackets, which means it's a mailbox or it's empty.

> the left hand side because of

I haven't seen an address literal in non-spam mail for over a decade 
either.  I'm pretty sure we can forget about them in A-R headers, too.

R's,
John

---

From: Damian Lukowski <[email-masked]>
Date: Sun, 22 Mar 2020 21:12:37 +0100

> No, see RFC 2821 or 5321 sec 3.3.  The reverse-path is what's between
> the brackets, which means it's a mailbox or it's empty.

Ok, I guess I didn't expect to have to go through a layer of english
while assembling a grammar from different sources, given that there was
an ABNF rule with a matching name.

Regards
 Damian

---

From: "John R Levine" <[email-masked]>
Date: 22 Mar 2020 17:13:28 -0400

> an ABNF rule with a matching name.

Figuring out the actual syntax of A-R is quite hard because there are so 
very many external references.

Even the ABNF is tricky -- I had to stare at the definitions of resinfo 
and propspec and pvalue for quite a while to confirm that your 
interpretation was correct.

A lot of people (not you) when reading RFC 5234 assume that there has to 
be only one way that the BNF can match a string.  That's not true, 
ambiguous matches are allowed, and sometimes can make the ABNF simpler.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Damian Lukowski <[email-masked]>
Date: Sun, 22 Mar 2020 23:04:08 +0100

Haha,

I just found
https://mailarchive.ietf.org/arch/msg/dmarc/FO6ImuO5jzr9ZvN6FcLAzMoH3-c/

Damian