Subject: [art] [Technical Errata Reported] RFC7601 (5435)

From: RFC Errata System <[email-masked]>
Date: Sun, 22 Jul 2018 17:05:58 -0700 (PDT)

The following errata report has been submitted for RFC7601,
"Message Header Field for Indicating Message Authentication Status".

--------------------------------------
You may review the report below and at:
http://www.rfc-editor.org/errata/eid5435

--------------------------------------
Type: Technical
Reported by: Peter Occil <[email-masked]>

Section: 2.2

Original Text
-------------
resinfo = [CFWS] ";" methodspec [ CFWS reasonspec ]
               *( CFWS propspec )

Corrected Text
--------------
resinfo = [CFWS] ";" methodspec [ CFWS reasonspec ]
            [ CFWS 1*propspec ]

Notes
-----
Since propspec includes optional CFWS at end, parsing the current version of resinfo will result in at most one propspec even if methodspec is followed by more than one propspec.

Instructions:
-------------
This erratum is currently posted as "Reported". If necessary, please
use "Reply All" to discuss whether it should be verified or
rejected. When a decision is reached, the verifying party  
can log in to change the status and edit the report, if necessary. 

--------------------------------------
RFC7601 (draft-ietf-appsawg-rfc7001bis-11)
--------------------------------------
Title               : Message Header Field for Indicating Message Authentication Status
Publication Date    : August 2015
Author(s)           : M. Kucherawy
Category            : PROPOSED STANDARD
Source              : ART Area General Application Working Group
Area                : Applications and Real-Time
Stream              : IETF
Verifying Party     : IESG

---

From: "John R. Levine" <[email-masked]>
Date: 22 Jul 2018 21:58:30 -0400

I think this erratum is wrong.  The CFWS at the end of propspec is 
actuall in the included pvalue rule and it's optional, so I think the ABNF 
is correct albeit clumsy.  ABNF doesn't have to be LL(1) or anything like 
that although I concur that it would be nice if it is where reasonably 
possible.

Regardless, we are in last call for draft-ietf-dmarc-rfc7601bis in the
dmarc WG so if there's anything to fix, that's the place to fix it.
The ABNF for resinfo and propspec haven't changed.

R's,
John

> "Message Header Field for Indicating Message Authentication Status".

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Peter Occil <[email-masked]>
Date: Sun, 22 Jul 2018 22:13:59 -0400

Speaking of last calls, the production “arc-info” in draft-ietf-dmarc-arc-protocol, which is also in last call, includes a semicolon after “instance”, which in turn has a semicolon at the end.  However, a great number of Arc-Authentication-Results header fields I’ve seen in practice include only one semicolon between the instance and the “authres-payload”; for example: “i=1; mx.example.com; …”.

And also, the erratum I submitted can be correct for handling certain parsers that skip the CFWS in the “pvalue”, set the pointer to after the CFWS (if any), and move on to parsing the next “CFWS propspec” within the (current) “resinfo” production (in other words, the parser does “greedy” matching).

--Peter

From: John R. Levine
Sent: Sunday, July 22, 2018 9:58 PM
To: RFC Errata System
Cc: Murray Kucherawy; [email-masked]; Alexey Melnikov; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [art] [Technical Errata Reported] RFC7601 (5435)

[…]

[…]

---

From: "John R. Levine" <[email-masked]>
Date: 22 Jul 2018 22:33:00 -0400

> And also, the erratum I submitted can be correct for handling certain 
> parsers that skip the CFWS in the “pvalue”, set the pointer to after the 
> CFWS (if any), and move on to parsing the next “CFWS propspec” within 
> the (current) “resinfo” production (in other words, the parser does 
> “greedy” matching).

ABNF doesn't have to be LL(k) or LR(k), so greedy parsers won't work, at 
least not without rewriting the rules to factor out ambiguities.

Regards,

---

From: Peter Occil <[email-masked]>
Date: Mon, 23 Jul 2018 08:19:48 -0400

You have pointed to an unexpected weakness of ABNF.  As a result, it's unfortunate that many ABNF productions in RFCs can't generally be used as is in order to build a parser, since ABNF itself has no strong concept of matching.  

For example, RFC 5234 sec. 3.2 is silent on which order alternative productions are to be matched (although this silence can be useful, I admit, in order for the incremental alternatives feature, sec. 3.3, to work).  The production "received-token" in RFC 5322 is illustrative:

   received-token  =   word / angle-addr / addr-spec / domain

Both "word" and "domain" include the production "atom".  Thus, whether a received-token parser parses an "atom" as a "word" or as a "domain" depends on the context in which the "atom" appears, the kind of parser (greedy or not), or whether earlier alternatives take precedence over latter alternatives or vice versa, none of which is defined in RFC 5234. "Obviously", if the "atom" is part of a domain, then the received-token should be the whole "domain" regardless of the details just mentioned.

[ Additional context: https://www.ietf.org/mail-archive/web/art/current/msg00571.html ]

From: John R. Levine
Sent: Sunday, July 22, 2018 10:33 PM
To: Peter Occil
Cc: Murray Kucherawy; [email-masked]; Alexey Melnikov; [email-masked]; [email-masked]
Subject: RE: [art] [Technical Errata Reported] RFC7601 (5435)

Regards,

---

From: "John R. Levine" <[email-masked]>
Date: 23 Jul 2018 10:49:02 -0400

On Mon, 23 Jul 2018, Peter Occil wrote:
> You have pointed to an unexpected weakness of ABNF.  As a result, it's 
> unfortunate that many ABNF productions in RFCs can't generally be used 
> as is in order to build a parser, since ABNF itself has no strong 
> concept of matching.

Of course they can, if you have adequate parser tools.  I shamelessly 
recomemnd "flex & bison", published by O'Reilly.  See the discussion of 
GLR parsers on pages 230-231.  Personally I'd refactor them to LL(1) 
or LR(1) but tastes differ.

More to the point, the IETF has used ABNF in its current form for over 20 
years, and it's not going to change now.  Its purpose is to specify valid 
syntax, even if the grammar parses that syntax ambiguously.  So please do 
send errata like the one on Received-SPF, where the ABNF didn't match the 
actual syntax, but if it's correct but ambiguous, that's not a bug.

Also, having written my share of compilers (try fitting a Fortran-77 front 
end into 48Kb on a PDP-11) I can assure you that I never was able to build 
a parser from an unmodified reference grammar.  You always need to 
refactor it.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Mon, 23 Jul 2018 08:40:59 -0700

On 7/23/2018 7:49 AM, John R. Levine wrote:
> More to the point, the IETF has used ABNF in its current form for over 
> 20 years,

41 years.

The changes made in the document since ABNF's inception in 1977 -- as 
part of RFC 733 -- are helpful but not fundamental.

And I'll take this opportunity to cite an error in credit I've been 
making over the years...

Within the set of RFC 733 authors, I was the newbie with no formal 
training and wound up as editor of the document.  I spent an afternoon 
in a technical bookstore, getting a little background on parsers.  There 
were lots of specifications being written in those days that did 
variants of BNF; ours was merely one more.

I remember being concerned that we hit a good balance between features 
(complexity) and utility (usability).  And I remember worrying quite a 
bit about the formatting of the ABNF, since that affects comprehension. 
(I was, at the time, in a nascent Human-Computer Interaction graduate 
program.)

What I didn't remember until reviewing the Introduction to RFC 733, 
recently, is that the core credit for creating ABNF goes to Ken Harrenstein.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 23 Jul 2018 21:19:18 +0200

On Jul 23, 2018, at 14:19, Peter Occil <[email-masked]> wrote:
> 
> Thus, whether a received-token parser parses an "atom" as a "word" or as a "domain" depends on the context in which the "atom" appears,

Hi Peter,

actually, this is not how BNF traditionally works.

BNF is a productive grammar that produces a language.

It doesn’t tell you how to “parse”, i.e., assign rules to sections of the strings that make up the language.

E.g.,

  Foo = Bar / Baz

  Bar = “abc”

  Baz = “ab” “c”

is perfectly fine as BNF (here in ABNF form modulo the smartquotes) and takes no position on whether “abc” is a Bar or a Baz; it is certainly part of the language (as is “aBc” but that is a different story).

Clearly, parser generators do generate “parsers" that do want to know, so they are happier if the decision can be made.

When writing software based on ABNF, I generally prefer to use PEG (parse expression grammar) parsers instead of traditional BNF-based parser generators; this sometimes means I have to massage the ABNF slightly.  I also try to deliver my ABNF in PEG-friendly form.  But none of that is supported by RFC 5234 (STD 68).

CDDL (the ABNF derivative for CBOR and JSON) does prefer a clear assignment of specific rules to a subtree of the data model item tree; we are using PEG semantics for this.

BTW, the first RFC to use BNF (without actually using that word) was RFC 5 of June 2, l969 [sic!]; this wasn’t Ken L. Harrenstien's version of BNF but already somewhat close.  Prepare for the “BNF in RFCs” fiftieth anniversary in 10.5 months…

Grüße, Carsten

---

From: [email-masked] (Dale R. Worley)
Date: Mon, 23 Jul 2018 22:36:29 -0400

I think that the erratum is, strictly speaking, incorrect.  I've not
worked through examples in detail, but you can see that each propspec
contains exactly one "=" (other than in certain quoted contexts), which
makes it unlikely that two successive propspecs could be parsed as a
single propspec.  If there really is an ambiguity, the way to
demonstrate it is to provide an example.

What is true is that the ABNF doesn't uniquely parse CFWS between
propspec's; some CFWS can be parsed as part of the resinfo production or
as part of the propspec production.  But nobody cares about that, i.e.,
those alternative parsings are (unspokenly) "semantically equivalent".

Someone else comments that ABNF, strictly speaking, isn't a
specification for parsing but rather a specification for *generating*.
But in actual use (running code?), that's not quite right either.  The
implicit assertion is that the ABNF shows permissible ways to parse the
strings (of necessity, strings that can be generated by the ABNF), and
that any ambiguities in parsing are either (1) explicitly dealt with by
the surrounding text, or (2) the differences between the parse trees are
"not significant" under the semantics that the text endows the parse
trees.

Dale

---

From: Peter Occil <[email-masked]>
Date: Tue, 24 Jul 2018 07:54:17 -0400

On further thought, it is indeed incorrect.  Therefore, the errata submission is withdrawn.

--Peter

From: Dale R. Worley
Sent: Monday, July 23, 2018 10:36 PM
To: RFC Errata System
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [art] [Technical Errata Reported] RFC7601 (5435)