Subject: [abnf-discuss] New Version Notification for draft-seantek-abnf-more-core-rules-03.txt

From: Sean Leonard <[email-masked]>
Date: Sat, 10 Oct 2015 09:39:25 -0700

Here is rev 3. I believe this addresses on-list and off-list concerns. 
Take a look...

Sean

-------- Forwarded Message --------
Subject: 	New Version Notification for 
draft-seantek-abnf-more-core-rules-03.txt
Date: 	Sat, 10 Oct 2015 09:30:32 -0700
From: 	[email-masked]
To: 	Sean Leonard <[email-masked]>

A new version of I-D, draft-seantek-abnf-more-core-rules-03.txt
has been successfully submitted by Sean Leonard and posted to the
IETF repository.

Name:		draft-seantek-abnf-more-core-rules
Revision:	03
Title:		Comprehensive Core Rules and Imports for ABNF
Document date:	2015-10-10
Group:		Individual Submission
Pages:		8
URL:            https://www.ietf.org/internet-drafts/draft-seantek-abnf-more-core-rules-03.txt
Status:         https://datatracker.ietf.org/doc/draft-seantek-abnf-more-core-rules/
Htmlized:       https://tools.ietf.org/html/draft-seantek-abnf-more-core-rules-03
Diff:           https://www.ietf.org/rfcdiff?url2=draft-seantek-abnf-more-core-rules-03

Abstract:
    This document extends the base definition of ABNF (Augmented Backus-
    Naur Form) to include comprehensive support for certain symbols
    related to ASCII, and defines an import syntax.

---

From: Riccardo Bernardini <[email-masked]>
Date: Sun, 11 Oct 2015 12:43:40 +0200

Let me check if I understood correctly your import proposal.  An example of
usage would be

   Letters = <ALPHA@[RFC5234]>

in order to define Letters as... the set of letters.  Alternatively

    ALPHA = <[RFC5234]>

has more or less the same effect.

I am fine with that, but I was thinking to a different approach. I am going
to describe it in order to have an alternative on the table...  Besides,
the two approaches are not mutually exclusive.

Let me be a bit formal (but not too much), in order to avoid
misunderstanding

* A ruleset is a set of related ABFN rules (e.g., the set of rules that
describes a format).  A ruleset can have a name.

* A document (e.g., an RFC) can contain one or more rulesets (for example,
if describes a file format and some protocol format)

* A ABFN pragma (please suggest a different name, if you do not like
"pragma") is a directive that can appear before the ABFN ruleset.  Pragmas
begin with '@' (CSS style? or C-style '#'?) so that they cannot be mistaken
for rule names

* The following pragmas (pragmata? :-) are defined

      @ruleset <name>
      @import  <document>
      @import  rulesets <list-of-names> from <document>
      @import  rules <list-of-names> from <document>

here <document> is a document reference ([RFC...] or an URI), <name> is a
string that matches current definition of "rulename" and list-of-names is a
list of <name> separated by commas and optional spaces.  Note that the
"from" in the last two pragmas cannot be mistaken for a rule name since it
is not preceded by a comma.

The action of @ruleset is to give a name to the set of rules that follows.
The action of the three flavours of import is to... import one or more
rules in the current document.  @import could act in two ways (to be
decided)

(1) C preprocessor-style, that is, it "virtually copies" inside the current
document the rules from the imported one.

(2) Ada "with" style, that is, the imported rules live in their own
namespace and they can be referred to by using a full path, for example,

   RFC5234.ALPHA

or, if RFC5234 had a  ruleset named "foo"

    RFC5234.foo.ALPHA

My personal preference goes to the Ada style since it does not pollutes the
local namespace and it makes it clear where the imported rule was defined.
If you are worried about having names too long, we can define also a pragma
@rename, for example,

    @rename RFC5234.foo.ALPHA as Letters

Alternatively, one could define @use. The drawback of the Ada solution is
that it requires a small change in the ABNF grammar by adding "." to the
set of characters allowed in a name.

Of course, there are many possible variants and things need to be
formalized better, but this was just to give a rough idea about what I had
in mind.

Cheers,

Riccardo

On Sat, Oct 10, 2015 at 6:39 PM, Sean Leonard <[email-masked]> wrote:

> URL:
> https://www.ietf.org/internet-drafts/draft-seantek-abnf-more-core-rules-03.txt
> Status:
> https://datatracker.ietf.org/doc/draft-seantek-abnf-more-core-rules/
> Htmlized:
> https://tools.ietf.org/html/draft-seantek-abnf-more-core-rules-03
> Diff:
> https://www.ietf.org/rfcdiff?url2=draft-seantek-abnf-more-core-rules-03
>
> Abstract:
> _______________________________________________
> abnf-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/abnf-discuss
>
>

---

From: Riccardo Bernardini <[email-masked]>
Date: Sun, 11 Oct 2015 15:16:49 +0200

A doubt just came to my mind.  Suppose that in fictional RFC8888 I define

  ALFA = <[RFC5234]>
  DIGIT = <[RFC5234]>
  id = ALFA  *(ALFA / DIGIT)

then in RFC9999 I write

   id = <[RFC8888]>

This would require the implicit import of ALFA and DIGIT too, right?  In
general, whenever I import a rule I should import all the rules the
imported rule depend on.

---

From: Sean Leonard <[email-masked]>
Date: Sun, 11 Oct 2015 08:45:43 -0700

Hi Riccardo,

On 10/11/2015 6:16 AM, Riccardo Bernardini wrote:
> This would require the implicit import of ALFA and DIGIT too, right?  
> In general, whenever I import a rule I should import all the rules the 
> imported rule depend on.

No, it would not. Conceptually, the ABNF is compiled / substituted /in 
situ/, so it definitely would not import (or need to import) the other 
rules. That is the point, to keep it clean.

It would be like:

In RFC8888:
ALFA = %x41-5A / %x61-7A
DIGIT = %x30-39
then:
id = (%x41-5A / %x61-7A) *(%x41-5A / %x61-7A / %x30-39)

Then, in RFC 9999:
id = (%x41-5A / %x61-7A) *(%x41-5A / %x61-7A / %x30-39)

The definitions of ALFA and DIGIT (and even id, in its current form) 
from RFC 8888 would not be imported into RFC 9999; instead the 
definitions are modified so that they have no dependencies. This keeps 
the spaces separate. It would be possible for RFC9999 to define ALFA to 
something else if it wants, for example.

I am thinking of proposing a simplified syntax without the < > brackets. 
Namely:

    rulename-local = rulename-foreign@[RFC8888]
    rulename-local = rulename-foreign@urn:ietf:rfc:3986
    rulename = @[RFC8888]
    rulename = @urn:ietf:rfc:3986

In this syntax, @ becomes the distinguishing token.

Mainly, the < > brackets are not necessary...it's true that a 
non-supporting ABNF compiler would fail on the existence of @ (instead 
of "skipping it"), but then again, a non-supporting ABNF compiler would 
fail on %i, %s [RFC7405], and would fail even on the existence of 
prose-vals in < > (because prose-vals are not analyzable). (In contrast, 
an ABNF lexer/parser, such as a syntax highlighter, would not fail.)

I am also going to propose that authors should put import rules at the 
top of their figures for human readability reasons. If people want to 
copy and paste the ABNF, it would be easier to omit the import rules if 
they are all at the top. Not a lexical requirement however.

Sean

>
>
> On Sat, Oct 10, 2015 at 6:39 PM, Sean Leonard <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Here is rev 3. I believe this addresses on-list and off-list
>     concerns. Take a look...
>
>     Sean
>
>
>     From: [email-masked] <mailto:[email-masked]>
>     To:     Sean Leonard <[email-masked]
>     <mailto:[email-masked]>>
>
>
>
>     [email-masked] <mailto:[email-masked]>

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 11 Oct 2015 14:22:05 -0400

> something else if it wants, for example.

Essentially you are proposing to import "by value" rather than "by name".

I thought about this problem some time ago, and considered this 
approach. Unfortunately, while it works in the case shown, there are 
cases where it doesn't work. In particular, it doesn't work for rules 
that have recursive definitions. And in other cases, while it might work 
in principle, in practice it would be ridiculous - the resulting rule 
definition would be unmanageably long and involved.

For example, consider:

    Request = <[request@RFC3261]>

This is what led me to conclude that namespaces are needed, so that you 
can import rules by name rather than by value, yet have them in a 
separate namespace, so that they don't conflict with the rule names you 
are defining.

Happily, it would be relatively easy to extend the abnf grammar for 
namespace qualified rulename references, using "." to separate namespace 
from rulename. (E.g. import the core rules into namespace 'core', and 
then reference 'core.ALPHA'.)

Another smaller issue is that the <[RFCnnnn]> syntax isn't self 
contained. It depends on the context in which it appears to supply the 
name of the rule to be imported. That

> In this syntax, @ becomes the distinguishing token.

Better.

But abnf is close to exhausting the possibilities for operator-based 
syntax using ascii special characters as operators. So one must be very 
careful about squandering the remaining possibilities. (I guess there is 
more potential in multi-character operator names, or perhaps for unicode 
characters outside of ascii.)

I am coming to the conclusion that it might be better to introduce some 
new possibilities for syntax. (E.g., something like the way "#" is used 
to get to preprocessor syntax in C.) This could provide some 
meta-control statements.

	Thanks,
	Paul

---

From: Sean Leonard <[email-masked]>
Date: Sun, 11 Oct 2015 18:22:42 -0700

On Oct 11, 2015, at 11:22 AM, Paul Kyzivat <[email-masked]> wrote:

> But abnf is close to exhausting the possibilities for operator-based syntax using ascii special characters as operators. So one must be very careful about squandering the remaining possibilities. (I guess there is more potential in multi-character operator names, or perhaps for unicode characters outside of ascii.)

Well, we appear to be making progress.

Regarding the point about “multi-character operator names”, that is exactly what RFC 7405 does: it defines %s"string" and %i"string". This leaves open %[^sibdx] (i.e., anything other than s = case-sensitive, i = case-insensitive, b = binary val, d = decimal val, x = hexadecimal val). From that perspective, we have a lot left over.

Regarding @ syntax: I am going to take this further as:

   import-rulename = [rulename] "@" (import-ref / uri-ref)

   import-ref = "[" VCHAR "]"

   uri-ref = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." ) ":" VCHAR

Essentially: @ can have something before it (rulename) or not; after it, the characters [ + ] and ALPHA + :  are well-defined. But this leaves open many additional options for @ + other thing combinations. There are still many punctuation characters left.

We can eliminate the [ ] option if we are confident that URIs can do the job, such as “urn:ietf:rfc:5234”. I think such unbridled optimism for URNs is misplaced at this time, but maybe that would kick the URNBIS WG to finish. ;)

Furthermore, we may observe that import-rulename can be a drop-in replacement (i.e., incremental alternative) for element:

   element   /= import-rulename

Therefore:

   URI-or-Mail = "{" ( URI@urn:ietf:rfc:3986 / addr-spec@urn:ietf:rfc:5322 ) "}"

Makes a good deal of sense, and gets the “namespaces” that Paul seems to want quite a bit. (Note the parallel between this construct and XML namespaces...) And for those who want a more compact alias for an import source than a full URI, “addr-spec@[RFC5322]” works pretty compactly, and the alias notably is under the control of a document author.

Sean