Subject: Re: ABNF sets and sequences

From: <[email-masked]>
Date: Wed, 12 Jul 2000 13:56:35 +0100

Keith,

> one problem I have with the
>
>         valid = { [ a ] b *c }
>
> notation is the use of [ ] and * to mean things that are actually
> quite different from the use of these symbols outside the { and }.
>
> while it's true that human languages often change the meaning of
> words or symbols based on subtle differences in context, a language
> like ABNF, intended for use in writing technical specifications, needs
> to be less ambiguous than that.   especially when in practice it seems
> likely that there will be a need to use both kinds of * or
> [ ] in close proximity to one another.

OK. I'm quite happy to replace "*" with let's say "+" within a set and
drop square brackets in favor of angle brackets giving us a set defined as
follows:

        set = "{" 2*( *c-wsp member ) *c-wsp "}"
        member = [repeatm] rulename / "<" rulename ">"
        repeatm = (*DIGIT "+" *DIGIT)

> that, and I can't see any way, using the notation above, to say
> something like "you can define your own names for additional parameters
> but none of these may appear more than once". as would presumably
> be the case for content-disposition parameters, content-type parameters,
> RFC 1894 per-recipient or per-message indicators, etc.

I agree that the above notation can't handle this.

> now, we could pick different characters to mean "it's an error if
> this symbol is matched more than once within this rule" (say %) or
> "it is an error if this symbol does not match at least once within
> this rule" (say &).  so we would then have
>
>                header = *(
>                                %&to_field /
>                                %from_field /
>                                %sender_field /
>                                %subject_field /
>                                received_field
>                )
>
> but offhand I still don't see a way to specify "sender_field must be
> present if from_field is missing" with a notation like this.

If we introduce a new type of alteration operator (say, "!") between
rulenames within parens within a set to mean non-exclusive OR, then using
my modified notation we could specify

        header = {
                to_field
                ( from_field ! sender_field )
                < subject_field >
                +received_field
        }

or using your notation

        header = *(
                 %&to_field /
                 %£( from_field ! %sender_field ) /
                 %subject_field /
                 received_field
        )

> and I still find it both clearer, and more useful from an implementor's
> point-of-view, to separate "parsing" from other kinds of syntax
checking.

I don't disagree with you here. In my mind the issue has always been
notational clarity to the reader, and I'm not sure despite the examples
above, using "!", or to a lesser extent "+" in place of "*", "<" in place
of "[" and ">" in place of "]" helps achieve this! At the same time, if we
could find a clean or even elegant way of dealing with unordered sets
notationaly, I think it would be a plus.

Nick

Nick Shelness, IBM Fellow
Chief Technology Officer
Lotus Development Corp
Tel: (617) 693-7484

Assistant: Mary McGurran
Tel: (617) 693-4321

---

From: Keith Moore <[email-masked]>
Date: Wed, 12 Jul 2000 09:33:14 -0400

interesting ideas for notation.

I'm all for increasing clarity to the reader, as long as we don't
lose precision in doing so.  There is a point (probably different 
points for different readers) at which additional notation does 
more harm than good.  We may have already reached this with the
current version of ABNF.  If we do extend ABNF we need to find 
a very 'clean' notation for doing so if we hope to increase the 
clarity.

I also think that there is a (somewhat surprising) tension between
writing a spec so that the reader can understand what is intended
and writing a spec so that different readers can accurately translate 
it into code that interoperates.  each is surely important, but an 
optimization for one case may be detrimental to the other.

Keith

---

From: <[email-masked]>
Date: Wed, 12 Jul 2000 15:02:55 +0100

Keith,

I think that we are in almost total agreement

> lose precision in doing so.

We absolutely cannot lose precision in this process. The only reason I am
devoting the effort to it that I am is that I believe that we are in need
of more rather than less syntactic precision, but this cannot be achieved
if the cost involves a loss of clarity.

> There is a point (probably different points for different readers)
> at which additional notation does more harm than good.

Agreed.

> We may have already reached this with the current version of ABNF.

I'm a bad judge of this as I'm very comfortable with ABNF's notational
style.

> If we do extend ABNF we need to find a very 'clean' notation for
> doing so if we hope to increase the clarity.

Agreed.

> it into code that interoperates.

In a previous life I was a production compiler implementor. My experience
in this space is that while most experimental compilers employ automated
parsers, almost all production compilers employ recursive descent. My gut
feeling is that the same is true in this space. Translating set notation
of the sort Dave, Robert and I are proposing into iterative case
statements with associated booleans (checked on iteration exit) and
counters checked on each case invocation is a pretty straight forward
process! Heck, it can even be automated.

> each is surely important, but an optimization for one case may be
> detrimental to the other.

Again, agreed.

Nick

Assistant: Mary McGurran
Tel: (617) 693-4321