Subject: #1047 and #1132 - ABNF for USEFOR 3.1.6

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 13 Sep 2005 10:10:15 +0200

Here's my current proposal, based on the discussion.

   path            =  "Path:" SP path-list CRLF

   path-list       =  [FWS]
                      *( ( path-identity / path-keyword / path-diagnostic ) 
[FWS]
                      path-delimiter [FWS] )
                      tail-entry [FWS]

   path-identity   =  ( ALPHA / DIGIT )
                      *( ALPHA / DIGIT / "-" / "." / "_" )

   path-keyword    = "POSTED" / "MISMATCH"

   path-diagnostic = 1*( ALPHA / DIGIT / "-" / "." / ":" / "_" )

   tail-entry      =  path-identity

   path-delimiter  =  "!" / "!!"

   A <path-identity> is a name identifying a site. It takes the form of a 
domain
   name having one or more components separated by dots.

   Each <path-identity> in the <path-list> (excluding the one in the
   <tail-entry>) indicates, from right to left, the successive agents
   through which the article has passed.  The <keyword> "POSTED"
   indicates that the agent to its left injected the article.  The use
   of the <path-delimiter> "!!" indicates that the agent to its left
   claims that the agent to its right was the verified source of the
   article (whereas the <path-delimiter> "!" implies no such claim).
   The <keyword> "MISMATCH" indicates that the agent to its right failed
   to be so verified.

A path-diagnostic is an item inserted into the Path header for purposes 
other than to indicate the name of a site.
One commonly observed usage is to insert an IP address. The colon (":") is 
permitted in order to allow IPv6 addresses to be inserted; note that this 
will cause interoperability problems at older sites that regard ":" as a 
path-delimiter and have neighbours whose names have 4 or fewer characters, 
and where all the characters are valid HEX digits.

[USEPRO] gives details on how to choose a path-identity and how to 
construct the Path: header.

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 13 Sep 2005 12:24:53 +0200

> path-diagnostic = 1*( ALPHA / DIGIT / "-" / "." / ":" / "_" )

Here you hae one problem:  Each <path-identity> is also a
<path-diagnostic>.  If the latter doesn't start with ALPHA or
DIGIT (and/or if it contains a colon) it's clear what it is.

Like Charles you also have an "_" in the <path-identity> for
the non-<bareword> case.

Charles versions have it clear that there are one or more
path-identities optionally followed by one "diagnostic"
per site (= <path-entry> in his proposals).

Your version skips this detail, in theory it allows adjacent
<path-diagnostic>s or <path-keyword>s like...

..!MISMATCH!MISMATCH!.. or ..!MISMATCH!abc::def!MISMATCH!.. or
..!abc::def!abc::def!.. or ..!abc::def!MISMATCH!abc::def!..

I'm not sure how bad that is.

Apparently you prefer the "let's not invent new stuff
in addition to '!!'" approach, taking "!MISMATCH!" and
"!POSTED!" as given.

That's in the direction of Charles' first variant.  The
concept of "one or more identity per site" in Charles'
versions isn't critical:

If a site wants this, it can just do it, and if it adds
diagnostics for its own multiple identities it's stupid
but no real problem.

With that simplification here's another version, a wild
mixture of your proposal and Charles' first version:

 path          = "Path:" SP [FWS] path-list [FWS] CRLF
 path-list     = *( path-entry path-sep ) tail-entry
 path-entry    = path-identity [ path-diag ]

 tail-entry    = path-identity  ; typically "not-for-mail"

 path-identity = ( 1*( label "." ) toplabel ) / bareword
 bareword      = ALPHA *( [ "-" / "_" ] alphanum )
 label         = alphanum [ *( alphanum / "-" ) alphanum ]
 toplabel      = ALPHA    [ *( alphanum / "-" ) alphanum ]
 alphanum      = ALPHA / DIGIT

 path-diag     = [ path-sep "POSTED" ]
                 [ path-sep ( IP-address / path-identity ) ]
                 [ path-sep [ "MISMATCH" ]]

 IP-address    = IPv4address / IPv6address  ; see [RFC 3986]
 path-sep      = [FWS] "!" [FWS]

The lost "!!" is actually still there:  "!MISMATCH!" without
the "MISMATCH" is "!!".

Showing the <source-identity> in the case of a match strikes
me as nonsense, and a !MISMATHCH! without <source-identity>
is also dubious, here's a cleaner solution:

 path-diag     = [ path-sep "POSTED" ]
                 [ path-sep (( IP-address / path-identity )
                               path-sep "MISMATCH"          ) ]

With that trick IPs appear only to the left of a "!MISMATH!",
or in other words, anything to the left of "!MISMATCH!" is a
<source-identity> (in Charles' terminology).

Anything else is a <path-identity> or the <tail-entry> or the
keyword "POSTED".  Bill's parser says okay for both variants
of my <path-diag>, but it won't see if there are too many or
not enough "!" somewhere.
                          Bye, Frank

P.S.:  <bareword> must start with ALPHA, is that as we want
       it, or is <alphanum> good enough ?

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 13 Sep 2005 13:07:06 +0200

>                  [ path-sep (( IP-address / path-identity )

Ugh... too easy to get it wrong, the MISMATCH case must be
optional:

   path-diag     = [ path-sep "POSTED" ]
                   [ path-sep [( IP-address / path-identity )
                                 path-sep "MISMATCH"          ] ]

Hiding the "!!" as "!...!MISMATCH!" without the "...!MISMATCH"
is bad, actually that would be [FWS] "!" [FWS] [FWS] "!" [FWS]

One [FWS] too many for my tastes.

   path-diag     = [ path-sep "POSTED" ]
                   [ ( path-sep ( IP-address / path-identity )
                       path-sep "MISMATCH" ) / ( [FWS] "!" ) ]

Now the "!!" case is [FWS] "!" path-sep, and that's the same
as [FWS] "!" [FWS] "!" [FWS]
                               Sorry, Frank

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 13 Sep 2005 13:18:00 +0200

> DIGIT (and/or if it contains a colon) it's clear what it is.

That's right.
Used as a parsing grammar and looking for names, that's not a problem - you 
may have a few more tokens to match against your neighbour list that were 
intended as diagnostics.

I thought that making the grammar more complex, without anything resembling 
a group consensus on what a diagnostic "should" look like, was not worth 
the effort.

> Like Charles you also have an "_" in the <path-identity> for

Yep. Allowing _ in hostnames seems to be an old tradition for USEFOR 
drafts, and I didn't change it for this.

> I'm not sure how bad that is.

Yes, it does. And it has to - by design, Charles' text for MISMATCH:

      If it does not match then it prepends instead two entries
      to that content; firstly the true established <path-identity> of
      the source followed by a '!', the <path-keyword> "MISMATCH" and a
      further '!', and then, to the left of that, its own <path-
      identity> followed by a '!!' <path-delimiter> as usual.

Given that the "true established path-identity" is likely to be a 
diagnostic (typically an IP address), not an identity, that will produce 
something that parses as

    <path-identity>!!<path-keyword>!!<path-diagnostic>

And given that Charles' examples showed no hard and fast rules for where 
these diagnostics pop up in Path, I think it's a stretch to mandate 
particular patterns of these beasts.

> in addition to '!!'" approach, taking "!MISMATCH!" and

I definitely tend to "less is more", yes - and not touching the previously 
established consensus for reserving MISMATCH and POSTED, and adding the !!.
I'd really like to get finished with this document.

No comments on the details of the alternate grammar.... it's definitely 
more specific, but I wonder if being more specific is good here.

                        Harald

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 14 Sep 2005 14:05:35 GMT

>a group consensus on what a diagnostic "should" look like, was not worth 

I think we need to make it clear what diagnostics are for. That's why I
called them "source-identities", and USEPRO will make it clear that they
can be prepended to indicate the observed identity of the sending site (as
opposed to the <path-identity> that the sending site claimed).

>    <path-identity>!!<path-keyword>!!<path-diagnostic>

No, that does not seem right. What my text produces is

     <path-identity>!!<path-diagnostic>!<path-keyword>!

[Note that my text is telling you to "prepend" two things, so they land up
in the opposite order.]

>particular patterns of these beasts.

I hope I had made it clear that they land up _after_ whatever
<path-identity>s were placed there by the site creating the <path-entry>.

>I'd really like to get finished with this document.

I think we need to discuss and decide upon the alternative suggestion
(Russ, myself, Forrest) for putting some distinctive syntax in the
diagnostic/source-identities (precise syntactic details negotiable).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 14 Sep 2005 13:46:06 GMT

>per site (= <path-entry> in his proposals).

The essential difference is that my grammar indicated what you should
generate (and gives a strong clue to what USEPRO will tell you when it
comes to creating a Path) and Harald's indicates more what should be
accepted. If we adopt the Harald style, then there will be somewhat more
to say in USEPRO.

However, I think Harald's present version is still too ambiguous. If we
are going to distinguish between <path-identity>s and <source-identity>s,
all built out of an assortment of barewords, FQDNs and IP-Addresses, then
the syntax should reflect this more. Hence I would support Frank's style
below if we are not going for my fuller version.

>mixture of your proposal and Charles' first version:

I have quoted this using Frank's correction in his second message.

> tail-entry    = path-identity  ; typically "not-for-mail"

See my previous message for why that is wrong

> toplabel      = ALPHA    [ *( alphanum / "-" ) alphanum ]

And see my previous message for why that is wrong. Essentially,
"123.123.123.foo" is neither an FQDN nor an IP-Address, so no need to
allow it. I suggest:

   path-identity = fqdn / bareword
   fqdn = 1*( label "." ) label
   label = ALPHA *( [ "-" / "." ] (ALPHA / DIGIT) )

> path-sep      = [FWS] "!" [FWS]

But now you have allowed "!FWS!" for "!!", and I don't think we want that.
I avoided that by only allowing [FWS] _after! a "!", not before it. But
other fixes are doubtless possible.

>is also dubious, here's a cleaner solution:

Unfortunately, showing the <source-identity> in the case of a match is a
widespread current practice. So I think we have to allow it (maybe with
keywords SEEN/MATCH). But I would be quite happy to explain, in USEPRO,
why it was a bad idea.

>       it, or is <alphanum> good enough ?

I think both <bareword> and <Label> (hence <fqdn>s) need to start with an
ALPHA. That way, you can always distinguish what you have got, and the
only remaining ambiguity in the grammar is whether what you are looking at
is a <path-identity> or a <source-identity>.

>concept of "one or more identity per site" in Charles'

Now my second variant seeks to make it syntactically clear whether you see
a <path-identity> or a <source-identity>. That is an orthogonal issue to
whether the syntax is written in Charles/Harald/Frank style, and we need
to discuss it on its merits.

It arose from a suggestion by Russ, which I worked up into a concrete
syntax (it was discussed some weeks back with some support) and it was
reinforced by a request from Forrest last week. Naturally, the precise
syntax, and whether you need all of "SEEN", "MATCH" and "MISMATCH" can be
discussed. But it would simplify the present discussions if we could
decide pretty soon whether we want to follow that route.