Subject: identifier in draft-ietf-httpbis-header-structure-01.txt

From: Alex Rousskov <[email-masked]>
Date: Sat, 29 Apr 2017 11:29:17 -0600

Hello,

    In draft-ietf-httpbis-header-structure-01.txt, "integer" is
"identifier":

> integer = ["-"] 1*19 DIGIT

> identifier = token  [ "/" token ]

> token          = 1*tchar
> tchar          = ... / DIGIT / ...

Does not that contradict the self-identification claim? If it does not,
then what do we mean by "self identification" exactly?

Thank you,

Alex.
P.S. There is a similar problem with "integer" and "timestamp", but I do
not want "all timestamps ought to be integers" arguments derail the
discussion. Clearly, not all integers are identifiers.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 29 Apr 2017 17:45:42 +0000

--------
In message <[email-masked]>, Alex

>then what do we mean by "self identification" exactly?

"Self-identification" refers to wrapping the entire header in > ... <

And yes, in HTTP/1.1 serialization you cannot tell if a particular
identifier is a word or a number, you need to definition of the
header in question to tell you that, that's the cost of a very
concise definition and backwards compatibility with standardized
headers.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Alex Rousskov <[email-masked]>
Date: Sat, 29 Apr 2017 17:22:01 -0600

On 04/29/2017 11:45 AM, Poul-Henning Kamp wrote:
> Alex Rousskov writes:
>>  In draft-ietf-httpbis-header-structure-01.txt, "integer" is "identifier":
>>

> "Self-identification" refers to wrapping the entire header in > ... <

Understood, thank you.

> concise definition and backwards compatibility with standardized

The "cost" part does not compute: A grammar without any "integer"
element would be even more "concise" and still backwards compatible.

I suspect you want to keep "integer" in the grammar (despite the fact
that it necessarily increases grammar complexity) to increase semantic
expressiveness/precision of some field definitions: Sometimes, you want
to say "integer" instead of vague "token". We do not have to make the
grammar ambiguous while achieving that particular goal! For example, we
could define "identifier" as a "token" that is not an "integer".

Do we care whether the grammar is ambiguous? We might. With an ambiguous
grammar, a generic >...< field parser would need "configuration" from
the semantics layer to build the right parsing tree. This configuration
increases generic parser API complexity (and often decreases parser
performance).

I realize that "integer" is not the only thing that makes the current
grammar ambiguous. I suspect we can get rid of all such ambiguity if we
want to, without sacrificing grammar "conciseness" and expressiveness.

IMHO, if the grammar remains ambiguous, then we should explicitly state
that because that property may seriously affect parser implementations.

Thank you,

Alex.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sun, 30 Apr 2017 07:48:19 +0000

--------
In message <[email-masked]>, Alex

>element would be even more "concise" and still backwards compatible.

The serialization in HTTP/2/HPACK+ or QUIC might distinguish between
identifiers and numbers for efficiency reasons.

>that because that property may seriously affect parser implementations.

Header Structure is not a syntactical specification, so the ambiguity
is not a problem.  HS is more like a vocabulary trying to prevent
future headers from being needlessly inventive.

---

From: Alex Rousskov <[email-masked]>
Date: Sun, 30 Apr 2017 10:09:08 -0600

> identifiers and numbers for efficiency reasons.

I do not see why "HTTP/2/HPACK+ or QUIC might distinguish" but HTTP/1
must not. HTTP/1 parsers do care about "efficiency".

> Header Structure is not a syntactical specification, 

You have fooled several people into thinking that the draft specifies
syntax rules (among other things). If the draft is not a syntactical
specification, then please remove all rules that contain ">", "<", ":",
'.', and other syntactical details and formats. Those things do not
belong to a pure "vocabulary" specification.

> so the ambiguity is not a problem.

Ambiguity is usually a "problem" for any specification, "syntactical" or
not. However, if you remove syntactical details, then there will be no
implied ambiguity in the draft. Keeping the abstract structure
unambiguous could then become a goal of various syntactical
specifications based on the draft (e.g., a draft covering HTTP/1 rules
could define "integer" syntax to never clash with "identifier", and
another draft for QUIC will accomplish the same goal using different
means). I am not implying you _should_ remove syntactical details, but
doing so would address my concerns.

> HS is more like a vocabulary trying to prevent

I fully support the intent, but the draft does not yet reflect it well
IMO. The Daft is currently full of vocabulary-unrelated details such as
specific delimiter characters and "formats".

IMHO, you have to decide where to stop:

Level 1: Specify vocabulary and associated abstract structure to help
minimize needless invention. Limiting the Draft scope isolates it from
syntax-related problems (such as needless ambiguity). It also naturally
limits its practical influence (but other drafts can cover the holes by
providing the necessary syntactical details).

Level 2: Add field syntax to help folks inventing fields and building
field parsers for specific protocols. The Draft ability to minimize
needless invention would significantly increase, but you would have to
deal with syntax-level concerns such as integer-vs-identifier ambiguity.

AFAICT, the current Draft works at Level 2 but you are dismissing
problems as if it is limited to Level 1.

Cheers,

Alex.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sun, 30 Apr 2017 18:50:12 +0000

--------
In message <[email-masked]>, Alex
 Rousskov writes:

>syntax rules (among other things). 
>[...]

Exactly!

That has been an issue with this document from the start.

The only syntax ("parser level") rule is "The data model of Common
Structure is an ordered sequence of named dictionaries."

I've often been wondering if that is a bridge too far for this
document.

Maybe it should stop instead at the "lexical" level, and only define
a family of "tokens", and leave it to the individual header-specifying
documents to define their precise order ?

One of the tokens would absolutely be 'dictionary', but the requirement
that the *only* compliant top-level structure of a header be 'list
of dictionaries' would disappear.

The argument "contra" is mainly that you will not be able to use the
same generic data structure to represent an arbitrary HS header.

The argument for the generic data structure is not very strong,
"IETF doesn't do APIs" and all the headers listed in A.5 would need
special-casing anyway.

The best argument "pro", apart from document clarity, is that we
can grandfather even more headers.  Not quite all of A.5, but it
brings some rather interesting headers into view of semantic
compression, notably all the ones with timestamps.

That way we could lift the art of HTTP header specifications a
couple of very usable steps above ABNF:

	"Range:" (
		"bytes" "=" 1# (
			<POSINT first-byte-pos> "-"
			/
			<POSINT first-byte-pos> "-" <POSINT last-byte-pos>
			/
			"-" <POSINT suffix-length>
		)
		/
		<TOKEN other-range-unit> "=" <1* VCHAR other-ranges-specifier>
	)

	"Retry-After:" ( <DATE when> / <POSINT delay-seconds> )

	"User-Agent:" * (
		RWS (
			<TOKEN product> [ "/" <TOKEN product-version> ]
			/
			<COMMENT comment>
		)
	)

I would *love* if the result was machine-readable.

The value of fuzzing-generators working directly from spec has been
documented for 40 years and needs no further argumentation.

It would certainly also be possible to have a code-generator spit
out H1 parsers/validators for the headers from this.

I can see how you could automatically produce definitions of
semantic on-the-wire compression schemes from this, but
security reasons would probably veto that.