Subject: Header Structure: goals

From: Mark Nottingham <[email-masked]>
Date: Wed, 14 Jun 2017 16:51:05 +0100

Picking this thread back up, my personal (not chair) observations...

I'd thought that the purpose of surveying existing headers in the draft was to learn commonly used structures, so as to help make what we come up more familiar (and thus easier to use), not to come up with something that will replace existing header definitions.

To me, the most important thing we're doing here is creating tools / guidelines for authors of *new* header fields to use, so that they don't a) fall into the many pitfalls of doing so, and b) reduce work/problems for implementers when generating/processing those headers.

(b) very well may involve specifying syntax.

Future consolidation of existing headers is a nice-to-have, but as discussed quite a bit, may require some awkwardness that makes it undesirable. My understanding was that this was a non-goal for this work.

Likewise, creating another header serialisation / compression that takes advantage of what we do is something we should keep an eye on, but isn't an immediate goal.

I don't *think* there's anything new above; it was discussed quite a bit in the threads leading up to PHK's draft (which AFAICT we adopted because it was going in a non-JSON direction). However, it seems like we're drifting back into that discussion again.

Chair hat back on -- Please discuss. We've gone back and forth significantly on this, so it'd be good to get a crisp set of goals that we can agree to in the draft.

Cheers,

> On 26 Apr 2017, at 1:04 pm, Amos Jeffries <[email-masked]> wrote:
> 
> On 26/04/17 11:16, Matthew Kerwin wrote:
>> 
>> 
>> On 26 April 2017 at 07:43, Poul-Henning Kamp <[email-masked]> wrote:
>> --------
>> In message <[email-masked]>, Amos Jeffries
>> writes:
>> 
>> >Reading section 7.1 I am wondering if this is significant enough to use
>> >HTTP/1.2 version number for 1.x agents as the signal that the sender can
>> >receive any header in self-describing Common Structure.
>> 
>> That's an interesting question...
>> 
>> 
>> ​Very interesting.  The request line is a hop-by-hop message, right?  So it couldn't be used as a signal for end-to-end headers.
> 
> I'm thinking that headers affected by this signal would have a 'legacy' format - for example Date. So this use-case would be hop-by-hop opportunistic performance optimization. Any agent using it would need to be able to map.
> 
>> How many middleboxen out there are likely to inspect the request line only as far as 'HTTP_VERSION > 1.0' and then barf if one of their response headers looks weird?  Although I suppose the answer is not that different from: how many will forward X-Accept-Fancy-Headers blindly and then barf...  Pity there was no strong hop-by-hop vs end-to-end flag on headers.
>> 
>> Cheers
>> -- 
>>   Matthew Kerwin
>>   http://matthew.kerwin.net.au/
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 14 Jun 2017 16:57:04 +0000

--------
In message <[email-masked]>, Mark Nottingham wri
tes:

>Future consolidation of existing headers is a nice-to-have, but as 
>discussed quite a bit, may require some awkwardness that makes it 
>undesirable. My understanding was that this was a non-goal for this 
>work.

It was my goal to at least not make it impossible.

>Likewise, creating another header serialisation / compression that takes 
>advantage of what we do is something we should keep an eye on, but isn't 
>an immediate goal.

Give the rapidly increasing volume of cryptographically whitened
data in headers, it might be a good idea to try to steer that data
towards a syntax which HPACKng/QUIC/whatever can recognize and
transmit as binary, to avoid the b64+compression detour.

Of course nothing prevents a compressor from speculatively looking
at a substring to see if it can be transmitted more efficiently by
un-b64'ing and then re-b64 it on the far side.

Somebody[tm] should really simulate that...

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Julian Reschke <[email-masked]>
Date: Fri, 16 Jun 2017 09:02:15 +0200

> Chair hat back on -- Please discuss. We've gone back and forth significantly on this, so it'd be good to get a crisp set of goals that we can agree to in the draft.

We are clearly not making sufficient progress solving the problem we 
wanted to solve - to make it easier to define new header fields with a 
syntax that is not "broken" and where processors can use off-the-shelf 
components to do the parsing.

Note that this is not a complaint vs PHK - it's a WG working item after 
all. Without concrete feedback, it's hard to make progress.

Best regards, Julian

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 16 Jun 2017 07:42:32 +0000

--------
In message <[email-masked]>, Julian Reschke writes

>components to do the parsing.

There are two possible overall aproaches to this:

1) The RFC which defines a new header includes a machine-readable
   specification you can copy&paste into your project, where through
   some mechanism of your choice, that turns into parser code.

   This makes the draft harder to produce, becaus we have to invent
   and specify that machine-readable specification language and
   provide reference implementations for a parser-generator and
   a syntax-checker tool for RFC writers.

   Advantages:

   * The parser and the representation of the parsed result can be
     optimized for your programming environment, for instance with
     respect to datatype for numbers etc.

   * Runtime performance.

   * HPACKng can use the same machine-readable specification(s) to
     optimize compression.

   * The parser can detect headers with RFC-extraneous fields.

   Disadvantages:

   * You get none of these advantages until you've read the RFC and
     copy&pasted the machine-readable spec into your project.

2) The RFC which defines a new header merely stays inside the dotted
   lines of HS, and a generic parser builds an internal represenation
   of each instance of the header, which your code can query, based on
   your reading of the RFC.

   That is basically the draft we have today, after scrutiny and
   text-processing.

   Advantages:

   * Your parser also supports RFCs you have not read.

   Disadvantages:

   * The represenation of the parse-tree will be bulkier and less
     efficient.

   * Probing of the parse-tree from the application is clumsy and
     more error-prone.

   * RFC-extraneous fields in a header will not be detected unless
     the application programmer does it manually.

Number two is close to impossible to avoid:  It will be reimplemented
by application programmers using regexps to take headers apart if
we don't provide for it.

So by and large, the question before us is if number one is worth
the effort, and I think that hinges on the HPACKng advantage.

The alternative take on HPACKng, would be a speculative semantic
compressor, which hunts for timestamps, numbers, and base64-encoded
substrings which can be transmitted more efficiently semantically
than by text-compression.

Again, the advantage of that model is that you don't need to read
the RFCs first to reap the advantage.

So summa summarum:  It probably depends more than anything on how
many new headers we expect and what they transmission volume and
semantic compression potential will be.

I'm still not sure where I personally land on this, but given the
WG activity on this draft, I guess going for number one would fall
almost entirely on me, in which case it won't happen...

>Note that this is not a complaint vs PHK - 

No worries.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 20 Jun 2017 12:21:14 +1000

Personally -- I'd be fine with #2; a machine-readable schema can come later. Schema languages have their own set of pitfalls, and defining one now seems... ambitious.

What I really want is a vocabulary that spec authors can use unambiguously, and a processing model that takes as many of the nasty questions about parsing headers off their shoulders as possible.

Cheers,

> On 16 Jun 2017, at 5:42 pm, Poul-Henning Kamp <[email-masked]> wrote:
> 
> --------

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 20 Jun 2017 21:14:16 +1200

>    * The represenation of the parse-tree will be bulkier and less

AFAIK the parsers in used today either do that already for a custom 
structure layout, or are loaded into memory themselves and do the parse 
as-needed.

So, it seems to me the generic parser being smaller and not needing to 
be loaded during the header lookups should counter that inefficiency 
somewhat. I was hoping to have some numbers of this from Squid by now, 
but life got in the way.

>    * Probing of the parse-tree from the application is clumsy and

That is the biggest issue IMO. But it is also one that is already 
present in code, so YMMV whether there is an overall gain or not.

Amos