Subject: Structured Headers: URI type (#782)

From: Mark Nottingham <[email-masked]>
Date: Thu, 2 May 2019 15:15:17 +1000

(Editor hat on)

<https://github.com/httpwg/http-extensions/issues/782>

PHK and I have discussed this, and I think we agree that this issue should be closed without any change to the specification. 

Any further discussion? We'd like to get this spec shipped.

Thanks,

--
Mark Nottingham   https://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Thu, 2 May 2019 07:39:23 +0200

> Any further discussion? We'd like to get this spec shipped.

I don't think that the discussion in the github issue really concluded.
It would be nice if Poul-Henning would follow-up on my replies.

FWIW, my latest comment was: "I think the ability to apply this to the
Link header field is pretty compelling.".

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 2 May 2019 15:45:19 +1000

Could you explain why it's compelling?

It's already possible to map a link header to the existing data structures (there are a couple of ways you'd do it).  It's true that the syntax wouldn't be identical to a Link header, but that hasn't been a goal for other parts of SH; in this sort of situation, we've assumed that some mapping function would be necessary (e.g., negotiating to send the new header as SH-Link, or some other mechanism).

I'd be much more supportive of doing a URI type if we could get agreement for *and* implementation of a common data model for URIs in SH. However, it appears that there's disagreement about that; browser folks want to reuse the WHATWG URI specification (which is sensible, from their viewpoint), whereas others don't like that spec. Exposing it as just a string doesn't seem to add much at all.

We could talk about this a lot more, but I suspect it would take at least a few months to conclude the discussion. I'd rather ship the spec; if we can agree on a new type down the road, we can always add it with an update.

Cheers,

> On 2 May 2019, at 3:39 pm, Julian Reschke <[email-masked]> wrote:
>

---

From: Julian Reschke <[email-masked]>
Date: Thu, 2 May 2019 07:58:21 +0200

> It's already possible to map a link header to the existing data structures (there are a couple of ways you'd do it).  It's true that the syntax wouldn't be identical to a Link header, but that hasn…

I understand it was not a goal, but that doesn't mean it wouldn't be
nice to get to that as closely as possible...

> I'd be much more supportive of doing a URI type if we could get agreement for *and* implementation of a common data model for URIs in SH. However, it appears that there's disagreement about that; b…

It indeed just adds a new syntax (no processing model is needed, true).
For URIs, it's kind of nice as it uses a delimiter that can't be used in
URIs anyway.

> We could talk about this a lot more, but I suspect it would take at least a few months to conclude the discussion. I'd rather ship the spec; if we can agree on a new type down the road, we can alwa…

That is true, but then updating both spec and implementations is always
more work then including things from the start. It's a trade-off.

 From a practical point of view, would you expect the spec to be
replaced by a new one, or should there be an extension point/hook so a
delta spec can be written?

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 2 May 2019 16:08:33 +1000

> nice to get to that as closely as possible...

True. It just seems like a poor tradeoff for just one header field.

> more work then including things from the start. It's a trade-off.

Of course. To me we're at diminishing returns here, and the friction against adding a new type later is pretty low (based upon the experience of building this set of types so far, in the spec, implementation, and tests).

> delta spec can be written?

I'd think it'd just be a matter of writing a spec that updates this RFC, defining an abstract type along with its textual parsing and serialisation algorithms. The biggest concern would be making sure that the first character of the textual serialisation identified the type (not a hard requirement, but it's the convention we've been using so far, so it'd need to be a good reason to break it, I think). In this case, that's pretty clearly "<".

Cheers,

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 02 May 2019 06:19:39 +0000

--------
In message <[email-masked]>, Julian Reschke writes

>It would be nice if Poul-Henning would follow-up on my replies.

I really dont have anything to add to what I said a month ago.

I can see the point, but not the compelling argument.

First, It's not like we cannot move a URI in SH.  Sh-string are
perfectly capable of that, and I put sh-binary in the spec as
"Kernighan-Escape"[1], should that fail.

Second,  If demarcating URIs in SH was important, somebody would
surely have spotted the deliberate mistake in the strawman syntax
I proposed a month ago ?

So no, I'm 100% with Mark that URIs are not worth another several
months of nothing much happening to this draft.

Poul-Henning

[1] https://www.lysator.liu.se/c/bwk-on-pascal.html section 2.6

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 2 May 2019 08:51:22 +0200

> I really dont have anything to add to what I said a month ago.

What I was referring to was the confusion about the syntax of the Link
header field.

> surely have spotted the deliberate mistake in the strawman syntax

I don't accept that as a serious argument.

> ...
Once again, please let's argue based on facts, not opinions. Unless you
can prove that it'll take months to add this.

Best regards, Julian

---

From: Matthew Kerwin <[email-masked]>
Date: Thu, 2 May 2019 17:25:08 +1000

> Any further discussion? We'd like to get this spec shipped.

For what it's worth, I'm still recovering from my ordeal with file://
so the thought of touching URLs again is unappealing.  The simplest
thing to do (aside from "nothing") would be to use whatever already
exists.  So if we were to define a URI type I would recommend using
whatever "Link:" already uses.  Looking at RFC 8288, that's
"URI-Reference" from 3986.

However, I think Link is a bad example to build off because it is a
"parameterised" thingy, but we've already restricted the set of
parameterisable thingies in SH to just sh-tokens.  So we couldn't
recreate Link in SH even if we wanted to, without even more work.  So
why bother adding a new type for it?  And we don't need a structured
type for Location, because that's not structured per se.

So I'm for closing with no action.

Cheers
-- 
  Matthew Kerwin
  https://matthew.kerwin.net.au/

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 02 May 2019 07:29:29 +0000

--------
In message <[email-masked]>, Julian Reschke writes

>What I was referring to was the confusion about the syntax of the Link

It didn't change my analysis, so I didn't specifically reply to that.

>I don't accept that as a serious argument.

It is nontheless a weighty part of my reasoning:  We have worked
on this spec for 30-ish months, URIs came up only two months ago
and nobody has paid the discussion much if any attention.

That doesn't communiciate "Important Oversight in Draft" to me.

>can prove that it'll take months to add this.

We would have to agree on a syntax, and if you think RFC3986 and
UTF-8 can be decided on 'facts not opinions' I have a cheap DIY kit
for a bikeshed I can sell you.

Once we have painted that, we need to write the text, and the parsing
algorithem, issue an new rev the draft, and wait for weeks before
WG last call, and then we're in the middle of summer vacation land.

QED.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 2 May 2019 09:55:27 +0200

> "URI-Reference" from 3986.

I would be ok with that, but I would also agree to just allow anything,
and to delegate error handling up one layer.

> So I'm for closing with no action.

That's actually a good point, but then there's the recent
<https://github.com/httpwg/http-extensions/issues/797>:

> in Signed Exchanges, @jyasskin asks about having Parameterised Lists whose parameterised identifiers are things other than Tokens.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 3 May 2019 17:15:22 +1000

> On 2 May 2019, at 5:55 pm, Julian Reschke <[email-masked]> wrote:
> 
>> 

I don't see the connection.

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 3 May 2019 19:04:43 +1000

> I don't see the connection.

'Link' is a parameterised list of URI references.  If we add a URI
reference type, and param-list is extended to allow URI references
alongside tokens, then it could maybe fit.

Though then there are "what to do if you receive a Link header value
with parameterised tokens instead of parameterised URI references"
stuff, which is why I'm still not 100% sold.

Cheers

---

From: Julian Reschke <[email-masked]>
Date: Fri, 3 May 2019 12:39:05 +0200

> alongside tokens, then it could maybe fit.

Right,

> stuff, which is why I'm still not 100% sold.

Why would that be a problem? It's just another error condition.

Best regards, Julian

---

From: Matthew Kerwin <[email-masked]>
Date: Thu, 9 May 2019 09:50:39 +1000

On Fri., 3 May 2019, 20:40 Julian Reschke, <[email-masked]> wrote:
> >>>> in Signed Exchanges, @jyasskin asks about having Parameterised Lists
whose parameterised identifiers are things other than Tokens.
> Why would that be a problem? It's just another error condition.

That's fair.

I keep thinking about, like, schemas or profiles or Java generics. I wonder
if there's some sort of extension or informational document to write that
could help, or at least offer guidance.

Cheers
-- 
Matthew Kerwin

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 09 May 2019 06:02:57 +0000

--------
In message <CACweHNCFSj-zGO-O=[email-masked]>

>could help, or at least offer guidance.

Mark and I talked about having a section about how other ID's should
use and point to SH, but sort of let it slip because A) we don't want
to write guidance based on zero experience and B) one wrong step and
you are ankle-deep in designing schemas.

At some point down the road a guidance document will probably be
appropriate.

PS: Reminds me: Why does ABNF not have an 'import' facility ?

This would be much clearer:

	import token68 from [RFC7235]

Than (from 7540):

	The ABNF [RFC5234] production for "token68" is defined in
	Section 2.1 of [RFC7235].

---

From: Michael Sweet <[email-masked]>
Date: Thu, 09 May 2019 07:35:43 -0400

> On May 9, 2019, at 2:02 AM, Poul-Henning Kamp <[email-masked]> wrote:
> ...

I've often wondered why the IETF doesn't publish the ABNF for RFCs someplace, e.g., "https://tools.ietf.org/abnf/rfcNNNN", so that such imports are not only possible but then automated tools can pull an official ABNF (with any typographical corrections applied that might slip through the RFC publication process) for use in validation, etc.

We started doing that for IPP specifications several years ago and it has been very useful in validating attribute values, etc.

_________________________________________________________
Michael Sweet, Senior Printing System Engineer

---

From: Julian Reschke <[email-masked]>
Date: Thu, 9 May 2019 14:20:52 +0200

>> 	import token68 from [RFC7235]

Has been suggested many times, but not specced (afair).

In HTTPbis, we use prose rules for that, such as

 > uri-host = <host, see [RFC3986], Section 3.2.2>

That at least makes the ABNF complete.

> We started doing that for IPP specifications several years ago and it has been very useful in validating attribute values, etc.

For many specs, you can simply extract it from the XML source.

For instance, we do that as part of building the HTTP specs, see
<https://github.com/httpwg/http-core/blob/master/httpbis.abnf>.

Best regards, Julian

---

From: Michael Sweet <[email-masked]>
Date: Thu, 09 May 2019 10:40:19 -0400

Julian,

> On May 9, 2019, at 8:20 AM, Julian Reschke <[email-masked]> wrote:
> ...

First, a large number of RFCs were never submitted as XML, and even for those that were (like the IPP updates that became RFCs 8010 and 8011, later STD 92) I can't seem to find the XML sources!

Second, you need to know which figures contain ABNF...

The point was more that there is currently no common repository of ABNF associated by RFC, so importing rules from an RFC is a manual effort...

---

From: Julian Reschke <[email-masked]>
Date: Thu, 9 May 2019 16:48:08 +0200

> First, a large number of RFCs were never submitted as XML, and even for those that were (like the IPP updates that became RFCs 8010 and 8011, later STD 92) I can't seem to find the XML sources!

I think you can obtain them from the RFC Editor if you can convince them
that you won't confuse them with the officially sanctioned plain text
files...

> Second, you need to know which figures contain ABNF...

<artwork type="abnf">

> The point was more that there is currently no common repository of ABNF associated by RFC, so importing rules from an RFC is a manual effort...

Understood and agreed for old specs. Once we move to the new RFC format
we should be able to do better.

Best regards, Julian

---

From: Tommy Pauly <[email-masked]>
Date: Tue, 14 May 2019 10:54:20 -0700

Hello all,

As with #781 on SH, we've had some discussion back and forth on this one, without clearly concluding the discussion.

We'd like to get a sense of the group's consensus so we can move forward with the document. Please reply with which option you think will be be best option to be able to ship the document:

A. Leave the document as-is, without specifying a URI type for SH; note that URI types can be added in a future revision
B. Specify a URI type for SH as a String

Thanks,
Tommy (chair hat on)

> On May 1, 2019, at 10:15 PM, Mark Nottingham <[email-masked]> wrote:
> 
> (Editor hat on)
>

---

From: Julian Reschke <[email-masked]>
Date: Tue, 14 May 2019 20:04:12 +0200

> B. Specify a URI type for SH as a String

My preference is B, in particular if we also allow non-tokens as
identifiers in parametrized lists (because then we could express the
Link header field as SH).

Best regards, Julian

---

From: Matthew Kerwin <[email-masked]>
Date: Wed, 15 May 2019 07:02:49 +1000

> B. Specify a URI type for SH as a String

A.

To be clear, B means: add a new type, and add that to sh-item ?  What
does it say about syntactic structure and restrictions of the new
type?  What value do we get out of doing it (assuming that means
"doing it now, quickly")?

Cheers

---

From: Kari Hurtta <[email-masked]>
Date: Tue, 21 May 2019 12:27:14 +0300 (EEST)

> A. Leave the document as-is, without specifying a URI type for SH;
>    note that URI types can be added in a future revision
> 

Weak preference is B.

I assume that this means

sh-item      += sh-reference
sh-reference  = "<" *(chr)  ">"

Analogical to

  sh-string = DQUOTE *(chr) DQUOTE

However  sh-reference can not be allowed exactly same 
characters. Specially < > can not allowed. And \ escaping
is not needed to be allowed because there %xx encoding 
can be used. Some clarify needed what this means?

( <https://github.com/httpwg/http-extensions/issues/782>
  seems suggesting

   sh-item      += sh-reference
   sh-reference  = "<" URI-reference ">"

   which imply that generic structured header parser
   check URI syntax.
)

That make sense if there is parameter values which
want literal string or alternatively allow giving
value as reference.

Using < > as separator then makes they syntactically
different.

Requiring that generic  structured header parser checks
URI syntax is perhaps too much, but if URI type for SH
is defined as String that is not required.

This B is very weak preference, because most URI parameters
seems want only URI ie there is no need for both literal
string and URI.

Because URI's occurs on http very often it however makes
sense to mark them with specific syntax.

/ Kari Hurtta