Subject: [art] Question regarding RFC 8089

From: Stephan Bergmann <[email-masked]>
Date: Fri, 14 Dec 2018 11:37:07 +0100

Is it intentionally so that file URIs do not support neither a query nor 
a fragment part?  At least, that is how I read RFC 8089 section 2 in 
combination with RFC 3986 section 3.

(Asking because it can occasionally be convenient to host HTML pages 
with "active content" that interacts with a URI's query part on a local 
file system, and try to access them from a browser with <file:/...?...> 
instead of <http://.../...?...> URIs.)

Thanks,
Stephan

---

From: Chris Lilley <[email-masked]>
Date: Mon, 17 Dec 2018 15:43:02 -0500

That would be convenient.

On the other hand, the definition of a fragment depends o the Internet 
Media type, which filesystems typically don't have. But there is "MIME 
sniffing" based on file extensions, I guess.

On 14-Dec-18 05:37, Stephan Bergmann wrote:
> Is it intentionally so that file URIs do not support neither a query 
> nor a fragment part?  At least, that is how I read RFC 8089 section 2 
> in combination with RFC 3986 section 3.
>
> with "active content" that interacts with a URI's query part on a 
> local file system, and try to access them from a browser with 
> <file:/...?...> instead of <http://.../...?...> URIs.)
>
> Thanks,
> Stephan
>
> _______________________________________________
> art mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/art

-- 
Chris Lilley
@svgeesus
Technical Director @ W3C
W3C Strategy Team, Core Web Design
W3C Architecture & Technology Team, Core Web & Media

---

From: Matthew Kerwin <[email-masked]>
Date: Tue, 18 Dec 2018 07:57:54 +1000

> combination with RFC 3986 section 3.

I would say: a bit yes, a bit no. There was definitely no support for
query or fragment in RFC 1630 nor 1738, and we never explicitly
*added* support in 8089.

The two goals of RFC 8089 were: to publish an RFC for file URIs with
unambiguous status, and to align the syntax/definition with RFC 3986.

According to the letter of the law (i.e. RFC 3986): "The query
component contains non-hierarchical data that, along with data in the
path component, serves to identify a resource..."  So as far as
aligning with that definition, it wouldn't have made sense to add a
query component.  Files are identified by purely hierarchical data.

As to the fragment, it's a bit murky, but the way I read RFC 3986,
section 3.5, "The fragment's format and resolution is ... dependent on
the media type of a potentially retrieved representation..." Since the
resource representation retrieved by dereferencing a file URI doesn't
have a media type (heuristics notwithstanding) it doesn't make sense
to include a fragment component in the URI scheme, since it would
always be undefined.

All that said, file URI syntax is now a subset of the generic syntax,
so you're technically able to parse a file URI with query and/or
fragment components.  If that's useful for you -- e.g. if you are
confident that your media type heuristics are good enough to make
fragment components workable -- then nothing stops you.  Clearly the
browsers support it; although I don't know what it would mean for
something like cURL or urllib to have to deal with a query component,
so I don't know how it could be added to the scheme definition.  You
could always just go with https://url.spec.whatwg.org/ instead...

A little bit of whataboutism occurs to me: what about data URIs?  They
*do* have media type information, but don't define fragments in the
URI scheme.

> instead of <http://.../...?...> URIs.)

That is a good point.  It's the sort of thing that makes me start
asking deeply philosophical questions, like: what is a URI?  How much
of the URI is now consumed by the resource representation as well as
(instead of?) the scheme's protocol?

My instinctual feeling is that RFC 3986 is out of date.  Anyone up for
a rewrite?

Cheers
-- 
  Matthew Kerwin
  https://matthew.kerwin.net.au/

---

From: Julian Reschke <[email-masked]>
Date: Mon, 17 Dec 2018 23:04:06 +0100

On 2018-12-17 22:57, Matthew Kerwin wrote:
> ...

URI schemes do not *need* to define fragment handling. It's inherent 
once the media type has been determined.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 18 Dec 2018 09:09:11 +1100

To add a bit -

URI scheme definitions are supposed *not* to define fragment identifiers, since their interpretation is dependent upon the media type, not the scheme.

This has caused confusion in the past; see <https://github.com/httpwg/http-core/issues/103>. 

Cheers,

> On 18 Dec 2018, at 9:04 am, Julian Reschke <[email-masked]> wrote:
> 
> URI schemes do not *need* to define fragment handling. It's inherent once the media type has been determined.
> 
> Best regards, Julian
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: John C Klensin <[email-masked]>
Date: Tue, 18 Dec 2018 03:17:04 -0500

However, Mark, it seems to me to be entirely appropriate, and
consistent with 3986, for a scheme definition to state whether
or not a fragment identifier is permitted.  That is very
different from specifying the details of the syntax or semantics
of that identifier if it is allowed and it is not clear to me
from reading 3986 whether the default, if the scheme definition
doesn't say anything, is "yes" or "no".

     john

--On Tuesday, December 18, 2018 09:09 +1100 Mark Nottingham
<[email-masked]> wrote:

> To add a bit -
> 
> URI scheme definitions are supposed *not* to define fragment
> identifiers, since their interpretation is dependent upon the
> media type, not the scheme.
> 
> This has caused confusion in the past; see
> <https://github.com/httpwg/http-core/issues/103>. 
> 
> Cheers,
> 
> 
>>>> Is it intentionally so that file URIs do not support
>>>> neither a query nor a fragment part?  At least, that is how
>>>> I read RFC 8089 section 2 in combination with RFC 3986
>>>> section 3.
>>>> 
>>> I would say: a bit yes, a bit no. There was definitely no
>>> support for query or fragment in RFC 1630 nor 1738, and we
>>> never explicitly *added* support in 8089.
>>> The two goals of RFC 8089 were: to publish an RFC for file
>>> URIs with unambiguous status, and to align the
>>> syntax/definition with RFC 3986. According to the letter of
>>> the law (i.e. RFC 3986): "The query component contains
>>> non-hierarchical data that, along with data in the path
>>> component, serves to identify a resource..."  So as far as
>>> aligning with that definition, it wouldn't have made sense
>>> to add a query component.  Files are identified by purely
>>> hierarchical data. As to the fragment, it's a bit murky, but
>>> the way I read RFC 3986, section 3.5, "The fragment's format
>>> and resolution is ... dependent on the media type of a
>>> potentially retrieved representation..." Since the resource
>>> representation retrieved by dereferencing a file URI doesn't
>>> have a media type (heuristics notwithstanding) it doesn't
>>> make sense to include a fragment component in the URI
>>> scheme, since it would always be undefined.
>>> All that said, file URI syntax is now a subset of the
>>> generic syntax, so you're technically able to parse a file
>>> URI with query and/or fragment components.  If that's useful
>>> for you -- e.g. if you are confident that your media type
>>> heuristics are good enough to make fragment components
>>> workable -- then nothing stops you.  Clearly the browsers
>>> support it; although I don't know what it would mean for
>>> something like cURL or urllib to have to deal with a query
>>> component, so I don't know how it could be added to the
>>> scheme definition.  You could always just go with
>>> https://url.spec.whatwg.org/ instead... A little bit of
>>> whataboutism occurs to me: what about data URIs?  They *do*
>>> have media type information, but don't define fragments in
>>> the URI scheme.
>>> ...
>> 
>> URI schemes do not *need* to define fragment handling. It's
>> inherent once the media type has been determined.
>> 
>> Best regards, Julian
>>

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 18 Dec 2018 19:47:28 +1100

Hi John,

I think that's correct -- and a worthy clarification for existing schemes.

Cheers,

> On 18 Dec 2018, at 7:17 pm, John C Klensin <[email-masked]> wrote:
>

---

From: Martin J. Dürst <[email-masked]>
Date: Tue, 18 Dec 2018 08:53:01 +0000

> doesn't say anything, is "yes" or "no".

The default is clearly "yes". And it's highly questionable whether a 
scheme specification can disallow fragments. RFC 3986 is very clear that 
it's none of a scheme definitions business to worry about fragment 
identifiers. Prohibiting them would just put spanners in the works where 
they make sense, and be unnecessary in cases where they don't make sense.

Please note the specific wording in RFC 6068 (mailto:):

    Note that this specification, like any URI scheme specification, does
    not define syntax or meaning of a fragment identifier (see [STD66]),
    because these depend on the type of a retrieved representation.  In
    the currently known usage scenarios, a 'mailto' URI cannot be used to
    retrieve such representations.  Therefore, fragment identifiers are
    meaningless, SHOULD NOT be used on 'mailto' URIs, and SHOULD be
    ignored upon resolution.  The character "#" in <hfvalue>s MUST be
    escaped as %23.

Please note the SHOULD NOT (not MUST NOT), and the reference to 
"currently known usage scenarios". If and when somebody comes up with a 
reasonable usage scenario, the SHOULD NOT will become irrelevant. Within 
the current infrastructure, it's difficult to guess how that might 
happen, but we don't want to exclude it.

The 'mailto:' scheme is about as non-standard an URI scheme as there is, 
and that's why the above text was put in, but even in this case, we 
don't prohibit fragment identifiers.

Regards,    Martin.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 18 Dec 2018 09:59:03 +0100

> they make sense, and be unnecessary in cases where they don't make sense.

Indeed.

See also <https://tools.ietf.org/html/rfc7595#section-3>:

>    A scheme definition cannot override the overall syntax for URIs.  For
>    example, this means that fragment identifiers cannot be reused
>    outside the generic syntax restrictions and that fragment identifiers
>    are not scheme specific.  A scheme definition must specify the scheme
>    name and the syntax of the scheme-specific part, which is clarified
>    as follows:

Best regards, Julian

---

From: Adam Roach <[email-masked]>
Date: Tue, 18 Dec 2018 11:24:05 -0600

> The 'mailto:' scheme is about as non-standard an URI scheme as there is,

Just for clarification, I'm not sure that's true. Off the top of my 
head, schemes like 'sip:' 'tel:', 'xmpp:', 'im:', 'iax:', 'jabber:', 
'h323:', 'msrp:', 'pres:', 'sms:', 'mms:', and their various secure 
variants have very similar properties for the purposes of this 
conversation (and that's ignoring a host of provisional schemes like 
'aim:' and 'facetime:').

/a

---

From: John C Klensin <[email-masked]>
Date: Tue, 18 Dec 2018 12:48:13 -0500

--On Tuesday, December 18, 2018 11:24 -0600 Adam Roach
<[email-masked]> wrote:

> Just for clarification, I'm not sure that's true. Off the top
> of my head, schemes like 'sip:' 'tel:', 'xmpp:', 'im:',
> 'iax:', 'jabber:', 'h323:', 'msrp:', 'pres:', 'sms:', 'mms:',
> and their various secure variants have very similar properties
> for the purposes of this conversation (and that's ignoring a
> host of provisional schemes like 'aim:' and 'facetime:').

Adam, you might have added "urn:", which has the added
distinction of being a name rather than a locator.   

In addition to the above, URNs add another complexity to the
discussion: RFC 2141 rather explicitly forbade fragments (and
specifically the "#" character) for use in "urn:" URIs,
reserving that character for future use.   Objections were not
raised at the time and 3986 does not update 2141 to change that
rule, so, while the default may be "yes", there is certainly
precedent for schemes banning the use of fragments.  (Of course,
for URNs, RFC 8141 removed that restriction but did it
explicitly.)

best,
   john

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 18 Dec 2018 13:49:32 -0500

> for URNs, RFC 8141 removed that restriction but did it

Perhaps this should have simply been considered a statement of fact 
rather than a normative requirement. Since the URN is a name rather than 
a value, there is nothing for a fragment to reference.

Similar arguments would apply to TEL, SIP, and some of the others.

	Thanks,
	Paul

---

From: John C Klensin <[email-masked]>
Date: Tue, 18 Dec 2018 19:18:48 -0500

--On Tuesday, December 18, 2018 13:49 -0500 Paul Kyzivat
<[email-masked]> wrote:

>> raised at the time and 3986 does not update 2141 to change
>> that rule, so, while the default may be "yes", there is
>> certainly precedent for schemes banning the use of fragments.
>> (Of course, for URNs, RFC 8141 removed that restriction but
>> did it explicitly.)
> 
> Perhaps this should have simply been considered a statement of
> fact rather than a normative requirement. Since the URN is a
> name rather than a value, there is nothing for a fragment to
> reference.
> 

In the case of URNs, it is more complicated than that.   While
some URN namespaces involve names and not objects (e.g., a
namespace could be set up as part of a registration procedure to
ensure uniqueness) others are object identifiers although
usually more abstract ones than traditional HTTPS URLs.  To use
an overused example, a URN for an ISBN namespace identifiers a
book, but in somewhat abstracted form because any book with the
number will satisfy an unadorned query.  One might even resolve
the URN into a URL that described exactly where to find the
book.  However, a fragment identifier associated with an ISBN
URN makes perfectly good sense to point to, e.g., a chapter of
the book although there are some messy issues associated with
whether the fragment identifier refers to the the URN itself or
to the object to which the URN points through one or several
levels of abstraction.  Getting that right is not obvious, but
that doesn't make the fragment on the URN any less reasonable.
See RFC 8141 from a more extensive discussion of these issues.

I do, however, agree with your "statement of fact" observations
about SIP, TEL, etc.

best,
   john