Subject: [urn] implementing f-components, consensus check

From: Sean Leonard <[email-masked]>
Date: Sun, 8 Nov 2015 20:22:31 -0800

This is a consensus check on our understanding of f-components.

A urn: URI is parseable into the following components:
"urn:"
NID
NSS
"?"
r-component
q-component
"#"
f-component

(Forget about the specific order for the time being. The list above is 
unordered.)

The f-component is a fragment [RFC3986]. Therefore, the semantics of the 
f-component are defined by the media type of the resource, if any.

A URN resolver is supposed to take NID, NSS, r-component, and 
q-component, and depending on various operations, ultimately return a 
set of URIs--or other kinds of resources or data items--with optional 
attributes. When all such things are fully dereferenced, the expectation 
is that you are supposed to get "content" in the MIME/Internet message 
sense. Therefore, a URI-using system can forward the f-component as-is 
to the software subsystem that processes the content according to its 
media type.

E.g., for <urn:ietf:rfc:3986?FOOOOOOOO?BARRRRRRRRRRRR#page=2> that is 
intended for display in a web browser that is application/pdf-aware, the 
web browser ultimately gets a content that is application/pdf, opens the 
PDF viewer, and has the PDF viewer component receive the fragment "page=2".

A URI "dereferencer" never needs to see the f-component [RFC3986]. It 
follows that a URN resolver never needs to see the f-component.

This understanding does not preclude resolution to non-"content". The 
urn: URI could ultimately get dereferenced to an octet stream (which has 
no media type--any media type would be guessed at with content-sniffing 
or other means), or to "metadata" (which for our purposes, is 
implementable as a blank URI with attributes). In such cases the 
semantics of f-component are undefined.

Agree? Disagree?

Sean

---

From: [email-masked] (Henry S. Thompson)
Date: Mon, 09 Nov 2015 10:32:36 +0000

Sean Leonard writes:

> The f-component is a fragment [RFC3986]. Therefore, the semantics of
> the f-component are defined by the media type of the resource, if any.

Yes (7231 would say "retrieved representation", not "resource",
i.e. what you below call "content").

> attributes. When all such things are fully dereferenced, the
> expectation is that you are supposed to get "content" in the
> MIME/Internet message sense. Therefore, a URI-using system can forward
> the f-component as-is to the software subsystem that processes the
> content according to its media type.

Not quite, see below.

> intended for display in a web browser that is application/pdf-aware,
> the web browser ultimately gets a content that is application/pdf,
> opens the PDF viewer, and has the PDF viewer component receive the
> fragment "page=2".

Yes.

> follows that a URN resolver never needs to see the f-component.

Yes.

> urn: URI could ultimately get dereferenced to an octet stream (which
> has no media type--any media type would be guessed at with
> content-sniffing or other means), or to "metadata" (which for our
> purposes, is implementable as a blank URI with attributes). In such
> cases the semantics of f-component are undefined.

(I think including 'sniffing' here is a distraction: if e.g. a browser
guesses a media type, processing is no different from when a media-type
is part of the response.)

> Agree? Disagree?

Your introductory description is a little too vague to be sure.  In
particular, taking your pdf example above, if we suppose a browser
encounters that URN in HTML content in, say, the 'href' attribute of an
'a' element, and accordingly (supposing the host page, or some higher
authority, has registered some resolvers against the 'urn' URI scheme),
an HTTP GET request is sent to one of them.  Some of your subsequent
discussion is then, in my view, open to misinterpretation, because it
leaves something important out.

That is, the crucial question wrt the f-component is the HTTP response
status code that comes back in response to a GET for a URN.  It is this
that determines whether the response body is "content" or not.  If the
response status code is 200, then the body is indeed "content", and the
f-component applies to it.  If it's _not_ a 200, then the body is _not_
"content", and the f-component does _not_ apply.

What I'm trying to get at here is that it's _not_ (or not primarily)
what's in an HTTP response body that determines whether it's "content"
or "metadata", but the response status code and other parts of the
response header.

This is not to say that much (anything?) has to change in urnbis, rather
just to try to clarify what is in fact out-of-scope for it, at least as
far as expectations when HTTP-mediated resolution is involved.

ht
-- 
       Henry S. Thompson, School of Informatics, University of Edinburgh
      10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44) 131 650-4440
                Fax: (44) 131 650-4587, e-mail: [email-masked]
                       URL: http://www.ltg.ed.ac.uk/~ht/
 [mail from me _always_ has a .sig like this -- mail without it is forged spam]

---

From: Keith Moore <[email-masked]>
Date: Mon, 9 Nov 2015 07:14:39 -0500

> cases the semantics of f-component are undefined.

Mostly agree.   A couple of minor exceptions:

- application/octet-stream is a media type, and it's possible to define 
f-components (e.g. ranges consisting of offset and length) that make 
sense with that media type.
- While we can't stop clients from "sniffing" to guess what that content 
"really" means, clients that try to be too smart about such things can 
also make life difficult for users.   So while you might be right about 
how a particular client would behave when presented with an octet-stream 
content, I wouldn't want us to specify that behavior.
- The expectation of URN resolution isn't inherently to obtain 
"content".  It is reasonable to resolve a URN with the goal of obtaining 
metadata, or information about related works, or access to a resource in 
some other way than to download all of its content (e.g. interacting 
with a remote database, or playing a video and having the video stream - 
but not the DVD content itself - sent back to one's display).   We 
should not presume that the only way to usefully use a resource is to 
download the entirety of that resource's content.

This expectation could be communicated explicitly via r-component or 
client request, or implicitly from the context in which the URN 
appears.   (e.g. a URN in a <IMG> SRC parameter probably requires access 
to the image content, whereas for a URN in an <A> HREF parameter used in 
an interactive application like a web browser it might be sufficient to 
permit the user to interact with that resource).

I also agree that semantics of an f-component with a URN that is 
resolved for the purpose of other than obtaining "content" is currently 
undefined.   I am not sure whether it should remain undefined, be 
declared inappropriate, or be declared an error to use f-component in 
this way.

Keith

---

From: [email-masked] (Henry S. Thompson)
Date: Mon, 09 Nov 2015 12:25:08 +0000

Keith Moore writes:

> resolved for the purpose of other than obtaining "content" is
> currently undefined.   I am not sure whether it should remain
> undefined, be declared inappropriate, or be declared an error to use
> f-component in this way.

On the one hand, neither server nor client can know what a user's
purpose is in attempting to resolve a URN.

On the other hand I think it follows from my earlier response that if a
server is configured so as not to respond with "content", i.e. with a
response status code of other than 200, then (modulo some subtleties to
do with 301 and 302 responses) the point is moot: there is no content,
so the f-component is irrelevant and, of necessity, ignored.

ht

---

From: Keith Moore <[email-masked]>
Date: Mon, 9 Nov 2015 07:27:40 -0500

> "content", and the f-component does _not_ apply.

Agree.   The f-component only applies if the operation to retrieve the 
content results in a successful download of that content.   It does not 
apply to an error document, or an incomplete transfer.

Note that since a URL fragment is specific to a particular kind of 
content-type, specifying an f-component of a URN (or fragment of a URL) 
that exists in multiple content-types isn't terribly useful without 
including some way to constrain the content-type obtained.

Keith

---

From: Keith Moore <[email-masked]>
Date: Mon, 9 Nov 2015 07:33:25 -0500

> purpose is in attempting to resolve a URN.

The client might know.   For instance, if a URN appears in a document, a 
web browser might allow a "right click" on the link or other displayed 
object, and offer the user options such as "display metadata", "find 
related works", etc.   In other cases the client can infer the purpose 
by context.   If the URN appears in an IMG tag, it's pretty clear that 
the document author intended that the image be displayed, which probably 
requires obtaining the content of the image file.

Also, the client is not necessarily an interactive web browser, and can 
potentially be any application that uses URIs.

> so the f-component is irrelevant and, of necessity, ignored.

Perhaps.  That's my current preference, anyway.

Keith

---

From: [email-masked] (Henry S. Thompson)
Date: Mon, 09 Nov 2015 12:38:50 +0000

Keith Moore writes:

> content-type, specifying an f-component of a URN (or fragment of a
> URL) that exists in multiple content-types isn't terribly useful
> without including some way to constrain the content-type obtained.

3986 attempts to address this by saying that it is the owner/server's
responsibility in such cases, in particular if content negotiation is
involved, to ensure that the fragment involved applies to _all_ the
potentially-returned content types. . .

ht

---

From: Keith Moore <[email-masked]>
Date: Mon, 9 Nov 2015 07:43:01 -0500

On 11/09/2015 07:38 AM, Henry S. Thompson wrote:
> involved, to ensure that the fragment involved applies to_all_  the
This is, of course, a sign of delusion.

Keith

---

From: John C Klensin <[email-masked]>
Date: Mon, 09 Nov 2015 13:09:53 -0500

--On Monday, November 09, 2015 12:25 +0000 "Henry S. Thompson"
<[email-masked]> wrote:

> Keith Moore writes:
> 
>> I also agree that semantics of an f-component with a URN that
>> is resolved for the purpose of other than obtaining "content"
>> is currently undefined.   I am not sure whether it should
>> remain undefined, be declared inappropriate, or be declared
>> an error to use f-component in this way.
> 
> On the one hand, neither server nor client can know what a
> user's purpose is in attempting to resolve a URN.
> 
> On the other hand I think it follows from my earlier response
> that if a server is configured so as not to respond with
> "content", i.e. with a response status code of other than 200,
> then (modulo some subtleties to do with 301 and 302 responses)
> the point is moot: there is no content, so the f-component is
> irrelevant and, of necessity, ignored.

Henry,

I think I agree -- the observation that I'm not sure is
symptomatic of some of the problems here--  but find two other
sources of confusion in your, and Sean's, analysis.  While I
obviously accept the importance of processing of HTML (or XML)
and interpretation and response codes in HTTP, I think that
trying to bind URNs (or, by the way, even RFC 3986) to those
functions, implying that they are the only plausible
representations or use contexts is a serious mistake.  The fact
that we've already got, e.g., XMPP URNs that are interpreted by
different protocols and used by them in different ways than
would be the case in an HTTP context is one of many indications
that URNs (or URIs generally) are not just for the web or just
to be defined in web-ish terms.

So, as soon as you (or others) start talking about, e.g., HTTP
response codes being definitive, I neither agree nor disagree; I
just think you are carrying out a conversation that is of little
interest to me in the context of this WG.

This is, perhaps, a generalization or different version of some
of what Keith has been saying, but I would go well beyond
"client is not necessarily an interactive web browser, and can
potentially be any  application that uses URIs".  For example, I
would have said "potentially be any application that uses URNs"
because, while 3986 can be interpreted as saying that any
application that accepts URIs must be able to accept URNs and do
something sensible, I am aware of nothing (in 2141bis or
elsewhere) that requires that any application that can accept
URNs be able to accept and process generalized URIs.

To go a step further, we've heard (and I think the WG has
largely accepted although I'm not trying to make a consensus
call) that the language of 3986 relative to fragments takes the
syntax and interpretation of them out of the hands of any
scheme.  If we were to interpret that as narrowly as possible,
2141bis should say absolutely nothing about f-components
(perhaps even avoid the term) and deal with fragments by
pointing to 3986, leaving it to that document to sort out what
they mean and how they are interpreted, especially in situations
where there is no media (content)-type or other framework for
interpretation.   I would further note that a narrow reading of
3986 probably doesn't even permit a URN processor to isolate an
f-component and concatenate it with a URL that it the target of
that URN because, when the "urn:" scheme is processed, if the
URNs resolves to a URL, URLs are not bound to a media type.  One
can get there, but only by interpreting "secondary resource"
from the first paragraph of Section 3.5 of RFC 3986 (and
subsequent paragraphs) as including "tertiary resources and
deeper recursion" as equivalent to it and ignoring parts of the
second paragraph.

Again, if any of this should lead to changes in 2141bis,
particularly if anyone would like the current discussion of
f-components and their sometimes-odd relationships to URNs that
never leave to resolution to anything that has a media type
expanded, please send text or otherwise be very specific.

    john

---

From: John C Klensin <[email-masked]>
Date: Mon, 09 Nov 2015 13:34:48 -0500

Apologies for not picking this up in my prior note, but...

--On Monday, November 09, 2015 07:43 -0500 Keith Moore
<[email-masked]> wrote:

>>> > Note that since a URL fragment is specific to a particular
>>> > kind of content-type, specifying an f-component of a URN
>>> > (or fragment of a URL) that exists in multiple
>>> > content-types isn't terribly useful without including some
>>> > way to constrain the content-type obtained.

>> 3986 attempts to address this by saying that it is the
>> owner/server's responsibility in such cases, in particular if
>> content negotiation is involved, to ensure that the fragment
>> involved applies to_all_  the potentially-returned content
>> types. . .

> This is, of course, a sign of delusion.

I don't want to go quite that far, but, if a particular URN in a
particular namespace can return -- based on usage context,
r-component contents, or other information-- both information
objects that are associated with media types and information
that is not associated with media types, then meeting that
condition is impossible (at least without a lot of handwaving).

As an example, consider 
   urn:ietf:rfc:3986

and assume the definition is expanded to include r-components
and fragments.  Now:
   urn:ietf:rfc:3986#Section-3.5

is valid and meaningful independent of whether what comes back
on HTTP-style resolution is the text form of the RFC or the PDF
form.  If the text of 3986 that doesn't permit schemes to limit
the use of fragments applies, then it is valid even under 2141
(without any 2141bis) requirements.  However, if that URN
resolves to whatever 
  http://www.rfc-editor.org/info/rfc3986

produces, that page has a media-type (presumably connected to
HTML), but that media type isn't either text/plain or
application/pdf.  More important, while the fragment is "valid"
because HTML allows fragments, there is no "Section-3.5" on that
page: to get to something that has such a section, the
compound/aggregate/possibly-recursive resolver has to somehow
have the intelligence to move beyond the "secondary resource"
(that page) and resolve some HTTP or HTTPS link that leads
somewhere for which a second number is meaningful.

And, as suggested in my prior note, if the "result" of resolving 
   urn:ietf:rfc:3986

(with or without the fragment) is actually 
   http://www.rfc-editor.org/info/rfc3986
then that "secondary resource" is a perfectly good URI, but URIs
are not generally considered to have media types.

To me, all of the above suggests even more strongly that 2141bis
should say as little about fragments as possible.  I'm ok with
what now appears in Section 3.3 of 2141bis-14 as long as I don't
look at it too closely and, e.g., realize that "the f-component
is copied verbatim to the fragment portion of the URL" probably
violates the provisions of 3986 that appear to require knowledge
of a media type to do _anything_ with a fragment.   If it is
necessary to say something else, including trying to more
carefully align that paragraph with 3986, then I suggest that it
may be prerequisite to get firm and clear definitions to a lot
of the issues and terminology suggested above and by my prior
note in this thread.    Again, send text.

    john

---

From: Keith Moore <[email-masked]>
Date: Mon, 9 Nov 2015 23:47:47 -0500

On 11/09/2015 01:09 PM, John C Klensin wrote:
> representations or use contexts is a serious mistake.

Assuming I understand what John is saying, I agree.  I'm not adverse to 
using HTTP as a URN resolution protocol, nor am I adverse to using 
existing HTTP protocol features (including redirects, result codes, and 
content-negotiation) where they make sense.   But I think it would be a 
serious mistake to insist that URN resolution protocol features be 
shoehorned into HTTP protocol features, and an even worse mistake to 
insist that a profile of HTTP be the only URN resolution protocol.

> The fact
It would be good to collect some of these examples as use cases, just so 
that we don't implicitly think of a single use case of URNs.   URNs 
aren't inherently persistent links in web documents (even though this 
was perhaps what most motivated their creation), nor inherently unique 
IDs that aren't bound to anything in particular (even if this is perhaps 
the use case that is visible to the largest number of users or IETF 
participants), nor inherently reserved for fixed media (even if perhaps 
the majority of URNs have been assigned for such purposes).   A URN can 
be any of these.   And a single namespace can potentially contain URNs 
from all of these categories.

> URNs be able to accept and process generalized URIs.
Agree.

> scheme.

Also agree.
> If we were to interpret that as narrowly as possible,
> interpretation.

I find it very useful to have different terms for f-component and 
fragment even though the f-component should map directly onto a fragment 
(and similarly for q-component and query).   Making a clear distinction 
between the two seems to make it easier to write clear text.   I also 
find that 3986 is confusingly written enough, especially where it 
addresses the topic of URNs or tries to shoehorn URNs into its 
assumptions, that I believe it's necessary for 2141bis to clarify some 
things - even though 2141bis tries to adhere to the spirit of 3986 as 
much as is reasonable.

> I would further note that a narrow reading of
> URNs resolves to a URL, URLs are not bound to a media type.
By that reasoning, fragments shouldn't be used with URLs either.

Given the way fragments were defined in an ad hoc manner on a 
per-media-type basis, and especially that some fragments aren't defined 
in terms of explicit markers in the resource, I have never believed that 
one could expect fragment meaning to be invariant across 
content-negotiation.   If one insists on this constraint, it essentially 
means that a resource can exist in only one media format.

Also, I don't see that URNs and URN resolution make this any worse than 
it already is in HTTP.

Keith

---

From: John C Klensin <[email-masked]>
Date: Tue, 10 Nov 2015 05:04:20 -0500

--On Monday, November 09, 2015 23:47 -0500 Keith Moore
<[email-masked]> wrote:

>...
>> they mean and how they are interpreted, especially in
>> situations where there is no media (content)-type or other
>> framework for interpretation.
> 
> I find it very useful to have different terms for f-component
> and fragment even though the f-component should map directly
> onto a fragment (and similarly for q-component and query).
> Making a clear distinction between the two seems to make it
> easier to write clear text.   I also find that 3986 is
> confusingly written enough, especially where it addresses the
> topic of URNs or tries to shoehorn URNs into its assumptions,
> that I believe it's necessary for 2141bis to clarify some
> things - even though 2141bis tries to adhere to the spirit of
> 3986 as much as is reasonable.

Wfm.

>> 3986 probably doesn't even permit a URN processor to isolate
>> an f-component and concatenate it with a URL that it the
>> target of that URN because, when the "urn:" scheme is
>> processed, if the URNs resolves to a URL, URLs are not bound
>> to a media type.

> By that reasoning, fragments shouldn't be used with URLs

No, because the relevant discussion of fragments in 3986 talks
about "primary" and "secondary" resources and seems, at least to
me on my most recent reading, to be fairly clear about that they
mean.  If my reading of that material is correct, attaching a
fragment to a URL, having a user agent use that URL to obtain a
resource-object (one with a media type), and then using the
fragment string to identify part of that resource-object works
just fine and is exactly how things are defined.  However, the
text does not provide for having a resolver pass a fragment from
one URI to another -- that operation would have to be done by a
client and clients don't do anything with fragments other than
apply them to objects that have media types.  Since URLs are not
objects, much less objects with media types, there is a
disconnect.

Now, to be clear, I think there are other ways to read 3986.
One can assume that the discussion of primary and secondary
resources and the use of the term "indirect referencing" implies
that recursion is permitted (and assumed), potentially to
arbitrary depth and that fragments, once "separated from the
rest of the URI prior to a dereference", are somehow kept
separate (including not being processed other than parsing by
any generic resolver) until a final object is obtained and then
applied, by the user agent, to the object.

But I think that making any sort of strong claim that the 3986
fragment text (primarily Section 3.5) requires any particular
behavior of URNs, such as either requiring that they resolve to
URLs or that a URN processor peel off the fragment component and
catenate it with the URLs that are produced is, well, a stretch.
 
> per-media-type basis, and especially that some fragments
> aren't defined in terms of explicit markers in the resource, I
> have never believed that one could expect fragment meaning to
> be invariant across content-negotiation.   If one insists on
> this constraint, it essentially means that a resource can
> exist in only one media format.

I think that is correct.  Coming back to the example used above,
the language of Section 3.5 clearly allows a URI-author to
construct 
   urn:ietf:rfc:3986#Page-24

despite years of warnings by the RFC Editor against basing
citations on page numbers.  While the "chapter" reference above
clearly works with text and PDF objects, if an HTML object, or
one optimized for some sort of electronic reader, is retrieved,
there may be no page numbers and the fragment may become
meaningless.

> Also, I don't see that URNs and URN resolution make this any
> worse than it already is in HTTP.

By specifying a specific location for an object, an HTTP URL
does not create some sort of choice mechanism on resolution.
One may reasonably argue non-persistence, but I'm not sure that
changes the expectation of retrieving an object that has a media
type.

best,
   john

---

From: [email-masked] (Henry S. Thompson)
Date: Tue, 10 Nov 2015 10:12:36 +0000

> representations or use contexts is a serious mistake.

Of course.  I entirely agree, and in a previous life spent rather a lot
of energy and political capital trying to keep the Internet safe for
user agents other than web browsers, ditto the use of URIs outside HTTP.

> The fact that we've already got, e.g., XMPP URNs that are interpreted
> by different protocols and used by them in different ways than would
> be the case in an HTTP context is one of many indications that URNs
> (or URIs generally) are not just for the web or just to be defined in
> web-ish terms.
>

That's too bad, because I think the _architectural_ points of my two
replies to Andy transcend the particular example of HTTP.  As soon as
we're in the request/response, message exchange, _protocol_ space at all
(and that's precisely how Andy framed his queries), questions of data
vs. metadata, content vs. protocol element and so on arise.

That 3986 depends on the representation/resource distinction for what it
says about the semantics of '#' is clear.  It follows that for a client
(in _any_ protocol-governed exchange), knowing whether the response it
gets to some request is content or not (error/metadata/forwarding
information/...) is essential to its ability to correctly use a fragment.
So of course HTTP response codes are only definitive for HTTP-governed
exchanges, but any _other_ protocol must have an equivalent mechanism.

> syntax and interpretation of them out of the hands of any

For in-principle resolvable URIs, yes.  

> URNs resolves to a URL, URLs are not bound to a media type.

Depends in part on how useful you think the HTTP model is.  What Keith
and I were agreeing on is that insofar as the f-component is treated as
HTTP treats fragments, then the client won't even _send_ the fragment to
the URN processor, so no question of concatenation or anything like it
arises.

I know that 3986 doesn't say this, and given that urnbis is not
explicitly addressing protocols (which I think we agree it shouldn't,
any more than 3986 does), this is _all_ just about what doors we leave
open versus what we shut.  Specifically, that generic URI processing via
HTTP or anything similar that strips f-components before attempting
resolution MUST NOT be _ipso facto_ in conflict with urnbis.

> expanded, please send text or otherwise be very specific.

For my money, this case (unlike the q-component case) is just violent
agreement on all sides.  So no, no changes required.

ht

---

From: [email-masked] (Henry S. Thompson)
Date: Tue, 10 Nov 2015 11:00:18 +0000

John C Klensin writes:

> application/pdf.

This is precisely the kind of example I was trying to get specific about
with my discussion of HTTP response status codes.  If whoever resolves
that request signals (with an HTTP response status code of 200 _or in
any other way_) that the response body (or whatever corresponds to that)
is a representation of the resource identified by "urn:ietf:rfc:3986",
then its inviting a client to interpret the f-component wrt that
text/html response, and that resolver is broken.

Because the response I get today to a request for
<http://www.rfc-editor.org/info/rfc3986> is _not_ a representation of
that RFC, it's a bunch of useful, interesting, IETF status information
regarding that RFC.  My view is that however your postulated resolver
behaves, it should _either_ be responding to the client as to how it can
redirect from <urn:ietf:rfc:3986> to <http://www.rfc-editor.org/info/rfc3986>,
in a way that makes clear that what it will find there is _not_ the
thing itself, but a source of metadata (what HTTP would use 303 "See
Other" for), or, if it does the HTTP retrieval itself, return the
resulting landing page representation packaged in a way that similarly
indicates that it's not the RFC itself).

> should say as little about fragments as possible.

Still violent agreement!

> I'm ok with what now appears in Section 3.3 of 2141bis-14 as long as I
> don't look at it too closely and, e.g., realize that "the f-component
> violates the provisions of 3986 that appear to require knowledge of a
> media type to do _anything_ with a fragment.

Interesting -- I read that as implicitly about what _clients_ do,
particularly given the immediately preceding "The f-component is intended to
be interpreted _by the client_" (emphasis added).n

That is, I read the bit you quote above as equivalent to "the
f-component is treated by the client as if it were an f-component of the
URL".  In particular, I don't see anything there that implies that the
_resolver_ has done anything at all with the f-component, or even seen
it.

> If it is necessary to say something else, including trying to more
> carefully align that paragraph with 3986, then I suggest that it may
> be prerequisite to get firm and clear definitions to a lot of the
> issues and terminology suggested above and by my prior note in this
> thread.  Again, send text.

Hmm.  I see that the quoted phrase actually occurs twice, once in the
3.3 introduction, and once in 3.3.3.

Sigh.  On re-reading all of 3.3, I'm concerned that the procedural
language takes us into dangerous territory.  In particular, it fails to
recognise in any way the distinction which HTTP makes with (200, 301 or
302) versus (303), that is, the distinction between (representations of)
the requested resource, and (representations of) relevant information
_about_ that resource, but not it.  As you say above, that means
treating a landing page as if it were the thing itself, which makes life
difficult in more ways than just f-component processing.

Less is more.  I need to think more about how to manage this -- given
that we surely _don't_ want to start talking about protocols, or (less
surely?) expanding the discussion of resolution to distinguish between
what one might call 'indirect' vs. 'direct' resolution (here's a locator
for what you asked for vs. here's what you asked for), exactly how to
best define the relationship of f- (and q-) components to the results of
resolution isn't obvious at all.  But talking explicitly about
concatenation seems to carry way too many assumptions about resolution
to be the right answer.

ht

---

From: [email-masked] (Henry S. Thompson)
Date: Tue, 10 Nov 2015 11:02:19 +0000

Keith Moore writes:

> Assuming I understand what John is saying, I agree.  I'm not adverse
> to using HTTP as a URN resolution protocol, nor am I adverse to using
> existing HTTP protocol features (including redirects, result codes,
> and content-negotiation) where they make sense.   But I think it would
> be a serious mistake to insist that URN resolution protocol features
> be shoehorned into HTTP protocol features, and an even worse mistake
> to insist that a profile of HTTP be the only URN resolution protocol.

Lest there be any residual doubt about this, I agree 100%

ht

---

From: [email-masked] (Henry S. Thompson)
Date: Tue, 10 Nov 2015 11:29:04 +0000

> applied, by the user agent, to the object.

That's certainly how the HTTP specs call for fragments to be
interepreted.  It's moderately important (to me!) that if you type

  http://www.w3.org/TR/REC-xml.html#sec-xml-wg

into a web browser or pass it to the Python or Java HTTP libraries that
the fragment is first removed, the resulting URI resolved, then the
resulting redirection Location resolved, and _then_ the fragment
semantics implemented.

> catenate it with the URLs that are produced is, well, a stretch.

Agreed.

ht

---

From: John C Klensin <[email-masked]>
Date: Tue, 10 Nov 2015 08:44:39 -0500

--On Tuesday, November 10, 2015 11:00 +0000 "Henry S. Thompson"
<[email-masked]> wrote:

> This is precisely the kind of example I was trying to get
> specific about with my discussion of HTTP response status
> codes.  If whoever resolves that request signals (with an HTTP
> response status code of 200 _or in any other way_) that the
> response body (or whatever corresponds to that) is a
> representation of the resource identified by
> "urn:ietf:rfc:3986", then its inviting a client to interpret
> the f-component wrt that text/html response, and that resolver
> is broken.

Ok, but... 

> <http://www.rfc-editor.org/info/rfc3986> is _not_ a
> representation of that RFC, it's a bunch of useful,
> interesting, IETF status information regarding that RFC.  My
> view is that however your postulated resolver behaves, it
> should _either_ be responding to the client as to how it can
> redirect from <urn:ietf:rfc:3986> to
> <http://www.rfc-editor.org/info/rfc3986>, in a way that makes
> clear that what it will find there is _not_ the thing itself,
> but a source of metadata (what HTTP would use 303 "See Other"
> for), or, if it does the HTTP retrieval itself, return the
> resulting landing page representation packaged in a way that
> similarly indicates that it's not the RFC itself).

However, and here is where your thinking and explanation get
HTTP-specific because there is no 3986 requirement for a
redirect function and that function is, indeed, rare for many or
most of the protocols for which URIs have been standardized.  

There are other issues embedded in that example that may be
worth some thought: First, for many years, any possible resolver
for <urn:ietf:rfc:3986> would point to
ftp://ftp.rfc-editor.org/in-notes/rfc3986.txt.   That requires a
two-stage resolution to get to an object (rather than
descriptive material) and, while the object would work perfectly
well with a "Section"-style fragment, but the FTP protocol (and
"ftp" URI scheme) do not identify or return media types, so the
fragment cannot be applied by the FTP client.    Second, if
anyone wants to pin down "persistence", they need to consider
whether the change from resolution to an FTP URL that points to
an object to resolution to an HTTP URL that points to a
description that, in turn, contains multiple links to objects is
reasonable or fundamentally undermines whatever we mean by
persistence.

>> To me, all of the above suggests even more strongly that
>> 2141bis should say as little about fragments as possible.
> 
> Still violent agreement!
> 
>> I'm ok with what now appears in Section 3.3 of 2141bis-14 as
>> long as I don't look at it too closely and, e.g., realize
>> that "the f-component is copied verbatim to the fragment
>> portion of the URL" probably violates the provisions of 3986
>> that appear to require knowledge of a media type to do
>> _anything_ with a fragment.
> 
> Interesting -- I read that as implicitly about what _clients_
> do, particularly given the immediately preceding "The
> f-component is intended to be interpreted _by the client_"
> (emphasis added).

> f-component is treated by the client as if it were an
> f-component of the URL".  In particular, I don't see anything
> there that implies that the _resolver_ has done anything at
> all with the f-component, or even seen it.

And this is where the difference in conceptual models may make a
difference, especially to the "universal URI resolver" or even
"universal URN resolver" themes.  Noting that I don't think
anyone has a problem with a "universal URI parser" that merely
divides a URI into components that are consistent with the
symtax of 3986, suppose a program encounters something it
identifies as a URI, one with an arbitrary scheme.  Now it seems
to me that there are two possibilities either of which is, in
principle, compatible with 3986.  

(i) The client program contains code that recognizes the scheme
and knows what to do with it, just as web browsers know what to
do with http, https, ftp, and a handful of other schemes.   If
it doesn't know what to do (or if the scheme is a URN -- see
below), then it either produces a "don't have a clue what to
about about that scheme" error or has to go to the second option.

(ii) The client program calls on a universal URI resolver that
can figure out what to do with almost any scheme and return
either an object, an error code, or an indication as to what to
do next.  Whether the client parses the URI, separates out any
fragment for later use, and passes the rest of the URI to the
universal resolver or whether it passes the entire URI string to
the universal resolver, gets back the fragment as a separate
piece of data structure and then applies it is a small matter of
programming (and terminology) that really does not affect either
standards-compliance or anything external to that client.

Ok so far?

However, URNs are different because, unlike, e.g., an http or
ftp URL, they inherently are not self-contained: a lookup or
resolution process has to involve the NID and NSS and there is
no universal mechanism for getting from the NID to the relevant
lookup or resolution process.  That turns option (i) above into
"recognize the scheme _and_ recognize the NID" and means that
the universal resolver hypothesized under (ii) has to be a lot
more sophisticated (and has to have a much more elaborate
unexpected condition reporting structure) than we normally
assume for URLs.  

>> If it is necessary to say something else, including trying to
>> more carefully align that paragraph with 3986, then I suggest
>> that it may be prerequisite to get firm and clear definitions
>> to a lot of the issues and terminology suggested above and by
>> my prior note in this thread.  Again, send text.
> 
> Hmm.  I see that the quoted phrase actually occurs twice, once
> in the 3.3 introduction, and once in 3.3.3.
> 
> Sigh.  On re-reading all of 3.3, I'm concerned that the
> procedural language takes us into dangerous territory.  In
> particular, it fails to recognise in any way the distinction
> which HTTP makes with (200, 301 or 302) versus (303), that is,
> the distinction between (representations of) the requested
> resource, and (representations of) relevant information
> _about_ that resource, but not it.  As you say above, that
> means treating a landing page as if it were the thing itself,
> which makes life difficult in more ways than just f-component
> processing.

Indeed.   And although I believe the HTTP case is the easy one,
that is why I'm trying to tease this out a bit more.

> Less is more.  I need to think more about how to manage this
> -- given that we surely _don't_ want to start talking about
> protocols, or (less surely?) expanding the discussion of
> resolution to distinguish between what one might call
> 'indirect' vs. 'direct' resolution (here's a locator for what
> you asked for vs. here's what you asked for), exactly how to
> best define the relationship of f- (and q-) components to the
> results of resolution isn't obvious at all.  But talking
> explicitly about concatenation seems to carry way too many
> assumptions about resolution to be the right answer.

Agreed.  I look forward to proposed text.

     john

---

From: John C Klensin <[email-masked]>
Date: Tue, 10 Nov 2015 08:48:32 -0500

--On Tuesday, November 10, 2015 11:29 +0000 "Henry S. Thompson"
<[email-masked]> wrote:

>> resources and the use of the term "indirect referencing"
>> implies that recursion is permitted (and assumed),
>> potentially to arbitrary depth and that fragments, once
>> "separated from the rest of the URI prior to a dereference",
>> are somehow kept separate (including not being processed
>> other than parsing by any generic resolver) until a final
>> object is obtained and then applied, by the user agent, to
>> the object.
> 
> into a web browser or pass it to the Python or Java HTTP
> libraries that the fragment is first removed, the resulting
> URI resolved, then the resulting redirection Location
> resolved, and _then_ the fragment semantics implemented.

And no one has questioned the appropriateness of the HTTP specs
--or any other spec of the "locator" family-- to be written that
way.

>> behavior of URNs, such as either requiring that they resolve
>> to URLs or that a URN processor peel off the fragment
>> component and catenate it with the URLs that are produced is,
>> well, a stretch.
> 
> Agreed.

Yep.  Again, the question is to how to tune the 2141bis text to
match what we seem to be agreeing on... and being sure that
agreement reflects WG consensus.

   john