Subject: [urn] Explanation of draft-ietf-urnbis-urns-are-not-uris- and call for review

From: John C Klensin <[email-masked]>
Date: Mon, 07 Apr 2014 17:46:51 -0400

Hi.

Noting that, even though I've edited a few documents and
contributed to some others, I have no "official" position in the
WG and speak only for myself.  That said, as everyone has
probably noticed, things have been stalled.  Like others, I've
been trying to get them unstuck (that should be no surprise to
anyone following the list in the last several months).  My
analysis of where we have been and where we are is:

* A major source of the blockage has been difficulties
	in figuring out how one can think about queries on, or
	portions of, a resource or its metadata while remaining
	consistent with the syntax and definitions of RFC 3986.
	Arguments about fragments have been very much part of the
    problem thinking and discussions.

* After a number of private discussions, several of us
	came to the conclusion that the root of the problem has
	been the conflation of 3986 (with its roots in URLs) with
	URNs and persistent identifiers.

* We have therefore prepared a draft that explains the
	issues and that asserts that considering URNs as a member
	of the set of generic URIs is inappropriate (and has been
	since RFC 3986 was published).  If approved, it will remove
	URNs from the scope of RFC 3986 Generic URIs.  That draft
	has been posted as draft-ietf-urnbis-urns-are-not-uris-00.  

* If the WG agrees with that approach, the next step
	will be to revise 2141bis to be self-contained (rather
	than depending on 3986) and to provide a framework for
	extensions appropriate to URNs.

Obviously the next step is WG review and comments on the new
draft.  I/we expect significant pushback to this "URNs are not
URIs" move so, if the WG isn't sure it wants it, the idea is
going nowhere... and people need to come up with some other idea
as to how to make progress and/or why Barry shouldn't shut us
down for failure to do so.

     john

---

From: [email-masked] (Dale R. Worley)
Date: Wed, 9 Apr 2014 12:21:05 -0400

I'm not sure I see the significance of this draft.

My understanding of "URIs" is:

    (1) A URI "indicates" some "resource" -- with both quoted words having
    a very broad range of meaning.

    (2) All URIs conform to a particular syntax, and in particular, the
    "schema" part of a URI is well-defined.

    (3) The semantics of a generalized URI is not defined, other than that
    the schema specifies how the remainder of the URI is to be
    interpreted.

    (3) Some URIs are "locators" and are called "URLs", which provide
    adequate information to access the resource over the network.  Certain
    URI schemas are considered to consistently indicate URLs.

    (4) Some URIs are "names" and are called "URNs", which provide unique
    and persistent indicators for well-defined resources.  URIs with the
    "urn" schema are all URNs.

Essentially, the only thing all URIs share is a syntax and that the
schema tells how to interpret the remainder of the string.

The fact that URIs all share the same syntax seems to be useful,
because there are situations where you don't want to have to specify a
priori that only URLs or URNs may be used.  E.g., "Namespaces in XML
1.0", section 2.1: "Definition: An XML namespace is identified by a
URI reference [RFC3986]", where both urn: and http: URIs are popular
as namespace names in practice.

This draft is arguing that the *syntax specification* of URNs should
be decoupled from the syntax specification of URLs, but it doesn't
seem to make any claims that the URI *syntax* is inadequate for URNs.

Of course, the semantics of URNs and URNs are quite different, but
that doesn't seem to have any bearing on the syntax question.

Dale

---

From: John C Klensin <[email-masked]>
Date: Wed, 09 Apr 2014 13:43:18 -0400

--On Wednesday, April 09, 2014 12:21 -0400 "Dale R. Worley"
<[email-masked]> wrote:

>     (1) A URI "indicates" some "resource" -- with both quoted
> words having     a very broad range of meaning.
> 
>     (2) All URIs conform to a particular syntax, and in
> particular, the     "schema" part of a URI is well-defined.
> 
>     (3) The semantics of a generalized URI is not defined,
> other than that     the schema specifies how the remainder of
> the URI is to be     interpreted.

3986 goes a little beyond that in that semantics (or at least
partial semantics) _are_ assigned to certain pieces of syntax.
To take a handy example that has become part of the URN problem/
argument (at least as several of us have read 3986), if there is
a "#" out there in the right end of a generic URI [1], it is
required to be treated as a fragment identifier with the
requirement that the resource be approved and then the fragment
identifier be interpreted.  Certainly the semantics of what that
identifier "means" is at least a function of the URI type/schema
and may be a function of the resource itself.  But the whole
concept of "retrieve all of it and then sort the fragment
identifier out" may be completely wrong for persistent,
abstracted-name-based, identifiers of digital objects,
specifically when those identifiers have nothing to do with
locations on the web.

>     (3) Some URIs are "locators" and are called "URLs", which
> provide     adequate information to access the resource over
> the network.  Certain     URI schemas are considered to
> consistently indicate URLs.

Note (3) twice.   I'm not sure what "consistently indicate URLs"
means, especially in the light of other recent developments [2]
> 
>     (4) Some URIs are "names" and are called "URNs", which
> provide unique     and persistent indicators for well-defined
> resources.  URIs with the     "urn" schema are all URNs.
> 
> Essentially, the only thing all URIs share is a syntax and
> that the schema tells how to interpret the remainder of the
> string.

If that were actually true, the draft would be unnecessary.
Instead, RFC 3986 effectively says "if these characters appear
in the syntax, they have the following meaning and must be
interpreted in the following way".  What follows doesn't specify
complete semantics, but is complete enough to be a big problem
if one departs from network-based locators to provide
identifiers (as that term is understood by the library,
information science, and related communities) for digital and
non-digital objects of various sorts.

> The fact that URIs all share the same syntax seems to be
> useful, because there are situations where you don't want to
> have to specify a priori that only URLs or URNs may be used.
> E.g., "Namespaces in XML 1.0", section 2.1: "Definition: An
> XML namespace is identified by a URI reference [RFC3986]",
> where both urn: and http: URIs are popular as namespace names
> in practice.

Viewed from that perspective, yes, as long as it doesn't force
conditions on the URNs which make them unsuitable for their
purpose.  (and as long as it doesn't force unreasonable
constraints on URLs either [2]).  A different way to say both
what I'm saying and what WHATWG and maybe W3C are saying is
that, it trying to define a generic syntax with partial
semantics that would apply to a very range of "identifiers"
(some of which, in information sciences terms, probably aren't)
was overreaching from the state of our understanding at the
time.  Especially in the light of other work [2], producing a
spec as an alternative to this one that would have said "3986
overreached, it applies only to URLs and that only temporarily"
would have been an alternative to this one, but it would clearly
have been beyond the scope of a URNBIS WG and would probably
have gotten everything thoroughly bogged down.

In the example you give, the processor (or comparison action)
has to know how to handle http URLs and URNs differently today.
For it, this changes nothing other than the vocabulary.
Conversely, if you tell me that the generic URI spec provides
enough information to compare those namespace identifiers
against a fixed or stored  string for identity, I'll claim it is
further evidence that there are semantics in the supposedly
semantic-free generic spec.

> This draft is arguing that the *syntax specification* of URNs
> should be decoupled from the syntax specification of URLs, but
> it doesn't seem to make any claims that the URI *syntax* is
> inadequate for URNs.

Yes, it does.  I didn't want to spell that out more than I did
(the above "fragment" issue is just one example) because that
seemed a job for 2141bis.  I also didn't spell something else
out: this WG or its successors would be insane to allow new URNs
that do not conform to the syntax of 2141 much less to
invalidate any now-valid one.  The question is how to create and
integrate extensions for functionality that now seems necessary.
A few of those functions were explicitly anticipated in 2141,
and that is where problems start to arise.  Of course, if the WG
agrees to the principle but wants more text in one place or the
other, that is easily done.
 
> Of course, the semantics of URNs and URNs are quite different,
> but that doesn't seem to have any bearing on the syntax
> question.

See above.
Thanks for reading it and commenting.
    john

[1] Deliberately avoiding use of the specific 3986 terminology
here.

[2] It is not an issue for this WG, but there are efforts in
WHATWG and elsewhere to redefine URLs and IRIs (IRLs?) in a way
that, apparently, would partially subsume the latter into the
former and then fork both from the 3986/398y specs.

---

From: [email-masked] (Dale R. Worley)
Date: Thu, 10 Apr 2014 23:15:54 -0400

> From: John C Klensin <[email-masked]>

> specifically when those identifiers have nothing to do with

Aye, yes.  I've spent quite a bit of time with the ABNF of 3986 but
never had to deal with the bits that discuss semantics.  And despite
its title of "Uniform Resource Identifier (URI): Generic Syntax",
there is text about semantics.

One semantic part is the process of interpreting relative URLs, which
forms a constraint on the semantics of any URI which can have a "/"
after the "scheme:".  But URNs are prohibited from doing that, so
that's not relevant for URNs.  Or rather, the "urn" scheme is
prohibited from doing that and it is not relevant for them.

Which leads to a subsidiary question, are there going to be other URN
schemes than "urn"?

The semantic part that seems problematic is the "fragment".  The text
of section 3.5 is somewhat vague but does make it clear that the use
of the fragment is determined by the media type of the object
referenced by the base URI, not by the scheme:  "Fragment identifier
semantics are independent of the URI scheme and thus cannot be
redefined by scheme specifications."  That's problematic if URNs want
to use the fragment in any other way.

But I don't see any other problems.  It seems to me that the problem
is to decouple the *semantics* of URNs from what little is said about
semantics in 3986.  We really, really do want to constrain future URNs
to stay within the 3986 syntax, or whatever the future syntax is for
the whole set of UR*s.

> I'm not sure what "consistently indicate URLs" means, especially in
> the light of other recent developments [2]

I was thinking of "http:".

> non-digital objects of various sorts.

Is there any problem other than the fragment?  Comparing the draft to
3986, I don't see it make a forceful argument that there are any
others.

> seemed a job for 2141bis.

I'm not tracking here.  The draft advocates for "this specification
separates the syntax for URNs from the generic syntax for Uniform
Resource Identifiers (URIs) specified in RFC 3986, updating the latter
specification accordingly."  But I don't see the argument for it.
I can certainly see the argument for separating the *semantics*,
especially in regard to the fragment (if people have plans for its use
in URNs).

BTW, are there plans for URNs to use fragment for some other purpose?
Why can't URNs simply not use fragment at all?  (Which is pretty much
what they do now, although I suppose a fragment is now defined if the
URN happens to specify a particular representation of a media object.)

Perhaps the point is that this draft is the output of a long
discussion which I haven't seen.  But in that case, why does it
include the "critical issues include" section?  It really reads as if
it's the beginning of a discussion, not the end of one.

> I also didn't spell something else
> invalidate any now-valid one.

True, but it seems to me those are the *most important* constraints on
change going forward, and it is important to list them.

Dale

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 10 Apr 2014 22:02:21 -0700

> prohibited from doing that and it is not relevant for them.

RFC 2141 says:

    RFC 1630 [2] reserves the characters "/", "?", and "#" for particular
    purposes. The URN-WG has not yet debated the applicability and
    precise semantics of those purposes as applied to URNs. Therefore,
    these characters are RESERVED for future developments.  Namespace
    developers SHOULD NOT use these characters in unencoded form, but
    rather use the appropriate %-encoding for each character.

So "prohibited" seems a bit strong.

> Which leads to a subsidiary question, are there going to be other URN

I don't think so.

> to stay within the 3986 syntax, or whatever the future syntax is for

I think that's the question here, or one of them. One thing that emerged 
from earlier discussion in the URNBIS WG is that there very well might 
be a need for something like a fragment identifier (or a pointer to a 
constituent part of an object), and whether it's best to do that with 
"#" or with something else (at one point the only character we found 
that would work was "/").

> URN happens to specify a particular representation of a media object.)

There has been much discussion of this topic (on and off) in this WG.

> it's the beginning of a discussion, not the end of one.

Yes.

Peter

---

From: John C Klensin <[email-masked]>
Date: Fri, 11 Apr 2014 15:40:07 -0400

For convenience, I'm responding to Peter's note (and leaving
things out that I think he has covered more than adequately)
although most of the comments are actually to Dale.  So consider
this a bit of perspective from the guy who held the pen on this
particular draft.

This note is over-long and parts of it probably belong in -01.
Apologies for that, but the questions seem to deserve thorough
answers.  Help in identifying those parts would be appreciated
(and is easier than the usual "send text" request).

--On Thursday, April 10, 2014 22:02 -0700 Peter Saint-Andre
<[email-masked]> wrote:

>>> partial semantics) _are_ assigned to certain pieces of
>...
>> Aye, yes.  I've spent quite a bit of time with the ABNF of
>> 3986 but never had to deal with the bits that discuss
>> semantics.  And despite its title of "Uniform Resource
>> Identifier (URI): Generic Syntax", there is text about
>> semantics.
>> 
>> One semantic part is the process of interpreting relative
>> URLs, which forms a constraint on the semantics of any URI
>> which can have a "/" after the "scheme:".  But URNs are
>> prohibited from doing that, so that's not relevant for URNs.
>> Or rather, the "urn" scheme is prohibited from doing that and
>> it is not relevant for them.
 
> So "prohibited" seems a bit strong.

Without debating "prohibited", the key issue that has
constrained the WG is that 2141 defined only what, for lack of a
better term, I'll call qualifier-free URNs and reserved some
bits of syntax for those future developments.  We are now in the
future. Developments require some qualifiers and syntax for
them.  And the "Generic URI" (actually, I now believe,
"Generalized URL") syntax (both gURL below) and semantics made
that impossible.

>> Which leads to a subsidiary question, are there going to be
>> other URN schemes than "urn"?
> 
> I don't think so.

There are actually two different questions in the above, at
least IMO.  One is whether there are actually types of Generic
URIs other than URLs and, at least until now, URNs.  That
question goes back to discussions in the original and
mostly-unlamented URI WG, but there is at least some case to be
made for a three-way split among locators (for books, think
library shelf locations), categories (think the Dewey, LC, and
other systems), and persistent names (e.g., ISBNs and, with the
understanding that "persistent" doesn't necessarily require
"unambiguous and unique", titles and associated author(s)).   I
don't want to start a discussion of URCs or other types now, but
it seems to me that we may be helping make progress by saying
"you can't start with URLs, wave your hands madly, and then
start making assertions about fully generic resource-thingies
that span the entire relevant space or set of spaces.

The other answer is that, if we don't allow the functionality of
URNs to expand as anticipated in 2141 and as experience has
demonstrated is necessary, we are almost certain to end up with
some other form of persistent identifier that does that same job
but that isn't under IETF change control and perhaps isn't
upward compatible from 2141.  The reality of the Internet, as we
have seen over and over again, is that, if there is a real and
legitimate need, the IETF doesn't get to prevent its being
satisfied by saying "no, evil" or "we are in charge".  IMO, that
would be a drastic and unfortunate solution.  In a way, this
step is an alternative to a URx which is like a URN but with
non-gURL syntax and semantics.  I think that, if we were to
design a new URN successor today, it would not conform to the
gURL syntax and semantics even to the extent that 2141 does --
from what we know from programming languages, etc., 3986 is
really a pretty bad syntax model, showing all of the
disadvantages of something that started out simple without a lot
of insight into future needs and that then growing by accretion
and patchwork add-ons, and with confidence that the next
required addition would be the last.  Just IMO, of course.

>> The semantic part that seems problematic is the "fragment".
>> The text of section 3.5 is somewhat vague but does make it
>> clear that the use of the fragment is determined by the media
>> type of the object referenced by the base URI, not by the
>> scheme:  "Fragment identifier semantics are independent of
>> the URI scheme and thus cannot be redefined by scheme
>> specifications."  That's problematic if URNs want to use the
>> fragment in any other way.
>> 
>> But I don't see any other problems.  It seems to me that the
>> problem is to decouple the *semantics* of URNs from what
>> little is said about semantics in 3986.  We really, really do
>> want to constrain future URNs to stay within the 3986 syntax,
>> or whatever the future syntax is for the whole set of UR*s.
> 
> I think that's the question here, or one of them. One thing
> that emerged from earlier discussion in the URNBIS WG is that
> there very well might be a need for something like a fragment
> identifier (or a pointer to a constituent part of an object),
> and whether it's best to do that with "#" or with something
> else (at one point the only character we found that would work
> was "/").

I agree with Peter, but let me give you a different,
complementary, answer.  "We" have been at this Uniform Resource
{Whatever] business for somewhat over two decades and, in some
parts of the work, have done a pretty good job of demonstrating
how naïve we are about all of it (see "patchwork add-ons"
above).  The library/ museum/ information management/
information sciences communities, who have many hundreds
(perhaps even a few thousand) years of experience with these
subjects are telling us that mixing and confusing locator-things
with persistent identifiers of "objects" is a _really_ bad idea.
Juha's example (in Section 2 of the draft unless I screwed it
up) of the difference between ISBNs and shelf locations was very
helpful to me and might be to others.

>...
>> Is there any problem other than the fragment?  Comparing the
>> draft to 3986, I don't see it make a forceful argument that
>> there are any others.

We don't know.  As you suggest below, one could also deal with
this by some patching of, and of implied semantic restrictions
from, the generic URI spec (3986).  Didn't seem on balance like
a good idea...  see below.

>>>> This draft is arguing that the *syntax specification* of
>>>> URNs should be decoupled from the syntax specification of
>>>> URLs, but it doesn't seem to make any claims that the URI
>>>> *syntax* is inadequate for URNs.
>>> 
>>> Yes, it does.  I didn't want to spell that out more than I
>>> did (the above "fragment" issue is just one example) because
>>> that seemed a job for 2141bis.
>> 
>> I'm not tracking here.  The draft advocates for "this
>> specification separates the syntax for URNs from the generic
>> syntax for Uniform Resource Identifiers (URIs) specified in
>> RFC 3986, updating the latter specification accordingly."
>> But I don't see the argument for it. I can certainly see the
>> argument for separating the *semantics*, especially in regard
>> to the fragment (if people have plans for its use in URNs).

Pragmatic decision.  Options similar to that were considered and
this one chosen.  In particular:

(i) If one accepts the argument that the fundamental problem is
that URLs and URNs were never part of the same creature and
didn't belong together (an argument that is outlined in the
draft), then that argument for as little separation as possible
just doesn't seem relevant.  Indeed, there are arguments for
making them look as different as they actually are and the
burden ought to be on demonstrating why they should be kept as
similar as possible. 

(ii) I admit to a personal bias that we are still close to the
dawn of a growing Internet with a long future ahead of it.  If
we decide that a particular construction is broken, I believe
that we should, insofar as possible, learn from our mistakes,
move beyond it, and build a solid foundation for the future.
Piling more patches and kludges on top of other kludges and
patches just doesn't seem to be the right way to proceed,
especially if we can make changes that do not invalid existing
uses.

(iii) Even if you disagree with the above (and reasonable people
may do so), my explorations of alternatives suggested that
saying "ok, 3986 doesn't apply to URNs any more" was
straightforward and each to get right".  Peeling the semantics
out of 3986 for URNs only --semantics that are sufficiently
subtle that, on first glance, many people don't believe they are
there at all-- is a much more difficult task with no greater
payoff.  If you (or anyone else on this list) believe that work
is worth doing, please post a draft that does things that way
and make a case for it.  As you consider doing so, please
consider two things:  First, you may write faster than I do, but
the current draft took me a couple of weeks after thinking about
it for a month or two.  A "separate out the semantics" approach
is, I believe, more work.  The stated intent of the WHATWG
effort is to obsolete 3986 entirely (in part because at least
some of those involved don't believe in URNs at all and never
have), so an effort to fine-tune 3986 would be likely to be
embroiled in considerable, time-consuming, controversy.

>> BTW, are there plans for URNs to use fragment for some other
>> purpose? Why can't URNs simply not use fragment at all?
>> (Which is pretty much what they do now, although I suppose a
>> fragment is now defined if the URN happens to specify a
>> particular representation of a media object.)
 
> There has been much discussion of this topic (on and off) in

To elaborate very slightly, "fragment", as defined in 3986, is
just the wrong model for various types of persistent
identifiers, in part because of the types and levels of
abstraction involved.  While one could change the 3986-defined
semantics and overload the syntax a bit, that would not only be
confusing but would immediately lead to a need for various types
of semi-fragments, meta-fragments, and fragment-like things,
each of which would either need its own syntax or some horrible
kludge.  One can, by the way, make the same argument about
queries (with or without fragments).   As Peter say, the WG has
been down that path and several of the options (or, in
retrospect, kludges) have been proposed.

>> discussion which I haven't seen.  But in that case, why does
>> it include the "critical issues include" section?  It really
>> reads as if it's the beginning of a discussion, not the end
>> of one.
> 
> Yes.