Subject: [urn] Thoughts on fragments, queries, and new URN namespaces

From: John C Klensin <[email-masked]>
Date: Thu, 13 Jun 2013 17:17:46 -0400

Hi.

I've been trying to watch this WG from a safe distance and not
comment in order to avoid repeating arguments from the original
URN design discussions or the more recent 2141 versus 3986 one.
But a review of the correspondence of the last several weeks has
convinced me that a few important principles are getting lost in
the shuffle.

So, some thoughts.  This note may turn out to be long.  If so
consider the text length averaged over the last few years.   

Summary:  While I believe that fragment identifiers on URNs are
architecturally difficult and unattractive and queries even more
so, I think prohibiting them is a mistake on interoperability
grounds. I think we can also extrapolate from other things that
have gone on with the Internet to conclude, however sadly, that
having the IETF try to either hold up or deny registration of
URN namespaces on aesthetic grounds or to assert authority over
namespaces for which it has no real knowledge will only lead to
the use of unregistered namespace names and the consequent
interoperability problems.

Details:

In an ideal world, URNs would point to atomic objects.    If
chapters of books are the relevant atomic objects, then URNs
should identify those chapters with some sort of aggregate URN
associated with the book and reflecting the chapter URNs, rather
than having a book URN and fragments or queries to identify the
chapters.  But I think we need to recognize that some people,
libraries, and publishers (perhaps many of them) won't do it
that way regardless of what we say.  The fact that URNs are
particular kinds of URIs and that fragments and queries are part
of the basic URI syntax definition doesn't help -- if URNs were
a completely different sort of creature, these things would be
easier to prevent.

Queries are even worse than fragments for the reasons discussed
by others on the list.

In the world in which we actually live, the effect of
prohibiting either fragments and queries given that people think
they need them will be to have them go ahead and use them
--locally or even in national or other profiles-- without
telling us (especially if we express no sympathy for their
needs). That is pretty much a guarantee of non-interoperability.
I think interoperability, rather than notions of architectural
purity (including the ones I've described above) had best be our
goal.

So, recommendations:

(1) Incorporate provisions for fragments and queries in the
syntax document, allowing them to be authorized on a
per-namespace basis.

(2) Strongly advise against namespaces allowing them except in
exceptional circumstances where the need is significant and they
can be tightly defined.

(3) Alter the registration template for new URN namespaces so
that it explicitly permits a template to allow fragments and/or
queries, pointing to the advice above to discourage their use.

(4) Modify  the registration procedure for URN namespaces to
follow the general model established for media types and allow
three types of registration instead of the current two.  The
third would be available to "recognized standards bodies".  That
term is deliberately undefined for media types and should be
undefined here but the intent would be to allow (at least)
bodies who have reasonable and open procedures for developing
stable standards-grade specifications and who are responsible
for the definition of underlying identification systems to take
lead responsibility for the related URNs.  The general procedure
would be that they would expose their proposed definitions to
the IETF for review and comment (so that we can provide advice
on syntax and usage) but the final decision to ask IANA to
register an appropriate namespace without a "urn-" prefix would
lie with them.   The usual "stable spec" rules would apply --
they could either provide IANA with a pointer to a specification
they published, ask for RFC publication of their spec as an
Informational document, or both.

And then we move on.

Suggestions and comments welcome.  More arguments for purity are
really not unless one is willing to to argue that
non-interoperability, or producing standards we know will be
ignored, are good things or at least ok.

best,
    john

I think URNs ought to point to

---

From: Keith Moore <[email-masked]>
Date: Thu, 13 Jun 2013 22:06:35 -0400

> syntax document, allowing them to be authorized on a

I don't think this makes any sense.  A fragment is a property of the 
resource.   A query is a property of the resource and the protocol used 
to access the resource.  Neither is, or should be, a property of the 
resource name.

Also, making fragments/queries a per-namespace attribute increases 
complexity for no gain.

> exceptional circumstances where the need is significant and they

URN namespaces already cannot allow them.   2141 syntax forbids either ? 
or # appearing in namespaces.   To change that would be an incompatible 
change to the protocol.  Nor is there any need to change the syntax.

> ignored, are good things or at least ok.

This isn't an argument about purity.   Making fragments and query 
strings properties of URN namespaces will harm interoperability rather 
than help it.

Keith

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 13 Jun 2013 22:27:54 -0600

On 6/13/13 8:06 PM, Keith Moore wrote:
> I don't think this makes any sense.  A fragment is a property of
> the resource.   A query is a property of the resource and the
> protocol used to access the resource.  Neither is, or should be, a
> property of the resource name.
> 
> URN namespaces already cannot allow them.   2141 syntax forbids
> either ? or # appearing in namespaces.   To change that would be an
> incompatible change to the protocol.  Nor is there any need to
> change the syntax.
> 
> strings properties of URN namespaces will harm interoperability
> rather than help it.

However, we know that people are going to use at least fragments and
probably queries.

I would really prefer to avoid the URN equivalent of LEIRI vs. IRI.

John's argument makes sense to me. I don't have to like the
conclusions he's reached to understand that consensus here might
require compromise.

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: Keith Moore <[email-masked]>
Date: Fri, 14 Jun 2013 00:57:22 -0400

> conclusions he's reached to understand that consensus here might
It's the wrong kind of compromise.  It will destroy the carefully 
designed functionality of URNs.

I'm currently writing up an Internet Draft that will explain the 
problem, and my proposal, in more detail.

Keith

---

From: John C Klensin <[email-masked]>
Date: Fri, 14 Jun 2013 12:49:27 -0400

--On Friday, June 14, 2013 00:57 -0400 Keith Moore
<[email-masked]> wrote:

>> However, we know that people are going to use at least
>> fragments and probably queries.
>> 
>> conclusions he's reached to understand that consensus here
>> might require compromise.

Peter, let me stress that I don't like those conclusions at all.
They make me uncomfortable and more than a little bit sad.  But
I think they constitute facing reality and dealing with it.
Unless, as I said, we like a model in which we effectively
encourage people to ignore what our specifications say and
create non-interoperable situations.

> It's the wrong kind of compromise.  It will destroy the
> carefully designed functionality of URNs.

Keith, I don't really see it as a compromise as much as facing
reality and trying to make the best of a bad situation.  In
hindsight, 2141 (or some supplemental document) should have been
a lot more clear about aspects of that "carefully designed
functionality" and the motivation for it.  It (or something
else) should have anticipated that there would be requirements
that didn't conform to the URN model, specified what to do to
satisfy them  with some non-URN mechanism, and motivated that
and the reasons to keep that mechanism and URNs clearly
separated.  The motivations should have been clear enough that
people and implementations would have conformed because doing so
was easy, straightforward, and obvious and because there were no
advantages to not doing so.  

It is worth noting that some of the things that are called URNs
but that do not conform to a narrow reading of RFC 2141 actually
go back to interpretations of discussions three or four years
earlier, some of which were recorded in RFC 1737.  As a
strategy, taking a term that has been used for something fairly
broad and then trying to reuse it to describe something more
specific in the hope that the prior uses will spontaneously
disappear rarely works well.  I believe history shows that the
IETF Terminology Police have been even less effective than the
IETF Protocol Police.  If you don't like how that worked out,
you should probably complain vigorously to the relevant AD when
2141 was completed and published.

People who don't want an analysis of the history of how we got
here, at least from my perspective, may want to stop reading
here and skip to the end of this note.

It is probably still controversial, but, again in hindsight, I
think it may have been a mistake to bind URNs to the URI concept
and syntax, at least without a richer elaboration and set of
implementations of the whole "Internet Information
Infrastructure Architecture (IIIA)" described in RFC 1737 and
other documents of that period.  I wish it had been possible at
the time to come up and carry out a plan to develop the IIIA and
fit URNs and some other URI types into it cleanly.   Instead, as
you will probably remember, the WG deteriorated into such
hopeless bickering about different visions of architectural
purity and the right ways to split hairs that, after several
discussions and warnings, I, as the relevant AD at that time,
notoriously decided that the only way to salvage anything was to
kill the URI WG and, with it, the unified and generalized URI
effort.   The intent of that shutdown was to force people to
regroup and get more focused rather than producing yet more
grand architectures for castles in the sky and other edifices
that would not, and probably could not, be built.   

I (and many others) knew at the time that a decision to force
separate and focused work efforts was likely to result in less
architectural elegance that people had hoped for, so the
conclusion, like this month's conclusions, saddened me.  But
there was clearly no consensus around how to move forward with
IIIA once people had to start talking about details and building
things rather than sketching architectures.  The choice was not
about preserving IIIA or any other flavor of generalized
architectural purity because they were already moribund.  The WG
shutdown action stopped the general theorizing and eventually
led to the far more focused effort that produced URNs and 2141.
The other major URI types contemplated by the IIIA went nowhere,
which is unfortunate, but the available choices did not involve
having them, architectural purity, URNs with a clear role, and
clear alternatives for other purposes.  It was between having
the URNs we have -- including a few warts that may have been the
result of trying to incorporate pieces of the architecture that
arguably should have gone into other URI types -- and nothing at
all other than DNS-based locators (at least nothing else that
would have been URI-ish).

In that context, I agree with you that RFC 3986 is an
unfortunate document, both because of the reasons you have
pointed out and because parts of it seem to represent a return
to the thinking of the RFC 1736-1738 period that I would
characterize as "if only one can generalize enough, everything
will fall into place".   But accepting it as part of our reality
isn't a "compromise" either... it is just accepting reality...
at least until someone can get traction on actually revising the
thing.

Now, if either the IETF time machine or my personal one worked
better, there are a number of things I'd like to go back and
change.  I would have liked to see the slightly clearer, better
explained, and even more focused URN model hinted at in my first
paragraph to you above.  I would have liked some other URI types
to be developed in parallel -- different persistency properties
and object models and maybe to have had URNs targeted a bit more
on named objects and less on abstract "stuff".  I would have
preferred a somewhat different specification than what ended up
as 3986.  And, because it seems obligatory, I'd like a pony.

But, in the real world, it is just too late, at least for the
IETF.  The Internet Architectural Philosophy Task Force might be
able to do better, but what has happened and is happening in the
real world is an inescapable engineering constraint.  I am not
suggesting that the IETF adopt and encourage things that we know
to be bad practices that threaten interoperability or effective
Internet operations -- I've argued strongly against the "people
are doing it, so we have to accept and standardize it" approach
elsewhere and will continue to do so.  

Now I'm making some assumptions that I should be explicit about.
I assume that having a well-defined and unifying set of
mechanisms for describing and handling stable identifiers is a
good idea (if it isn't, it is really hard to defend URNs rather
per-object-type URIs or some other per-object mechanism).  I
assume that a key condition for such stable identifiers is that
we have a sufficient registry structure to vastly reduce the
odds of two parties using the same namespace identifier for
different purposes (I don't know how to maintain a useful family
of stable identifiers without such arrangements, but would be
interested if you have a model for doing so).   

I'm also assuming that the IETF's saying "these URN namespaces
are a good and proven way to handle stable identifiers, people
should consider them when a stable identifier type is needed"
and then saying "but unless your type and definitions meet our
norms and requirements, which we reserve the right to set
without actually understanding your problem space, we won't let
you register your namespace and therefore you can't use it"
would be delusional.  Unless you think there is something
special about URNs, I would suggest the long history of people
squatting on other identifier keywords (in many different areas)
when we've created high barriers to registration makes that
particular assumption a very safe one, no matter how much we
would wish otherwise.

Picking up a few points from your earlier note...

 
> I don't think this makes any sense.  A fragment is a property
> of the resource.   A query is a property of the resource and
> the protocol used to access the resource.  Neither is, or
> should be, a property of the resource name.

Despite your comment elsewhere, that is precisely what I've
described, I believe accurately, as argument for purity.  No
matter how much you (or even I) might prefer it were not the
case, the marketplace has decided otherwise.  Adjusting things
as experience accumulates is what Proposed Standards are
supposed to be about, remember?  But, of course, one can made
the case that, contrary to the way the IETF Standards process
normally works (and especially that property of Proposed
Standards), 2141 freezes the name, definition, and syntax of
URNs in place for all time.  If so, then Peter, myself, and some
others we could round up might reasonably take the existing
draft documents and recast them into a new definition for USIs
--Uniform Stable Identifers-- used those recast documents to
explicitly deprecate 2141 and move it to historic on the ground
that the marketplace had gone in another direction and
2141-style URNs were no longer relevant, and then go to the IESG
and argue for a new or rechartered WG that would focus on the
new specs and efficiently moving 2141-style URNs to the dustbin
of protocol history.

I think that would be a worse alternative than readjusting the
URN definition to reflect what we understand today (and may not
have understood 16 years ago) because I think the predictable
result in the marketplace would be a mixture of:

	* Use of new format, syntax, definitions and name
	* Use of 2141-conformant URNs despite the IETF's
	  deprecating them and encouraging migration
	* Use of the new new format, syntax, and definitions,
	  but calling them URNs

and, probably,
	* Use of some things, under one URI identifier or
	another, that conform to neither spec because we've
	confused people enough that they no longer care.

I think a forward-compatible change that recognizes evolving
understanding and applications and that legitimizes some uses
that are now deployed but non-conforming would be a better plan.
YMMD, of course.
	
> Also, making fragments/queries a per-namespace attribute
> increases complexity for no gain.

See below.
 
>> exceptional circumstances where the need is significant and
>> they can be tightly defined.
> 
> URN namespaces already cannot allow them.   2141 syntax
> forbids either ? or # appearing in namespaces.   To change
> that would be an incompatible change to the protocol.  Nor is
> there any need to change the syntax.

Here is where we clearly disagree and what I think may be the
essence of the disagreement.  First, what you see as an
"incompatible change to the protocol" is what I see as a
forward-compatible extension.   If a tail containing a "#" or
"?" is passed to a generic URN processor today, it will
presumably be rejected.  As far as I can tell, 2171 does not
require that it be rejected with a specific "invalid syntax"
status.  Consistent with how we usually specify protocols, that
is just outside the protocol requirements.  We also know that
some things that claim to be URN processors will accept these
things without obvious harm and that generic URI syntax
processors that conform to Internet Standard STD 66 (aka RFC
3986) will accept the syntax.  

Moveover, 2141 doesn't say that "?" and/or "#" are treated as
normal characters and can be used that way in URN strings.
_That_ would leave us with the sort of incompatible change to
which you refer.   It doesn't even put them on the "excluded
characters" list of Section 2.4, which might be an issue.
Instead, what it says (Section 2.3) is that these characters are
reserved for future use associated with the uses of those
characters specified in RFC 1630.  Then it says (Section 2.3.2):

	"The URN-WG has not yet debated the applicability and
	precise semantics of those purposes as applied to URNs.
	Therefore, these characters are RESERVED for future
	developments."

That does not sound like "chiseled into stone forever" or even
"incompatible with the whole idea of URNs" to me.  It says,
something more like "when some future URN-related WG comes along
and assigns uses and semantics, we have reserved the characters
to make that possible".  I read that as explicitly allowing for
and authorizing the extensions we are now discussing, not
consigning them to eternal damnation as your notes on the
subject seem to imply.

> This isn't an argument about purity.   Making fragments and
> query strings properties of URN namespaces will harm
> interoperability rather than help it.

Do you have worked examples to back up that assertion?
Especially given that, as far as I can tell, 2141 explicitly
predicts such extensions?

Let's consider a slightly hypothetical case.  There is a
community that has an extremely stable identifier for books.
That identifier is a well-recognized international standard in
use by almost every serious book publisher worldwide.  If one
counts specifically-identified objects, that identifier set is
probably much more used than unrelated standardized URNs and
much more used outside the URN context than in it.  They have
even solved, to the satisfaction of their community, the always
difficult object naming problem of determining when two alleged
instances of something are the same object and should get the
same identifier and when they are different.   Now a significant
constituency within that community comes along and says "we need
to use a fragment identifier to point to specific chapters" and
intend to do it by using

URN:Our-Book-Identifier-Name:Our-Internationally-Specified-Identifier-code#ChapterN

Now you propose to tell them that they can't do that.   Why do
you believe they will pay any attention to you?    Why do you
believe that your telling them that and their ignoring you will
hurt the Internet?   Why do you think it is better to have them
do that and not document it in any way that is visible to the
IETF community than to have the usage clearly-specified and
documented?

I just don't get it.

As far as the "per namespace" part of my suggestion, I suggest
to you that we have lots of protocols (including the one used to
transport this message) for which we have specified syntax in a
general way and then imposed contextual and semantic
restrictions on how and where that syntax is actually used and
permitted.  I don't see the difference between that and saying,
e.g., "fragments are permitted in URNs using the following
general syntax, but the particular form and semantics of
fragment identifiers, including whether they are permitted in
practice, is part of the namespace definition".  Again, I just
don't see another practical way to do that, especially given
that, which "#Chapter5" (or some similar syntax) might make
perfectly good sense for a book, it makes little or no sense for
a scientific journal, less sense for an article in such a
journal, and still less sense for, e.g., a photograph.

> I'm currently writing up an Internet Draft that will explain
> the problem, and my proposal, in more detail.

I look forward to seeing that.  Given that we are some 15 or 16
years into the deployments and use of URNs for various purposes
and the spectrum of fully conforming and less conforming
implementations, I hoe that your I-D will also explain where you
expect to see your proposal implemented and deployed in actual
name-assigning and user communities.

   john

---

From: SM <[email-masked]>
Date: Fri, 14 Jun 2013 12:47:24 -0700

>IETF community than to have the usage clearly-specified and

I don't think that it is a good idea to tell people that they cannot 
do that as it does not solving the problem they would like to 
solve.  If there isn't a convincing reason, in their opinion, not to 
do that I view it as the IETF community failing to find solutions.

Regards,
-sm

---

From: Keith Moore <[email-masked]>
Date: Fri, 14 Jun 2013 15:51:23 -0400

Without responding point-by-point (which I might do later or in private 
mail), I think it's pretty simple.

1.  RFC 2141 defined a type of identifier which it called Uniform 
Resource Names or URNs.   It also defined a syntax for URNs which 
happens to begin with 'urn:', and rules for creating and assigning and 
not reassigning URNs.   Granted, there had been discussions for years 
prior to that which used the term URN more loosely and/or which proposed 
different rules and different syntaxes, but 2141 represents a 
rough-consensus result of those long discussions aimed at understanding 
what URNs really should be.

2. For various reasons, some people weren't happy with the URN syntax or 
framework that IETF decided on.   One result was that the authors of RFC 
3969 tried to broaden the definition of URNs in that document.

3.  Even if you accept (as RFC 3969 states) that the name URN applies to 
things other than those defined in 2141, the situation we were left with 
is both confusing and cumbersome to discuss. The identifiers defined in 
RFC 2141 have unique properties, by design, which do not necessarily 
apply to other persistent URI-like identifiers.   And it's cumbersome to 
discuss RFC 2141 identifiers specifically (for the purpose of updating 
RFC 2141, or for any other purpose) while still being consistent with 
the language in RFC 3969.   You end up either saying "for the purpose of 
this document, URN refers to the identifiers defined in RFC 2141, 
language in RFC 3969 notwithstanding" (inviting confusion from those who 
miss that restriction), or you end up saying something like "URNs as 
defined in RFC 2141" every time you need to refer to that kind of 
identifier.

Regardless of what RFC 3969 says, the identifiers most often associated 
with the term URN are undoubtedly those that begin with 'urn:' and have 
a syntax consistent with RFC 2141.   Trying to say that there are URNs 
that don't begin with 'urn:' is like saying that there are other HTTP 
URLs that don't begin with 'http:'. Yes, it's true in a sense, but it's 
just silly and confusing and there's no good reason to define things 
that way.

Also, regardless of what RFC 3969 says, URNs as defined in RFC 2141 
weren't designed to be used with fragment identifiers or query strings - 
or at least, we didn't manage to define how that would work, and the 
reason that we didn't define how that would work is because there wasn't 
an obvious interpretation that didn't kill the properties we wanted for 
URNs.   So we declined to do that in RFC 2141, and in trying to 
generalize URI syntax, RFC 3969 didn't address those issues either.

It is of course possible to define other URIs with different syntax or 
rules that have persistence properties similar to, and perhaps even 
better than, those of URNs.   Many people claim to have done so.   Those 
identifiers, and URNs, should each succeed or fail on their own 
merits.   However, those other URIs will inevitably have subtly 
different properties than URNs as defined in RFC 2141.    (If someone 
wants a name to refer to all such identifiers, I nominate "persistent 
URIs".)   Properties of URNs shouldn't be automatically assumed to apply 
to other kinds of persistent URIs, nor vice versa.

4.  URNs (and when I say URN, I always mean the 2141 definition), above 
all else, are intended to be persistent.   This is the fundamental 
property of URNs - not only that they are persistent, but that the 
presence of the 'urn:' tag is an indicator that the identifier may be 
interpreted as persistent, and also that the persistence of that 
identifier is an important property that should be maintained.   
Extending the concept of URNs in such a way that the resulting 
identifier is no longer persistent would break the essential and 
fundamental property of URNs.

5. It was always intended that URNs should be applicable to existing 
resources such as HTML documents, or HTTP-based search engines, though 
by no means limited to those resource types.

6. Fragment identifiers as used in existing content-types were not 
designed to be persistent across changes to the document.   For 
identifiers to have persistence there needs to be some discipline in 
assigning meaning to them initially and in not reusing them in ways 
inconsistent with their originally assigned meanings.   If there is any 
such convention for fragment identifiers, I'm not aware of it, but it 
certainly isn't widely used.   Given the existence and utility of other 
kinds of identifiers which are not persistent and should not be so, I 
also believe that persistent identifiers need to be readily 
distinguished from non-persistent identifiers.   Again, I'm not aware of 
a convention for doing this, though I hypothesize that one could exist.

7. Thus, the combination of a URN and a fragment identifier has no 
assurance of persistence.   It follows that the combination of a URN and 
a fragment identifier cannot be a URN.

8. One can argue that the persistence of an identifier consisting of a 
URN and a query string could actually be persistent, if the resource 
named by the URN were defined in such a way as to make such queries 
persistent.    For instance, if ISBNs are persistent identifiers, then a 
service that returned citation data associated with an ISBN, and which 
accepted the ISBN as a query string, would appear to have approximately 
the persistence properties necessary for the combination of a service 
URN and a query string, to itself be considered a URN.   But defining 
those properties in a way that is generally applicable to resources that 
accept queries is problematic.   In addition, it is desirable to be able 
to write an identifier that consists of a URN and a query string even 
for resources for which queries do not have a consistent meaning over time.

9. At any rate, existing resources that accept query strings do not in 
general assure persistence of the results of such queries.   Thus, in 
general, a combination of a URN used to name an existing resource, and a 
query string, provides no assurance of persistence, and the combination 
should not be considered a URN.

The above, I submit, is reality.   (There are probably some other 
relevant and salient points which are also defensible as reality.)
------------------------------------------------------
What's below this line is brainstorming.

Now, given the above, I can see the need for up to four kinds of 
identifiers consisting of a "base" URN and a query string or fragment 
identifier:

1.  URN + fragment identifier, where there is no assurance of the 
persistence of the fragment identifier over changes to the named 
resource.   This combined identifier is not a persistent identifier, and 
thus not a URN.

(One can argue that such an identifier isn't really useful.   But I 
think it's potentially useful in the same sense that a reference to a 
document with section or page number can still be useful if that 
document was revised and the old section or page number is no longer 
quite correct... at least it gets you to the right document.)

2.  URN + query string, where there is no assurance of the persistence 
of the results of the query service (e.g. that the query syntax wouldn't 
change or that the interpretation of the query wouldn't change).   This 
combined identifier is not a persistent identifier, and thus not a URN.

3. URN + fragment identifier, under some hypothetical scheme for naming 
fragments that assured the persistence of the fragment identifier over 
changes to the resource, and for which the fragment identifiers were 
themselves recognizable as being persistent.   (For example, one could 
define a new URN for each fragment.)
This combined identifier would qualify as a URN.

4. URN + query string, where there were some (as yet undefined) 
assurance of persistence of query results (for all possible query strings).

Of these, cases 1 and 2 are easily described.  Cases 3 and 4 need more 
work (perhaps much more work).   It's important that each of these four 
kinds of identifier be readily distinguished from one another.  I can 
think of several different ways to do that.

Common URI syntax, and common practice, is to combine a base URI with a 
fragment identifier or query string by appending the latter to the 
former.   Using this convention for cases 1 and 2 seems likely to cause 
confusion, because the resulting identifier would begin with 'urn:' but 
not have the persistence properties required and expected of URNs.   
Using this convention for cases 3 and 4 might also cause confusion, 
because people see these identifiers, and it wouldn't be obvious to most 
people that a string like urn:foo:bar?zot was not simply the result of 
appending the query string ?zot to an ordinary URN.

I'm not committed to this view, but I suspect that using the existing 
convention for either cases 1 and 2 or cases 3 and 4 would promote 
widespread misunderstanding.   (And URNs are already suffering from a 
tremendous amount of misunderstanding...).   So I'm guessing that we 
need (at least) two separate syntactic conventions for deriving 
identifiers that are based on URNs, neither of which is simply URN + 
fragment id or query string. (One can argue for additional conventions 
to serve as input to URN resolution, but that would require a much more 
rigid definition of URN resolution than has yet been attempted, so I 
think we can safely say that's a topic for future study.)

One approach might be to define two new kinds of identifier: 
non-persistent URN-derived names and persistent URN-derived names.   
Assign them distinct prefixes like nudn: and pudn:. Define rules for (a) 
creating a NUDN from URN + query string or URN + fragment ID, and (b) 
interpreting an NUDN by extracting the embedded URN, resolving it, and 
interpreting the query string or fragment id in the context of the 
resource resulting from resolution of the URN.   Perhaps the definition 
of 'pudn:' should be left for further study, but it might still be 
useful to mention them as a way of distinguishing them from NUDNs.  And 
reinforce the notion that neither '#' and '?' should appear in a URN; 
and if they do appear, the result is at best undefined, or perhaps 
defined to be invalid.

Strawman proposal for NUDN syntax (just to have something concrete to 
think about, or throw tomatoes at, not because I'm in love with it):

nudn =  'nudn:' + base64-encoded-URN + '?' + query-string
           /   'nudn': + base64-encoded-URN + '#' + fragment-id

I do think it's highly desirable that identifiers from cases 1 and 2 NOT 
begin with 'urn:'

I don't think there's any way to solve this problem that's not ugly, but 
some ways are less ugly than others.

Keith

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 14 Jun 2013 14:43:02 -0600

On 6/14/13 1:47 PM, SM wrote:
> I don't think that it is a good idea to tell people that they cannot do
> that as it does not solving the problem they would like to solve.  If
> there isn't a convincing reason, in their opinion, not to do that I view
> it as the IETF community failing to find solutions.

Yes, I think that is how it will be perceived. It's the same story as
LEIRI vs. IRI and I would dearly like to avoid that fate here!

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Keith Moore <[email-masked]>
Date: Fri, 14 Jun 2013 16:56:56 -0400

> LEIRI vs. IRI and I would dearly like to avoid that fate here!
Without your being more specific about what you mean here, I don't know 
how to evaluate it.

Keith

p.s. responding to "it's not a good idea to tell people that they cannot 
do <something>":

URNs were invented for particular purposes, and the standards for URN 
syntax and use were carefully tailored to fit those purposes. If someone 
wants to define a kind of identifier that isn't consistent with those 
purposes, URNs are not what they need.

It happens all the time that we and others define solutions that turn 
out to not be appropriate for some unanticipated use case, or that some 
use cases were ruled out of scope.   People might not like the result, 
but it's necessary and inevitable that it happens sometimes.   To make 
things too general is a sure path to failure.

Similarly, if we've somehow failed to be sufficiently general in how 
we've defined URNs, so that the syntax or other rules governing URNs 
don't permit uses of URNs that were nevertheless consistent with those 
intended purposes, it's a bit late to change those standards.   That's 
not to say that we absolutely cannot do so.  But if we attempt to do so, 
we risk of harming all other uses of URNs, and the risk of that has to 
be balanced against the perceived benefit of changing the URN 
specifications in an incompatible way.

---

From: Julian Reschke <[email-masked]>
Date: Sat, 15 Jun 2013 13:02:40 +0200

On 2013-06-14 21:51, Keith Moore wrote:
> ...
> ...

s/3969/3986/ I assume.

FWIW, RFC 3986 says in 
<http://greenbytes.de/tech/webdav/rfc3986.html#URLvsURN>:

"1.1.3. URI, URL, and URN

A URI can be further classified as a locator, a name, or both. The term 
"Uniform Resource Locator" (URL) refers to the subset of URIs that, in 
addition to identifying a resource, provide a means of locating the 
resource by describing its primary access mechanism (e.g., its network 
"location"). The term "Uniform Resource Name" (URN) has been used 
historically to refer to both URIs under the "urn" scheme [RFC2141], 
which are required to remain globally unique and persistent even when 
the resource ceases to exist or becomes unavailable, and to any other 
URI with the properties of a name.

An individual scheme does not have to be classified as being just one of 
"name" or "locator". Instances of URIs from any given scheme may have 
the characteristics of names or locators or both, often depending on the 
persistence and care in the assignment of identifiers by the naming 
authority, rather than on any quality of the scheme. Future 
specifications and related documentation should use the general term 
"URI" rather than the more restrictive terms "URL" and "URN" [RFC3305]."

If you really really believe there's something to fix over here then 
please make a concrete proposal.

> things other than those defined in 2141, the situation we were left with

Actually, it recommends to avoid the term. All it says about is what it 
was used for historically.

> defined in RFC 2141" every time you need to refer to that kind of

I believe you read something into RFC 3986 which isn't there.

> ...
> ...

RFC 3986 just defines generic syntax. It doesn't say anything about URI 
schemes having to support queries. What it *does* say is how queries, 
when present, affect resolution against a base URI.

It also doesn't say anything specific about fragments in URI schemes. 
What it says (<http://greenbytes.de/tech/webdav/rfc3986.html#fragment>) is:

"The semantics of a fragment identifier are defined by the set of 
representations that might result from a retrieval action on the primary 
resource. The fragment's format and resolution is therefore dependent on 
the media type [RFC2046] of a potentially retrieved representation, even 
though such a retrieval is only performed if the URI is dereferenced. If 
no such representation exists, then the semantics of the fragment are 
considered unknown and are effectively unconstrained. Fragment 
identifier semantics are independent of the URI scheme and thus cannot 
be redefined by scheme specifications."

Do you disagree with this?

> to other kinds of persistent URIs, nor vice versa.

Right. I don't see any disagreement about that.

> identifier is an important property that should be maintained. Extending
> the concept of URNs in such a way that the resulting identifier is no
> longer persistent would break the essential and fundamental property of
> URNs.

Right. I don't see any disagreement about that either.

> by no means limited to those resource types.

Yes.

> a fragment identifier cannot be a URN.

I agree that it would be a bad idea. So is this about whether we allow a 
URI using the "urn" scheme carrying a fragment identifier to be called a 
URN?

> for resources for which queries do not have a consistent meaning over time.

As per RFC 3986, the query component is used to identify the resource, 
just like the path. It's just syntax. Whether two URIs that only differ 
in the query part have anything in common depends solely on the URI 
scheme definition.

> general assure persistence of the results of such queries.   Thus, in

...so this kind of terminology is misleading. In general, you use both 
path and query to locate the resource.

> relevant and salient points which are also defensible as reality.)

I don't like that direction.

A URI scheme can allow queries. In that case, not making the query part 
of the scheme syntax would be totally confusing.

Not *calling* it a URN might work, but still: it's part of the URI, but 
not part of the URN? Things might work better if you mint a new term for 
the URN part that *is* persistent. Or better yet, don't go there: do not 
allow queries purposes that conflict with the intent of URNs.

> not have the persistence properties required and expected of URNs. Using
> this convention for cases 3 and 4 might also cause confusion, because
> people see these identifiers, and it wouldn't be obvious to most people
> that a string like urn:foo:bar?zot was not simply the result of

Indeed.

Before we go there: do we have a concrete use case that we would test 
these ideas with?

Best regards, Julian

---

From: John C Klensin <[email-masked]>
Date: Sat, 15 Jun 2013 09:15:53 -0400

Keith,

It is starting to feel as if we are either reading different
documents with the same names and identifiers or that we are
somehow reading the same documents very differently.  Key
examples and some other discussion inline below.

Note to impatient readers: there is a proposal for specific text
(a proposed new section of 2141bis and update to 1737) at the
end of this over-long note.

--On Friday, June 14, 2013 15:51 -0400 Keith Moore
<[email-masked]> wrote:

>...
> 1.  RFC 2141 defined a type of identifier which it called
> Uniform Resource Names or URNs.   It also defined a syntax for
> URNs which happens to begin with 'urn:', and rules for
> creating and assigning and not reassigning URNs.   Granted,
> there had been discussions for years prior to that which used
> the term URN more loosely and/or which proposed different
> rules and different syntaxes, but 2141 represents a
> rough-consensus result of those long discussions aimed at
> understanding what URNs really should be.

My reading of draft-ietf-urnbis-rfc2141bis-urn-05 is entirely
consistent with that.  It excludes the other uses of the term
"URN" in favor of talking about 2141-type URNs, contains
explicit statements about minimal necessary chances from 2141,
etc.  Now, if I thought 3986 were as bogus as you apparently do,
or even found it mildly distasteful (which I do), and were
holding the pen on 2141bis rather than Peter, I'd probably try
to structure the introductory paragraphs to sound less like the
main reason for the document was to bring 2141 into conformance
with 3986.  But, even if Peter rephrased things that way, they
wouldn't change the spec significantly in most areas, only the
vocabulary used to describe it.

> 2. For various reasons, some people weren't happy with the URN
> syntax or framework that IETF decided on.   One result was
> that the authors of RFC 3969 tried to broaden the definition
> of URNs in that document.

I assume you mean "3986" in the above and a few places below
that you mention 3969.

Whether that characterization is accurate or not --it goes
almost without saying that some people (maybe exclusively the
same "some people", maybe not) would disagree-- 3986 has now
stood at a full Internet Standard for more than eight years
without any significant challenge to its validity as a
specification or claim to be an Internet Standard.  Given that,
I think URNBis and rfc2141bis are obligated to be consistent
with it, at least unless we want to make the claim that 2141
(and 2141bis) URNs are really not URIs.  I don't see any basis
in the URNBis charter for either challenging the "URNs are URIs"
assumption or for simply ignoring an applicable Internet
Standard that is much later than 2141.

> 3.  Even if you accept (as RFC 3969 states) that the name URN
> applies to things other than those defined in 2141, the
> situation we were left with is both confusing and cumbersome
> to discuss. The identifiers defined in RFC 2141 have unique
> properties, by design, which do not necessarily apply to other
> persistent URI-like identifiers.   And it's cumbersome to
> discuss RFC 2141 identifiers specifically (for the purpose of
> updating RFC 2141, or for any other purpose) while still being
> consistent with the language in RFC 3969.   You end up either
> saying "for the purpose of this document, URN refers to the
> identifiers defined in RFC 2141, language in RFC 3969
> notwithstanding" (inviting confusion from those who miss that
> restriction), or you end up saying something like "URNs as
> defined in RFC 2141" every time you need to refer to that kind
> of identifier.

Yes, that is a bit of an editorial challenge.  I don't see it as
a serious substantive problem because I don't think anyone is
claiming that draft-ietf-urnbis-rfc2141bis-urn should be
anything but 2141bis... with adjustments made to conform
2141-style URNs to the requirements of 3986.  I don't see that
3986 requires draft-ietf-urnbis-rfc2141bis-urn to adopt any
fundamental definition for URNs other than that of 2141 and
don't think the draft contains such a definitional change.  So
I'm not sure where we are disagreeing.

> Regardless of what RFC 3969 says, the identifiers most often
> associated with the term URN are undoubtedly those that begin
> with 'urn:' and have a syntax consistent with RFC 2141.
> Trying to say that there are URNs that don't begin with 'urn:'
> is like saying that there are other HTTP URLs that don't begin
> with 'http:'. Yes, it's true in a sense, but it's just silly
> and confusing and there's no good reason to define things that
> way.

But, as far as I can tell, draft-ietf-urnbis-rfc2141bis-urn-05
doesn't do that and in fact carefully avoids it.  So the above
is either an attack on 3986 (and hence out of scope for the WG)
or just isn't relevant.

> Also, regardless of what RFC 3969 says, URNs as defined in RFC
> 2141 weren't designed to be used with fragment identifiers or
> query strings - or at least, we didn't manage to define how
> that would work, and the reason that we didn't define how that
> would work is because there wasn't an obvious interpretation
> that didn't kill the properties we wanted for URNs.   So we
> declined to do that in RFC 2141, and in trying to generalize
> URI syntax, RFC 3969 didn't address those issues either.

And here we get to what I think are two of the core issues.
Let's separate them into two questions:

	(i) If there were no syntax restrictions imposed by
	2141, 3986, or anything else, would fragment identifiers
	and/or queries be appropriate for URNs?
	
	(ii) Is 3986 required to authorize fragments or queries
	in the URN syntax and, if so, is that a serious and
	problematic incompatible change?

Let me address the second here and the first below.

You read the statements in 2141 (or perhaps some oral tradition
to which the rest of us don't have access) as prohibiting
fragment identifiers and queries.  When I read what I hope are
the same statements, I conclude that there are lots of ways to
say "prohibit".  One of them involves the term "excluded", which
is exactly what 2141 says about a number of characters in its
Section 2.4.    But what it says about "?" and "#" is to
identify them with purposes defined in RFC 1630 -- no appeal to
presumed 3986 revisionism is required-- and "has not yet debated
the applicability and precise semantics of those purposes as
applied to URNs".  It then says "these characters are RESERVED
for future developments".  To me, those are statements that
imply that those future developments are anticipated, even
though no timeframe is given for them (and they might not
happen).

When the spec explicitly says those sorts of things about future
definition of semantics and future use, it seems to me that
comments like "weren't designed to be used..." (with the
implication of "designed to _not_ be used") are a real stretch.
"didn't manage to define how that would work, and the reason
that we didn't define how that would work is because there
wasn't an obvious interpretation that didn't kill the properties
we wanted for URNs" might be true, but there is no evidence at
all for it in 2141.  All I can get from 2141 is that there
wasn't consensus on particular semantics but that there was no
particular reason to expect that semantics and consensus would
not emerge in the future.  Again, if the intent had been to say
"we concluded that this was impossible without violating
fundamental URN design decisions" or just "really bad idea" how
do you explain 2141 not just saying that rather than talking
about future use?

Your memory or that of others as to what you thought the intent
was at the time notwithstanding, 2141 is now a 16-year-old spec.
I think what it actually says has to be taken at face value,
especially if it seems to contradict what you believe was the
intent.

>...
> 4.  URNs (and when I say URN, I always mean the 2141
> definition), above all else, are intended to be persistent.
> This is the fundamental property of URNs - not only that they
> are persistent, but that the presence of the 'urn:' tag is an
> indicator that the identifier may be interpreted as
> persistent, and also that the persistence of that identifier
> is an important property that should be maintained.
> identifier is no longer persistent would break the essential
> and fundamental property of URNs.

I don't think anyone is disagreeing about that.  At least I'm
not and I don't see anything in the current version of 2141bis
that does either.

>...
> 6. Fragment identifiers as used in existing content-types were
> not designed to be persistent across changes to the document.
> For identifiers to have persistence there needs to be some
> discipline in assigning meaning to them initially and in not
> reusing them in ways inconsistent with their originally
> assigned meanings.   If there is any such convention for
> fragment identifiers, I'm not aware of it, but it certainly
> isn't widely used.   Given the existence and utility of other
> kinds of identifiers which are not persistent and should not
> be so, I also believe that persistent identifiers need to be
> readily distinguished from non-persistent identifiers.
> Again, I'm not aware of a convention for doing this, though I
> hypothesize that one could exist.

I think you are creating a strawman here and then demolishing
it.  There is no question, in my mind, at least, that using,
e.g., a character offset fragment identifier would be truly
stupid in a context that requires  persistence, URN or
otherwise.  But that doesn't imply that, for some URN types
(namespaces) stable fragment identifiers cannot be properly
identified and defined.  Second, even after rereading 2141 and
1737, I'm not sure how far one can go in the direction of an
identifier that is "persistent across changes in a document"
because even that statement takes one very far in the direction
of needing a universal theory about what changes are still "the
same document" and what changes make new documents. 

More important, as soon as you say "persistent identifier" and
"changes to object" in the same context, you transport us all to
to the edge, not of rathole, but of a bottomless pit.
"Persistence" of an identifier is clear when there is a single,
unique, object and the only thing that changes is its location.
That is where one of the major threads that led to URNs started
when web objects were considered -- URLs were just not right
when one considered content that might be relocated, unchanged,
from one server (and DNS name) to another.  But, as soon as one
talks about two objects that are alleged to be identical or
changes in one object, the "persistence" and object-binding
validity of the identifier become fairly deep questions that
have plagued archivists, classifiers, and philosophers for
centuries... questions that ultimately have no clear answers
except in the axioms and postulates of object-type-specific
axiomatic systems.  Another key reason why I shut down the
original URI WG in the hope of saving the work as that several
of the efforts that were underway appeared to have comprehensive
solutions to the "can it change and be the same thing" and "are
two objects actually identical" questions in the critical path
of a protocol or identifier type questions -- a WG that could
safely be predicted to go on for years and indeed centuries and
never converge was just not considered acceptable in the IETF of
the time.

Note that the above has nothing to do with fragments or queries.
The problem exists in its full glory with URNs and URN-object
bindings as soon as you say that something _is_ "designed to be
persistent across changes to the document".  Whether fragments
make things any worse depends on what the namespace looks like
and how things are designed.  

Let's take the fairly familiar example of a book.  The
publisher, library, and archival communities have established
conventions about when two copies of a book are "the same".  If
they are "the same" then we would except either copy to
represent a correct binding (and resolution response) to a given
URN (or instances of that URN that match the 2141 equivalence
rules).  It is important to understand that "the same"
represents conventions and that pushing the limits too hard
leads to confusion and/or high-minded arguments.  For example,
two different editions are almost always considered different
books for identification or classification purposes.  Two
different printings in which a few obvious typographical efforts
are corrected in the later one are usually considered the same
book.  If one physical instance of the book is autographed by
the author and another is not, they are the "same book" for many
purposes but are certainly not "the same".   

The only possible definitive, convention or axiom-free
definition of "the same" requires agreeing that all objects are
unique and that no URN can be satisfied by more than one
volume-object. (Of course, that is pretty close to an axiomatic
statement too, but of a different type than we we talk about
multiple satisfying objects.)  But "all physically distinct
volumes are unique" would, as a rule, make book-URNs useless for
most of the purposes to which one would like to put them.

Now, given that hypothetical book-URN (i.e., "urn:book:..."),
whether a transformation of the content from bound paper form to
electronic (and non-page-image) form is a change that still
allows both forms to satisfy the same URN is another one of
those philosophical questions that can be resolved only by
convention.  If the convention is that they are the same, than
any fragment identifier that utilizes page numbers is obviously
trash (and cannot be "persistent" across the two forms).  But
one that utilizes chapter numbers or even names doesn't make the
URN any less persistent than it would be if fragment identifiers
are not used or not allowed.  

Suppose, instead, that the convention is established that a
translation is still "the same book".  Now that convention would
make me and probably others very anxious but I can find nothing
specific in 2141 or even 1737 that prohibits it. The nervousness
arises from the URN and namespace definition and not from the
presence or absence of fragments.  But it would _constrain_ the
types of fragment identifiers that make any sense at all.  For
example, using the example above, chapter numbers would still be
sensible as fragment identifiers (at least modulo a few i18n
issues) but chapter names almost certainly would not.

(Aside for the record: whatever that hypothetical "book" URN
might be and how it might be defined,
draft-ietf-urnbis-rfc3187bis-isbn-urn is not it.  The latter
identifies, among other things, a particular set of conventions
about uniqueness and who gets to determine it together with the
consequent rules about object-bindings.  By doing so, it
"solves" a lot of the more general issues described above but
also covers over some differences that might be important for
other purposes.)

> 7. Thus, the combination of a URN and a fragment identifier
> has no assurance of persistence.   It follows that the
> combination of a URN and a fragment identifier cannot be a URN.

That does not follow at all.  If it does, it leads to the
interesting conclusion that a URN cannot be persistent enough to
be a URN unless it names only a single and unique object with no
possibility of changes to the object itself.    It does follow
that fragment identifiers to be used with (or as part of) URNs
have to be designed with far more care about the nature of the
namespace and what that namespace is used to identify than
fragment identifiers for, e.g., URL-identified web pages have
often been in the past.

> 8. One can argue that the persistence of an identifier
> consisting of a URN and a query string could actually be
> persistent, if the resource named by the URN were defined in
>...

I think that query strings associated with URNs are far more
problematic than fragment identifiers because fragment
identifiers (at least by historical convention) point to
something _within_ and object or some subdivision of it.
Queries, as your example (not quoted here) suggests (at least as
I interpret it), can, in principle, be used to take the rest of
the URI as input and return something completely different.
Defining such a situation in a way that would assure persistence
is hard at best.  

Extending your example a bit and combining it with my
hypothetical "book" URN, one could imagine
   urn:book:....?reverse-citation-index

which would return all of the known books or articles that cite
that book.  Since a new citation could be added at any time, the
practical persistency problems of that query are horrible even
if the query could be well-defined (it can't, at least without a
definition of the sources to be searched, but that is just a
property of my choice of a simple, but sloppy, example).

> 9. At any rate, existing resources that accept query strings
> do not in general assure persistence of the results of such
> queries.   Thus, in general, a combination of a URN used to
> name an existing resource, and a query string, provides no
> assurance of persistence, and the combination should not be
> considered a URN.

This is where I wish the IETF could make a bam ban assertions
that are not backed up by citations or evidence that is visible
to the community.  

Let me state that in a way that more closely aligns with the
reality I've seen and that has been pointed out to me.

	"Some existing resources that accept query strings (or
	fragment identifiers) do so in ways that are
	ill-considered and that do not assure persistence of the
	results.  Other existing resources and uses do and are
	fine.  The question is whether it is appropriate to try
	to ban the latter because there are a certain number
	(even a large number) of bad examples or if we should
	try to define things so that the good cases are allowed
	and we are more clear about why the bad cases are bad.".

> The above, I submit, is reality.   (There are probably some
> other relevant and salient points which are also defensible as
> reality.)

Note that, while our realities may differ, part of mine is that
I am extremely positive that, if the IETF says "don't do that,
we think it is evil", we will mostly be ignored and fragments
and queries in things that people will persist (sic) and that
people will persist (sic) in calling URNs and using
"urn:namespace:..." syntax to describe.  If we say "the syntax
is valid but one must be really, really, careful about how the
things are used to ensure that persistence is maintained" and,
ideally, explain why that is important, then we will affect the
behavior of at least some of those who are trying to do The
Right Thing.  If se say "don't do it because we said so" and ban
the syntax, it is nearly certain that we will be ignored by
existing uses of fragments and/or queries and by lots of
potential ones.  And saying "even though that thing that
conforms to the URI syntax and starts in 'urn:', it isn't a URN
and you are forbidden to call it one" is even more useless.  At
least in my pragmatic, observational, reality.

In the interest of even a weak approximation to brevity, I'll
skip comments on your brainstorming for now.

I think this does suggest that 2141bis needs an additional
section that says something like the following.  I've written
this first cut on the assumption that we will allow fragment
identifiers in queries in the syntax, but I believe, for the
reasons explained above, that most of the material is needed
regardless.  Even if we decide to not allow fragment identifiers
and/or namespaces, the relevant text below could probably
usefully be adapted into a "why not" explanation (rather than
having to rely on an IETF assertion of authority).  Some of the
other material above may be useful; I'd be happy to see it in
the document if Peter and the WG believe that would help.   I
believe that, if this type of material is added to 2141bis, it
should be explicitly identified as updating RFC 1737 by
clarifying issues associated with the "requirements" of that
document.

	"The notion of 'persistency' of a URN and  its
	relationship to whatever resource it identifies is key
	to the nature of URNs as defined in this document and in
	the original functional specification [RFC1737].  That
	notion and the associated relationships are, however,
	somewhat elusive and are likely to depend, in practice,
	on conventions and the properties of particular
	namespaces.  For example, if one can speak of replicated
	versions of a resource, transformation of a resource
	into a different form without affecting its content or
	nature, or even changes to a resource that don't alter
	what a URN identifies, one must either establish very
	specific conventions or move into the fundamental
	philosophical problem of when two objects can properly
	be considered "the same".  In more practical terms, if
	replicated objects are considered different for some
	purposes but the same for others, the "Global
	uniqueness" criterion of RFC 1737 Section 2 may easily
	be violated. so the conventions about identity and
	uniqueness are important parts of the namespace
	definition even though, in practice, they may be better
	articulated for some namespaces than for others.
	
	"It is important to note that universal conventions are
	almost certainly impossible: there is no reason to
	assume that the conventions that apply for one namespace
	will apply to another.
	
	"These issues are a large part of what make fragment
	identifiers and queries problematic for many URN
	namespaces and create a requirement for very careful and
	namespace-sensitive definitions in the namespaces where
	they are allowed.  A badly-designed fragment identifier
	may be inconsistent with the stability and persistence
	of a putative URN if replication or any changes all all
	to the names object are allowed.  A badly-designed query
	string may require reference to information or
	resolution of objects outside the namespace, thereby
	undermining multiple key URN properties as identified in
	this document, RFC 1737, and elsewhere.  In addition if
	ether were to follow trends common in contemporary usage
	of queries and sometimes fragments in URLs, the
	requirements of Section 3 of RFC 1737, especially those
	for Human transcribability and Simple comparison, could
	easily be violated.
	
	"To the extent feasible, definitions of particular URN
	namespaces should be clear about the relationships
	between the URN, the namespace, and the underlying
	objects; about the implications of replication and
	various changes to the named resources; and, if fragment
	identifiers or queries are allowed, how they should be
	constructed and constrained to preserve identifier
	persistency and to meet the other requirements of this
	specification and RFC 1737."

That is obviously just a first cut, but maybe it will help us
understand at least some of where we disagree, what the problems
actually are, what problems can and cannot be solved (especially
in a general, rather than per-namespace, way), and how to move
forward.

best,
    john

---

From: Keith Moore <[email-masked]>
Date: Sat, 15 Jun 2013 09:58:49 -0400

On 06/15/2013 07:02 AM, Julian Reschke wrote:
> A URI can be further classified as a locator, a name, or both. The 
> term "Uniform Resource Locator" (URL) refers to the subset of URIs 
> that, in addition to identifying a resource, provide a means of 
> locating the resource by describing its primary access mechanism 
> (e.g., its network "location"). The term "Uniform Resource Name" (URN) 
> has been used historically to refer to both URIs under the "urn" 
> scheme [RFC2141], which are required to remain globally unique and 
> persistent even when the resource ceases to exist or becomes 
> unavailable, and to any other URI with the properties of a name.
>
> An individual scheme does not have to be classified as being just one 
> of "name" or "locator". Instances of URIs from any given scheme may 
> have the characteristics of names or locators or both, often depending 
> on the persistence and care in the assignment of identifiers by the 
> naming authority, rather than on any quality of the scheme. Future 

If I see a politically feasible way to fix it, I'll recommend it. Until 
then, I'm just going to recommend that this group's documents strictly 
limit their scope to URNs as defined in 2141.

/But the main thing is, let's not get bogged down in discussions of RFC 
3986. /  It's a rathole.   We should first figure out what changes are 
needed for RFC 2141 URNs, and only when we have that figured out should 
we worry about how to clearly describe that to people in light of the 
language in RFC 3986.

[ skipping a lot of discussion that I think is a distraction... ]
> I agree that it would be a bad idea. So is this about whether we allow 
> a URI using the "urn" scheme carrying a fragment identifier to be 
> called a URN?

Yes.  I started out thinking that if we had a string of the form 
urn:foo:bar#zot that we would just say the URN portion is urn:foo:bar 
and the fragment ID
is #zot, but I eventually realized that people would inevitably treat 
the entire string urn:foo:bar#zot as the URN, and that would be bad.

Now the question in my mind is, is it acceptable to use the urn: scheme 
for a URN carrying a fragment identifier if there's some assurance that 
the combination of the URN and the fragment identifier is persistent?    
Or will the existence in documents of URNs that look like 
urn:foo:bar#zot (but meeting the persistence criteria) encourage others 
to append fragment IDs that aren't persistent, to URNs that are 
persistent?     (My suspicion is that the right answer for political 
purposes is the wrong answer if we want to encourage robustness and 
interoperability.)

[skipping a bit more]

> path and query to locate the resource.

That's one way to look at it.  But in practice people generally treat 
the search engine as a resource independent of the query string, so (for 
example) if they believe that they understand the nature of the queries, 
they feel free to substitute one query string for another.   And this 
potentially has implications for the persistence properties of URNs 
containing query strings.

> A URI scheme can allow queries. In that case, not making the query 
> part of the scheme syntax would be totally confusing.
>

I've come to the same conclusion that "not making the query part of the 
scheme syntax" would be confusing, but that doesn't meant that we can't 
invent a new kind of URI, with a different prefix, that consists of a 
base URN and a query scheme or fragment identifier.

> Not *calling* it a URN might work, but still: it's part of the URI, 
> but not part of the URN? Things might work better if you mint a new 
> term for the URN part that *is* persistent. Or better yet, don't go 
> there: do not allow queries purposes that conflict with the intent of 
> URNs.

My concern is that if people see URNs that contain query strings, 
they'll assume that those query strings work just like those for http: 
URLs, and they'll start appending query strings to ordinary URNs.   
(Actually I'm much more concerned that they'll do this for fragment IDs 
than for query strings; I think the two cases are significantly 
different and the potential for fragment IDs to cause problems is much 
greater.)

> Before we go there: do we have a concrete use case that we would test 

Use cases:

1.  Joe would like to assign a URN to a resource that currently takes 
the form of an HTML document which is accessible via HTTP. That document 
contains fragment identifiers which aren't themselves maintained as 
persistent identifiers.   Joe would like to be able to use that URN as a 
persistent reference to that resource and still be able to refer to 
particular fragments of that resource, even though he (hopefully) 
realizes that the combination of the URN and fragment ID are not persistent.

2. Carol would like to assign a URN to a resource that accepts query 
strings and returns different results depending on the query, in order 
to provide a persistent reference to that resource.    She would also 
like to make it possible for URI references to that resource to contain 
query strings.   However she has no assurance that the interpretation of 
those query strings by that resource will not change over time.

3. Jack would like to assign a URN to a resource that currently takes 
the form of an HTML document which is accessible via HTTP. However, Jack 
also wants to assign persistent identifiers to the named fragments of 
that document, and is willing to ensure that the fragment IDs in that 
document have the same persistence properties as URNs do, that fragment 
IDs are never reassigned within that document, and that the meanings of 
those fragment IDs are retained should that resource be produced in any 
other format that supports fragment IDs.    (If necessary, Jack is even 
willing to register each of the combinations of base URN + fragment ID 
as separate URNs.... though whether things should work that way is a 
separate question.)

4. Alice would like to assign a URN to a resource that accepts query 
strings and returns different results depending on the query. However 
she also wants URIs that refer to that resource and contain a query 
string to also be persistent, and she's willing to provide assurances 
that the resource's interpretation of the query string will not change 
over time.

5. Ted knows of a resource that is named by a URN.   That resource 
contains named fragments.   Ted is authoring a document that references 
a particular portion of that resource and would like to provide in the 
document that he is authoring, a stable reference to a specific fragment 
of that resource.   He's unsure as to whether he can simply append the 
fragment ID to the URN, but other URIs work that way....

6. Bob knows of a resource that is named by a URN and which accepts 
query strings.   Bob has found a particular query result interesting and 
would like to provide a reference that result by somehow composing a URI 
from that URN and query string.

---

From: Keith Moore <[email-masked]>
Date: Sat, 15 Jun 2013 11:51:02 -0400

> examples and some other discussion inline below.

I also get the impression that some of us are talking about different 
things and/or working from vastly different assumptions. It appears to 
me that 3986 is a significant source of the confusion.   I might be 
wrong, but I doubt it -  because people keep talking about 3986 and 
quoting text from it as if its authors actually understood URNs and the 
implications of what they were saying for URNs, when it's clear to me 
that they did not.

Note that I haven't mentioned draft-ietf-urnbis-rfc2141bis-urn-05 (or 
any version) at all.   To me it seems premature to wordsmith that 
document (at least about these issues) until the issues regarding 
persistence of URNs in combination with fragment identifiers and query 
strings are well understood.

(I have referred to "2141bis" but was using it in a generic sense of 
"the successor to RFC 2141", not to a specific I-D.   I can see how that 
might have caused confusion.)

> 2141-style URNs to the requirements of 3986.

I seriously doubt that the latter is even possible, the misunderstanding 
of URNs in 3986 is so great.  But let's table that issue for now, figure 
out how to deal with query strings and fragment IDs in URNs without 
harming the persistence properties of URNs, and then see if it's 
possible to do that without 2141bis taking exception to 3986.

(I do think it's reasonable for 2141bis to update 3986 if there's a need 
to do that.)

> is either an attack on 3986 (and hence out of scope for the WG)

The only reason I'm "attacking" 3986 is that it seems to be a 
significant source of confusion that's making this WG have a more 
difficult time understanding the current topics of discussion.    We 
can't address the problems with URNs and fragment IDs and query strings 
unless/until we have a shared understanding of what those problems are, 
and it seems to me that 3986 has muddied the waters considerably.

But as I just said in another reply, I think we should just stop talking 
about 3986, as it has become a rathole.

> though no timeframe is given for them (and they might not

Yes, I have a similar interpretation, and agree that RESERVED does not 
imply "prohibited for all time".   But of course the reason they were 
RESERVED is that we didn't know at the time how to make them work and 
retain the persistence properties of URNs.   It's not yet clear to me 
that we know how to do that now, but at least I think I personally 
understand the issues better than I did then.

(I actually misread the ABNF in 2141 the first time I looked at it 
recently and missed the <reserved> term in the production for <trans>, 
so it appeared to me that '?' and '#' could not appear in an NSS.)

> otherwise.

And yet, people are accustomed to being able to compose URIs out of base 
URIs and fragment IDs without having to worry about whether the result 
will be valid.   They don't know that it's stupid to do that, and 
nothing that 2141bis says is likely to affect the vast majority of users 
that are currently accustomed to appending fragment IDs to URIs.

> But that doesn't imply that, for some URN types
> identified and defined.
I doubt that it's appropriate to make this a namespace-specific 
property.   That seems to have a lot of really ugly consequences, like 
requiring both URN-parsing software and ordinary users to know which URN 
namespaces permit use of fragments and query strings. Though of course 
there's nothing wrong with doing a detailed analysis of those consequences.

> Second, even after rereading 2141 and

This problem already exists for URNs even without fragment IDs or query 
strings, and I don't think I've ever seen it satisfactorily articulated.

(Actually it exists for naming in general - a name for something is 
generally too brief to including information about how specific it is.)

In my mind at least, a URN shouldn't be associated with a document or 
resource, but rather with a resource definition that explains exactly 
what is being named, and that resource definition is part of what you 
should get back when you resolve the URN... so that the client has a 
better idea of how specific that particular URN is (e.g. is it a 
particular document or a particular version of a document or a 
particular rendering of a particular version of a document, or something 
narrower or broader?)

But in practice we tend to talk about URNs being persistent across 
changes in the document when we actually mean something more subtle than 
that, much as we talk about URNs being persistent when we're really 
referring to a property of the binding between the URN and the resource 
[definition].

> axiomatic systems.

Indeed, those are very deep and long-standing questions, and I've been 
involved in many long discussions about them over the years. But for 
better or worse, it was an explicit and strong consensus of the original 
URN working group that URNs weren't just to be bound to a specific 
document independent of location or replication, that they could 
reasonably refer to resources whose content intentionally changed over 
time but was still the same in some sense.   The example of "today's 
weather" was often cited.   Perhaps it's not readily apparent from RFC 
2141, but if memory serves, other RFCs about URNs reflect that consensus.

(FWIW, I've also had it forcefully argued to me that you can't make a 
useful identifier that only refers to a specific unchanging 
representation of content independent of location or replication, that 
if such identifiers were widely used, pressures would inevitably cause 
such identifiers to be used more broadly than intended.   I don't agree 
with that view, but it's not like the argument was completely without 
merit.)

> Another key reason why I shut down the

There's lots of both history and philosophy here, certainly more than we 
can reasonably relate to this group via email.   (and my, how times have 
changed!)

If anyone else is actually interested in rehashing history or 
philosophy, there's a high probability that I'll be in Berlin and I'd be 
happy (or at least willing) to discuss those things in person.   But 
I've seen them rehashed in email list discussions enough times to 
convince me that there's very marginal utility in doing so, especially 
these days when a smartphone is the device most frequently used to read 
email, and people's attention spans for email seem to have diminished 
considerably.   (not that I'm not also guilty of that.)
> That does not follow at all.

I realized after I sent that message that I didn't state it precisely 
enough.   What I meant was that if you have a URN for some resource (not 
one that you control) and you know of a fragment ID within that 
resource, it's perfectly reasonable for you to want to create a 
reference (URI) that combines the two - but there's no assurance that 
such a reference (URI) will be persistent.   And so, for example, if 
you're used to seeing HTTP URLs constructed as base URL + fragment ID, 
and you think you can construct a reference consisting of a URN and a 
fragment ID in a similar way, what you end up producing is something 
that looks like a URN (and begins with urn:) but doesn't have the 
properties of URNs.  And that, I think, is a significant problem.

Note that if you don't care about users being able to compose specific 
URIs from a base URN + a fragment ID, and you don't care about being 
able to extract the base URN from a URI consisting of a URN and a 
fragment ID, there's no issue.   Simply assign a separate URN to each 
fragment of the document, in addition to the URN for the entire 
document, with no obvious relationship between the URNs used to name 
fragments and the URN used to name the whole document. Then when the 
URNs used to name fragments get resolved, they get resolved to URLs that 
contain fragment IDs that are appropriate for the currently accessible 
version of the document.    Similarly, a URN without a query string 
could resolve to a URL that contained a query string, so you could 
define any number of URNs for specific queries made to a resource.

It's only because we want to be able to compose URIs by adding fragment 
IDs and query strings, or decompose URIs that contain fragment IDs and 
query strings, that we have this problem of trying to figure out how to 
shoehorn fragment IDs and query strings into URNs.   So being able to 
compose those URIs is an important property, and not just for the 
resource owner.

> property of my choice of a simple, but sloppy, example).

Yes, but the _meaning_ of the URN would remain the same, and (as far as 
I can tell) that's what's important.   I don't see how that's different 
than the oft-cited URN for the current weather map.

(Interesting example, though)

To me main problem with making URNs with query strings persistent is 
that the very act of assigning a URN to the resource puts responsibility 
on the resource maintainer to not change the interpretation of query 
strings, even (I suspect) to the point of defining results for query 
strings whose results were previously-undefined.   In other words, you 
can't extend the query language.

Perhaps more to the point, I'm wary of saying that the act of assigning 
a URN to a resource constrains how that resource can evolve.   In 
particular, I don't think there should be any constraint on who can 
assign a URN to something.   Whoever assigns the URN doesn't necessarily 
control the resource.

But I'll readily admit (and these examples illustrate) that it might be 
difficult to get consensus on what URNs that contain query strings mean, 
and that this is probably more subtle than defining what it means to 
have persistent URNs that contain visible fragment IDs.

> 	fine.

This doesn't contradict what I wrote above.   The point is, people 
composing URIs out of base URIs and query strings can't be assumed to be 
the people who maintain the resources being referred to.   And those 
people generally don't know whether the resource is constructed in such 
a way as to assure persistence of the resource across the entire space 
of possible query strings.

> The question is whether it is appropriate to try

I don't want to try to ban any practice.   I think there are valid use 
cases for all of these practices.   What I want to do is figure out how 
to permit these practices (including the practice of users composing 
URIs from base URIs + query string that they supply) without degrading 
the notion that urn: means "persistent identifier" because I think 
that's a very important property of URNs.

> least in my pragmatic, observational, reality.

I don't want to have to defend myself against implied accusations that I 
was trying to "ban" something, when what I'm really trying to do is 
preserve and improve the utility of URNs.   Banning useful practices is 
not my goal.   Trying to encourage those useful practices to be done in 
ways that don't cause problems is my goal.

> 	namespaces.

Again, I really don't think this is naturally, or should be defined as, 
a property of the namespace.   To the extent that it appears to be a 
property of the namespace, it might be because some namespaces are 
associated with particular kinds of resources.   But that's a corner 
case, not how URNs are supposed to work in general.

In principle, any URN can refer to any kind of resource.  If there are 
particular namespaces that constrain themselves as to only assign URNs 
to specific kinds of resources that have specific persistence 
properties, that's their business, but we shouldn't define general URN 
behavior in the presence of fragment IDs and/or query strings in such a 
way that it only works for those corner cases.

Keith

---

From: John C Klensin <[email-masked]>
Date: Sat, 15 Jun 2013 17:52:29 -0400

--On Saturday, June 15, 2013 09:58 -0400 Keith Moore
<[email-masked]> wrote:

>...
>> Not *calling* it a URN might work, but still: it's part of
>> the URI,  but not part of the URN? Things might work better
>> if you mint a new  term for the URN part that *is*
>> persistent. Or better yet, don't go  there: do not allow
>> queries purposes that conflict with the intent of  URNs.
> 
> My concern is that if people see URNs that contain query
> strings, they'll assume that those query strings work just
> like those for http: URLs, and they'll start appending query
> strings to ordinary URNs.   

I want to address the points I see in the above because it seems
symptomatic of a number of other things.  The subsequent
sentence (below)  identifies an interesting (to me) point of
disagreement.

I think we can agree that there are ignorant and stupid people
in the world who will do stupid things, and that some of those
stupid things will be the result of someone's looking at one
case or example and making an inference from it that will lead
them astray.   While a mechanism for stamping out ignorance and
stupidity, and even ordinary bad inferences, would create an
opportunity to vastly improve the world, such a mechanism is out
of scope and almost certainly outside the competence of this WG.

The question is then what to do about those ignorant and/or
stupid people and their potential for harm.  One answer,
consistent, IMO, with some of what you are suggesting, is to
prohibit features that might be perfectly reasonable in context
simply because they might be misinterpreted and used to justify
something that is problematic.  There is an argument for that
sort of approach: in at least one cultural tradition,
prohibiting something reasonable because an ignorant person
might mistake it for something else and make a bad inference is
known as "building a wall around" something that is considered
important.  Enough of that sort of wall-building and lead to
absurd conclusions and, worse, prohibiting enough obviously
reasonable situations that educated and reasonable people decide
to ignore the rules entirely.   In protocol design, that outcome
is sometimes known as "shooting oneself in the foot" because it
leads the smart people who want to faithfully implement the
protocol to not do so while doing little to cure or alleviate
ignorance or stupidity.

Another approach is to fiddle around with names and categories.
"That creature out there may look like a duck, walk like a duck,
and sound like a duck, but you are forbidden to call it a duck."
It doesn't work either: the smart, careful, and sophisticated
people who are trying to produce good-faith implementations
don't need the terminology games and the ignorant and stupid
ones (and those who are easily confused) will get confused and
ignore those distinctions.  Worse, if someone does something
stupid or harmful, it will make absolutely no difference to the
Internet whether they call it a duck, a goose, or a URN.

In addition to the above and the stupidity and ignorance
factors, I suggest that this is just a less theoretical world
than it once was.  If someone has something that they sincerely
believe is a real customer requirement that would be satisfied
with, e.g., a fragment identifier with a persistent resource
identifier and have seen URNs, they are extremely likely to use
a fragment identifier with the URN.  If they even manage to read
out spec and all we have to say is "don't do that", we will, in
the vast majority of cases, lose.  Like it or not, in today's
Internet, perceived customer requirements will trump theoretical
models of how things are supposed to work almost every time.

I am suggesting a different plan and suggesting it because it
has reasonable odds of being successful among those who actually
might pay attention of us.  It is that we make these things
valid and do so in a way that is as simple and straightforward
as possible, with no hair-splitting.  If we have to hold our
noses to do that, we get good at nose-holding rather than
getting good at telling folks what they cannot do.  Then we
explain, with as much clarity as possible, some examples of ways
to use those identifiers that will work well, some that will be
problematic, and why the latter are a bad idea.  The "bad idea"
part of that explanation should focus more on "bad for customers
and users" than on "bad for the Internet" and should focus more
on "bad for the Internet" than on "offends IETF religious
beliefs" or even "violates the principles of RFC 1737" because,
frankly, almost no one cares.  

For those who have any respect for what we do and/or who choose
to read our documents, that will help because we will be giving
them good advice about what will help them and their customers
and they will be inclined to follow it.  For those who don't...
well, nothing will make any difference and we just need to
accept that as part of our reality.

> (Actually I'm much more concerned
> that they'll do this for fragment IDs than for query strings;
> I think the two cases are significantly different and the
> potential for fragment IDs to cause problems is much greater.)

Interesting.  I would have said exactly the opposite.  I think
it is possible to write advice about fragment identifiers that
makes it clear that:

	* The URN without the fragment identifier must be
	persistent, stable, and meet the criteria of 2141 and
	1737.
	
	* The proposed fragment identifier must be structured as
	a pointer into the resource identified by the rest of
	the URN, not something that lies outside that resource
	or modifies it in any way.
	
	* The semantics of the proposed fragment identifier must
	be natural to the resource type and must neither change
	the semantics of the resource nor require that it have
	characteristics that are not natural to the resource.
	URN:Rabbit:...#Chapter3 must fail, not identify the
	right rear leg.
	
	* The semantics of the proposed fragment identifier must
	be stable and persistent until any transformation of the
	resource that does not invalidate the URN itself.  (See
	my earlier example about the difference between book
	chapters and page references if transformation of book
	content to different media are considered to not change
	the resource.)

	* The fragment identifier, by the nature of fragment
	identifiers, should not require a process for
	interpreting it that is not natural, internal, and
	natural to the resource.

Now, that may not be the right set of rules, but the point is
that rules can be written because the natural use of a fragment
identifier is to select a part of a resource by some marker or
property that is intrinsic to that resource.  I don't know how
to write a similar set of rules for queries because they often
(perhaps usually) imply some process or processor that is
external to the resource.  As soon as one says something like
"search engine" in conjunction with a query, a high standard for
persistence and predictability will require that one identify
the search mechanism and its properties and that those be stable
too.   The final proposed rule above could be restated as
"fragment identifiers must not be search specifications in
disguise".

So, personally, I'm a lot more afraid of queries than I am of
fragments.  To go a bit further, I don't think there is any
point second-guessing 3986 and its predecessors at this point,
but I think we would have been a lot better off with a query
syntax that embedded a URI rather than a query that is part of a
URI.  The observation that may URIs that contain queries today
contain embedded URIs as part of the query only reinforces that
belief.

   john

---

From: Keith Moore <[email-masked]>
Date: Sun, 16 Jun 2013 02:13:30 -0400

On 06/15/2013 05:52 PM, John C Klensin wrote:
> [...]

I'm not worried about stupid people doing stupid things.  That is a 
given, and there's nothing we can do about that.  I'm worried about 
reasonably intelligent people doing things that look like obviously the 
right thing, which happen to be the wrong thing and to cause harm.   Now 
obviously the world would be a better place if everybody thoroughly read 
the relevant specifications before composing URIs or writing code that 
deals  with URIs, and all of the specifications were complete and well 
written, and there were no contradictions in the specifications.   But 
that's not the real world, or even close to it.

So I believe that technical standards work better in practice if they're 
designed so that either (a) doing the obvious thing turns out to be the 
right thing (happy accident) or (b) doing the obvious thing fails in 
such a way as to force the implementor to figure out what the right 
thing is.  It seems to me that protocol standards that have been widely 
successful and have a low barrier to implementation, are usually ones 
where the obvious thing is the right thing a great deal of the time.   
(and maybe that there's a low investment to figuring out what the 
"obvious" and right thing is in the context of that protocol.)

Now it happens that the very nature of URNs (even without considering 
fragments or query strings) makes using them so subtle that the obvious 
thing (at least to people not intimately familiar with library science 
or similar disciplines that have had to deal with this sort of thing) is 
often the wrong thing.   For instance, putting any kind of 
human-meaningful information inside an NSS is nearly always the wrong 
thing, because doing that quite often creates incentives (sooner or 
later) to make the name non-persistent.    Adding fragments or query 
strings just makes this problem worse and introduces more cases where 
the obvious thing is the wrong thing.

(Though it pains me to admit it, this is one thing that DOIs got right - 
the DOI equivalent of an NSS is deliberately meaningless to humans.   
The advantage that URNs have, is that by not restricting the contents of 
the NSS it becomes possible to grandfather any kind of persistent name 
into URN-space.   But that advantage comes with a curse, which is that 
other parties then want to change URNs in order to suit their own 
requirements.)

But back to URNs and fragments and query strings.   If we can define 
things in such a way that the obvious thing works well often enough, 
that's wonderful.   (How often is "often enough"?  I'm not sure.) If we 
can't do that, we should consider making the obvious thing fail and 
providing a non-obvious way to do things that works well. And in any 
context where other URIs are permitted, the "obvious" way of composing 
URIs out of URNs and fragments or query strings is likely to be the way 
it works for HTTP.

I should probably say at this point that I don't have a firm opinion 
about which is the best thing to do.   Right now I'm trying to 
understand the different options in the solution space.   There are 
clearly more options than either (a) just say NO, or (b) recommend a 
syntax just like the one used for other URIs and hope that people don't 
abuse it.

(I started writing a internet-draft to recommend one of those options, 
and in the process of doing so, decided that I needed to do more case 
analysis before recommending a solution.   That's the benefit of looking 
at things in detail rather than trying to make decisions based on emails 
that need to be more brief.)

> something that is problematic.
I don't think I suggested anything nearly that strongly, and I don't 
agree with that characterization.   Again, my goal is to find a way to 
satisfy various reasonable use cases (which I attempted to list a 
message or three ago) and to have URNs work reliably for those cases.   
But that's not the same thing at all as either prohibiting features, or 
recommending "obvious" ways of doing things that will likely cause problems.

> There is an argument for that
> important.  [...]
> [...]

It appears that your idea of what I imagine to be the possible 
solution-space is very different than mine.    Forgive me if I don't 
want to argue about your analysis of your idea of what I'm thinking, 
especially since what I'm thinking is still a moving target.  I think it 
makes more sense to argue about concrete proposals.  At least then we 
are more likely to be arguing about the same thing.

> the vast majority of cases, lose.

It appears that we have a test case for that, as RFC 2141 essentially 
says "don't do that (yet)".   And perhaps that is the reason that URNs 
haven't so far been as successful as many of us had hoped.    Maybe 
figuring out how to add fragment IDs and query strings will make URNs 
more broadly applicable and thus more successful.    I'm all for that, 
as long as we can do it without destroying the intended utility of URNs 
in the process.   (Though I expect the real gap isn't lack of support 
for query strings and fragment IDs, but rather lack of a general purpose 
and widely-usable URN resolution solution.)

> getting good at telling folks what they cannot do.

Assuming I understand what you mean by hair-splitting, I agree that it 
generally causes problems.   Let's try to avoid introducing subtlety 
where it isn't needed.   On the other hand, we may also need to 
recognize where there is inherent subtlety that needs to be reflected in 
the protocol.

I'm not worried about how things smell.  I'm worried about how well they 
work in practice.    Things can smell bad and still work well.

I also think that people may have trouble following technical arguments 
about vague generalities, and that one person's idea about what is 
hair-splitting or smelly or subtle might be different than the next 
person's.   (Also, if you're trying to dismiss something that I see as a 
valid technical concern, it would probably help to actually name it 
precisely rather than refer to it in vague terms.)

> Then we

Actually I think there's an important middle group here - people who 
will read our documents if necessary to get things working.   If things 
appear to work when people do the obvious thing, but they actually fail 
in subtle ways (or they work when initially tried but fail at some 
arbitrary time in the future), it will be hard to get people to 
recognize their implementation errors.  But if people do the obvious 
thing, try it, and it fails in such a way as to give them a clear error 
indication, they'll be more likely to refer to the specification.

Something that I've observed for a long time is that plain text 
protocols - like RFC 822, SMTP, HTML, HTTP, XML, json, and so forth, and 
also URIs - have a huge advantage in deployability because people don't 
think that they have to actually read the specifications in order to 
implement them.   But for the same reason, they tend to break whenever 
any subtlety is required in how they're used.

Again, I'm still trying to understand the solution space, so I don't 
have a concrete proposal yet.   But it seems to me that this could be 
part of an answer.

> property that is intrinsic to that resource.

And the funny thing about that assumption is that the "natural" use of a 
fragment identifier is exactly one that is unlikely to be persistent 
except for very specific kinds of resources.   For instance, "chapter3" 
works as a persistent fragment ID for a book as long as the publisher 
never revises the book in such a way as to renumber chapters.   That 
might actually turn out to be likely in the case of paper books named by 
ISBNs, but perhaps only because ISBNs get used as inventory control 
numbers.   So if a publisher revises a book significantly enough that 
the chapter numbers change, it seems likely that the book will get a new 
ISBN anyway, to distinguish old stock from new stock.   But that's a 
corner case. A rule to not renumber chapters is not a constraint that 
we'd want to impose on URNs in general, and maybe not even on electronic 
resources named by ISBN.

So "chapter3" might be a good example of an inappropriate fragment ID, 
or a case where doing the obvious thing turns out to be the wrong thing, 
or one that only works well in a corner case that needs to be 
well-understood.

Or maybe we just need a way to compose URNs with fragment IDs such that 
the resulting URI isn't seen as a URN.    For the users who are 
demanding that URNs support fragment IDs, how important is it to them 
that those composed URIs also be URNs and thus persistent?

>   I don't know how
> too.

Perhaps.   I also suspect that this is a "hard problem".   It's easy to 
get close to defining what persistence means for URNs containing query 
strings, but hard to nail down exactly what is required.

But something I don't know is - for the use cases that people envision, 
how important is it that the composition of a URN and a query string be 
persistent?   Will a non-persistent URI that combines a URN and a query 
string suffice for most cases?

> The final proposed rule above could be restated as

Indeed, and the problem is much worse than that.    The conventions of 
the web along with its user interface paradigm really encourages a view 
of the world where, outside of some very specific contexts, GET is the 
only thing you can expect to be able do with a URI.   So you end up 
trying to coerce almost everything that you need to do into a GET.   And 
that's hugely limiting, not only for URNs but for URIs in general.

Keith

---

From: John C Klensin <[email-masked]>
Date: Sun, 16 Jun 2013 12:25:29 -0400

(note aggressively trimmed to concentrate on issues I haven't
already said more than enough about)

--On Saturday, June 15, 2013 11:51 -0400 Keith Moore
<[email-masked]> wrote:

>> Yes, that is a bit of an editorial challenge.  I don't see it
>> as a serious substantive problem because I don't think anyone
>> is claiming that draft-ietf-urnbis-rfc2141bis-urn should be

> I seriously doubt that the latter is even possible, the
> misunderstanding of URNs in 3986 is so great.  But let's table
> that issue for now, figure out how to deal with query strings
> and fragment IDs in URNs without harming the persistence
> properties of URNs, and then see if it's possible to do that
> without 2141bis taking exception to 3986.

Wfm but see Julian's note.  While I dislike several things about
3986, I don't see it as nearly as URN-hostile or, for that
matter, different from its predecessors, as you obviously do.
Nonetheless, I think that getting this right first and then
seeing what, if necessary has to be done to align things with
3968 (or vice versa) is a reasonable strategy.

That said, I think we need a 2174bis draft to use as the basis
for discussion.  Both in the the absence of a posted alternative
and because I belief that most of what is needed is there, I
would prefer to treat draft-ietf-urnbis-rfc2141bis-urn as that
draft.

>...
> Yes, I have a similar interpretation, and agree that RESERVED
> does not imply "prohibited for all time".   But of course the
> reason they were RESERVED is that we didn't know at the time
> how to make them work and retain the persistence properties of
> URNs.   It's not yet clear to me that we know how to do that
> now, but at least I think I personally understand the issues
> better than I did then.
> 
> (I actually misread the ABNF in 2141 the first time I looked
> at it recently and missed the <reserved> term in the
> production for <trans>, so it appeared to me that '?' and '#'
> could not appear in an NSS.)

Again, if 2141 had said "prohibited" or the equivalent, we would
be having a discussion that would be at least somewhat
different.  

> And yet, people are accustomed to being able to compose URIs
> out of base URIs and fragment IDs without having to worry
> about whether the result will be valid.   They don't know that
> it's stupid to do that, and nothing that 2141bis says is
> likely to affect the vast majority of users that are currently
> accustomed to appending fragment IDs to URIs.

Ok.  That, to me, is an argument for regularizing them,
explaining why they are a problem in a URN context unless
principles that we identify are followed, and then hoping for
minimal damage.

>...
> I doubt that it's appropriate to make this a
> namespace-specific property.   That seems to have a lot of
> really ugly consequences, like requiring both URN-parsing
> software and ordinary users to know which URN namespaces
> permit use of fragments and query strings. Though of course

It would be, IMO, a terrible mistake to try to trap fragment
identifiers in parsing.  The issue should be whether they can be
dealt with appropriately when the URNs are derefernced.  The
problem there is ultimately no different from the URL case: a
meaningless fragment identifier associated with a given URI
won't work and will presumably fail (see below) whether the URI
is

	http://www.iana.org/assignments/urn-serviceid-labels/urn
	-serviceid-labels.xml#sos-sub-services

or 
  URN:birds:...#horse

Now, that suggestion _does_ raise some issues with the Fragment
definition in 3986 Section 3.5.  

First, 3986 ties fragment "format and resolution" to media type,
which is, in the general case, meaningless for URN-type
resources (independent of namespace because the namespace
essentially implies whether or not a media type will even be
present.  

Second, it says "Fragment identifier semantics are independent
of the URI scheme and thus cannot be redefined by scheme
specifications".  That is one of those "predict and constrain
the future in very broad terms" that gets us in trouble, sooner
or later, almost every time we try to do it in an IETF
specification.   It never should have been permitted in an
Internet Standard.  But I didn't catch it and object (much less
appeal after objecting), you didn't, either, and I don't believe
anyone else did either.  Too bad, but it can always be updated
and amended either for URNs or more generally.   

Third, parts of the "special role" discussion (fifth paragraph
by my count) would get one laughed out of most serious
discussions with skilled informational scientists.  

Fourth, while it may not be our problem, the notion of binding
fragment identifiers to media types would, in any rational
world, require that the media type registry indicate whether
fragment identifiers are permitted and what their syntax and
semantics are. RFC 3986 doesn't appear to specify that registry.
RFC 6838, Section 4.11 says "can specify how applications should
interpret fragment identifiers..." and "Media types are
encouraged to adopt fragment identifier schemes..." but stops
far short of a requirement.  That provision was included in the
medial type registration spec for the first time in 6838, i.e.,
this year; prior that there wasn't even a slot in the template
for the information and I note with amusement that no one has
bothered to update even the text/html type registration at
http://www.iana.org/assignments/media-types/text/html with
fragment information.

Finally, while the text includes a discussion that includes "not
found" as an example, some media types are clearly interpreted
in some places as "if the fragment identifier doesn't map to the
resource, just return the resource as if no fragment was
specified" (the first example above will, if dereferenced,
return the relevant web page as if no fragment had been
specified while a fragment that actually matches that page, e.g.,

	http://www.iana.org/assignments/urn-serviceid-labels/urn
	-serviceid-labels.xml#urn-serviceid-labels-2
	
returns the location the user presumably intended.

So, yes, if we go down this path, some updating of 3986 is
likely to be needed.  But, as you (I think) suggested earlier,
let's figure out what we need to do and then worry about what
else needs to be patched up to permit it.  That seems to me to
be much more of a way forward than spending energy on denouncing
3986.

What is more important than either the analysis of what users
typing things in might do or of the issues with 3986 and common
practice is that the very nature of a fragment identifier
requires that a document author or URN-assigner do something
that makes it interpretable.  Meaningless fragment identifiers
just fail to do anything useful but may not cause much harm
beyond confusing users about when happened when they specify
what 3986 calls a "sub-resource" and get the resource instead. 

So, ignoring 3986 and reprising somewhat, I am suggesting that
we 

(1) Allow fragment identifiers for URNs with the "#" delimiter
(because switching delimiters to clearly denote these as
"something else from what one might infer from 3986" would cause
far more problems and incompatibility.

(2) Without restricting syntax, make the presence or absence of
fragment identifiers a namespace issue (not a parsing one).
2141-compliant UNR processors, and processors associated with
older namespaces, will presumably get some flavor of "syntax
error - reserved character" errors or some other failure
condition.  I don't see that as a problem.

(3) _Require_ that processors for namespaces that do not allow
fragment identifiers respond to their presence with an error
condition, whether that is a slightly-bogus "syntax error" or
"character/fragments not allowed" report (for backward
compatibility) or some flavor of "not allowed" or "not found".

(4) _Require_ that processors for namespaces that perform
dereferencing that do allow fragment identifiers (note that, if
2141 were followed, all of these would be new or revised) reject
the things if they don't match identified fragment elements in
the relevant resource.  I expect this requirement will be
ignored in namespaces that are already using fragments, but at
least we will have a clear line about compliance.

(5) As discussed earlier, warn about some types of fragment
identifiers are inconsistent with persistence and prohibit them
in URNs for that reason.  I expect this may be ignored as well,
but it will give us a fairly clear line about compliance and at
least won't make anything worse or make the IETF look stupid.

(6) Make sure that, from a syntax standpoint, fragment
identifiers are clearly part of the URN and including in the
matching rules.   None of 

    URN:feline:...lion
    URN:feline:...lion#African
    URN:feline:...lion#Asian

Match any of the others.  That means that users who go around
attaching fragment identifiers to random URNs that are not
dereferenced (either in their contexts or generally) are pretty
likely to find themselves in unnecessary "no match" situation.
I believe that is harmless at best and probably A Good Thing.
Note that this rule is not backward-incompatible because the
second and third forms are now prohibited by 2141 so can't match
anything.

I am agnostic about whether different media types that are used
within (or dereferenced from) the same namespace should be able
to impose their own restrictions on fragments as long as they
don't broaden the rules above.  Clearly exactly what is
permitted to be useful in a fragment (uing the book example
again, consider "#Chapter3" versus "Chapter-III") is
resource-dependent, not media type dependent.   I'd prefer to
avoid the complexity of an additional layer of rules so will ask
for use cases, but I ultimately don't care very much.

>...
> In my mind at least, a URN shouldn't be associated with a
> document or resource, but rather with a resource definition
> that explains exactly what is being named, and that resource
> definition is part of what you should get back when you
> resolve the URN... so that the client has a better idea of how
> specific that particular URN is (e.g. is it a particular
> document or a particular version of a document or a particular
> rendering of a particular version of a document, or something
> narrower or broader?)
> 
> But in practice we tend to talk about URNs being persistent
> across changes in the document when we actually mean something
> more subtle than that, much as we talk about URNs being
> persistent when we're really referring to a property of the
> binding between the URN and the resource [definition].

To me, the distinction you are trying to make has been a problem
with the definition of URNs in the beginning and is part of what
is wrong with 1737, not just 2141.  A little hard to fix today,
but see the end of this note.
 
> Indeed, those are very deep and long-standing questions, and
> I've been involved in many long discussions about them over
> the years. But for better or worse, it was an explicit and
> strong consensus of the original URN working group that URNs
> weren't just to be bound to a specific document independent of
> location or replication, that they could reasonably refer to
> resources whose content intentionally changed over time but
> was still the same in some sense.   The example of "today's
> weather" was often cited.   Perhaps it's not readily apparent
> from RFC 2141, but if memory serves, other RFCs about URNs
> reflect that consensus.

Yep.  See above and below.

>...
> There's lots of both history and philosophy here, certainly
> more than we can reasonably relate to this group via email.
> (and my, how times have changed!)

Yep.  See below.
 
> philosophy, there's a high probability that I'll be in Berlin
> and I'd be happy (or at least willing) to discuss those things
> in person.   But I've seen them rehashed in email list
> discussions enough times to convince me that there's very
> marginal utility in doing so, especially these days when a
> smartphone is the device most frequently used to read email,
> and people's attention spans for email seem to have diminished

Keith, I think it would be a real contribution for some group of
people to put together a comprehensive description of the
history and philosophical debates and considerations that got us
to where we are today.  I'd hope the Independent Submission
Editor would be receptive to such a document; if not, I can
identify at least a few other places that would be.  But, even
while I've been rehashing some of those discussions because your
remarks seemed to make it necessary, I'd like to separate them
as much as possible from the question of what this WG needs to
do to  more forward in a way that is consistent with today's
realities.

> I realized after I sent that message that I didn't state it
> precisely enough.   What I meant was that if you have a URN
> for some resource (not one that you control) and you know of a
> fragment ID within that resource, it's perfectly reasonable
> for you to want to create a reference (URI) that combines the
> two - but there's no assurance that such a reference (URI)
> will be persistent.   And so, for example, if you're used to
> seeing HTTP URLs constructed as base URL + fragment ID, and
> you think you can construct a reference consisting of a URN
> and a fragment ID in a similar way, what you end up producing
> is something that looks like a URN (and begins with urn:) but
> doesn't have the properties of URNs.  And that, I think, is a
> significant problem.

But I think it is inherent is treating URNs as just another type
of URIs.

> Note that if you don't care about users being able to compose
> specific URIs from a base URN + a fragment ID, and you don't
> care about being able to extract the base URN from a URI
> consisting of a URN and a fragment ID, there's no issue.
> Simply assign a separate URN to each fragment of the document,
> in addition to the URN for the entire document, with no
> obvious relationship between the URNs used to name fragments
> and the URN used to name the whole document. Then when the
> URNs used to name fragments get resolved, they get resolved to
> URLs that contain fragment IDs that are appropriate for the
> currently accessible version of the document.    Similarly, a
> URN without a query string could resolve to a URL that
> contained a query string, so you could define any number of
> URNs for specific queries made to a resource.

And that is the "URNs only point to atomic objects" approach,
which I mentioned last week and suggested long ago, in a
slightly different guise.

> It's only because we want to be able to compose URIs by adding
> fragment IDs and query strings, or decompose URIs that contain
> fragment IDs and query strings, that we have this problem of
> trying to figure out how to shoehorn fragment IDs and query
> strings into URNs.   So being able to compose those URIs is an
> important property, and not just for the resource owner.

Ok.  See below.

>...
> To me main problem with making URNs with query strings
> persistent is that the very act of assigning a URN to the
> resource puts responsibility on the resource maintainer to not
> change the interpretation of query strings, even (I suspect)
> to the point of defining results for query strings whose
> results were previously-undefined.   In other words, you can't
> extend the query language.
> 
> Perhaps more to the point, I'm wary of saying that the act of
> assigning a URN to a resource constrains how that resource can
> evolve.   In particular, I don't think there should be any
> constraint on who can assign a URN to something.   Whoever
> assigns the URN doesn't necessarily control the resource.
> 
> But I'll readily admit (and these examples illustrate) that it
> might be difficult to get consensus on what URNs that contain
> query strings mean, and that this is probably more subtle than
> defining what it means to have persistent URNs that contain
> visible fragment IDs.

Right.  And that was what got me to the point I'm at.

> I don't want to try to ban any practice.   I think there are
> valid use cases for all of these practices.   What I want to
> do is figure out how to permit these practices (including the
> practice of users composing URIs from base URIs + query string
> that they supply) without degrading the notion that urn: means
> "persistent identifier" because I think that's a very
> important property of URNs.

Sure.  But a different way to look at your comments above is
simply to say that, as soon as one takes a URN and embeds it as
part of a composed URI, it becomes impossible to make global
statements about the stability and persistency of the aggregate
result.  I think that is inevitable even in theory; it is
certainly inevitable if one considers all of the sloppy things
that we know happen and will happen and that we can't prevent.
I suppose that, by making enough rules and restrictions, we
could figure out how to say that URNs that are fully conformant
with IETF specifications and intent remain stable when embedded
in URIs that are fully conformant with IETF specifications and
intent.  But, in the absence of ability to test URNs and URIs
for conformance at that level without dereferencing anything, I
don't think such a statement would be useful even in theory...
and that it would be useless in practice, even if true.

>...
> In principle, any URN can refer to any kind of resource.  If
> there are particular namespaces that constrain themselves as
> to only assign URNs to specific kinds of resources that have
> specific persistence properties, that's their business, but we
> shouldn't define general URN behavior in the presence of
> fragment IDs and/or query strings in such a way that it only
> works for those corner cases.

Given the above, where we disagree is only about what is a
corner case and what is, or is going to be, common practice.  If
we identify everything we don't want to see in URNs as a corner
case and dismiss it, then what "URN:..." means will be
determined in the marketplace with no regard to your (or my or
the IETF's) beliefs about what constitutes reasonable and
appropriate uses.  If we are going to stick with URNs as a
concept, I'd rather let go of some of that, admit that some URNs
are going to be a lot more persistent and well-behaved than
others.  I'd like to see us try to tie that difference to
namespaces --some of which will certainly be more persistent and
well-behaved than others no matter what we do-- than to leave
the users of URNs to guess on a per-URN basis.

I do think we have fundamental problems with where we are and
where we are headed.  One possible conclusion from both your
remarks and concerns and mine is that trying to establish a
single persistent identifier model for arbitrary resources may
have exceeded the IETF's knowledge and competence if, indeed, it
is possible.  If that is even partially true, then trying to
define them as an element of a syntax-sharing "uniform resource
identifier" concept was probably a worse idea whether those
identifiers are the URIs described in 3986 and its predecessors
or something else.  It would have therefore been much easier
(and more plausible) to define our persistent resource
references separately from persistent resource-class references
and to do so in a way that would be less likely to lead the
naive but well intentioned (much less those who don't think or
don't care) into situations that invite abuse and undermining
the concept.   But, even if that very pessimistic analysis of
the problem is the right one, the question today remains how we
move forward to salvage the best of a bad situation, accepting
the actually practices in the field and the potential for
competing standards (or at least IETF standards that are ignored
in practice) as part of the reality we have to consider.

I've made my proposal as to how to thread that needle; I look
forward to yours.

I've found this discussion very educational, thought-provoking,
and illuminating.  It has certainly clarified my thinking.  I
suspect from their silence that most of the rest of the WG's
participants are less enthused or perhaps just waiting for your
promised specific proposal.  I don't think that the two-way
conversation will help much more until others step in or your
I-D is posted.  So I'm going to go back to lurking and hoping
for the best until one of those things happens.

best,
   john

---

From: John C Klensin <[email-masked]>
Date: Sun, 16 Jun 2013 16:27:14 -0400

--On Saturday, June 15, 2013 13:02 +0200 Julian Reschke
<[email-masked]> wrote:

> A URI can be further classified as a locator, a name, or both.
> The term "Uniform Resource Locator" (URL) refers to the subset
> of URIs that, in addition to identifying a resource, provide a
> means of locating the resource by describing its primary
> access mechanism (e.g., its network "location"). The term
> "Uniform Resource Name" (URN) has been used historically to
> refer to both URIs under the "urn" scheme [RFC2141], which are
> required to remain globally unique and persistent even when
> the resource ceases to exist or becomes unavailable, and to
> any other URI with the properties of a name.
> 
> An individual scheme does not have to be classified as being
> just one of "name" or "locator". Instances of URIs from any
> given scheme may have the characteristics of names or locators
> or both, often depending on the persistence and care in the
> assignment of identifiers by the naming authority, rather than
> on any quality of the scheme. Future specifications and
> related documentation should use the general term "URI" rather
> than the more restrictive terms "URL" and "URN" [RFC3305]."
> 
> If you really really believe there's something to fix over
> here then please make a concrete proposal.
>...
> Actually, it recommends to avoid the term. All it says about
> is what it was used for historically.

Julian,

While I don't agree with what I think is Keith's perception that
3986 is the root of many of the troubles we are now facing (or
at least I don't see as much trouble there), a recommendation
that the term "URN" be avoided seems to contradict the charter
for this WG (https://datatracker.ietf.org/wg/urnbis/charter/).
That document seems pretty clear that URNs are something
specific and at least rooted in 2141.  

As Keith and I have said to each other in our rather long
thread, I think it is better than the WG figure out what we want
to do, reflect it in [some] 2141bis and then figure out what, if
anything, needs to be adjusted in 3986 to get things back into
alignment.

However, it seems to me that the material quoted above will need
to be adjusted to take the stance that calling things "uniform
resource names" that are not part of the URN scheme is just an
error and source of confusion rather then deprecating the usage
entirely.  The seems necessary in order to make this WG and any
possible output from it meaningful.   In retrospect, that change
should have been prerequisite to even chartering the WG, but it
is rather late for that now.

I think that one of the places that Keith's and my reasoning
intersect is that, syntax aside, URNs are a rather specific type
of beast. Because of the persistence concept and maybe ideas of
what "resources" are about, that beast is different from, rather
simply a subset of, generic URIs.  Perhaps the WG can find a
path that makes that not the case and makes URNs just a URI
scheme.  I don't see that possibility right now without in the
process, making IETF-defined URNs a lot less useful, but I'm
open to the idea.  Unless it can be done, I think 3986 will need
some adjusting (even more general than the above) to note that
exception.

Again, I'll defer a more specific proposal until we have 2141bis
under control, but I thought I should at least respond to your
challenge by telling you where I think I'm headed.

I discussed the URN fragment issues in my note to Keith earlier
today.

best,
   john

---

From: Keith Moore <[email-masked]>
Date: Sun, 16 Jun 2013 16:31:51 -0400

On 06/16/2013 04:27 PM, John C Klensin wrote:
> simply a subset of, generic URIs.
I certainly concur.

Keith

---

From: SM <[email-masked]>
Date: Sun, 16 Jun 2013 17:48:20 -0700

>for the best until one of those things happens.

I think I might not have understood some parts of what Keith has been 
trying to explained.    I would like to go over the messages which 
John and Keith have exchanged and some of the old messages to this 
mailing list.  It's a significant amount of material for me.  I would 
have wished that the matter was brought up earlier but we cannot 
change the past. :-)

Regards,
-sm

---

From: Keith Moore <[email-masked]>
Date: Mon, 17 Jun 2013 00:43:49 -0400

[more aggressive trimming to address a relatively narrow point]

> essentially implies whether or not a media type will even be

I don't think that's true, or maybe I don't understand what you're 
claiming.   URN namespaces certainly aren't inherently specific to media 
types.   Even if some legacy namespaces were developed in the context of 
physical media, there seems to be a growing need for identifiers that 
refer to content independent of media type (say for both physical media 
and digital content).

And though perhaps it doesn't state it clearly, I think that 3986 is 
essentially correct that fragments are artifacts of specific media types 
(i.e. content-types).   Many content-types don't have the notion of 
fragments, and among those that do have some concept like that, 
different media types treat them differently.   HTML fragments are 
technically regions of the document, and they can be nested. I seem to 
recall that other media types define the ability to reference a 
particular point in a document (like a page, or a line, or the beginning 
of a chapter or paragraph) but not so much a region.   And different 
content-types may impose different constraints on how to name fragments.

So there's a tension that results whenever you want to refer to fragment 
IDs for any resource that might exist in multiple formats - how do you 
keep the fragment ID meaning the same thing across multiple 
representations of a resource?   This problem exists even in HTTP in the 
presence of content negotiation, but it also exists in the context of 
URNs that can resolve to multiple representations in different 
content-types.

At least, that's what comes to mind when I read those sections of 3986.

> and amended either for URNs or more generally.

I think they just wanted to have a general URI parser that could 
separate fragments and query strings from the remainder of the URI.   
And that idea mostly works except for URNs, because the urn: prefix was 
supposed to be an indicator of persistence, and fragments tend to break 
persistence unless there's a lot of discipline in how the resource is 
maintained.    So when I read this, I just think that the 3986 authors 
didn't really grasp the implications of URNs, or perhaps (based on 
recent rereading of some old email) simply didn't want to believe that 
such things should exist.

> semantics are.
Well, we generally do want media type registrations to reference 
published specifications.    But we don't try to generalize aspects of 
media types much beyond that; we just trust that when content is 
presented that there will have to be some media type specific code that 
knows how to present them.   The fragment ID is just a parameter to be 
passed to that code.   Offhand I don't see how significant benefit would 
be derived from trying to extract that information from the published 
specification of each media type and putting it into the media type 
registry.    This is the sort of content-specific detail that I don't 
want to have to know about when trying to deal with a URI at a general 
level.    I want it to be isolated to the routines that actually do 
interpretation and/or presentation of that content.

Now, granted, if correct interpretation of a URI with a fragment ID 
requires selecting the "right" version of the content - the one for 
which that fragment ID is meaningful - that's a problem.   But I don't 
think that problem can be fixed by having the URI lookup or URN 
resolution routines consult a registry.   I think it just indicates that 
the notion of fragments in the web architecture just wasn't well 
thought-out, and now we're stuck with it.   (On the other hand, there 
are certainly aspects of the URN architecture that aren't well 
thought-out, and we'll be lucky if we can fix them.   So I don't feel 
like throwing rocks over this one.)

> what 3986 calls a "sub-resource" and get the resource instead.
Mumble.  If at all possible I'd like to impose the burden for ensuring 
URN persistence on the party that assigns the URN and maintains the 
resolution data for that URN, rather than the party that maintains the 
resources that the URN resolves to.   I think that's an important 
separation of function.

Maybe I'm sensitive to this because I've experienced too many times in 
which a DNS RRset was out of sync with a host's configuration. That 
experience has made me very aware that what a resource is/does, and what 
associated with the name of a resource, are two distinct things.   It's 
important to (a) not get the two confused and (b) architect things so 
that the consequences of things being out of sync are minimized.   As 
applied to URNs I think it means this: Just because someone can 
associate a URN with a resource, doesn't mean that that someone is in a 
position to impose constraints on that resource and/or how it is 
maintained.   Instead, the person who creates that URN and assigns a 
meaning to it, incurs an obligation to keep that meaning consistent over 
time (despite changes to the resource), either by adjusting the metadata 
associated with the URN and/or deleting it.

But one implication of that might be that fragment names in URNs (at 
least those for which the resulting composed URI are expected to be 
persistent) need to be registered as part of that URN's associated 
metadata, and included in URN resolution results, so that such fragment 
IDs from such persistent URIs can be mapped to the current fragment 
names in the actual resource.   (That might also address some of the 
issues with multiple resource representations.)

> far more problems and incompatibility.

See my forthcoming list of proposals for how to deal with fragments, 
queries, etc. in URN syntax.   There are several concerns that need to 
be balanced.   Some of those proposals do use "#" and "?" as they are 
used in current URIs, and some don't.  I'm not sure at the moment which 
ones I prefer, but I think the choices are interesting - especially the 
possibility of getting away from the notion (enforced by current syntax 
conventions) that the only things you can do to modify a URI (or how the 
indicated resource is presented) are send it a query string or reference 
a fragment of it.

> condition.  I don't see that as a problem.

I still don't see why this is at all a namespace issue.... other than 
possibly that some specific parties associated with namespaces are 
insisting on it.  Architecturally I don't think we want different 
interpretations of fragments for different namespaces. If they're valid 
for one namespace, they're valid for all of them. But this of course 
reflects my assumption that it's reasonable and desirable for ordinary 
users to compose references to specific fragments (bookmarks if you 
will) of things that are referenced by a URN, independently of what the 
URN maintainer does.   We just don't want to confuse the resulting 
identifiers with persistent identifiers.

> compatibility) or some flavor of "not allowed" or "not found".

Well, that would have fragments in the context of URNs mean something 
very different than fragments appearing in other URIs - and I doubt that 
that's desirable.   For other URIs, interpretation of fragments is 
entirely specific to content-type.    Even if somehow a fragment ID were 
used as input to URN resolution, I think it would be "get the metadata 
for the base URN, see what it says about mapping persistent fragment IDs 
to current fragment IDs, and then produce a URI that can be used to 
access the result".   But really I think that it's just simpler to treat 
fragments as being entirely about presentation/interpretation of the 
resource.

And I still think it's reasonable for users to want to refer to 
fragments of things named by URNs, even if the resulting URN+fragment 
URIs aren't persistent.

> least won't make anything worse or make the IETF look stupid.

I am doubtful that it's reasonable to define exactly what types of 
fragments are, or aren't, consistent with persistence.   Any kind of 
fragment ID can be persistent if the resource never changes (or the URN 
that points to the resource is deprecated when it does) and only exists 
in one representation.   And any meaningful section of a resource is 
something that might need to be refactored over time.

> second and third forms are now prohibited by 2141 so can't match

I actually believe it is a Bad Thing.   I would rather say that for 
resolution purposes the fragment ID and/or query string are first 
stripped (or ignored) and that the fragment ID or query string are to be 
interpreted in the context of the resulting resource.

I'm not so concerned (though others are) about whether the fragment ID 
is "part of" the URN, because no matter how you define it, you end up 
having to explain in more detail what is actually needed to make it 
work.   Whichever explanation is simpler and easier to interpret 
correctly is ok with me.

> but see the end of this note.
Though it might be "a little hard" to fix, I think it's worth a try.

> and that it would be useless in practice, even if true.

But that's true for URNs in general even without fragments or queries.   
We tried to define them in such a way as to encourage them to be 
persistent, but we have no way of testing that - certainly not in 
advance and perhaps not even over time.     It's just that it's much 
more difficult to ensure persistence when you need to not only maintain 
persistence of a single name for a resource, but also persistence of 
every possible variant of that name that a user can concoct.

> the users of URNs to guess on a per-URN basis.

I think that users might well develop a sense of which namespaces are 
more persistent than others, and in particular they might find that URNs 
with certain NSIs probably just shouldn't be used.   But that shouldn't 
change the architecture.  URNs, regardless of namespace, are supposed to 
assure persistence.   They're supposed to be distinguishable from other 
kinds of URIs that presumably don't make assurances of persistence.  The 
whole point in developing a URN umbrella was to be able to provide that 
level of assurance without users and software having to know 
characteristics of different URI types.   And we want software to be 
able to treat all URNs the same at some level, without having to deal 
with subtle differences in what persistence means for each.

Now if people have to adjust to the notion that maybe URNs with 
fragments/queries don't come with an assurance of persistence (even 
though they start with urn:), or that URNs with fragments/queries need 
an explicit indication of whether they are persistent, that might be 
ok.   Either one of those seems preferable to expecting users to know 
which namespaces assure one kind of persistence and which namespaces 
assure a different kind of persistence.

> is possible.

While that might be true for other reasons, I don't think it's true 
because we have different ideas on how to indicate persistence of 
URIs.   Those issues are going to emerge no matter who is looking at 
them, as long as they're looking in sufficient depth.   Granted some 
parties might solve things in such a way as to optimize their use cases 
at the expense of others', but that doesn't mean they'd do a better job.

Keith