Subject: [urn] syntax and registration: a proposal

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Thu, 16 Oct 2014 15:11:47 -0600

The URNBIS WG seems to be at an impasse. Having reviewed the discussions 
at and after IETF 90, I'd like to make a proposal.

I don't think we will quickly come to agreement on the semantics of 
paths, queries, and fragments in the context of the "urn:" scheme. (In 
fact, I suggest that we call them "p-components", "q-components", and 
"f-components" here so that we don't get bogged down in how their 
meaning in URNs might be different from their meaning in URIs.)

I remain hopeful that we might someday come to such agreement, but as 
far as I can see it's not productive for us to wait for such agreement 
before making progress on the two primary tasks on the charter of this 
WG: updating the URN syntax definitions from RFC 2141 and updating the 
URN namespace identifier registration procedures from RFC 3406.

I think we can complete those tasks in the relatively near future if we 
decide to do one thing: let go of semantics. That is, I suggest that we 
update the URN syntax to allow p-components, q-components, and 
f-components, but leave the semantics of those elements up to 
specifications that define specific NIDs.

Now, some folks might think that's reckless, but I think it's pragmatic. 
Right now we don't have a clear and distinct idea of how various 
URN-using communities might put p-components, q-components, and 
f-components to work in their context. In particular, the needs of 
various information sciences communities are still not well-known in 
this WG, despite the heroic efforts of Juha Hakala and a few other folks.

This pragmatic approach would enable those communities to complete their 
work in venues more conducive to involvement by the relevant 
subject-matter experts. If they are able to make progress there and 
define the semantics for their use of p-components, q-components, and 
f-components, then we will have some examples from which we can abstract 
broader definitions of semantics (if and when we decide that such an 
effort might be necessary).

In the meantime, the semantics might differ by NID. Although it's good 
to be aware of that possibility, I think the risks involved with 
NID-specific semantics are smaller than the risks involved with not 
completing our work, leaving the syntax as it is in RFC 2141, and then 
having URNs forked because communities that are trying to solve 
real-world problems will use q-components and f-components (and perhaps 
p-components) no matter what RFC 2141 says.

I see at least two open issues here:

First, how will comparison work? I suggest that q-components and 
f-components would not be factored into comparison, but that 
p-components would be factored in (if they end up being used).

Second, how is the syntax within p-components, q-components, and 
f-components defined? I suggest that it isn't, at least not in 2141bis. 
Particular NID specifications could, however, define intra-component 
syntax to meet their needs.

Third, under this approach would we still be comfortable with a NID 
registration policy of Expert Review? I would be, but I can't speak for 
anyone else.

(Naturally, there might be other open issues, as well.)

I'm wondering what WG participants think about this approach. If it is 
acceptable, I can update 2141bis and 3406bis accordingly (I think that 
the changes would be relatively minor, and I could make them before the 
I-D submission deadline on October 28th).

Thanks for your consideration.

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: John C Klensin <[email-masked]>
Date: Thu, 16 Oct 2014 18:04:16 -0400

Hi.

Speaking only for myself, I've come to conclusions about the
state of the WG and the likelihood of making significant
progress in the near future similar to Peter's.  Much as I'd
like to see a Grand Unified URN model, I think we are too
divided about expectations of what URNs are likely to do for us
and how to reach consensus conclusions about that any time soon.
Peter's suggestion builds on some things I started to explore on
list and in draft-ietf-urnbis-semantic-clarif but he figured out
the logical next steps which I was unable to put together.

A few comments inline below.

--On Thursday, October 16, 2014 15:11 -0600 Peter Saint-Andre -
&yet <[email-masked]> wrote:

> The URNBIS WG seems to be at an impasse. Having reviewed the
> discussions at and after IETF 90, I'd like to make a proposal.
> 
> I don't think we will quickly come to agreement on the
> semantics of paths, queries, and fragments in the context of
> the "urn:" scheme. (In fact, I suggest that we call them
> "p-components", "q-components", and "f-components" here so
> that we don't get bogged down in how their meaning in URNs
> might be different from their meaning in URIs.)

> I remain hopeful that we might someday come to such agreement,
> but as far as I can see it's not productive for us to wait for
> such agreement before making progress on the two primary tasks
> on the charter of this WG: updating the URN syntax definitions
> from RFC 2141 and updating the URN namespace identifier
> registration procedures from RFC 3406.

While I remain hopeful too, I'm now pessimistic about whether it
will ever be possible to reach such an agreement on other than a
"cluster of uses" or "community of users" basis.  It appears to
me that the list of things for which people intend to use URNs
is just too diverse for much unification unless we somehow make
draconian rules and expect people to follow them.  I think there
is ample evidence that the latter won't happen if those rules
violate a particular community's idea of good sense.

> I think we can complete those tasks in the relatively near
> future if we decide to do one thing: let go of semantics. That
> is, I suggest that we update the URN syntax to allow
> p-components, q-components, and f-components, but leave the
> semantics of those elements up to specifications that define
> specific NIDs.

Note that this requires not only letting go of semantics, as
Peter points out, but letting go of notions of syntax control
across namespaces about how p/q/f-components are structured
internally, making that a per-namespace matter.   That is, IMO,
consistent with RFC 3986 which rather carefully avoids
specification of the internal structure of the corresponding
generic URI components.

>...
> In the meantime, the semantics might differ by NID. Although
> it's good to be aware of that possibility, I think the risks
> involved with NID-specific semantics are smaller than the
> risks involved with not completing our work, leaving the
> syntax as it is in RFC 2141, and then having URNs forked
> because communities that are trying to solve real-world
> problems will use q-components and f-components (and perhaps

Concur.

> First, how will comparison work? I suggest that q-components
> and f-components would not be factored into comparison, but
> that p-components would be factored in (if they end up being
> used).
> 
> Second, how is the syntax within p-components, q-components,
> and f-components defined? I suggest that it isn't, at least
> not in 2141bis. Particular NID specifications could, however,
> define intra-component syntax to meet their needs.

Agreed.  See above.  This will imply that we need to put even
more emphasis on clear and comprehensive registration
descriptions, but we have, at least IMO, been moving in that
direction anyway.

> Third, under this approach would we still be comfortable with
> a NID registration policy of Expert Review? I would be, but I
> can't speak for anyone else.

I am, but with a fairly liberal interpretation of Expert Review.
In particular, I think we should be looking at a sort of
editorial advisory board model in which the role of the reviewer
or review team had much more to do with explaining the system
and advocating for completeness and clarity than making
pass/fail decisions.

     john

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Thu, 16 Oct 2014 16:51:23 -0600

Thanks, John. A few additional comments inline.

> &yet <[email-masked]> wrote:

<snip/>

>> I see at least two open issues here:

Actually I listed three...

> descriptions, but we have, at least IMO, been moving in that

Good point. We'll need to revisit 3406bis on this topic. I suspect that 
it will require further work.

> and advocating for completeness and clarity than making

Yes. Over time, I would hope that the review board members would find 
patterns across registrations so that they could encourage sharing of 
information across the "communities of use" you mentioned. I don't know 
that this would be a formal suggestion to such a board, but I at least 
would hope for it as an outcome.

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: Juha Hakala <[email-masked]>
Date: Fri, 17 Oct 2014 14:57:49 +0300

Hello Peter,

I hope that what you suggest below provides a basis for (rough) 
consensus in the WG. As far as I can tell, what you are suggesting would 
enable the information sciences community to create a new generation of 
URN resolvers, without creating a fork - our own, enriched version of 
the URN syntax and other URN specifications which would be managed 
outside IETF - in ISO, for instance.

Some comments below.

> meaning in URNs might be different from their meaning in URIs.)

If this the only way this WG can complete its task, then so be it.

> I think we can complete those tasks in the relatively near future if 
> we decide to do one thing: let go of semantics. That is, I suggest 
> that we update the URN syntax to allow p-components, q-components, and 

Since there is a need to harmonize things done with q-component across 
multiple namespaces, it is an additional burden to create the necessary 
specifications at NID level. But doing what Peter suggests does not 
prevent several  namespaces from sharing the same specification, which 
may or may not be published by IETF.

>
> Now, some folks might think that's reckless, but I think it's 
> pragmatic. Right now we don't have a clear and distinct idea of how 
> various URN-using communities might put p-components, q-components, 
> and f-components to work in their context. In particular, the needs of 

If this approach  allows my community to proceed in such a way which 
meets our needs, fine. I look forward to leaving philosophical 
discussions concerning semantics of URIs, URNs etc. to IETF. My 
community did not create that mess, and I am glad if that Gordian knot 
does not prevent us from moving on any more. If it turns out that even 
the approach Peter advocates does not enable the WG to reach rough 
consensus, then it is likely that we shall go our own way, like the 
Handle community before us.

>
> This pragmatic approach would enable those communities to complete 
> their work in venues more conducive to involvement by the relevant 
> f-components, then we will have some examples from which we can 
> abstract broader definitions of semantics (if and when we decide that 
> such an effort might be necessary).
>
> real-world problems will use q-components and f-components (and 
> perhaps p-components) no matter what RFC 2141 says.
>

OK.
> f-components defined? I suggest that it isn't, at least not in 
> 2141bis. Particular NID specifications could, however, define 
> intra-component syntax to meet their needs.

This is what we would do for e.g. ISO standard identifiers which have 
registered URN namespaces.

> registration policy of Expert Review? I would be, but I can't speak 
> for anyone else.

Such review would be more complicated, but not impossible.

> the changes would be relatively minor, and I could make them before 
> the I-D submission deadline on October 28th).

Please proceed.

All the best,

Juha

-- 

  Juha Hakala
  Senior advisor

  The National Library of Finland
  Library Network Services
  P.O.Box 26 (Teollisuuskatu 23)
  FIN-00014 Helsinki University
  Tel. +358 9 191 44293
  Mobile +358 50 3827678

---

From: John C Klensin <[email-masked]>
Date: Fri, 17 Oct 2014 13:34:45 -0400

--On Friday, October 17, 2014 14:57 +0300 Juha Hakala
<[email-masked]> wrote:

> Hello Peter,
> 
> I hope that what you suggest below provides a basis for
> (rough) consensus in the WG. As far as I can tell, what you
> are suggesting would enable the information sciences community
> to create a new generation of URN resolvers, without creating
> a fork - our own, enriched version of the URN syntax and other
> URN specifications which would be managed outside IETF - in
> ISO, for instance.

I hope Peter will agree, but let me say the above a little
differently to be sure we are talking about the same thing.  The
expectation would be that your "own, enriched version..." would
conform to the basic URN syntax.  That syntax would be pretty
basic, essentially reflecting the same restrictions as RFC 3986.
You would also be expected to register each new namespace and
NID.  I'd hope that, if your community (and others) created a
common set of principles and rules for how "your" URNs were
structured and would work, new namespaces that conformed to them
could be fast-tracked through the registration procedure based
on commonality of references and practices.  The broad issues
about registration practices will still need to be sorted out in
this WG's discussions and consensus reached, but Peter and I
hope that is a much less difficult problem than trying to sort
decisions about URN behavior out globally and across different
types of uses and communities.

More below.

> If this the only way this WG can complete its task, then so be

I see it as more positive than that, even if that is also true.
It recognizes that the WG, at least in my opinion, has been
moving much more toward a registration model than an IETF
approval one.  That change is consistent with a growing
understanding of the limits to the IETF's subject matter
expertise in naming and identification areas as well as
acceptance that, for many communities, the IETF's saying "we
don't think you should do that" is advice and not a commandment.
Many of us had hoped that we could preserve at least a common
syntax and set of comparison runes and operations for, and
within, "fragments" and/or "queries" so that all uses of the
"urn:" scheme would behave in the same way, but it appears that
opinions and communities of interest, as well as concerns about
the applications we haven't seen yet, are sufficiently diverse
to make consensus at that level of detail.   From that
perspective, this is just acceptance of what can be specified
and standardized scheme-wide and what is more properly left to
communities of uses and users who can define their needs and how
they should be implemented more effectively than the IETF can.

Put differently, this looks to me like the next logical step in
figuring out the role the IETF should properly play in or with
specific namespaces and URNs generally.

>> future if  we decide to do one thing: let go of semantics.
>> That is, I suggest  that we update the URN syntax to allow
> Since there is a need to harmonize things done with
> q-component across multiple namespaces, it is an additional
> burden to create the necessary specifications at NID level.
> But doing what Peter suggests does not prevent several
> namespaces from sharing the same specification, which may or
> may not be published by IETF.

Exactly.  If convenient, the IETF (or the RFC Editor) can
generally publish (or republish) specifications from other
bodies when that publication serves the needs of the community.
If there is a publicly and generally-available specification
published elsewhere, that may not be necessary.

A few of us have also kicked around the idea of explicitly
grouping specifications with significant similarities, such as
sharing the same basic specification for what components are
used and how, e.g., q-components are used in some detail, so
that a registration document could reference that group
specification rather than repeating all of the details.  Again,
the WG would need to sort that out, but my personal view is
that, while it would be convenient for the IETF to ask IANA to
maintain a registry of group names that could then be referenced
from the registry of namespaces (NID names), it would not need
to have significant other involvement in the process.

>> pragmatic. Right now we don't have a clear and distinct idea
>> of how  various URN-using communities might put p-components,
>> q-components,  and f-components to work in their context. In
>> particular, the needs of  various information sciences
>> communities are still not well-known in  this WG, despite the
>> heroic efforts of Juha Hakala and a few other folks.
> 
> If this approach  allows my community to proceed in such a way
> which meets our needs, fine. I look forward to leaving
> philosophical discussions concerning semantics of URIs, URNs
> etc. to IETF. My community did not create that mess, and I am
> glad if that Gordian knot does not prevent us from moving on
> any more. If it turns out that even the approach Peter
> advocates does not enable the WG to reach rough consensus,
> then it is likely that we shall go our own way, like the

I, and I think I can say "we", are aware of that risk and, in
part because there are other communities with the same problem
and needs and perspectives different from yours, we are trying
to cut said knot by narrowing the definition of how much all
URNs have to have in common.

>> not in  2141bis. Particular NID specifications could,
>> however, define  intra-component syntax to meet their needs.
> 
> This is what we would do for e.g. ISO standard identifiers
> which have registered URN namespaces.

That is my hope, i.e., we get the IETF out of your way as
suggested above.

>> a NID  registration policy of Expert Review? I would be, but
>> I can't speak  for anyone else.
> 

Again, if a common specification is established by some
relatively established group -- especially if that group is a
recognized standards body-- I'd hope the IETF review process
could examine that specification, offer whatever advice it has,
and then provide a very rapid and low-pain registration process
for NIDs that conformed to that specification.  The main value
of that registration process for such groups would be the
classic one: preventing two different namespaces from using the
same NID by accident and in a confusing way.

best,
   john

---

From: [email-masked] (Dale R. Worley)
Date: Fri, 17 Oct 2014 17:54:30 -0400

> From: Peter Saint-Andre - &yet <[email-masked]>

> specifications that define specific NIDs.

This strikes me as consonant with how the IETF operates in many
situations.  There aren't many operations that need to work
consistently on URNs qua URNs:  What is the scheme?  What is the
namespace?  A universal syntax restriction (so all URNs can be parsed
from their contexts).  A universal equality test (that may yield false
negatives but must not yield false positives).  We need to ensure that
those features work consistently within a framework that lets
interested parties explore alternative solutions to the problems on
which we clearly have no consensus.

> From: Juha Hakala <[email-masked]>
> 

And I don't see that as being very difficult -- One RFC can reference
another.  The major limitation is that it's difficult to do if the
first RFC isn't written in a way that allows a clear external
reference to the particular component.  (My personal example of this
is that the description of the SIP "digest authentication scheme"
references the HTTP digest authentication scheme in a clumsy way that
requires unnecessary work to understand.)

> From: John C Klensin <[email-masked]>
> 
> used and how,

I don't think that deliberate grouping would be very fruitful at this
stage unless a group was generated naturally by the proposers of the
NIDs in the group.  I would expect that these groups would develop
organically over time, and eventually there would be a series of -bis
RFCs regularizing their similarity.  (By analogy, there is no official
grouping of all the SIP RFCs.)

> From: Peter Saint-Andre - &yet <[email-masked]>

> p-components would be factored in (if they end up being used).

Without any additional input, I suggest that the generic comparison
includes all q-/f-/p-components.  That is because the current generic
comparison errs on the side of false-negatives and but avoids
false-positives.  NIDs are allowed to define lexical comparison rules
that cause more URNs to be "equal" than the generic comparison rule,
but cannot cause more URNs to be "not equal" than the generic
comparison rule.  If the generic comparison rule includes all
additional components taken literally, then we can maintain this
pattern regardless of how the equality definition evolves in
particular NIDs.

> syntax to meet their needs.

That makes sense.  That is, the component syntax is defined simply as
the iteration of a particular set of characters (that does not include
any of the demarcating characters).

> registration policy of Expert Review? I would be, but I can't speak for 

Again, that makes sense to me.  Expert Review is rather flexible.
Certainly, it is expected to include "editorial oversight" to make
sure that the definition is clear and comprehensive (per JCK).  But
experts also make suggestions as to how a new NID definition can be
improved.  E.g., ensuring that it has extensibility, making its
constructions similar to those of other NIDs, etc.  And the writers of
NIDs generally adopt such suggestions.

> From: John C Klensin <[email-masked]>
> 
> I'd hope the IETF review process could examine that specification,
> offer whatever advice it has, and then provide a very rapid and
> low-pain registration process for NIDs that conformed to that
> specification.  The main value of that registration process for such
> groups would be the classic one: preventing two different namespaces
> from using the same NID by accident and in a confusing way.

And additionally, the IETF can advise on matters within its specific
expertise, such as ways to maximize regularity of the entire URN
universe and ensureing that the proposal has good extensibility
properties.

Dale

---

From: Julian Reschke <[email-masked]>
Date: Sat, 18 Oct 2014 09:16:13 +0200

> which we clearly have no consensus.

Well, if URNs can be used everywhere URIs are used, there is at least 
one more common operation which needs to be defined somewhere: 
resolution of a reference against a base URN.

Best regards, Julian

---

From: John C Klensin <[email-masked]>
Date: Sat, 18 Oct 2014 04:32:33 -0400

--On Saturday, October 18, 2014 09:16 +0200 Julian Reschke
<[email-masked]> wrote:

> 
> Well, if URNs can be used everywhere URIs are used, there is
> at least one more common operation which needs to be defined
> somewhere: resolution of a reference against a base URN.

Given that there is no requirement that URNs be resolved at all
and that some URNs are not intended to be resolved, I don't
understand your comment.  Could you explain further?

   john

---

From: Julian Reschke <[email-masked]>
Date: Sat, 18 Oct 2014 10:37:17 +0200

> understand your comment.  Could you explain further?

The other kind of "resolution" -- see 
<http://greenbytes.de/tech/webdav/rfc3986.html#reference-resolution>.

Best regards, Julian

---

From: John C Klensin <[email-masked]>
Date: Sat, 18 Oct 2014 05:42:28 -0400

--On Saturday, October 18, 2014 10:37 +0200 Julian Reschke
<[email-masked]> wrote:

>> Given that there is no requirement that URNs be resolved at
>> all and that some URNs are not intended to be resolved, I
>> don't understand your comment.  Could you explain further?
> 

Julian,

To the best of my knowledge, there has never been such a thing
as a "relative URN".  Certainly RFC 2141 doesn't mention such a
thing and its syntax doesn't appear to me to allow for it.  It
seems to me that this takes us back to the circle we have been
going around, so, at the risk of reviewing the core of that
impasse in this context.

(1) Any assumption that URNs were required to have partial or
relative forms was intended to be eliminated by
draft-ietf-urnbis-semantics-clarif.  It that document has to
enumerate such assumptions, I would appreciate help in making
the list.   But the "nothing but the syntax" approach would seem
to eliminate Section 5 of RFC 3986 because it is not about
syntax but a relatively complex set of rules for interpreting
one URI into another.

(2) RFC 3986 does not update or obsolete RFC 2141.  It appears
to me that the closest it comes to requiring that all URI
schemes conform to everything it mentions is the discussion in
Section 1.1.3 that concludes 'Future specifications and related
documentation should use the general term "URI" rather than the
more restrictive terms "URL" and "URN" [RFC3305].'.  That
doesn't strike me as imposing any requirement other than a
terminology suggestion.

(3) Even if that were not the case, 3986 introduces the concept
of "relative reference" in Section 1.2.2 with an introduction
that makes them, and "resolving" them, "common terminology for
use", not "requirement to support".  Since that takes us back
into the exercise of 3986 exegesis, the relevant sentences are 

	"Such operations are defined by the protocols that make
	use of URIs, not by this specification.  However, we do
	use a few general terms for describing common operations
	on URIs."

The paragraph there, starting "A relative reference ((Section
4.2) refers to a resource by describing the difference within a
hierarchical name space...."  does not impose a requirement that
all URI types support relative references either.   Indeed, it
goes on to say 

	"As relative references can only be used within the
	context of a hierarchical URI, designers of new URI
	schemes should use a syntax consistent with the generic
	syntax's hierarchical components unless there are
	compelling reasons to forbid relative referencing within
	that scheme."

Now, I suppose we could get far into the domain of protocol
lawyers and debate "should" and what constitutes "compelling
reasons", but I note that this is not a conformance statement
but a recommendation and that "urn:" is certainly not a "new URI
scheme" nor was it a new URI scheme at the time of 3968's
writing.

(4) I don't see anything in Section 4 that requires all URI
schemes to support relative references either.  It appears to me
to just define a syntax model for them if they are allowed and
used.

(5) Finally, Section 5 ("Reference Resolution"), particularly in
5.2, contains the same type of "if this is not applicable, one
can try that" language that appears in Section 6 ("Normalization
and Comparison").  I think that any argument that Section 6 is
just guidance and suggestions abut what might be done, rather
than a normative requirement on all URIs (much of the discussion
in Toronto if I understood it correctly) has to be taken as
applying equally to Section 5.1 and 5.2.

So, either I still don't understand what you are suggesting or
it seems to me that it is not relevant to URNs.  Again, if more
words are required in draft-ietf-urnbis-semantics-clarif, please
suggest text.

    john

---

From: John C Klensin <[email-masked]>
Date: Sat, 18 Oct 2014 06:14:52 -0400

Dale,

One area of clarification, one of possible disagreement...

--On Friday, October 17, 2014 17:54 -0400 "Dale R. Worley"
<[email-masked]> wrote:

> I don't think that deliberate grouping would be very fruitful
> at this stage unless a group was generated naturally by the
> proposers of the NIDs in the group.  I would expect that these
> groups would develop organically over time, and eventually
> there would be a series of -bis RFCs regularizing their
> similarity.  (By analogy, there is no official grouping of all
> the SIP RFCs.)

Dale, we may be thinking about this a bit differently, so let me
explain what I meant by "grouping" (an attempt to choose a
neutral term).   As Juha pointed out, the library community (and
its close collaborators) may respond to our going down this path
by generating their own specification for how a set of
namespaces (present and future) that meet their needs will
behave.  It is clear to me that they understand their needs even
if many of the rest of us have been unsuccessful in being
confident that we shared that understanding.  It is clear that
they require more than one namespace: after all, there are three
of them (ISBN, ISSN, and NBN) specifically called out in this
WG's original charter.  Now, suppose that community produces
"Grand Unified Library URN Specification #1" (with no
expectation that there will ever be a number #2, but that is not
the IETF's problem or anything the IETF can or should control).
I'd assume they would then rewrite the ISBN, ISSN, NBN and
probably DOI URN specifications in thems fo that one and that
those that reference it and established International Standards
would be very short.    That is, in the sense in which I used
the term, a "group" of URN namespaces, one that shares a
definition that is much more specific about what components are
allowed, what syntax they use internally, and how they are used
than 2141 or 2141bis but that is still consistent with the
latter.

> Without any additional input, I suggest that the generic
> comparison includes all q-/f-/p-components.  That is because
> the current generic comparison errs on the side of
> false-negatives and but avoids false-positives.  NIDs are
> allowed to define lexical comparison rules that cause more
> URNs to be "equal" than the generic comparison rule, but
> cannot cause more URNs to be "not equal" than the generic
> additional components taken literally, then we can maintain
> this pattern regardless of how the equality definition evolves
> in particular NIDs.

I think that misses the point.  If one uses the completely
generic URI comparison model laid out in 3986, then the above is
what you get: lots of false negatives, few if any false
positives, and, along with it, some vague ideas that the
ultimate way to compare URIs is to resolve them and look at the
results.  That is what one would expect a generic URI engine
that is not scheme sensitive to do and I read 3986 as saying
just that.  But we'd heard arguments on this mailing list that
there are some information elements (independent of binding to
syntax components) that really should be included in comparisons
(if they don't match, the URNs don't either) and others that
should not (if they don't match, it isn't important and the URNs
are equal anyway).  So, remembering that 2141 does not allow
q-components, f-components, or (past-NSS) p-components, this is
out opportunity to say 

	-- these components must match for the URN to be equal;
	those don't count in an equality comparison.
	
	-- designers of URN namespaces that utilize any of
	f/q/p-components must make decisions about what
	information or requests go in to what queries based on
	an understanding of their equality comparison
	implications.

It seems to me that is a perfectly reasonable position to take
on a per-scheme basis and that it is actually a position
consistent with 3986.  It doesn't prevent even stronger rules if
one has NID-specific information but, if a system knows about
the "urn:" scheme, it seems to me that one loses a lot,
especially for namespaces that are not resolved or whose
resolution involves subtle issues, by saying what amounts to
"there are not equality comparisons other than bit-for-bit
string matching".

>> I'd hope the IETF review process could examine that
>> specification, offer whatever advice it has, and then provide
>> a very rapid and low-pain registration process for NIDs that
>> conformed to that specification.  The main value of that
>> registration process for such groups would be the classic
>> one: preventing two different namespaces from using the same
>> NID by accident and in a confusing way.
> 
> And additionally, the IETF can advise on matters within its
> specific expertise, such as ways to maximize regularity of the
> entire URN universe and ensureing that the proposal has good
> extensibility properties.

As long as it is advice and timely, no problem.  Especially
after this long period of little progress and much confusion, I
am still concerned that, if the IETF starts telling others what
to do, they will ignore us, create their own independent
definitions, and not bothering trying to register the NIDs
associated with those namespaces.

     john

---

From: Julian Reschke <[email-masked]>
Date: Sat, 18 Oct 2014 12:44:53 +0200

> words are required in draft-ietf-urnbis-semantics-clarif, please

With the risk of repeating myself. If URNs are supposed to appear where 
URIs can go, resolution of a relative reference against a URN base URI 
will need to be defined somewhere (and yes, it'll be ok if it's exactly 
what RFC 3986 says).

Concretely: what do you suggest what Java should do in

http://docs.oracle.com/javase/7/docs/api/java/net/URI.html#resolve%28java.net.URI%29

for a URN base URI?

(And the same thing needs to be answered of xml:base, or the base 
attribute in HTML).

Best regards, Julian

---

From: Sean Leonard <[email-masked]>
Date: Sat, 18 Oct 2014 07:00:40 -0700

On 10/18/2014 3:44 AM, Julian Reschke wrote:
> With the risk of repeating myself. If URNs are supposed to appear 
> where URIs can go, resolution of a relative reference against a URN 
> base URI will need to be defined somewhere (and yes, it'll be ok if 
> it's exactly what RFC 3986 says).
>

Absolutely nothing.

I don't see any conceptual problem with this when compared with the cid: 
URL [RFC2392] or the data: URL [RFC2397]. Check those out.

As I understand it, URNs are not hierarchical in nature. They identify 
things of interest to Internet users in a universal, context-free way.

Consider urn:ietf [RFC2648]:
urn:ietf:rfc:2141
urn:ietf:mtg:90

What is the base URI for these URNs? Nothing. Are we so lazy that 
instead of writing "urn:ietf:rfc:3986" you have to say <base 
href="urn:ietf:rfc"> and <a href="3986"> ? The whole point is to make 
URNs context-free, and that means hierarchy-free. Hierarchy is context.

"ietf:rfc" and "ietf:mtg" illustrate a convenient means to divide the 
namespace logically. But URNs are not retrievable in the protocol sense. 
If anything, "urn:ietf:rfc:2141" should refer to the plain text file, 
which is a format utterly lacking in support for URIs, so the point is 
moot. In fact "urn:ietf:rfc:2141" names an abstract 
thing-of-interest-to-the-Internet-community, namely: an IETF-named 
object, which is an RFC, which has the number 2141. Now that we have 
named the abstract thing, we can /resolve/ it to more concrete things 
(HTTP URL, text/plain, text/html, etc.) depending on what we want.

Another point: well-designed URNs are short. Whenever an identifier gets 
long, there are natural impulses to try to shorten it (e.g., make 
relative URIs). So, keep URNs short, and you'll never fret about using 
them in their full forms.

Thoughts?

Sean

---

From: Julian Reschke <[email-masked]>
Date: Sat, 18 Oct 2014 23:46:08 +0200

>> http://docs.oracle.com/javase/7/docs/api/java/net/URI.html#resolve%28java.net.URI%29

What should it return, or should it throw an exception?

> URL [RFC2392] or the data: URL [RFC2397]. Check those out.

I agree that all of these should get the same treatment; and that 
treatment is defined in RFC 3986 (and yes, I realize that java.net.URI 
does not do that).

> things of interest to Internet users in a universal, context-free way.

Yes, but that doesn't answer the question.

> What is the base URI for these URNs? Nothing. Are we so lazy that

"base URI" of a "URI" doesn't make any sense at all.

> URNs context-free, and that means hierarchy-free. Hierarchy is context.

You don't have to, but our specs still should define what the above means.

> namespace logically. But URNs are not retrievable in the protocol sense.

Unless you define a resolver.

> If anything, "urn:ietf:rfc:2141" should refer to the plain text file,

Why?

> which is a format utterly lacking in support for URIs, so the point is

If you mean fragments, not URIs: yes, they are defined.

> (HTTP URL, text/plain, text/html, etc.) depending on what we want.

See :-)

> them in their full forms.

It doesn't answer the question I asked.

Resolving a relative reference against a URI is a common operation. 
Please do not pretend it doesn't need to be defined just because the URI 
is a URN.

Best regards, Julian

---

From: Sean Leonard <[email-masked]>
Date: Sat, 18 Oct 2014 14:56:10 -0700

On 10/18/2014 2:46 PM, Julian Reschke wrote:
>>> [SNIP]

from that same URI (lol):

    publicURI  <http://docs.oracle.com/javase/7/docs/api/java/net/URI.html>  resolve(URI  <http://docs.oracle.com/javase/7/docs/api/java/net/URI.html>  uri)

    Resolves the given URI against this URI.

    If the given URI is already absolute, or if this URI is opaque, then
    the given URI is returned.

Definition:

    An/opaque/URI is an absolute URI whose scheme-specific part does not
    begin with a slash character ('/'). Opaque URIs are not subject to
    further parsing. Some examples of opaque URIs are:

        mailto:[email-masked] 	
        news:comp.lang.java 	
        urn:isbn:096139210x

The documentation itself uses URN as an example! :) The end.

>> If anything, "urn:ietf:rfc:2141" should refer to the plain text file,

Only because the plain text file is (currently) the canonical form of 
RFCs (at least I thought that they are, by default). Perhaps I should 
have said "if you NEED to assume that all urn:ietf:rfc named items are 
some kind of Internet media type content, it is safest to assume that a 
text/plain content for it exists". (Which is a much more elliptical 
statement.)

> If you mean fragments, not URIs: yes, they are defined.

I mean that plain text is an abstract sequence of characters. The text 
http://foo is not guaranteed to be processed by a given text editor or 
"cat" command at the terminal in any particular way. Similarly, in HTML, 
<p>Hello I am http://foo</p> does not contain a "supported URI". 
<p>Hello I am <a href="http://foo">foo</a></p> contains a supported URI, 
because it will treat that URI as a URI and not an undifferentiated 
sequence of abstract characters.

-Sean

---

From: Julian Reschke <[email-masked]>
Date: Sun, 19 Oct 2014 00:20:58 +0200

> The documentation itself uses URN as an example! :) The end.

OK, so did you just change your position to "yes, it should be defined, 
and the definition from the java.net.URI javadocs makes sense to me"?

In which case, we ought to discuss whether the scheme-agnostics 
resolution defined in RFC 3986 needs to be revised.

BTW: I happen to believe that Java's definition yields surprising 
results for, for instance, data URIs.

base: data:text/html/...stuff...
rel: #x

I would expect the result of the resolution to be

data:text/html/...stuff...#x

> text/plain content for it exists". (Which is a much more elliptical

I note that the RFC Editor seems to believe that "The RFC status page 
for that RFC" is a better answer (I happen to disagree with that).

> sequence of abstract characters.

Oh, I see what you meant to say, but it's irrelevant here. The media 
type doesn't need to support URIs in any way -- resolution of relative 
references still needs to be defined.

Best regards, Julian

---

From: Sean Leonard <[email-masked]>
Date: Sat, 18 Oct 2014 15:40:50 -0700

On 10/18/2014 3:20 PM, Julian Reschke wrote:
>>     publicURI 
>> <http://docs.oracle.com/javase/7/docs/api/java/net/URI.html> 
>> resolve(URI 
>> <http://docs.oracle.com/javase/7/docs/api/java/net/URI.html> uri)
>>
> OK, so did you just change your position to "yes, it should be 
> defined, and the definition from the java.net.URI javadocs makes sense 
> to me"?
>

The most telling thing is that Java's definition yields "surprising 
results".

Overall the thing is that URIs that are not path-oriented do not support 
a concept of relative references. You can ask the question, but the 
answer will not make sense. We need not inquire further about what Java 
or any other particular implementation would do.

Example 1:
base URI:
mid:[email-masked]

given relative URI:
../foo.html

What does it mean when you combine them?
Answer: Don't do that. It doesn't make sense.

Example 2:
base URI:
data:text/html;base64,PGh0bWw+PGhlYWQ+PHRpdGxlPkhlbGxvPz8/PzwvdGl0bGU+PC9oZWFkPjxib2R5PkhpIHRoZXJlLjwvYm9keT48L2h0bWw+

given relative URI:
../foo.html

What does it mean when you combine them?
Answer: Don't do that. It doesn't make sense.
Your algorithm might see ...Pz8/Pzwvd... and slice the URI at that point.
Don't do that. It doesn't make sense.

> The media type doesn't need to support URIs in any way -- resolution 
> of relative references still needs to be defined.

Not obvious to me. Where are you going to get the relative references 
from? If you're not getting it from the content, where else are you 
getting it?

Sean

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Sat, 18 Oct 2014 19:52:28 -0600

> (And the same thing needs to be answered of xml:base, or the base

Julian, are you implying that there might be different answers to those 
three questions?

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: Julian Reschke <[email-masked]>
Date: Sun, 19 Oct 2014 10:04:15 +0200

> Answer: Don't do that. It doesn't make sense.

I agree mostly. But I disagree that we can pretend not to have to answer 
the question.

> What does it mean when you combine them?

Something weird.

> Don't do that. It doesn't make sense.

Yes.

But what if the relative reference is "#a" (and assuming the HTML 
contains that anchor)? Still meaningless?

> from? If you're not getting it from the content, where else are you

I see your point. Yes, if the format doesn't have this information, 
relative reference resolution will not happen frequently. It doesn't 
mean it never happens, though.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Sun, 19 Oct 2014 10:09:07 +0200

> Julian, are you implying that there might be different answers to those

In a perfect world, there'lll be a single definition that makes sense 
for hierarchical URIs (for some value "hierarchical") and does something 
predictable (but maybe weird) for all other schemes. Optimally, that 
definition would be the one in RFC 3986.

The problem here is that if groups get away with the idea that they just 
can define what they want (and are not bound by the base spec), we 
*will* end up with different definitions. That's why I tried to make 
clear that even if relative reference resolution isn't *important*, it 
still needs to be defined.

Best regards, Julian

---

From: Andrew Newton <[email-masked]>
Date: Mon, 20 Oct 2014 10:14:38 -0400

On Sun, Oct 19, 2014 at 4:09 AM, Julian Reschke <[email-masked]> wrote:
> The problem here is that if groups get away with the idea that they just can
> define what they want (and are not bound by the base spec), we *will* end up
> with different definitions. That's why I tried to make clear that even if
> relative reference resolution isn't *important*, it still needs to be
> defined.

I agree that we should not gloss over relative reference resolution
(i.e. it is important). But I am unclear as to what you feel we should
do. Should we have a definition for URNs? Is it ok if we say this
behavior is undefined? Or are we to somehow align ourselves with the
mess that already exists with URIs in general?

-andy

---

From: Julian Reschke <[email-masked]>
Date: Mon, 20 Oct 2014 16:41:50 +0200

> mess that already exists with URIs in general?

I believe you should say that URIs using the "URN" scheme work exactly 
the same way as any other URI, and that that behavior is defined by RFC 
3986.

And no, it's not a mess.

Best regards, Julian

---

From: [email-masked] (Dale R. Worley)
Date: Mon, 20 Oct 2014 10:49:56 -0400

> From: Julian Reschke <[email-masked]>

> http://docs.oracle.com/javase/7/docs/api/java/net/URI.html#resolve%28java.net.URI%29

To me, the immediate analogy is, what should Java do if the base URI
is a sip: URI?

Within the sip: scheme, there is no hierarchy expressed in the syntax,
and (as far as I've ever heard) no conceptual hierarchy system within
which a relative URI could be assigned a meaning.

What Java *does* is outside my ken, but quite naively, I'd expect the
function to throw an exception.  Basically, "You've attempted to use
this URI as if it has a hierarchical structure, but it doesn't."
(Describing how you really want to handle that situation within the
design of the Java language may be different.)

Now there is a caveat:  Perhaps some future NIDs of URNs will have a
hierarchical structure defined for them, and within that a relative
URI has a meaning.  In which case, the function can be extended to
handle those in the conceptually correct way.

Dale

---

From: John C Klensin <[email-masked]>
Date: Mon, 20 Oct 2014 11:04:24 -0400

--On Monday, October 20, 2014 10:14 -0400 Andrew Newton
<[email-masked]> wrote:

> I agree that we should not gloss over relative reference
> resolution (i.e. it is important). But I am unclear as to what
> you feel we should do. Should we have a definition for URNs?
> Is it ok if we say this behavior is undefined? Or are we to
> somehow align ourselves with the mess that already exists with
> URIs in general?

And, noting Dale's comments, if 3986 is taken to imply that all
URIs are expected to support relative forms, then what should be
done about SIP and other URIs, some very heavily-used, that
don't support relative references either?  For URNs, would a
revised version of draft-ietf-urnbis-semantics-clarif do the job
of clarifying that relative references are not applicable, or
does something have to be done to 3986?

    john

---

From: [email-masked] (Dale R. Worley)
Date: Mon, 20 Oct 2014 11:08:00 -0400

> definition would be the one in RFC 3986.

Actually, that sort of thing is not universal in a perfect world.
There are numerous situations in programming languages where the
outcome of an operation is not strictly defined, and it is agreed that
the programming language is better for it.  And there are various
flavors of "not strictly defined".  In practice, they all mean that
your program should avoid attempting to make such an operation happen
unless it takes special precautions.  An even more elaborate version
is the definition in the VAX CPU architecture definition of what is
and is not allowed within the definition of "the result of the
operation is not defined".

> still needs to be defined.

I'd say that we need to make it clear what is defined and what is not
defined.  If someone wants to extend the official definition with an
unofficial one, that's their problem.

> contains that anchor)? Still meaningless?

If we decide to extend the use of the syntax "#a" to URNs to indicate
fragments when the URN denotes (one or more) MIME-typed data objects,
then it would make sense to extend the rules of relative URI
resolution to interpret a relative URI like "#a" on that URN as a base
in the way described in RFC 3986; it becomes the fragment-part of the
target URI that is generated by the resolution process.

Dale

---

From: [email-masked] (Dale R. Worley)
Date: Mon, 20 Oct 2014 11:12:02 -0400

> From: John C Klensin <[email-masked]>

> But we'd heard arguments on this mailing list that there are some
> information elements (independent of binding to syntax components)
> that really should be included in comparisons (if they don't match,
> the URNs don't either) and others that should not (if they don't
> match, it isn't important and the URNs are equal anyway).

I see the designation at this time that a component is not significant
for equality comparison as risky.  If in the generic URN specification
all components are significant, then NID definitions can later say
that particular components are not significant.  But if we say in the
generic URN specification that a particular component is not
significant, then all NID definitions inherit that rule.

I am willing to be swayed if somone proposes that a *particular*
component should be non-significant, and everyone agrees that that is
so.  But in a discussion of this high level of generality, I don't see
that happening.

Saying in the generic specification that all components are
significant is the safe choice with regard to future extensibility.

Dale

---

From: Julian Reschke <[email-masked]>
Date: Mon, 20 Oct 2014 17:16:56 +0200

> does something have to be done to 3986?

I believe that a URI scheme shouldn't say "they aren't applicable". What 
it *can* say is "they don't make sense, so be careful".

Best regards, Julian

---

From: John C Klensin <[email-masked]>
Date: Mon, 20 Oct 2014 11:21:19 -0400

--On Monday, October 20, 2014 16:41 +0200 Julian Reschke
<[email-masked]> wrote:

> I believe you should say that URIs using the "URN" scheme work
> exactly the same way as any other URI, and that that behavior
> is defined by RFC 3986.

Just trying to understand here...

In other words, you believe that all URIs are required to be
hierarchical and to support relative references into that
hierarchy?

How do you reconcile that with the distinctly non-hierarchical
URNs specified in RFC 2141, the many documented and deployed
URNs that follow that model, including the ones I've described
as "indicators" (i.e., the presence or absence of the URN
conveys information without any further semantics associated
with its value), and other URIs, such as the SIP ones Dale has
referenced, that don't have enough implicit or hierarchical
structure for relative references to make any sense?  Are you
suggesting that all of those URIs became non-conforming and
invalid with the publication of RFC 3986, even those that
appeared after it?

    john

---

From: Julian Reschke <[email-masked]>
Date: Mon, 20 Oct 2014 17:40:42 +0200

> hierarchical and to support relative references into that

No.

I believe that the rules for the relative reference resolution need to 
be independent of the URI scheme. It's fine if a URI scheme isn't 
hierarchical, in which case these rules will yield "surprising" results.

Best regards, Julian

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Mon, 20 Oct 2014 11:24:57 -0600

> hierarchical, in which case these rules will yield "surprising" results.

Julian, do you think that 2141bis needs to have some text on this point?

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: Julian Reschke <[email-masked]>
Date: Mon, 20 Oct 2014 20:03:11 +0200

> Julian, do you think that 2141bis needs to have some text on this point?

It it's completely silent that would be technically ok. However, it 
might cause confusion among some readers that somehow come to the 
conclusion that RFC 3986 does not apply.

Thus, it would be good to state once for all and upfront that "URN" URIs 
are really URIs with all implications stated by RFC 3986.

Best regards, Julian

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Mon, 20 Oct 2014 12:09:35 -0600

> are really URIs with all implications stated by RFC 3986.

I think that 2141bis already effectively says this, but I will review 
the text again to make sure that the next version is as clear as possible.

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: John C Klensin <[email-masked]>
Date: Mon, 20 Oct 2014 14:26:53 -0400

--On Monday, October 20, 2014 17:16 +0200 Julian Reschke
<[email-masked]> wrote:

> I believe that a URI scheme shouldn't say "they aren't
> applicable". What it *can* say is "they don't make sense, so
> be careful".

--On Monday, October 20, 2014 17:40 +0200 Julian Reschke
<[email-masked]> wrote:

> I believe that the rules for the relative reference resolution
> need to be independent of the URI scheme. It's fine if a URI
> scheme isn't hierarchical, in which case these rules will
> yield "surprising" results.

Ok. At least now I understand what you are suggesting.  That is
progress.

However, it identifies a small (or maybe not-so-small)
philosophical disagreement.  

First, I think that, when Section 1.2.3 of RFC 3896 says

	"For some URI schemes, the visible hierarchy is limited
	to the scheme itself: everything after the scheme
	component delimiter (":") is considered opaque to URI
	processing."

it implies that we can say 

	"For the 'urn' scheme, there is no URI processing beyond
	scheme identification.  Therefore relative reference
	processing (which is definitely URI processing) does not
	apply and, incidentally, any discussion of 3986 beyond
	basic parsing of <uri> into components is irrelevant."
	
Second and more fundamentally, I believe that designing
protocols with the intent that even the most carefully-written
implementations will cause potentially-nasty surprises is just
wrong, to the point that words like "evil" and "immoral" occur
to me.  If 3986 actually requires that any system evaluating
URIs apply relative reference processing even when it will yield
bizarre and meaningless results, then I would contend that is a
seriously bad specification.

I don't see any such requirement in 3986.  What is see instead
is a set of rules for applying and interpreting relative
references for schemes for which they are relevant.  I wish that
were a lot more clear (either way), but don't see a problem with
my reading.

I wish this had come up in Toronto when it looked like
separating URNs more clearly from 3986 semantics --or, if one
prefers, treating all of the semantics in 3986 as advisory
rather than mandatory issues to be decided on a per-scheme
basis.  If I had thought about relative references at all at
that time, I would have assumed they were part of what we were
discussing as semantics because they are clearly not part of the
<uri> ABNF or even discussions of it.  But here we are, back to
discussing what 3986 requires URNs to be or do.  And, fwiw, even
if we simply discarded "urn:" (something that would not fly in
the marketplace and would just lead to non-IETF URN specs) and
started moving forward with some new URx", we would still have
this relative reference problem.

That said, if it would get us unstuck and this were absolutely
the last "URNs can't do or be that because of what 3986 can be
interpreted to say" discussion we needed to have, I could live
with "they don't make sense, so be careful" language in the
spec.  I wouldn't be happy, but we really do need to get unstuck
unless the goal is kill URNs, or at least an IETF URN
definition, by raising objections one at a time until everyone
gives up.

     john

---

From: John C Klensin <[email-masked]>
Date: Mon, 20 Oct 2014 15:14:30 -0400

--On Monday, October 20, 2014 11:12 -0400 "Dale R. Worley"
<[email-masked]> wrote:

>> But we'd heard arguments on this mailing list that there are
>> some information elements (independent of binding to syntax
>> components) that really should be included in comparisons (if
>> they don't match, the URNs don't either) and others that
>> should not (if they don't match, it isn't important and the
>> URNs are equal anyway).
> 
> I see the designation at this time that a component is not
> significant for equality comparison as risky.  If in the
> generic URN specification all components are significant, then
> NID definitions can later say that particular components are
> not significant.  But if we say in the generic URN
> specification that a particular component is not significant,
> then all NID definitions inherit that rule.

Yes, they do.   On the other hand, at least to my knowledge, no
one has yet shown a requirement for a p-component for any NID,
at least a requirement that cannot be satisfied by a q-component
or f-component.  I've argued for allowed p-components precisely
to allow a piece of syntax that makes a URN-wide comparison
distinction that is not covered by the NID and NSS.  If there is
no need to have such a distinction -- that all URNs are either
character-for-character identical (see 3986 Section 6.2.1) or
they will not compare equal by anything that doesn't understand
the details of the NSS -- then I don't see a need for the
syntactic clutter of allowing p-components at all.

Again, because of the generic comparison rules in 3986, a
processor that is not specific to the URN scheme is free to
treat just about anything it likes other than an exact
bit-for-bit comparison as not-matching so it isn't even clear
that the rule you are asking for has any practical effect.

So, unless you are thinking about example cases that haven't
occurred to me (in which case your explaining them would help),
what problem do you see with:

(i) Allowing p-components, remembering the RFC 2141 does not
allow them (or q- or f-components) at all.

(ii) Telling designers of new URN namespaces and associated
syntax that they are not allowed to use p-components unless they
want this particular comparison behavior.   If they want the
comparison behavior you think appropriate, they should confine
themselves to the use of q- and/or f-components (or avoid
components in addition to the NID:NSS combination entirely).

It seems to me that allows namespace URN syntax designers
maximum flexibility without costing us any functionality, but
maybe I'm missing something.

There _is_ another difficulty wrt p-components, which is
associated with the way the syntax of RFC 3986 is defined.  If
you look at Section 3 of that document, and compare the
"hier-part" syntax in its first paragraph with the syntax in
Section 3.3 and the examples in the third paragraph of Section
3, you will discover that...

	(i) There is no ABNF production with <path> on the RHS
	but the beginning of Section 3 has <path-abempty>,
	<path-absolute>, <path-rootless>, and <path-empty> on
	the RHS of <hier-part>.  Then Section 3.3 has <path> on
	the LHS and five separate "<path->* definitions on the
	RHS.  The second paragraph of Section 3 does say "five
	different ABNF rules..." and points to 3.3, so it is
	plausible to believe that the omission of
	<path-noscheme> was just an error in the <hier-part>
	list, but it then seems to invoke an RHS
	pattern-matching operation that is definitely not
	allowed by anything I can find in RFC 5234.    Given the
	absence of a RHS pattern matching in ABNF, and since
	<path> has no RSH anchor, it is not clear that
	<path-noscheme> is valid for use in URIs at all or how
	it connects to the <hier-part> syntax if it is.
	
	(ii) The examples and small piece of artwork identify
	URNs as being part of the "path", with no authority.  If
	we accept that, then Peter's p-component is really not
	the whole path, but only the part of the path that
	follows the first slash if it is present, i.e., that the
	generic syntax for the URN scheme is 
	   "urn:"NID ":" NSS ["/" p-component] ["?" q-component]
	["#" f-component] 
	and not 
	   "urn:" path  ["?" q-component] ["#" f-component]
	as one might infer from 3986.

> I am willing to be swayed if somone proposes that a
> *particular* component should be non-significant, and everyone
> agrees that that is so.  But in a discussion of this high
> level of generality, I don't see that happening.

First, note that 3986 requires that "two URIs that differ only
by the suffix "#" are considered different regardless of the
scheme".  While that very global statement is buried in Section
6.2.3, it can be read as applying to all comparisons that are
not strictly character by character equal.   On the other hand,
if what is really means is that two URIs, one of which has no
"#" and the other of which has "#" and an empty fragment (which
3986 appears to allow because <fragment> in not in brackets on
the RHS of <uri> but has an RHS starting in "*" and not, e.g.,
"1*" on its RHS in Section 3.5), and two URNs with  different
non-empty fragements might still be the same on a scheme basis,
then we might define it either way for URNs. [1]

But I think what is being proposed is exactly that, for
comparison):

 * The NID is significant
 * The NSS is significant
 * Any p-component that is present is significant (of
	course the NID definition may make any URN-string
	containing a p-component invalid)
 * As part of what 3986 describes as Scheme-based
	Normalization, any q-component that is present is not
	significant (with the same comment about the NID -- two
	URNs comparing equal or not equal doesn't make either
	one valid).
 * If we follow what I infer from the above is the intent
	of RFC 3968 (see above), any f-component that is present
	is significant.  If not, we can treat it either way but
	probably should be clear about it.
	
> significant is the safe choice with regard to future

Yes.  But is is also the worst choice with regard to
functionality and to any URN that really wants comparison to
succeed, regardless of the value or presence or absence of some
of those components.  See the remarks about 3986 above, but also
note that Juha and others have made what I believe is a strong
case for some syntax out on the right end of a URN, I think
particularly fragments, not counting toward matching whether two
URNs are equivalent.  We can push it down and specify that on a
per-NID basis... or maybe we can't, because:

People in this WG should take a careful look at the
recently-published RFC 7320, which appears to me to impose very
strong constraints on what can be specified below the scheme
level.   If that is a correct reading, then we really cannot
make comparison rules per-NID and, if comparison is important,
may be in deep trouble (or at have to justify violating an
alleged best practice).   I didn't comment on it earlier because
I really don't have time to follow even this WG carefully and
because, when it was under active discussion, I assumed it
wouldn't apply to URNs [2].

best,
    john

[1] FWIW at this point, it is this exact type of discussion,
including the one about relative references (which I had not
spotted earlier) that led me to believe six months ago that the
only way for the URN effort to move forward was to completely
separate it from whatever 3986 said or whatever people could
read into it.  I understand and agree with the principle that a
generic URI parser should be able to parse a URN, at least to
the extent represented by the <urn> production in 3986.   But,
as we are taken into territory in which we have to worry about
whether something is permitted (or prohibited) on an NID basis
rather than for all URNs or about whether 3986 requires support
for particular operations, it seems to me that 3986 and
discussions about it are impeding actual discussion about URN
requirements, rather than moving us forward.  As we came out of
the Toronto discussion, I hoped, as did others, that "URI syntax
only" would untie the knot without the far more drastic approach
of "URNs are now URIs".  But I'm now starting to wonder.

---

From: Julian Reschke <[email-masked]>
Date: Mon, 20 Oct 2014 21:50:15 +0200

> philosophical disagreement.

I wouldn't call it "philosophical".

> 	basic parsing of <uri> into components is irrelevant."

I think that would be misleading, as relative referencing *does* happen.

> seriously bad specification.

It would be helpful if you could propose a less "immoral" way to resolve 
relative references that is both scheme-agnostic *and* consistent with 
what the world expects from widely deployed URI schemes (such as FTP and 
HTTP).

> this relative reference problem.

Could you clarify what the actual "problem" is that you are referring to?

Best regards, Julian

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Mon, 20 Oct 2014 14:18:01 -0600

> what the world expects from widely deployed URI schemes (such as FTP and

Morality aside, I don't understand why it is the responsibility of the 
URNBIS WG to define a scheme-independent and consistent way to resolve 
relative references in URIs.

Consider the transition from RFC 2368 to RFC 6068 for the "mailto" 
scheme. RFC 6068 doesn't talk about relative references, and it surely 
doesn't attempt to clarify RFC 3986 in this regard. All it does it 
modernize the definition of the "mailto" scheme from the pre-3986 world 
to the post-3986 world. Why would the transition for the "urn" scheme be 
any different in this respect?

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: Andrew Newton <[email-masked]>
Date: Mon, 20 Oct 2014 16:34:30 -0400

> relative references in URIs.

I agree. It is not the job of this WG to fix generic URIs problems.
Relative referencing seems to be problematic for many URI schemes. If
we'd like to clarify how it works with URNs, that is a different
matter.

-andy

---

From: Julian Reschke <[email-masked]>
Date: Mon, 20 Oct 2014 23:34:09 +0200

> relative references in URIs.

I agree.

My comment was with respect to John:

"If 3986 actually requires that any system evaluating URIs apply 
relative reference processing even when it will yield bizarre and 
meaningless results, then I would contend that is a seriously bad 
specification."

I read between the lines that he had some idea how this could have been 
done in a better way; that's why I was asking for a concrete proposal.

> any different in this respect?

The reason why we got here was Dale Worley saying:

"This strikes me as consonant with how the IETF operates in many
situations.  There aren't many operations that need to work
consistently on URNs qua URNs:  What is the scheme?  What is the
namespace?  A universal syntax restriction (so all URNs can be parsed
from their contexts).  A universal equality test (that may yield false
negatives but must not yield false positives).  We need to ensure that
those features work consistently within a framework that lets
interested parties explore alternative solutions to the problems on
which we clearly have no consensus.  "

To which I replied:

"Well, if URNs can be used everywhere URIs are used, there is at least 
one more common operation which needs to be defined somewhere: 
resolution of a reference against a base URN."

And yes, I failed to mention that this *is* defined by RFC 3986, so we 
just need to make sure we don't either screw that up.

Best regards, Julian

---

From: Sean Leonard <[email-masked]>
Date: Mon, 20 Oct 2014 16:48:09 -0700

> I agree. It is not the job of this WG to fix generic URIs problems.

+1

> _______________________________________________
> urn mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/urn

---

From: Sean Leonard <[email-masked]>
Date: Mon, 20 Oct 2014 17:13:21 -0700

On Oct 19, 2014, at 1:04 AM, Julian Reschke <[email-masked]> wrote:

> But what if the relative reference is "#a" (and assuming the HTML contains that anchor)? Still meaningless?

Having taken a brief hiatus from this thread, I see that it has now blown up. :)

Let’s be clear: RFC 3986 says that a relative reference (Section 4.2) can have some (including none or all) of: p-component (which may or may not be / delimited), q-component, and f-component (fragment).

Relative references only make sense in certain kinds of resources that are identified by URIs.

If the resource is something like HTML and has well-defined fragment semantics, the fragment component alone, like #a, doesn’t even need to invoke URI resolution. It doesn't matter (practically) what the base URI is because navigation will occur entirely within the document, without resorting to URI dereferencing machinery.

Schemes like mailto: don’t identify resources that can contain relative references. Generally speaking, URIs can be used to launch computing processes or identify things of interest (not necessarily “resources” of interest). tel: identifies telephone numbers of interest and generally causes a telephony application to launch; mailto: only causes a new e-mail message to be created (its purpose really is not to identify e-mail addresses in any practical sense).

So too it is with URNs. The error in reasoning is to assume that a text/html “resource” is available at urn:ietf:rfc:3986. *If* such a resource were available, and the resource has a relative reference like <a href=“#a”>...</a>  and <a name=“a”>…</a>, then mechanically, urn:ietf:rfc:3986#a means something.

But urn:ietf:rfc:3986 doesn’t represent a text/html content like http does. You’re supposed to take the information blob “urn:ietf:rfc:3986” and URN-resolve it to something else (possibly the text/html document itself, in which case, I would argue that the base URI is nothing at all). It is like saying “If I am standing on a cloud and am told to go one step left, where will I be standing?” You can’t stand on clouds in the sky.

To try to bring this back on-point, it is probably best if URNs are not considered suitable for base URIs. Just like mailto: .

Think of a URN as a compact universal nickname, shortcut, or pointer to somewhere else. *Anywhere* else. Just not itself.

It is an interesting thought experiment whether q-components have meaning in the context of the URN scheme. Then <a href=“?getmimetype=application/pdf#page=6”>Turn to page 6 in the paper version of this RFC</a> could make sense. The thought experiment is defused by saying that URNs can’t be base URIs. Since a URN doesn’t serve as a base URI—just a pointer to other things—you wouldn’t write that <a> reference anyway.

Sean

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Mon, 20 Oct 2014 19:52:13 -0600

On 10/20/14, 6:13 PM, Sean Leonard wrote:
> Let’s be clear: RFC 3986 says that a relative reference (Section 4.2)
> can have some (including none or all) of: p-component (which may or
> may not be / delimited), q-component, and f-component (fragment).
>
> If the resource is something like HTML and has well-defined fragment
> semantics, the fragment component alone, like #a, doesn’t even need
> to invoke URI resolution. It doesn't matter (practically) what the
> base URI is because navigation will occur entirely within the
> document, without resorting to URI dereferencing machinery.
>
> Schemes like mailto: don’t identify resources that can contain
> relative references. Generally speaking, URIs can be used to launch
> computing processes or identify things of interest (not necessarily
> “resources” of interest). tel: identifies telephone numbers of
> interest and generally causes a telephony application to launch;
> mailto: only causes a new e-mail message to be created (its purpose
> really is not to identify e-mail addresses in any practical sense).
>
> So too it is with URNs. The error in reasoning is to assume that a
> text/html “resource” is available at urn:ietf:rfc:3986. *If* such a
> resource were available, and the resource has a relative reference
> like <a href=“#a”>...</a>  and <a name=“a”>…</a>, then mechanically,
> urn:ietf:rfc:3986#a means something.
>
> But urn:ietf:rfc:3986 doesn’t represent a text/html content like http
> does. You’re supposed to take the information blob
> “urn:ietf:rfc:3986” and URN-resolve it to something else (possibly
> the text/html document itself, in which case, I would argue that the
> base URI is nothing at all). It is like saying “If I am standing on a
> cloud and am told to go one step left, where will I be standing?” You
> can’t stand on clouds in the sky.
>
> To try to bring this back on-point, it is probably best if URNs are
> not considered suitable for base URIs. Just like mailto: .
>
> Think of a URN as a compact universal nickname, shortcut, or pointer
> to somewhere else. *Anywhere* else. Just not itself.
>
> It is an interesting thought experiment whether q-components have
> meaning in the context of the URN scheme. Then <a
> href=“?getmimetype=application/pdf#page=6”>Turn to page 6 in the
> paper version of this RFC</a> could make sense. The thought
> experiment is defused by saying that URNs can’t be base URIs. Since a
> URN doesn’t serve as a base URI—just a pointer to other things—you
> wouldn’t write that <a> reference anyway.

To address this and other points, I propose that we make the following 
change to the section "Handling of URNs by URI Processors" in 2141bis:

OLD
    The URN syntax has been defined so that URNs can be used in places
    where URIs are expected.

NEW
    Because a URN is, syntactically, a URI under the "urn" scheme, in
    theory a URN can be placed in any protocol slot that allows for a URI
    (e.g., an XML namespace name [XML-NAMES]).  However, this does not
    imply that, semantically, it makes sense in practice to place a URN
    in a given URI protocol slot; in particular, because a URN does not
    specify the location of a resource, it is not appropriate to place a
    URN in a URI protocol slot that points to a resource (examples
    include the 'href' and 'src' attributes and the <base/> element in
    HTML, as well as the 'xml:base' attribute in XML [XML-BASE]).

Where the (informational) references are as follows:

[XML-BASE] = http://www.w3.org/TR/2009/REC-xmlbase-20090128
[XML-NAMES] = http://www.w3.org/TR/2009/REC-xml-names-20091208

I would include an informational reference for HTML, but at this point I 
have no idea what to cite as a stable reference for the HTML 
specification...

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: Julian Reschke <[email-masked]>
Date: Tue, 21 Oct 2014 07:40:21 +0200

>     specify the location of a resource, it is not appropriate to place a

...until a resolver is defined.

>     HTML, as well as the 'xml:base' attribute in XML [XML-BASE]).

I'm ok with "doesn't make sense", but you're still avoiding the answer 
to "what if it *is* used that way". Can we just clarify that this 
doesn't overrule RFC 3986?

> have no idea what to cite as a stable reference for the HTML

I hear that the W3C is trying to get HTML5 to Full Recommendation before 
the end of the year.

Best regards, Julian

---

From: Sean Leonard <[email-masked]>
Date: Mon, 20 Oct 2014 23:08:01 -0700

>    HTML, as well as the 'xml:base' attribute in XML [XML-BASE]).

Despite my rather strongly opinionated series of e-mails, I actually 
disagree with this new text. Let me explain.

Because URNs have a sort of "wildcard" nature about them, they are very 
useful as a permanent identifier for an abstract resource that might 
change over time. Therefore, even though it does not make sense for a 
web browser to receive an HTML page with a purported base of 
urn:ietf:rfc:3986, or link to <a href="urn:ietf:rfc:3986">, or whatever, 
it makes sense for a content management system to record those URNs in 
those places for dynamic substitution when served.

Consider, for example, a book recommendation service like Wikipedia. 
Wikipedia as a non-profit institution probably does not want to be 
accused of bias by promoting amazon.com links over bn.com links (or 
whatever bookstore is popular in some local market). Rather than 
recording hard links to amazon.com AND bn.com and fifteen other 
bookstores and Google Books and iBooks and whatever, Wikipedia can just 
record <a href="urn:isbn:978-0345803481"> in the content. When served, 
Wikipedia can dynamically substitute the URNs for actual, locatable URIs 
(URLs) appropriate to the client.

A "URN Resolver" service might be said to be the missing link that 
performs this just-in-time service! Yay.

That at least deals with the issue of outbound links. Using a URN in a 
<base> element is a bit harder to justify off-hand...the overall point 
being, however, that it might be okay as long as it is substituted for a 
true URL that is appropriate to the destination context at the 
appropriate time. (Which, by the way, is probably a good reason to use 
this <base> stuff as sparingly as possible in the first place, since a 
resource retrieved by a functioning URL will supply a healthy dose of 
context.)

Sean

---

From: [email-masked] (Dale R. Worley)
Date: Tue, 21 Oct 2014 12:22:56 -0400

> the same way as any other URI, and that that behavior is defined by RFC 

I haven't thoroughly checked all the relevant text, but my reading of
RFC 3986 in regard to the processing of relative references against a
base URI that conforms to RFC 2141 yields this:

- The relative reference consists of a "path" part alone, and that
  path consists of one "segment" (because it does not contain '/').

- The base URI consists of a "scheme" part and a "path" part alone,
  and the path consists of one "segment".

- The target (what the relative reference resolves into) is the scheme
  of the base URI with the path part of the relative reference.

That is, the target URI is the same as the relative reference, with
"urn:" prepended.

So it seems to me that it's pretty unlikely that anyone has attempted
to use in practice relative references relative to a urn: base URI.
(Although it appears to be possible in principle.  If a document was
named by a URN, and retrieved by some mechanism, that URN could be the
base URI of relative references in the document.)

If we were to extend 2141 to allow "query" and "fragment" parts -- or
anything we wanted to consider to be analogous to those -- the direct
application of RFC 3986 says that the quasi-query and quasi-fragment
parts of the target URI would be copied from those of the relative
reference.

Which leaves us in the situation where a relative URI provides
essentially no additional semantic power.

(All of this stems from the fact that, in the notation of RFC 3986
sectoin 5.2.2., (1) R.scheme is undefined (i.e., this is "really" a
relative reference, because it has no scheme part), (2) R.authority is
undefined (because URNs have no authority part), (3) R.path is
non-empty (because the NID is part of the path), (4) R.path does not
start with '/', and (5) Base.path does not contain '/'.  Given those
facts, the resolution algorithm collapses into essentially "T = R".)

Now, given this as background, it's not clear to me what you're
pushing for.  It seems that the current situation is that RFC 3986
applies to relative references against a urn: base URI, and there's no
reason for anyone to ever exploit that.  And that the current
proposals don't seem to change either fact.  I suspect that I'm
overlooking something.

Dale

---

From: John C Klensin <[email-masked]>
Date: Tue, 21 Oct 2014 13:18:51 -0400

--On Monday, October 20, 2014 23:08 -0700 Sean Leonard
<[email-masked]> wrote:

> Because URNs have a sort of "wildcard" nature about them, they
> are very useful as a permanent identifier for an abstract
> resource that might change over time. Therefore, even though
> it does not make sense for a web browser to receive an HTML
> page with a purported base of urn:ietf:rfc:3986, or link to <a
> href="urn:ietf:rfc:3986">, or whatever, it makes sense for a
> content management system to record those URNs in those places
> for dynamic substitution when served.
> 
> Consider, for example, a book recommendation service like
> Wikipedia. Wikipedia as a non-profit institution probably does
> not want to be accused of bias by promoting amazon.com links
> over bn.com links (or whatever bookstore is popular in some
> local market). Rather than recording hard links to amazon.com
> AND bn.com and fifteen other bookstores and Google Books and
> iBooks and whatever, Wikipedia can just record <a
> href="urn:isbn:978-0345803481"> in the content. When served,
> Wikipedia can dynamically substitute the URNs for actual,
> locatable URIs (URLs) appropriate to the client.
> 
> A "URN Resolver" service might be said to be the missing link
> that performs this just-in-time service! Yay.
> 
> That at least deals with the issue of outbound links. Using a
> URN in a <base> element is a bit harder to justify
> off-hand...the overall point being, however, that it might be
> okay as long as it is substituted for a true URL that is
> appropriate to the destination context at the appropriate
> time. (Which, by the way, is probably a good reason to use
> this <base> stuff as sparingly as possible in the first place,
> since a resource retrieved by a functioning URL will supply a
> healthy dose of context.)

Sean,

Whether the text Peter proposes is adequate or not is another
question, but it seems to me that your discussion is mixing
"resolve relative references to base URIs", "resolve some UNRs
in various creative ways", and a certain amount of DWIM activity
together.

We also many need some new terminology to avoid confusing
ourselves because forms of "resolve" at used for too many
different things.

  urn:isbn:978-0345803481

is just a name or, if you prefer, an embedding of a name.  The
only thing it "resolves to" globally is
    ISSN: 978-0345803481

Unless there is mechanism in HTML5 that I don't know about, "<a
href...>" isn't nearly powerful enough to do what you want.
Moreover, especially if we want the web to be global, the notion
of Wikipedia being able to "resolve" an ISBN URN to a list of
one or more booksellers on the basis of the <a> element alone
leads us into bad trouble.  What you are looking for would be
more reasonably represented by something akin to 

http://www.favority-bookseller-list.wikipedia.com/?urn="isbn:1-4012-9876-1"

Which is a perfectly valid http-URL.  See Appendix B.3 of
draft-ietf-urnbis-semantics-clarif-00.txt for more discussion of
that general model

More generally, and noting 3986's comments about the loose
binding between "scheme" and "protocol", the thing that makes
the "urn" scheme different from most or all other URIs (and
certainly from the URL ones) is that, for the others, the
resolution mechanism is bound to the scheme name, often in
conjunction with the authority.  In the URN case, the scheme
just tells you that everything else is a name, there is no
authority, and the NID (which a hypothetical generic URI
processor could not parse out or otherwise recognize as more
than just part of a no-authority path) cannot be bound to a
resolution mechanism because:

(1) The hypothetical generic URI processor doesn't work that
way; it can use schemes at most because of the 3986
scheme-dependent opacity rule.

(2) Unless we develop a whole different theory about what URNs
and NIDs are about, any binding of an NID to a resolver must be
essentially universal.  In other words, Wikipedia cannot, at
least without significantly more information that I wouldn't
characterize as "resolution" map
 
    urn:ietf:rfc:3986

as an href argument of an <a> element into either

http://www.favorite-bookseller-list.wikipedia.com/?isbn=1-4012-9876-1

or 

   http://bn.com/?isbn=1-4012-9876-1

while Juha maps the same thing into a shelf reference and
someone else maps it to the actual ISBN registry record.

Expecting those to work requires a huge dose of DWIM pixie dust
or what would essentially be an HTML preprocessor that would
seek out hrefs with urn schemes and map them into something else
(or some family of other things).

We _do_ expect URNs to be used in exactly that way, but some
mechanism has to be used to specify how they are to be
dereferenced (not necessarily "resolved") to something else.
We've talked about that mechanism as part of an extended
URN-string (i.e., as part of a p/q/f-component) and about
expressing it in the embedded style mentioned above and
discussed in the  Appendix B of "semantics-clarif-00".   I've
probably forgotten some other "resolution" discussions -- there
have been a lot of them (although not as many distinct ones).
Context, as in an information retrieval system used to access
specific types of things or that collects information to do its
work beyond the URN itself, works too.   But HTML <a>, at least
unless HTML5 contains provisions for magic, just doesn't.

And, AFAICT, none of the above has anything to do with relative
references except to further illustrate why they really make no
sense, or at least no consistent and predictable sense, for URNs.

     john

p.s. I also believe Dale's "T = R" conclusion and that Andy's
consensus note yesterday and draft-ietf-urnbis-semantics-clarif
make discussion of relative references essentially moot for URNs
unless someone wants to explicitly introduce and require them.

---

From: Sean Leonard <[email-masked]>
Date: Tue, 21 Oct 2014 11:03:32 -0700

On 10/21/2014 10:18 AM, John C Klensin wrote:
> Unless there is mechanism in HTML5 that I don't know about,

This is not about HTML5. Nor is it about being particularly creative. It 
is about processes using syntactically valid inputs and transforming 
them in such a way that is appropriate for other consumers.

urn:isbn:978-0345803481 is a stand-in for any resource related to a book 
or publication identified by the ISBN. If you have a server side process 
that resolves it into a http: URL (universally compatible) or a data: 
URL (only compatible with the majority of web browsers since ~2009), the 
client web browser will be able to do something useful with it. If you 
know that the client specifically handles urn:isbn namespaces, you can 
send it to the client and the client will do its preferred thing with it.

I don't see an operational difference between this and, say, charset 
capabilities. If a server has text in some charset (iso-2022-jp) and 
knows that the client doesn't support iso-2022-jp, it can transform the 
text into a form that the client can understand.

>   "<a
> leads us into bad trouble.

It's not done on the basis of the <a> element alone. It's done with 
additional information in the surrounding context or environment--what 
RFC 2276 identifies as "hints".

>    What you are looking for would be

No, because:
a) that HTTP URL has a hard-coded authority of 
"www.favority-bookseller-list.wikipedia.com";

b) how can the operator of "www.favority-bookseller-list.wikipedia.com" 
store its content in a universal way, even if the operator changes its 
host name or path structure? A very convenient strategy is to use 
urn:isbn:... directly internally, and transforming that into appropriate 
references that a client can understand when the content (HTML) leaves 
the server.

URNs are (or can be) effectively the "environment variable" $(ENV_VAR) 
of URIs. The specific location can change based on context but the name 
"ENV_VAR" remains the same.

> resolution mechanism because

We are in agreement here. URNs are perhaps most useful when they are not 
definitionally bound to any *particular* resolution mechanism. But they 
can be resolved. The resolution is context-dependent.

Sean

---

From: Julian Reschke <[email-masked]>
Date: Tue, 21 Oct 2014 20:40:46 +0200

> proposals don't seem to change either fact.  I suspect that I'm

What I'm pushing for is a spec that is sufficiently clear that the 
relative resolution algorithm defined by RFC 3986 indeed applies to URN 
URIs, even though the results it yields for most URN schemes might not 
be very useful.

Best regards, Julian

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Tue, 21 Oct 2014 13:42:05 -0600

> URIs, even though the results it yields for most URN schemes might not

I propose the following text:

    Despite the fact that URNs are not hierarchical and are not
    appropriate for use as a base URI (see Section 5.1 of [RFC3986]), the
    relative resolution algorithm specified in Section 5.2 of [RFC3986]
    still applies to the "urn" URI scheme; implementers need to be aware,
    however, that running the algorithm against URNs can lead to results
    that are unexpected or not useful.

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: Julian Reschke <[email-masked]>
Date: Tue, 21 Oct 2014 22:05:59 +0200

>     that are unexpected or not useful.

+1.

Thanks!

---

From: John C Klensin <[email-masked]>
Date: Tue, 21 Oct 2014 19:19:23 -0400

--On Tuesday, October 21, 2014 07:40 +0200 Julian Reschke
<[email-masked]> wrote:

>>     URN in a URI protocol slot that points to a resource
>>     (examples include the 'href' and 'src' attributes and the
>>     <base/> element in HTML, as well as the 'xml:base'
>>     attribute in XML [XML-BASE]).
> 
> I'm ok with "doesn't make sense", but you're still avoiding
> the answer to "what if it *is* used that way". Can we just
> clarify that this doesn't overrule RFC 3986?

Julian,

This will be my last note on the subject until after I've
absorbed Peter's new version and text (he is holding the pen for
this one at least), but...

(i) I think draft-ietf-urnbis-semantics-clarif already overrides
3986 in this area.   If it is what the WG wants to do, I won't
like it but don't have a problem with a document that associates
relative reference processing with URNs, describes how to do it
(possibly by reference to 3986 and I hope at least consistent
with it) and mentions that it may lead to silly states
sometimes.   But my hope in Toronto and with the
semantics-clarif document has been that we could put all
arguments based on "we have to do that to conform to 3986" to
rest unless the issue is strictly about syntax (and not
processing or interpretation of that syntax).

(ii) The answer to "what if it is used that way [anyway]" is
that weird things will happen, things that both 3986 and, I
hope, the URN specs will simply treat as invalid and that things
that encounter them will need to deal with.  I would have the
same answer if someone asked "what if someone writes 'urn::bar'
or  "urn:/foo-".  I don't know whether they are relative
references or not (but see Dale's analysis), but I'm pretty
confident that, absent some preprocessor that has rules for
doing string processing that can make up an NID and maybe an
NSS, they are going to trigger an error somewhere.

The proposed text that Peter just posted meets the above
criteria as far as I'm concerned and so I can live with it, but
I'm still very unhappy about it. 

--On Tuesday, October 21, 2014 11:03 -0700 Sean Leonard
<[email-masked]> wrote:

>...
> I don't see an operational difference between this and, say,
> charset capabilities. If a server has text in some charset
> (iso-2022-jp) and knows that the client doesn't support
> iso-2022-jp, it can transform the text into a form that the
> client can understand.

The difference (and the reason I brought HTML and <a> up) is
that we have all sorts of mechanisms around the web for
specifying charset alternatives and even a new "encoding" spec
working its way through the WHATWG and W3C processes that, among
other things, specifies what to do in cases where client and
server capabilities (or what is specified in HTML and what the
client knows how to display) don't match.    On the other hand,
if one is in an environment in which the originating client and
the destination server cannot negotiate in real time (like
email) the arrival of a message that uses a charset that the
destination system doesn't understand (remembering that knowing
enough about a charset to map it to another one is fairly far
along the "understanding" curve) will result in some sort of
error state (perhaps such as the recipient seeing complete
nonsense of having her head explode).  There may be less
difference between that view and URNs than you expect, but it is
a different view from the one you describe above in which, as I
understand it, the server knows (or can find out) the
capabilities at the client and munge things around until they
match.

>...
> URNs are (or can be) effectively the "environment variable"
> $(ENV_VAR) of URIs. The specific location can change based on
> context but the name "ENV_VAR" remains the same.

I think this gets us to the difference between our points of
view.  As I understand it, you are seeing (at least some) URNs
as placeholders that get bound to other information at the
latest possible point and then transformed or, if you will,
resolved.  I see them (or at least some of them) as very stable
and very real identifiers that often won't be either resolved or
turned into (or "resolved" into) any thing else.  

The range of things that people seem to want to do with URNs are
such that we are probably both right.  I note that pure
indicators that are never resolved (and for which "resolved" is
clearly not meaningful) fit with my perspective, but not well,
and perhaps don't fit yours at all (see below).  These
combinations make text hard to write even without the
complications of being expected to conform ot pieces of 3986
that, IMO, are better matched to locators that are not only
expected to be resolved but are self-contained wrt resolution
mechanisms.

> We are in agreement here. URNs are perhaps most useful when
> they are not definitionally bound to any *particular*
> resolution mechanism. But they can be resolved. The resolution
> is context-dependent.

Well, some of them can be resolved.  Some can't unless one picks
a really creative definition of "resolved" and, in any event,
are not intended to be.

But, if someone chose to make a relatively straightforward
statement like "URNs represent a significantly higher level of
abstraction than [most other] URI"  I would agree (and be fairly
uninterested in debating whether the words in brackets belong
there or not).   Modulo those words, I think the above indicates
that you would agree too.

    john

---

From: Julian Reschke <[email-masked]>
Date: Wed, 22 Oct 2014 08:10:07 +0200

> processing or interpretation of that syntax).

Well, I continue to disagree on that point.

> that encounter them will need to deal with.  I would have the

I don't think RFC 3986 treats anything as invalid here.

> references or not (but see Dale's analysis), but I'm pretty

They are not.

"If the URI-reference's prefix does not match the syntax of a scheme 
followed by its colon separator, then the URI-reference is a relative 
reference." -- <http://greenbytes.de/tech/webdav/rfc3986.html#uri-reference>

> NSS, they are going to trigger an error somewhere.

That is possible.

Best regards, Julian

---

From: [email-masked] (Dale R. Worley)
Date: Wed, 22 Oct 2014 16:46:31 -0400

> URIs, even though the results it yields for most URN schemes might not 

OK, I think I understand you there.  But why do you care?  It would
seem to me better that we leave it in the state "Technically, RFC 3986
describes how you resolve a relative URI against a base urn: URI, but
it's very unlikely to be useful in practice, and probably nobody has
ever done it."

If we do that, we reserve the ability to say (if it becomes
necessary):

    [That] may not be backward-compatible with the specification, but it
    is backward-compatible with reality.  -- Francois Audet

Enhancing our commitment to a definition that seems completely useless
in practice isn't going to gain us anything.

*However*, if we introduce "/" in urn: URIs, then all of this changes:
Then the 3986 algorithm can do useful things.

Dale

---

From: Larry Masinter <[email-masked]>
Date: Thu, 23 Oct 2014 05:25:50 +0000

I'm hoping we can converge 3986 and url.spec.whatwg.org, see 

http://intertwingly.net/stories/2014/10/20/Url.xhtml
http://intertwingly.net/blog/2014/10/21/pegurl-js

Also, please separate "fragment" because to allow delegation of
fragment to the NID requires a substantial change to 3986,
Which currently already delegates fragment another way.

I think doing so is valuable, but I don't know if there is
consensus to make the needed changes to 3986 (to let
the scheme, and 'urn:' in particular, to delegate the
fragment to the representation returned by GET.

---

From: John C Klensin <[email-masked]>
Date: Thu, 23 Oct 2014 03:16:15 -0400

> fragment to the representation returned by GET.

Larry,

Just to be sure, I think the above means that you believe that,
despite the WG's decision to use 3986 syntax only and bypass the
semantics (see Andy's recent postings), you believe that we
cannot use fragments (or "f-components") without following the
3986 rules about how fragments are "delegated".  

Is that correct?   If it is, do you believe similar delegation
requirements for p-components and/or q-components?

FWIW, I just grepped 3986 and variations on the term "delegate"
appear to occur only in the following contexts:

 * A general statement about "Identifier" in Section 1.1
	that indicates that identification is delegated to each
	scheme specification (without any language that I can
	immediately find that restricts the scheme specification
	from delegating it further).
	
 * Discussion about Authority in Section 3.2 (including
	Host in 3.2.2).  Those discussions would appear to be
	irrelevant to URNs because they do not use an Authority
	component, at least as the syntax is defined in 3986.

That is all.   "Delegation" terminology is not used about
fragments (or queries, etc.) at all.  

Again, FWIW, this sort of thing, including usage in discussions
of it by one of it listed authors, is one of the reasons why
some (perhaps many) of us find 3986 and its terminology and
coverage confusing.

     john

---

From: Larry Masinter <[email-masked]>
Date: Thu, 23 Oct 2014 14:45:22 +0000

> Just to be sure, I think the above means that you believe that, despite the WG's
> decision to use 3986 syntax only and bypass the semantics (see Andy's recent
> postings), you believe that we cannot use fragments (or "f-components")
> without following the

> requirements for p-components and/or q-components?

My belief is that currently the scheme's definition has control
over the syntax (within 3986 limitations) and complete control
over the interpretation of / and ? delimited parts
 ('p-components' and 'q-components') but that, currently,
 the scheme's definition cannot define the syntax or interpretation
of any # delimited part (the f-component).

> Again, FWIW, this sort of thing, including usage in discussions of it by one of it
> listed authors, is one of the reasons why some (perhaps many) of us find 3986
> and its terminology and coverage confusing.

I apologize profusely for our failure of clarity for those trying
to do a close reading of the exact words of the text. I am not
among those who believe 3986 is perfect and should be carved
into stone tablets or engraved on nickel disks. 

In this case, section 3.5 of RFC 3986 uses "defined by X" and
"dependent on X" and not "delegated to X", an oversight
which I admit might be confusing to some.

Humbly,

Larry
--
http://larry.masinter.net

---

From: John C Klensin <[email-masked]>
Date: Thu, 23 Oct 2014 11:45:58 -0400

(top post) 
Larry,

Thanks for the clarification.  

While I believe that the 3986 properties of fragments are
semantics and not part of the syntax and that distinction was
part of the motivation for Peter's introducing "f-component",
just as I believe that questions of the perceived 3986
requirement for evaluating relative references are semantics and
not syntax, I think I'm now clear on what you are asking for and
why.  That is a big help.

best,
   john

--On Thursday, October 23, 2014 14:45 +0000 Larry Masinter
<[email-masked]> wrote:

>> Just to be sure, I think the above means that you believe
>> that, despite the WG's decision to use 3986 syntax only and
>> bypass the semantics (see Andy's recent postings), you
>> believe that we cannot use fragments (or "f-components")
>> without following the
>  the scheme's definition cannot define the syntax or
> interpretation of any # delimited part (the f-component).
> 
> 
>  
>> FWIW, I just grepped 3986 and variations on the term
>> "delegate" appear to occur only in the following contexts:
>> 
>> Again, FWIW, this sort of thing, including usage in
>> discussions of it by one of it listed authors, is one of the
>> reasons why some (perhaps many) of us find 3986 and its
>> terminology and coverage confusing.
> 
> I apologize profusely for our failure of clarity for those
> trying to do a close reading of the exact words of the text. I
> am not among those who believe 3986 is perfect and should be
> carved into stone tablets or engraved on nickel disks. 
>

---

From: Keith Moore <[email-masked]>
Date: Thu, 30 Oct 2014 16:12:17 -0400

> specifications that define specific NIDs. 

I am emphatically opposed to this proposal.  I find it completely 
unacceptable to allow different NIDs to define different semantics for 
these things.   That would essentially force software to treat every NID 
differently, to implement special code for each NID, and likely result 
in very spotty implementation.   In other words, it would circumvent the 
entire point of URNs.

Keith

---

From: Keith Moore <[email-masked]>
Date: Thu, 30 Oct 2014 16:19:31 -0400

On 10/21/2014 02:40 PM, Julian Reschke wrote:
> relative resolution algorithm defined by RFC 3986 indeed applies to 
> URN URIs, even though the results it yields for most URN schemes might 
> not be very useful. 

RFC 3986 is fundamentally broken in numerous ways, especially (but not 
exclusively) as related to URNs.  The URN WG can't fix that, but it 
should point it out and leave the actual fixing to others.   The 
alternative - to try to reconcile reality to RFC 3986 - is a fool's errand.

Keith

---

From: Julian Reschke <[email-masked]>
Date: Thu, 30 Oct 2014 21:36:37 +0100

> alternative - to try to reconcile reality to RFC 3986 - is a fool's errand.

It would be helpful if there was more than hand-waving; a *concrete* 
list of things you think are broken, please.

Best regards, Julian

---

From: John C Klensin <[email-masked]>
Date: Thu, 30 Oct 2014 19:25:22 -0400

--On Thursday, 30 October, 2014 16:19 -0400 Keith Moore
<[email-masked]> wrote:

> RFC 3986 is fundamentally broken in numerous ways, especially
> (but not exclusively) as related to URNs.  The URN WG can't
> fix that, but it should point it out and leave the actual
> fixing to others.   The alternative - to try to reconcile
> reality to RFC 3986 - is a fool's errand.

I would not have said "reconcile" but rather to force reality
into RFC3986's mold (or straightjacket), but otherwise I agree. 

"Point it out, bypass it for URNs, and move on" was exactly what
the "URNs are not URIs" approach tried to take.  The WG didn't
like it, so I thought (and still think) that "syntax only" went
far enough in that direction and that it was what was agreed to
in Toronto.   Given that, I continue to be confused by positions
that seem to start from "3986 imposed this non-syntax
requirement, so URNbis must conform".

    john

---

From: John C Klensin <[email-masked]>
Date: Thu, 30 Oct 2014 19:37:13 -0400

--On Thursday, 30 October, 2014 21:36 +0100 Julian Reschke
<[email-masked]> wrote:

> It would be helpful if there was more than hand-waving; a
> *concrete* list of things you think are broken, please.

Julian,

I've supplied several entries for that list in separate notes.
I think others have too.  My impression is that Larry (as a
co-author) has even agreed to a few of the issues I and others
have raised.  From my perspective, the two "separation" drafts
were both attempts (agreed to by the WG) to permit URNs to move
forward without putting "revise 3986" or even "reach agreement
on the details of what is wrong with 3986" into the URN critical
path.

Once we get URNs under control, I'll be happy to find time to go
back through my notes and the archives and produce a draft of a
consolidated list.

Until then, while I assume it is not what you intend, this whole
discussion subtopic increasingly has the appearance of a
delaying tactic to keep the WG from accomplishing anything; a
"why you can't do that" discussion rather than part of a
constructive "what do URNs need and how is that accomplished"
one.  And, until Andy explicitly suggests otherwise, I'm
finished with trying to have it.

     john

---

From: John C Klensin <[email-masked]>
Date: Thu, 30 Oct 2014 19:51:41 -0400

--On Thursday, 30 October, 2014 16:12 -0400 Keith Moore
<[email-masked]> wrote:

> I am emphatically opposed to this proposal.  I find it
> completely unacceptable to allow different NIDs to define
> different semantics for these things.   That would essentially
> force software to treat every NID differently, to implement
> special code for each NID, and likely result in very spotty
> implementation.   In other words, it would circumvent the
> entire point of URNs.

Predictions of spotty implementation aside (and, IMO, it is too
late to prevent that) we are there already.    For example, we
can't talk about resolution because some URNs don't resolve.  We
can't talk about fragments or queries because we either find
ourselves back in discussions about what 3986 requires or
hopelessly bogged down about details or what is and is not
generic or, alternately, per-NID.  We could just say "no syntax
beyond 2141" but that takes us straight to loss of all IETF
control because some important communities are convinced that
they need some of that added syntax, are perfectly capable of
adopting their own (if necessary conflicting) standards, and
ignoring all IETF philosophy discussions, including those about
3986, while doing so.

I've said this before but it seems to me that we have three
choices:

(1) Adopt a proposal very much like Peter's, ignoring "3986
won't allow that" discussion that move beyond the ability of a
basic URI parser to parse, and let others flesh out details on a
per-NID basis.  That would require several of us to accept a
conclusion that is, at best, "least bad".

(2) Conclude that we can't reach consensus on anything and shut
the WG down.

(3) Continue going around these circles forever.

The second and third have the same outcome --others adopting
their own standards or calling things URNs, probably without
even a consolidated registration and description model, and even
more probably with complete disregard of what you, me, or the
IETF think.   And those decisions would likely be made on a
per-NID, or per set of NIDs, basis in forums in which neither
you nor Julian (as representatives of what seem to be opposite
positions today) are likely to have a voice.

I'd rather grit my teeth, wish we could have done better, and go
with (1) or a close variation on it. 

YMMD but, again, be careful that you aren't ultimately forcing a
position that y9u'd like even less.

     john

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 31 Oct 2014 08:55:06 +0900

>> URN URIs, even though the results it yields for most URN schemes might

The relative resolution algorithm applies to any and all schemes 
(including URNs) where it is applied. I'm not aware of any URI scheme 
spec that mentions this explicitly. I'm not sure that needs to be called 
out in the URN spec, or in other scheme specs or registration templates. 
It would look very repetitive.

If you think it should, it would make sense to add this to the 
requirements. You should make a comment to that effect on 
https://tools.ietf.org/html/draft-ietf-appsawg-uri-scheme-reg-04 on the 
Apps WG list.

In practical terms, I think that if an actual document that gets 
returned when somehow resolving an URN contains relative references, the 
producers of that document will quickly learn to add a <base> element 
(or whatever the equivalent in formats other than HTML).

> alternative - to try to reconcile reality to RFC 3986 - is a fool's errand.

Can you please be specific? Otherwise, we end up with the situation that 
everybody thinks it's broken, but in different ways. Which in other 
words may mean that it's just a reflection of rough consensus.

Regards,   Martin.

---

From: Keith Moore <[email-masked]>
Date: Thu, 30 Oct 2014 19:58:45 -0400

On 10/30/2014 07:55 PM, "Martin J. Dürst" wrote:
> (including URNs) where it is applied.

Sorry, that's just insane.  It makes no technical sense whatsoever. 3986 
is simply incorrect, and needs to be obsoleted and move to Historic.

Keith

---

From: Keith Moore <[email-masked]>
Date: Thu, 30 Oct 2014 20:02:52 -0400

> (3) Continue going around these circles forever.

I don't accept that these are the only alternatives, but either (2) or 
maybe even (3)  :-) is better than (1).   A bad recommendation is worse 
than none at all.   There's nothing inherently wrong with URNs as they 
are now; the only problems are that (a) 3986 is misleading in the 
extreme and (b) URNs as they are now don't do everything people want 
them to do.

I actually think that the WG's biggest problem is insufficient 
participation, so trying to have the discussion in a way that brings 
more people to the table seems to me like what needs to happen.   If 
that means ceding change control of URNs to another party, so be it, 
though I'd like for IETF to still be able to provide some technical 
guidance as to what makes sense on a protocol level.

Keith

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 31 Oct 2014 10:19:34 +0900

For the record, I agree with Larry re. leaving out fragment 
identifiers/f-components.

<aside type='important'>
To Larry's and RFC 3986's defence, words like "defined by X", "dependent 
on X", and "delegated to X" are very close (if not synonyms) in this 
context.

Also, they are words with general meaning, where the value of carefully 
agreeing to a single term and even maybe defining formally it is low 
when compared to the effort to agree on one term and its definition. 
Also, the fact that these are words that can just be read with their 
everyday meaning means that even co-authors such as Larry won't 
necessarily remember the exact word and insist on its use. That on the 
other hand means that a simple grep won't be enough. Looking at the 
appropriate section (3.5 Fragments) would have worked. And I can also 
highly recommend it for other aspects of RFC 3986, as well as of course 
other RFCs. That's why we have sections, and TOCs.
</aside>

Regards,   Martin.

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 31 Oct 2014 10:40:00 +0900

[discussing this strictly with respect to URNs]

> is simply incorrect, and needs to be obsoleted and move to Historic.

Please note that I wrote "where it is applied". And indeed, if I create 
an HTML document with <base href='urn:isbn:...'>, and in the same 
document have some relative URIs, then if I activate one of these 
relative URIs, indeed the relative resolution algorithm (or some browser 
variation thereof) will be applied.

The result of that may be called "useless" or "insane" depending on your 
choice of vocabulary, but it's what will happen. And it's fair game to 
blame the document author.

Regards,   Martin.

---

From: Keith Moore <[email-masked]>
Date: Thu, 30 Oct 2014 21:52:30 -0400

On 10/30/2014 09:40 PM, "Martin J. Dürst" wrote:
>
> Please note that I wrote "where it is applied". And indeed, if I 
> create an HTML document with <base href='urn:isbn:...'>, and in the 
> same document have some relative URIs, then if I activate one of these 
> relative URIs, indeed the relative resolution algorithm (or some 
> browser variation thereof) will be applied.

With a document named exclusively by a URN, there's essentially zero 
chance that it's applicable.   It's difficult enough to make and mantain 
persistent identifiers for single resources. Trying to make persistent 
identifiers with persistent relationships between resources embedded in 
those identifiers is extremely difficult and almost certainly to fail.

> The result of that may be called "useless" or "insane" depending on 
> your choice of vocabulary, but it's what will happen. And it's fair 
> game to blame the document author. 

I don't think so.   It's not wrong for a document to have multiple 
names, one being a URN and another being a URL.   Nor is it the document 
author's fault if a document that contains relative URLs is given a URN 
as one of its names.   The safest thing to say about such relative URLs 
in the context of URNs is that they are extremely unlikely to be 
resolvable when the document is referenced entirely as a URN, without 
any intervening URN to URL resolution step.   If on the other hand there 
is a URN to URL resolution step, there's at least one context in which 
the relative URL potentially makes sense - but the result of applying 
that relative URL to the base URL is still a URL, not a URN.

Keith

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 31 Oct 2014 14:25:08 +0900

Hello Keith,

> those identifiers is extremely difficult and almost certainly to fail.

That may well be the case, and would be good advice to heed by everybody 
involved.

> as one of its names.

I'm not sure you are aware of this, but the <base> element is *inside* 
the document. Of course there are various ways to get it in there, 
including complicated CMS stuff, but in the wider sense, it's the 
author/originator that got it there. My advice would be that if the 
document has an URN and an URL, then in the <base> element use the URL. 
In the example I gave, that's not followed.

> The safest thing to say about such relative URLs

I agree. I haven't checked URN resolvers, because I don't know any, but 
I'd expect that it would return the document with something like an http 
Content-Location: header field, which could contain an URL, and that URL 
should be taken as the base for the document if there isn't any <base> 
information inside the document. (Julian should be able to tell us 
whether Content-Location: indeed works, or is supposed to work, like this.)

Regards,   Martin.

---

From: Keith Moore <[email-masked]>
Date: Fri, 31 Oct 2014 01:35:08 -0400

On 10/31/2014 01:25 AM, "Martin J. Dürst" wrote:
> That may well be the case, and would be good advice to heed by 
> everybody involved.
>
> document has an URN and an URL, then in the <base> element use the 
> URL. In the example I gave, that's not followed.

Yes, sorry I was interpreting "base URI" in a more general context, 
rather than taking your example literally.  I'm sure you were being 
precise, and I was interpreting your example more broadly than you intended.

For the specific case of a <base> element containing a URN, and the same 
document containing references to relative URIs, I would essentially 
call that a syntax error at least for those references.    A validator 
should mark those references as invalid.   (I mean, if you can detect 
such errors at their source, why wait until document evaluation time?)

> I agree. I haven't checked URN resolvers, because I don't know any, 
> but I'd expect that it would return the document with something like 
> an http Content-Location: header field, which could contain an URL, 
> and that URL should be taken as the base for the document if there 
> isn't any <base> information inside the document. (Julian should be 
> able to tell us whether Content-Location: indeed works, or is supposed 
> to work, like this.)

Nothing (that I'm aware of) says that a resolver has to use HTTP, or if 
it uses HTTP, that it has to use its response header fields in this 
way.   I would prefer a richer representation of resource description 
than is accorded by HTTP response header fields, even if in most 
situations it's going to be subsetted.   But in general, resolving a URN 
to a URL has always been at least one of the anticipated mechanisms for 
gaining access to resources named by URNs.

Keith