Subject: [urn] urnbis 14-seantek text (incorporates discussions)

From: Sean Leonard <[email-masked]>
Date: Mon, 16 Nov 2015 13:09:59 -0800

URNies:

Please find a revision of urnbis-rfc2141bis attached (if these attachments get accepted), which you can refer to as the “14-seantek” (or “14s”) revision. I used the XML version posted directly on ietf.org and edited that. I am also including the output of rfcdiff.

This is what I want to see added to the draft, based on discussions over the last ~4 to 6 months. People (notably the chairs) have been asking to “send text” so here is the text.

Key points:
* (informative) resolution-to-resources model discussed, illustrating interaction of assigned-name, r-component, q-component, f-component
* fixed the ambiguity between q-component and r-component
* r-component syntax and semantics defined
* a variety of small editorial fixes
* updated text/uri-list registration
* fixed and improved ABNF

Please: read the text before commenting on these points. If you don’t like the direction, take it up with the text, and *send different text*.

Thank you and I look forward to productive discussion.

Kind regards,

Sean

PS If you want to use rfcdiff <https://xml2rfc.ietf.org/rfcdiff>, use this URL for the text source:

https://dl.dropboxusercontent.com/content_link/mi4xwZzmQgkdmK94enMHvod9ptoSUPOpnRBgATxa9iYoAWNiB1TBoSZuWYy9NsgR/file

PPS I also have a brainstorming scratch text file where I put extensive notes on why and how the syntax is derived to be as it is in the 14-seantek draft. If people want to see it I can send it or paste it in. Symbols were carefully chosen to eliminate ambiguity, ease parsing, maximize compatibility, and maintain compatibility with existing APIs.

---

From: John C Klensin <[email-masked]>
Date: Fri, 20 Nov 2015 16:54:01 -0500

--On Monday, November 16, 2015 13:09 -0800 Sean Leonard
<[email-masked]> wrote:

> URNies:
> 
> Please find a revision of urnbis-rfc2141bis attached (if these
> attachments get accepted), which you can refer to as the
> "14-seantek" (or "14s") revision. I used the XML
> version posted directly on ietf.org and edited that. I am also
> including the output of rfcdiff.
> 
> This is what I want to see added to the draft, based on
> discussions over the last ~4 to 6 months. People (notably the
> chairs) have been asking to "send text" so here is the
> text.
>...

Sean,

Partially as an explanation of why you (and the WG) are going to
see two or more responses from me to other notes before I
respond in more detail or with changes to the draft related to
this one, while I really appreciate your good intentions and
willingness to supply text, please do not, in future, do so by
posting an alternate document.  It doesn't give me (or Peter, or
the WG or its leadership) enough information about what problem
you are trying to solve and why you believe the proposed new
text helps and, because other changes have already been made to
the working draft, it creates a more complex and time-consuming
merge requirement than is really needed.

This should probably be a private, off-list, note, but I think
it may be helpful for the WG to see it to get a better sense of
what is going on (and to more quickly tell me I'm out of line if
I am).

There are several of your changes that I really like even though
I'm nervous about teasing out intent and unanticipated changes
in some cases and have major issues in some others.  Others are
more difficult.

As an example, your proposed ABNF is dependent on
draft-seantek-abnf-more-core-rules-04.    I really don't have an
opinion about that document, but I have a very strong preference
that we avoid normative references to I-Ds that are not under
control of the WG because such references regularly turn into
another block to timely publication.  I can probably fix that
for this case, but it implies more work and creates a new risk.
Both Peter and I have said "I am not an ABNF expert" several
times.   I can't speak for Peter, but, to be completely candid,
my problem with ABNF stems from a deep dislike of it that is, in
turn, based on many years of work with metalanguage design and
application to formal language definition and standardization.
>From that point of view --which I will freely admit is partially
about aesthetics and religious conviction rather than clear
technology boundaries-- ABNF is reasonably good for descriptive
purposes but fairly bad for definitional ones.  It gets worse
when people think "regular expression matching" and then try to
write ABNF, with 3986 and its "will match one of the following"
construction unfortunately being a poster child for that
problem.   

It is almost independent, but, when I'm looking at that small
bit of text, I completely agree with your "N.B." comment.   But
I think that, if we try to address it, or even get entangled
with it, in 2141bis, we go straight down a rathole from which
their may be no escape (if you are interested in the problem,
try to have a look at the W3C i18n "encoding" draft -- I don't
remember if there is a reasonably recent public version, but, if
you can't find it, write me offlist and I'll try to track it
down and work something out).

In any event, if you provide the WG with variant ABNF that
doesn't quite work with 5334 or that raises extra issues with
3986, it just about guarantees a slow response (and,
unfortunately, my having "can't deal with that right now"
reactions and finding something else to do).

Your "exemplary model" approach is, IMO, elegant and provides
one way to deal with the "metacomment" note I'll be posting
shortly (to some small extent, it contributed to it).   However,
that approach is rarely taken in the IETF and I've seen
documents shot down because they said "this is one example of
how to do it but other examples are conforming as long as they
produce similar or identical results".   So I think it
ill-advised to splice it into the document without a discussion
with Andy and the WG and more or less explicit sign-off from
Barry and maybe the other ARD ADs.  I've got some similar
concerns about, e.g., normative dependency on X.660.

In other places, you've rewritten whole sections and taken the
document into places, such as with your "rop" model, that the WG
has not, IMO, discussed at all, much less agreed on.

So I'm going to pull out what I can from the text for -15.  I've
already spent about six hours trying to go through it and
separate the editorial changes from major new proposals and
directions.  That feels like a lot of time.  Especially if no
one else seems inclined to comment in detail on the new bits, I
thing I need instructions from Andy as to how to proceed with
the rest.

Thanks again for the attempt and considerable useful text even
if I think you want a bit off into areas that are unnecessary or
inappropriate to the WG's scope.  

best,
   john

---

From: Sean Leonard <[email-masked]>
Date: Fri, 20 Nov 2015 16:55:04 -0800

On 11/20/2015 1:54 PM, John C Klensin wrote:
> willingness to supply text,

Ok in that case I will hold off on any further responses until you are 
finished with your e-mail batch. Thanks.

Note however that the "problems trying to be solved" are 
straightforward, as they are covered in the Key Points.

As it stands, urnbis-14 is significantly unimplementable. (I have tried 
to implement URNs before, and have a lot of experience with implementing 
URI handlers and processing URIs in various contexts.) No matter what 
the WG agrees or disagrees about things like "??" vs "?" or whatever, if 
the spec is not implementable, nobody is going to use it (and in 
particular, use it to build interoperable systems). The overriding goal 
of this effort, therefore, has been to make the unimplementable parts of 
the spec actually implementable. "Implementable" means not only 
literally that a system can be implemented, but that systems can be 
implemented to do something actually useful, and can interoperate.

Here are the Key Points, with associated Problems:

> PROBLEM: people are talking about resolvers and resolution without an implementable model of what is actually going on. Need an implementable model, even if that is not the only model.
> PROBLEM: can't see where q-component and r-component begin or end. Not implementable.
> PROBLEM: Keith Moore, myself, and others (Juha Hakala perhaps 1/2? they can speak for themselves), want to see the r-component actually **do something**. The term "facet" has been used, along with "profile". An r-component with no structure or semantics is not implementable (see above).
> PROBLEM: In an effort to propose an implementable system, the system needs a data model of how URNs turn into "final" resources. RFC 2483 provided such a model. But text/uri-list is old. Some people (Keith Moore etc.) want attributes. So we could create some new thing, or use our existing thing and make it better.
> PROBLEM: Current ABNF too vague and not really implementable, certainly not in an unambiguous way

> PROBLEM: There are editorial problems. A specific list of editorial problems:

>> Changes to solve the above problems resulted in text that no longer makes sense. For example:
2.  The syntax of URNs assigned within the namespace, including
  whether q-components, r-components, or f-components are allowed.

has been changed to:
2.  The syntax of URNs assigned within the namespace, including the
  anticipated effects of q-components, r-components, and f-components.

...that is because: the namespace can't really *prohibit* things like f-components--the discussion about all that is already in the -14 text. For example, the namespace may identify certain kinds of things such as "despotic leaders in history" that, since whoever is administering the namespace never intended to resolve to Internet media type content, can't have fragments and therefore is not expecting f-components for urn:despotic:stalin. But someone actually implementing the URN can interpret it however he/she/it wants, including going to some content that has chapters or whatever where fragments are meaningful.

Another example:
Removed "Note that" in "NIDs are case insensitive..."
...that is because: It's nice to note things, but it's not a note. Case-sensitivity (or not) is a bedrock interoperability issue that affects the equivalence of urn: URIs.

>> Example URNs such as "urn:example:foo-bar-baz-qux#somepart" are terribly uninformative.

>   
> In other places, you've rewritten whole sections

Nope. Only subsections. In particular I only reordered q-component and 
r-component subsections (3.3.1 and 3.3.2). For that matter, the 
q-component section is 95% the same. Section 8.3 was added (not 
rewritten) for text/uri-list, but 95% of that is a copy-and-paste job 
from RFC 2483.

I suggest checking the rfcdiff HTML output on that one...

>   and taken the

Oh, it's been discussed all right. See last 2-6 months of messages...the 
syntax has some novelties/proposals, but the premise has been there for 
awhile. (Actually the reason for any "novelties" is because people were 
handwavy about implementability, so I needed to propose something that 
is actually implementable.) The main thing is that the chairs kept on 
saying, "Send Text, Send Text", so now it's sent. I acknowledge that 
agreement is not clear: that is the reason why I sent text so that 
people can see what such implementable systems would look like.

Best regards,

Sean

---

From: Andrew Newton <[email-masked]>
Date: Mon, 23 Nov 2015 13:34:14 -0500

On Fri, Nov 20, 2015 at 7:55 PM, Sean Leonard <[email-masked]> wrote:
> As it stands, urnbis-14 is significantly unimplementable. (I have tried to
> implement URNs before, and have a lot of experience with implementing URI
> handlers and processing URIs in various contexts.) No matter what the WG
> agrees or disagrees about things like "??" vs "?" or whatever, if the spec
> is not implementable, nobody is going to use it (and in particular, use it
> to build interoperable systems). The overriding goal of this effort,
> therefore, has been to make the unimplementable parts of the spec actually
> implementable. "Implementable" means not only literally that a system can be
> implemented, but that systems can be implemented to do something actually
> useful, and can interoperate.

In your opinion, is 2141 unimplementable as it stands today? Or is
2141bis making URNs unimplementable? Or is it that r-components are
simply unimplementable?

> handwavy about implementability, so I needed to propose something that is
> actually implementable.) The main thing is that the chairs kept on saying,
> "Send Text, Send Text", so now it's sent. I acknowledge that agreement is
> not clear: that is the reason why I sent text so that people can see what
> such implementable systems would look like.

And thanks for sending text.

I have a few questions. Do you believe your ROP proposal is necessary
for 2141bis, or can standardization of ROP wait until later?

-andy

---

From: Martin J. Dürst <[email-masked]>
Date: Tue, 24 Nov 2015 16:40:24 +0900

Hello Sean,

I have not really been following the URN work very closely lately, but I 
strongly suggest that you follow general IETF practice and submit your 
modified draft as a real Internet Draft.

Regards,   Martin.

On 2015/11/17 06:09, Sean Leonard wrote:
> _______________________________________________
> urn mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/urn
>

---

From: Sean Leonard <[email-masked]>
Date: Tue, 24 Nov 2015 02:19:47 -0800

On 11/23/2015 10:34 AM, Andrew Newton wrote:
> In your opinion, is 2141 unimplementable as it stands today?

2141 is implementable (and many implementations exist) for the purpose 
that it claims: universal durable identifiers.

That do nothing.

I.e., "resolution" in 2141 is not specified, so it's not part of the scope.

>   Or is
> 2141bis making URNs unimplementable?

As of 2141bis-14, the identification part is mostly fine. Stuff like "/" 
got included in identification, while "?" and "#" and such got turned 
into delimiters for other non-identification components, consistent with 
RFC 3986. Progress.

But the things that ? and # do, are not currently implementable. 
2141bis-14 is now including "resolving" in its scope, in a way that 2141 
does not. So, we have to say, at a minimum, what the parts are going to do.

>   Or is it that r-components are
> simply unimplementable?

The WG has come to an agreement that r-components, q-components, and 
f-components should be (and therefore are) distinct. Progress.

But for them to be implementably distinct, we have to say what they do, 
i.e., how they are different. If you say "ok these are q and r and f but 
by the way q and r, you can't really distinguish them, I guess they have 
?? but sometimes ?? might just be part of q or r", that is not 
implementable. Furthermore, what is the point of distinguishing q and r, 
if you throw both q and r at the "Resolver" and have the Resolver look 
at both of them? No point. May as well smush them back together.

To distinguish them, we have to say what they do: specifically, what 
*conformant implementations* SHALL do when they parse them out. If we 
can't say what they do then we aren't really saying much of anything at all.

Make sense?

> for 2141bis, or can standardization of ROP wait until later?
Yes, I believe "rop" is necessary.

For my part, I do not find q-components to be implementable in -14 or in 
-14-seantek.

However, people really like the idea of q-components as this part of the 
syntax that the resolver doesn't see, but some other component (which I 
conceptualize as the URI handler on the client side) sees, and manages 
to smush together with whatever comes back from the resolvers. (In 
-14-seantek, I refer to this "whatever comes back" as "products of the 
resolution services", which is handwavy, but gets the job done.)

Therefore, I left q-components mostly alone, to "Die Another Day". I 
mainly care that q-components stay out of r-components.

We need to strive for interoperability when defining r-component. That 
means that two different resolver implementations can look at a URN spec 
(the portion in its purview), get the r-component, and do the same thing 
with it. Leaving r-component as a blank canvas is not conducive to 
interoperability.

At a minimum, r-component needs to tell what the resolver to *do*. For 
two implementations to interoperate (or provide interoperable 
functionality), they both need to look at the r-component string and 
isolate the part that says to do something, and to interpret that part 
to do the same thing between the both of them.

Hence, resolution operation, or "rop". Not complicated. The rest of the 
details of "rop" boil down to identifier assignment choices.

The other parts of r-component are r-authority and r-params. Some people 
called for r-component to have name-value pairs (I believe Keith Moore 
was a/the chief proponent). So that's r-params. The r-authority 
production just seemed to flow out of syntax, and the desire to have 
predictable (and therefore, interoperable) functionality.

Best regards,

Sean

---

From: Andrew Newton <[email-masked]>
Date: Tue, 24 Nov 2015 14:05:06 -0500

On Tue, Nov 24, 2015 at 5:19 AM, Sean Leonard <[email-masked]> wrote:
>
> 2141 is implementable (and many implementations exist) for the purpose that
> it claims: universal durable identifiers.
>
> That do nothing.
>
> As of 2141bis-14, the identification part is mostly fine. Stuff like "/" got
> included in identification, while "?" and "#" and such got turned into
> delimiters for other non-identification components, consistent with RFC
> 3986. Progress.
>
> But the things that ? and # do, are not currently implementable. 2141bis-14
> is now including "resolving" in its scope, in a way that 2141 does not. So,
> we have to say, at a minimum, what the parts are going to do.

This is just my opinion, so feel free to disabuse me of any of it.
With regards to resolution, I believe your assessment of 2141 is
accurate. Resolution of URNs today requires software to key off of the
NID with namespace-specific behavior (though a lot of namespaces will
resolve with nearly identical behavior). In other words, there are a
lot of things software can do with all URNs, but resolution is not one
of them.

What I fail to understand is how r-components change that assumption.
That is, even with r-components the resolution of a URN requires
namespace-specific behavior. And that leads me to this question: why
do we need a set of standard r-components for all namespaces if
resolution is still a namespace-specific operation? Why can't the
r-components be namespace-specific since the resolution is also
namespace-specific?

-andy

---

From: John C Klensin <[email-masked]>
Date: Tue, 24 Nov 2015 15:26:04 -0500

Sean,

There are parts of this that I think I can respond to, more as a
long-time participant in this effort than as co-editor.  Other
parts go more to WG operation and strategy; for those I need to
defer to Andy.

--On Tuesday, November 24, 2015 02:19 -0800 Sean Leonard
<[email-masked]> wrote:

>>> As it stands, urnbis-14 is significantly unimplementable. (I
>>> have tried to implement URNs before, and have a lot of
>>> experience with implementing URI handlers and processing
>>> URIs in various contexts.) No matter what the WG agrees or
>...
> 2141 is implementable (and many implementations exist) for the
> purpose that it claims: universal durable identifiers.
> 
> That do nothing.
> 

But we have been trying very hard to not define "resolution" in
2141bis.  To the extent to which we have been doing so, it is
because attempts to define q-components and r-components (and
maybe f-components) to everyone's satisfaction have ended up
with statements about "resolvers" and "resolution functions".
I'd like to think that is a difficulty with the way we have been
constructing those definitions rather than trying to define
resolution in a way that doesn't work.    Suggestions welcome
(but see below).

> As of 2141bis-14, the identification part is mostly fine.
> Stuff like "/" got included in identification, while "?" and
> "#" and such got turned into delimiters for other
> non-identification components, consistent with RFC 3986.
> Progress.
 
> But the things that ? and # do, are not currently
> implementable. 2141bis-14 is now including "resolving" in its
> scope, in a way that 2141 does not. So, we have to say, at a
> minimum, what the parts are going to do.

Let me try to recast that a little differently and see if you
(and others) can agree.  Especially if q-components, like
f-components, are to be treated as opaque strings, we need to
specify what happens to them and/or where they go in the cases
(NID-denoted namespaces) where they are relevant at all.  That
is, btw, consistent with RFC 3986, which makes general comments
about what fragments and queries are and might be used for, but
carefully avoids specifying internal syntax or semantics for
either, much less "what they are going to do".    

IMO, this gets tied up with the fundamental nature of URLs and
URNs.  For URLs for most schemes, we've got a piece of syntax
that is, more or less:

    Scheme:xxxxx?query-string@fragment-string

The scheme and hier-part imply, for many schemes, some sort of
object-finder and the information needed to find it.  The
query-string qualifies that operation, supplying additional
information about what is to be found or returned (3986, as I
read it, says less than that), and the fragment is applied to/
into the object itself.  Three abstraction layers with a
slightly-fuzzy boundary between the first two.

Now URNs, at least the subset of URNs that yield a URL (see the
case list in my recent response to Juha), add an extra layer to
that model.  For that extra layer, r-components have, in
principle, exactly the same relationship to the URN-> URL
process that queries have to the URL-> object process.  We have
been struggling to define that boundary better for a variety of
reasons and have talked about "resolvers" in the process, but
I'm not yet convinced that, in order to have URNs "work", we
need to deeply define either resolver functions or r-component
semantics... any more than 3986 defines query semantics.

> The WG has come to an agreement that r-components,
> q-components, and f-components should be (and therefore are)
> distinct. Progress.
> 
> But for them to be implementably distinct, we have to say what
> they do, i.e., how they are different. If you say "ok these
> are q and r and f but by the way q and r, you can't really
> distinguish them, I guess they have ?? but sometimes ?? might
> just be part of q or r", that is not implementable.

Other than the issue of where they are applied and to what, how
is that different from 

   http://some-domain?chapter=5
versus
   http://some-domain#chapter5

> Furthermore, what is the point of distinguishing q and r, if
> you throw both q and r at the "Resolver" and have the Resolver
> look at both of them? No point. May as well smush them back
> together.

I agree with that, actually.  But suppose you say to the
"resolver":

	"Here is this r-component stuff and it is information
	for you to do with as you find appropriate
	(appropriateness possibly even depending on NID); here
	is this q-component stuff which you need to either treat
	as opaque and pass to something else or need to perform
	some well-described and only scheme-dependent lexical
	operations on and then pass the result to something else
	(note the resemblance between the latter and IRI
	processing)? "

In that same vocabulary, f-component becomes a string that is
completely opaque with not even that possible lexical
transformation.
 
> To distinguish them, we have to say what they do:
> specifically, what *conformant implementations* SHALL do when
> they parse them out. If we can't say what they do then we
> aren't really saying much of anything at all.
> 
> Make sense?

Yes, but, again, I'm not sure that what we have been trying to
do with them  and specify about them is any different from what
3986 says about queries and fragments.  I don't happen to think
that is really implementable either, but it sets the norms and
boundaries in which we need to operate. 

>>> Oh, it's been discussed all right. See last 2-6 months of
>>> messages...

I apologize but, while I saw a lot of comments, I didn't
understand that you were suggesting a typology and protocol
framework for what I've been describing as a universal resolver
(or structured family of them).   There is a lot about your
suggestions that I like and the specifics were a huge help in
the regard.  But I'm resisting getting it into the critical path
of this WG if only because of the purely pragmatic concern that
requiring agreement on it in order to move 2141bis forward will
doom both.

That, however, falls into Andy's territory.

>>> the syntax has some novelties/proposals, but the
>>> premise has been there for awhile. (Actually the reason for
>>> any "novelties" is because people were handwavy about
>>> implementability, so I needed to propose something that is
>>> actually implementable.) The main thing is that the chairs
>>> kept on saying, "Send Text, Send Text", so now it's sent. I
>>> acknowledge that agreement is not clear: that is the reason
>>> why I sent text so that people can see what such
>>> implementable systems would look like.

>> And thanks for sending text.

My thanks too, in case that hasn't been clear.

>> I have a few questions. Do you believe your ROP proposal is
>> necessary for 2141bis, or can standardization of ROP wait
>> until later?
> For my part, I do not find q-components to be implementable in
> -14 or in -14-seantek.
> 
> However, people really like the idea of q-components as this
> part of the syntax that the resolver doesn't see, but some
> other component (which I conceptualize as the URI handler on
> the client side) sees, and manages to smush together with
> whatever comes back from the resolvers. (In -14-seantek, I
> refer to this "whatever comes back" as "products of the

It does indeed, if we need to go down the path to defining a
"resolution service" and it actions.  FWIW, I think DDDS is more
or a "resolution service finder" than a "resolution service", so
it would be interesting to see how DDDS and ROP would mesh.

> Therefore, I left q-components mostly alone, to "Die Another
> Day". I mainly care that q-components stay out of r-components.

I think we agree about that too.  But see Juha's most recent
note and my response.  It may be _very_ hard to establish a
universal boundary that works for all URNs without constraining
URNs and their applications more than the WG has so far been
willing to do.

> We need to strive for interoperability when defining
> r-component. That means that two different resolver
> implementations can look at a URN spec (the portion in its
> purview), get the r-component, and do the same thing with it.
> Leaving r-component as a blank canvas is not conducive to
> interoperability.

I can perhaps agree... but that gets me back to both the
critical path issue and to the definitions of queries (and
fragments) in 3986.  What is specified is where they are
interpreted, not what they do, much less a requirement that they
be interpreted in the same way for different resources.

> At a minimum, r-component needs to tell what the resolver to
> *do*. For two implementations to interoperate (or provide
> interoperable functionality), they both need to look at the
> r-component string and isolate the part that says to do
> something, and to interpret that part to do the same thing
> between the both of them.

I think that is true iff one requires that it be possible to
apply multiple, independently-developed, resolvers to the same
namespace and expect to get the same result.  That was not a
requirement of 2141; if we have made it a requirement in
2141bis, I suggest that was a writing/editing error.

> Hence, resolution operation, or "rop". Not complicated. The
> rest of the details of "rop" boil down to identifier
> assignment choices.
> 
> The other parts of r-component are r-authority and r-params.
> Some people called for r-component to have name-value pairs (I
> believe Keith Moore was a/the chief proponent).

I think I'm at least as much to blame as Keith is, but it was
clear to me that the WG didn't like the idea, so I gave up.

>  So that's
> r-params. The r-authority production just seemed to flow out
> of syntax, and the desire to have predictable (and therefore,
> interoperable) functionality.

When you characterize ROP that way (I think there is more to it
and I'm pleased about that) a different option (with some
handwaving about quoting cnoventions) would be to define

   r-component = URI

since a URI also involves a choice of identifier, an authority,
and a set of things that can be thought of as parameters.   The
difficulty, as Juha pointed out in a slightly different context,
is that specifying all of that information in or with a URN is
bad for long-term stability (and maybe "persistence").

best,
    john

---

From: John C Klensin <[email-masked]>
Date: Tue, 24 Nov 2015 15:29:20 -0500

Andy (and Sean and WG),

I wrote most of my note about this before this arrived and
finished and sent it before seeing this.  FWIW, I agree with
Andy's analysis and conclusion even though, as my note should
make clear, I come at the question from a somewhat different
perspective.

    john

--On Tuesday, November 24, 2015 14:05 -0500 Andrew Newton
<[email-masked]> wrote:

>> 2141 is implementable (and many implementations exist) for
>> the purpose that it claims: universal durable identifiers.
>> 
>> That do nothing.
>> 
> This is just my opinion, so feel free to disabuse me of any of
> it. With regards to resolution, I believe your assessment of
> 2141 is accurate. Resolution of URNs today requires software
> to key off of the NID with namespace-specific behavior (though
> a lot of namespaces will resolve with nearly identical
> behavior). In other words, there are a lot of things software
> can do with all URNs, but resolution is not one of them.
> 
> What I fail to understand is how r-components change that
> assumption. That is, even with r-components the resolution of
> a URN requires namespace-specific behavior. And that leads me
> to this question: why do we need a set of standard
> r-components for all namespaces if resolution is still a
> namespace-specific operation? Why can't the r-components be
> namespace-specific since the resolution is also
> namespace-specific?
> 
> -andy

---

From: "Hakala, Juha E" <[email-masked]>
Date: Wed, 25 Nov 2015 08:51:59 +0000

Hello Andy, 

> -----Original Message-----
> From: urn [mailto:[email-masked]] On Behalf Of Andrew Newton
> Sent: 24. marraskuuta 2015 21:05
> Resolution of URNs today requires software to key off of the NID with
> namespace-specific behavior (though a lot of namespaces will resolve with
> nearly identical behavior). In other words, there are a lot of things software
> That is, even with r-components the resolution of a URN requires namespace-
> specific behavior. And that leads me to this question: why do we need a set of
> standard r-components for all namespaces if resolution is still a namespace-
> specific operation? Why can't the r-components be namespace-specific since
> the resolution is also namespace-specific?

If resolution (and services provided via resolution) were entirely namespace specific, applicability of RFC 2483 would have been very limited.  

Sure, each namespace may have 1-n resolution services which are - at least in the beginning - only supported by that namespace. But other namespaces may develop support for those services later. And there are generic services which may / will be supported in many if not most namespaces. Some of them are included in RFC 2483, including retrieval of the identified resource; retrieval of 1 or many URLs related to the resource, and retrieval of (descriptive and other kind of) metadata about the resource (admittedly RFC 2483 specifies this service in a way which is not really useful, since it is vitally important to specify the metadata format used). Since URNs are used in different technical environments the actual implementation of these generic services may vary (retrieval of resource metadata means different things for librarians and archivists), but different namespaces could still share the same r-component syntax to request these services from resolvers. For instance, a librarian might ask for metadata about the identified resource and specify in a parameter that Dublin Core metadata record syntax is preferred. An archivist could do the same, except that his preference would be EAD record syntax. A NASA engineer would probably prefer yet another record syntax, and so on. The service would still be the same, although details differ. And whatever protocol is then used to retrieve metadata from the library / archive information system is not relevant for URNBIS / URN syntax; mapping from URN + r-component to the URI(s) needed for supplying the service is something the resolver developers will cope with.  

URN implementers have already built resolvers which deal with multiple namespaces (ISBN and NBN, for instance). There are also resolver networks in which resolution related data is harvested from participating systems to "union resolvers". This kind of co-operation becomes more complicated than it needs to be if each namespace has its own guidelines for constructing r-components. From technical point of view, co-ordination of r-component syntax and semantics is both desirable and possible. But it is not necessary to specify r-component in details in 2141bis; that can be done later. Or, more precisely, we need an RFC or other standard document which specifies how to maintain information (syntax, semantics and if needed, pragmatics) about URN / persistent identifier resolution services. As long as applications with which digital resources are managed are developing fast, also the resolution services available (and their features) will change constantly, and the URN / PID community needs something more flexible than rfc2141bis or a mere facelift of RFC 2483 to keep up. 

Best regards,

Juha

---

From: Sean Leonard <[email-masked]>
Date: Wed, 25 Nov 2015 07:06:43 -0800

> r-components be namespace-specific since the resolution is also

r-components can be namespace-specific. r-components can also be 
namespace-agnostic. As with all things, it depends on what the rough 
consensus/trend is in this WG, which ought to be informed by real-world 
experience.

The trend is that resolvers (computing implementations that take in urn: 
URI parts) take the assigned-name part and the r-component part, but 
*not* the q-component or f-component parts. Resolvers return some 
"products", which can be 0-n URLs (see Juha's statements) but could be 
other types of metadata or whatever.

An implementation that is not a "resolver" can look at the assigned-name 
part and discard the rest, in order to extract information about the 
assigned-name, and compare assigned-names (aka URNs) for equivalence. 
This is obvious from Section 4.

It follows that the r-component is "resolution-specific"--how it is 
structured depends on what exactly we want resolution to *do*, and 
whether we want to build resolvers OR interoperable resolver networks 
that can resolve multiple/most/all assigned-names.

What we ought to appreciate is that a resolver sucks in the 
assigned-name portion as an input parameter, and then can do whatever it 
wants with it, such like a search engine that takes input of "ieTf 
rfc3986 site:ietf.org" and "IETF RFC 3986". They are different strings 
but will likely (ultimately) return the same information. A resolver may 
do the exact same thing (i.e., return the exact same results) with 
"urn:ISSN:XXX", "urn:ietf:rfc:YYYY", and "urn:doi:.....", because the 
resolver internally is smart enough to map all three of these to the 
same object.

Juha just wrote [I will respond to the rest of Juha's post later]:

Sure, each namespace may have 1-n resolution services which are - at least in the beginning - only supported by that namespace. But other namespaces may develop support for those services later. And there are generic services which may / will be supported in many if not most namespaces.

<<
and:

Since URNs are used in different technical environments the actual implementation of these generic services may vary (retrieval of resource metadata means different things for librarians and archivists), but different namespaces could still share the same r-component syntax to request these services from resolvers.

<<
and:

URN implementers have already built resolvers which deal with multiple namespaces (ISBN and NBN, for instance). There are also resolver networks in which resolution related data is harvested from participating systems to "union resolvers". This kind of co-operation becomes more complicated than it needs to be if each namespace has its own guidelines for constructing r-components.

<<

Yay! Implementation experience!

Making r-components *always* be namespace-specific makes them more 
complicated. Syntactically, we already have a namespace-specific part: 
the NSS. That should be enough. I hope we are clear now that there are 
resolution behaviors (aka, operations) that cut across many--possibly 
all--namespaces.

It is more modular, and therefore more implementable (I would state more 
plainly: implementable vs. unimplementable) when r-components are not 
*always* namespace-specific. In the majority of cases, their semantics 
should be orthogonal to the namespace in the assigned-name component.

Even if r-components are "namespace-specific", it's important to 
appreciate that a namespace that defines r-components (i.e., defines 
resolution operations) is engaging in a different kind of 
identifier-assignment process than its primary purpose, which is to 
define the NSS part. If a "namespace" for books defines an operation 
"getpdf", that is a different identifier and assignment than a namespace 
for books that defines the book identifier 99992929212.

(Syntactically, both can be accomplished with the same NID--this is 
something that is developed in draft-14-seantek.)

Sean

---

From: Andrew Newton <[email-masked]>
Date: Wed, 25 Nov 2015 11:05:51 -0500

So to paraphrase both Sean and Juha, while resolution may still be
namespace specific including the behavior of r-components, it is quite
helpful to implementers of resolution services for multiple namespaces
if the r-components had similar structure, etc...

Where Sean and Juha seem to differ is that Sean wants the r-components
standardized now and Juha sees that as something that can be done
later.

Let me know if I've misunderstood.

Anybody else care to comment?

-andy

---

From: Sean Leonard <[email-masked]>
Date: Wed, 25 Nov 2015 08:38:07 -0800

> standardized now and Juha sees that as something that can be done

I guess that is a reasonable paraphrase, at least of my part.

Consider the resolution operation "info", which I have defined in 
-14-seantek to mean "get information about the thing identified by the 
assigned-name component". Would we rather that urn:isbn:XXX?info, 
urn:ietf:rfc:XXX?info, urn:ietf:params:XXX?info, and 
urn:oid:0.1.2332?info return markedly different things? Use the 
principle of least surprise..."info" should have a roughly equivalent 
meaning across all different namespaces (or, as many different 
namespaces as possible), even if for many things identified in the 
namespaces, it's not applicable.

I would much rather do the standardization now, because (among other 
things) that way different resolvers can interoperate with each other as 
well as with non-resolver components (clients, etc.). Also, we have a 
proposed syntax and momentum to discuss it. If we don't get it done now, 
I fear it will be another 4-20 years before it happens. Just looking at 
the history on this thing...

Sean

---

From: John C Klensin <[email-masked]>
Date: Wed, 25 Nov 2015 12:08:16 -0500

--On Wednesday, November 25, 2015 11:05 -0500 Andrew Newton
<[email-masked]> wrote:

> So to paraphrase both Sean and Juha, while resolution may
> still be namespace specific including the behavior of
> r-components, it is quite helpful to implementers of
> resolution services for multiple namespaces if the
> r-components had similar structure, etc...

See the distinction between lexical standardization and various
flavors of semantic standardization below and see if that helps.

> Where Sean and Juha seem to differ is that Sean wants the
> r-components standardized now and Juha sees that as something
> that can be done later.

I think there is a middle ground, which is to standardize, not
all aspects of r-components, but enough of the syntax and
structure to make sure that future standardization of specifics
is possible.  I'm not happy about going down that path, if only
because I think we have been there before and backed away and
because I think it will be hard to get agreement, but it is a
possibility.

> Let me know if I've misunderstood.

I don't think so.  See above and below.

--On Wednesday, November 25, 2015 07:06 -0800 Sean Leonard
<[email-masked]> wrote:

Sean,

It is going to take me a while to assimilate parts of this and I
may not be able to do that until after the holiday, but a
comment on a subset of it below (I reserve the right to change
my mind after understanding the rest):

> r-components can be namespace-specific. r-components can also
> be namespace-agnostic. As with all things, it depends on what
> the rough consensus/trend is in this WG, which ought to be
> informed by real-world experience.

I'm going to switch terminology here because I'm not 100% sure I
understand what you are saying and what I'm about to say may be
at least partially orthogonal.   I would consider an r-component
to be "namespace-independent" if its interpretation is the same
for all namespaces.  That sameness can actually exist at any of
three (or four) different levels:

(i) Component Lexical: All r-components are structured the same
way and can be parsed and elements identified in the same way
for all namespaces.  The elements themselves may differ from
namespace to namespace.  "Element" is used loosely to describe
what r-components are parsed into; by definition, the term is a
function of the lexical operations and independent of namespace
as well.

(ii) Element Lexical: In addition to the component-level lexical
sameness, elements are reserved across namespaces s.t., for any
given namespace, a particular element (and any keywords or
structure) is either meaningless or has the same structure as it
does for every other namespace for which it is meaningful.
Different namespaces are, however, allowed to interpret the same
element in different ways either across the namespace or for
particular NID:NSS-identified content objects

(iii) Semantic: In addition to the component-level and
lexical-level sameness, elements are reserved across namespaces
s.t., for any given namespace, a particular element (and any
keywords or structure) is either meaningless or has the same
interpretation as it does for every other namespace for which it
is meaningful.  There is no such thing as namespace-specific of
object-specific interpretations: an element is expect to either
be interpreted in the One True, Standardized, way or it must be
treated as meaningless.

(iv) There is a fourth level in which every element is expected
to be meaningful for every assigned-name, but I think we figured
out long ago that wasn't practical if only because it introduces
nasty race conditions into registries and operations.

I believe that even the third case implies a dynamic registry
that every "resolver" can check in real time.   I think the WG
has considered that option in other contexts and concluded it is
impractical.  Given the wide range of content-objects and
intermediary locators and objects,  I also don't think it is
plausible, but YMMD.

(i) requires only that we define the internal syntax of
r-components.  That is not impossible, but something we have
avoided so far.  (ii) and (iii) imply that we need to
standardize and create a keyword (or element) registry.

Any elements or characteristics of r-component that don't
conform to one of those levels of "sameness" makes r-components
namespace-specific at that level and to at least some extent.

Now, one can have a combination by adopting the criteria of (i)
above and then either:

(a) Reserving some set of elements (or element keywords or
equivalent) and giving them standardized treatment and treating
any unreserved element as namespace-specific.  That is, one
level further down, the "some reserved keywords" approach that
drove us to the r-component/ q-component distinction and has the
same disadvantages or either requiring a non-extensible list or
a plan as to what happens with older implementations when more
elements are added to the reserved list.

(b) Making a lexical distinction, either between standardized
and namespace-specific elements (while the latter would not be
in any sense "experimental", a review of the IETF's experience
with "X-" headers and keywords and/or the boundary between
public and private port identifiers would probably be helpful in
considering that option) or between two types of r-components,
the ones that are expected to be generic and well-defined across
all namespaces for which they are meaningful and the ones that
are considered namespace-specific.   Independent of the syntax
chosen (including ordering), that takes us another step down the
path from 
   query       to
   q-component, r-component   to
   q-conponent, r1-component, r2-component

and I have to wonder whether it ends there or there are still
more divisions possible.

Another variation would allow something else that might be
interesting, which would be to structure either r-component
(more or less as now defined in 2141bis-14) or r1-component (as
above) so that a resolver standard or family could be identified
in some way that could be standardized.  If, instead of
expecting all resolvers to be, or conform to the syntax and
semantic assumptions of, ROP (or, for that matter, DDDSng) that
would bind the resolution mechanisms and r-component
interpretation to the URN-string rather than to the namespace
definition without, in the process, requiring that the same
mechanism be used for all namespaces (NIDs) or assigned-names.

> The trend is that resolvers (computing implementations that
> take in urn: URI parts) take the assigned-name part and the
> r-component part, but *not* the q-component or f-component
> parts. Resolvers return some "products", which can be 0-n URLs
> (see Juha's statements) but could be other types of metadata
> or whatever.

While I think implementation experience is very valuable and
important, I'm very concerned when one starts talking about
"trends" at a pre-standardization phase, if only because it
substitutes "we got here first and therefore you must
standardize whatever we decided to do" for analysis and
long-term and balanced thinking.

More when I have time and energy.

    john

---

From: John C Klensin <[email-masked]>
Date: Wed, 25 Nov 2015 12:22:00 -0500

--On Wednesday, November 25, 2015 08:38 -0800 Sean Leonard
<[email-masked]> wrote:

>...
> Consider the resolution operation "info", which I have defined
> in -14-seantek to mean "get information about the thing
> identified by the assigned-name component". Would we rather
> that urn:isbn:XXX?info, urn:ietf:rfc:XXX?info,
> urn:ietf:params:XXX?info, and urn:oid:0.1.2332?info return
> markedly different things? Use the principle of least
> surprise..."info" should have a roughly equivalent meaning
> across all different namespaces (or, as many different
> namespaces as possible), even if for many things identified in
> the namespaces, it's not applicable.
> 
> I would much rather do the standardization now, because (among
> other things) that way different resolvers can interoperate
> with each other as well as with non-resolver components
> (clients, etc.). Also, we have a proposed syntax and momentum
> to discuss it. If we don't get it done now, I fear it will be
> another 4-20 years before it happens. Just looking at the
> history on this thing...

Sean,

I would _really_ like to see something like the above.  But,
unless you have far better ideas about how to create a
comprehensive list of keywords that are appropriately treated
that way than I do, I don't see how to get there.   If the list
is open-ended (at least to the extent of allowing additions over
time), then assume, e.g., that we standardize "info" now and you
build it into a resolver at version 1.

Now suppose that resolver encounters  
    urn:isbn:XXX?zork    and
    urn:issn:xxxx?inf0

Are "zork" and/or "inf0" syntax errors?  Bad keyword errors? A
sign that it is time to update the resolver to version 2?  Some
race condition with mirrors of the registry?

And note that, if the resolver is to recognize and process it,
not just get itself into a "yes, I've see 'zork' before as an
action keyword but have no clue how to process it" state, and do
so by updating tables rather than changing code, it implies an
executable description language, not just a keyword registration
and definition pointer.  

I just don't think we understand the issues and the range of
possible namespaces well enough to do that.

 best,
   john

p.s. I'm probably now off until Sunday or Monday at least.

---

From: "Hakala, Juha E" <[email-masked]>
Date: Thu, 26 Nov 2015 08:42:00 +0000

Hello, 

> From: urn [mailto:[email-masked]] On Behalf Of John C Klensin
> Sent: 25. marraskuuta 2015 19:08
> To: Sean Leonard <[email-masked]>; Andrew Newton <[email-masked]>
> Cc: [email-masked]
> Subject: Re: [urn] urnbis 14-seantek text (incorporates discussions)
> 

Correct. 

> See the distinction between lexical standardization and various flavors of
> semantic standardization below and see if that helps.
> 

Right. There isn't much that we can do about q-components. They may be either standard based, like URIs based on SRU or other well-known search protocols, or proprietary.  Clients will find it difficult to maintain proprietary q-components even in the short term. As regards r-components we can and should avoid the situation in which each namespace has its own semantics and syntax for r-components. The challenge is that r-components can get quite complex when the services available multiply. 

Just to illustrate the problem: Sean has defined resolution service "info" (without parameters) with which it would be possible to request metadata (that is, information) about the identified thing. 

This might work in an environment where there is just one system which holds relevant information about the thing. But for instance my library has four systems: national bibliography for descriptive metadata, digital asset management system for the documents themselves, license database for rights metadata and digital archive for long term preservation purposes (and administrative metadata). So it is essential to define what kind of information the user wants, since that has an impact on which system the resolver should pass the request. So we would need both "info" and parameter or keyword specifying what kind of information is required. Several years ago Alfred Hoenes suggested on this list a syntax like 

s="service name"&p="parameter"

Each resolution service can have a of 0-n parameters specific to it. For info (or, to maintain backwards compatibility with RFC 2483, I2C or I2CS) these parameters could be for instance acronyms of metadata formats or metadata types or just "all". Then we would get something like 

s=I2C

if the client/user is happy with what is the resolver / namespace default,

s=I2C&p=dc

if the user wants descriptive metadata in Dublin Core, 

s=I2C&p=premis

if the user wants preservation metadata in PREMIS format from the digital archive, 

s=I2C&p=rights 

if the user wants licensing information and other rights metadata from the licensing system, and 

s=I2CS&p=all

if the user wants all the metadata / information there is about the thing (and there will be a lot).  

So in the community I represent, just asking for "info" will not do. Our resolvers must know more precisely what the client/user wants.  Other communities may have similar requirements for some other resolution services libraries, archives and museums regard as simple. For instance, just retrieving the identified resource is a challenge if a user wants a dataset containing hundreds of terabytes of data.      
 
> I think there is a middle ground, which is to standardize, not all aspects of r-
> components, but enough of the syntax and structure to make sure that future
> standardization of specifics is possible.  I'm not happy about going down that
> path, if only because I think we have been there before and backed away and
> because I think it will be hard to get agreement, but it is a possibility.

Like John, I am not happy if we choose this approach, because deciding exactly how much (and what) we need to say about r-components in rfc2141bis is not going to be easy. One option is to specify syntax based on for instance names of services and service related parameters as I have done above, and then illustrate this approach using the resolution service names from RFC 2483 (which does not know anything about service parameters). But the most important thing is to tell how the list of services and service parameters will be maintained. Trying to incorporate that too into 2141bis would delay us a lot. Besides, it would be good if other persistent identifier systems use the same resolution service names and parameters as URN when possible, but it will be harder to convince them to do that if all r-component related information is  "hidden" in 2141bis.   

> It is going to take me a while to assimilate parts of this and I may not be able
> to do that until after the holiday, but a comment on a subset of it below (I
> reserve the right to change my mind after understanding the rest):
> 
> >...
> > consensus/trend is in this WG, which ought to be informed by
> > real-world experience.
> 
> I'm going to switch terminology here because I'm not 100% sure I understand
> what you are saying and what I'm about to say may be
> to be "namespace-independent" if its interpretation is the same for all
> namespaces.  That sameness can actually exist at any of three (or four)
> different levels:
> 
> (i) Component Lexical: All r-components are structured the same way and can
> be parsed and elements identified in the same way for all namespaces.  The
> elements themselves may differ from namespace to namespace.  "Element"
> is used loosely to describe what r-components are parsed into; by definition,
> the term is a function of the lexical operations and independent of namespace
> as well.

This kind of syntactic interoperability would mean that if r-component contains e.g. a service name and service parameter, any resolver should be able to parse the string. But the resolver would not need to understand what is being required. 
> 
> (ii) Element Lexical: In addition to the component-level lexical sameness,
> elements are reserved across namespaces s.t., for any given namespace, a
> particular element (and any keywords or
> structure) is either meaningless or has the same structure as it does for every
> other namespace for which it is meaningful.
> Different namespaces are, however, allowed to interpret the same element in
> different ways either across the namespace or for particular NID:NSS-
> identified content objects

OK. We can use keyword "info" or "I2C" to enable clients to request information / metadata about the identified thing. But depending on the namespace or target system, metadata retrieved may differ. Only after the parameter has been added, level (iii) is reached and the interpretation should be always the same. 
> 
> (iii) Semantic: In addition to the component-level and lexical-level sameness,
> elements are reserved across namespaces s.t., for any given namespace, a
> particular element (and any keywords or structure) is either meaningless or
> has the same interpretation as it does for every other namespace for which it
> is meaningful.  There is no such thing as namespace-specific of object-specific
> interpretations: an element is expect to either be interpreted in the One True,
> Standardized, way or it must be treated as meaningless.

This level can be reached by r-components like "info&dc" or "s=I2C&p=dc" which specify exactly what is being requested. It is possible to add even more details; for instance, r-component like "s=I2C&p=MARC21&level=5" means that the client wants a complete MARC 21 record, not an abridged one. But only some namespaces or target systems will be able to deal with this level of specificity. It will be up to resolvers to know what kind of r-component usage is OK for each target system, since resolvers will map the URN + r-component into 1-n requests which appropriate target(s) can deal with. For instance, if client wants all metadata, multiple requests must be generated. 

> (iv) There is a fourth level in which every element is expected to be
> meaningful for every assigned-name, but I think we figured out long ago that
> wasn't practical if only because it introduces nasty race conditions into
> registries and operations.

Yes, we did. 
> has considered that option in other contexts and concluded it is impractical.
> Given the wide range of content-objects and intermediary locators and
> objects,  I also don't think it is plausible, but YMMD.

A global dynamic registry will  be too difficult to build. There are some limited areas where useful things can be done. For instance, we can harvest configuration information from libraries' SRU servers, and map them to r-components in URN resolvers. 

> (i) requires only that we define the internal syntax of r-components.  That is
> not impossible, but something we have avoided so far.  (ii) and (iii) imply that
> we need to standardize and create a keyword (or element) registry.

>From rfc2141bis point of view, specifying internal syntax might be within scope. But I am not sure if we are ready yet to do that.  Trying to specify keywords (elements) and their parameters a'la RFC 2483 is not practical. After URN syntax has been modernized, we should outline a mechanism with which the registry is maintained.  

> Any elements or characteristics of r-component that don't conform to one of
> those levels of "sameness" makes r-components namespace-specific at that
> level and to at least some extent.
> 
> Now, one can have a combination by adopting the criteria of (i) above and
> then either:
> 
> equivalent) and giving them standardized treatment and treating any
> unreserved element as namespace-specific.  That is, one level further down,
> the "some reserved keywords" approach that drove us to the r-component/ q-
> component distinction and has the same disadvantages or either requiring a
> non-extensible list or a plan as to what happens with older implementations
> when more elements are added to the reserved list.

Service mnemonics from RFC 2483 could and should be a starting point for creation of this set. In addition to that we need parameters, such as metadata formats for I2C and I2CS.  
> 
> (b) Making a lexical distinction, either between standardized and namespace-
> specific elements (while the latter would not be in any sense "experimental",
> a review of the IETF's experience with "X-" headers and keywords and/or the
> boundary between public and private port identifiers would probably be
> helpful in considering that option) or between two types of r-components, the
> ones that are expected to be generic and well-defined across all namespaces
> for which they are meaningful and the ones that
> chosen (including ordering), that takes us another step down the path from
>    query       to
> and I have to wonder whether it ends there or there are still more divisions
> possible.

Both services and service parameters can be experimental or internal. For instance, requesting information about things is going to be one of the most common services, but this information may be available only in a proprietary or experimental format.

Experimental services may eventually become commonplace, so we need a mechanism for upgrading the status of a service or a service parameter.  
> 
> Another variation would allow something else that might be interesting,
> which would be to structure either r-component (more or less as now defined
> in 2141bis-14) or r1-component (as
> above) so that a resolver standard or family could be identified in some way
> that could be standardized.  If, instead of expecting all resolvers to be, or
> conform to the syntax and semantic assumptions of, ROP (or, for that matter,
> DDDSng) that would bind the resolution mechanisms and r-component
> interpretation to the URN-string rather than to the namespace definition
> without, in the process, requiring that the same mechanism be used for all
> namespaces (NIDs) or assigned-names.
> 
> 
> > The trend is that resolvers (computing implementations that take in
> > urn: URI parts) take the assigned-name part and the r-component part,
> > but *not* the q-component or f-component parts. Resolvers return some
> While I think implementation experience is very valuable and important, I'm
> very concerned when one starts talking about "trends" at a pre-
> standardization phase, if only because it substitutes "we got here first and
> therefore you must standardize whatever we decided to do" for analysis and

When it comes to creation of smart URN resolvers in general and r-components in particular, nobody "has gotten there" yet. Other PIDs, most notably ARK and Handle, do provide some resolution services, but they have been implemented in a manner which cannot be utilized with URNs.  

There are no existing URN implementations with features discussed here. But there is an increasingly sophisticated technical environment with which electronic resources are managed, and URNs must fit into it. Libraries, archives and museums develop and maintain some of these systems, and if there is anything that makes us different from many other organizations active in this area it is probably the need to preserve digital resources for very long term (decades or even centuries). Persistent identifiers & functionality based on them will be essential in this work.

All the best, 

Juha

---

From: Sean Leonard <[email-masked]>
Date: Thu, 26 Nov 2015 04:06:51 -0800

On 11/25/2015 9:22 AM, John C Klensin wrote:
> I would _really_ like to see something like the above.

Great!

> But,
> that way than I do, [...]

Already did that.

See draft-14-seantek. Let's talk about that text, and in particular, 
discuss the merits of the different types of "rops". Let me know. (Hint: 
alpha-op, OID-op, URN-op, NS-op.)

> If the list
> Are "zork" and/or "inf0" syntax errors?

No

> Bad keyword errors?

Just unrecognized.

> A
> sign that it is time to update the resolver to version 2?

Maybe, or just add support for the operations. (How would this be 
different from asking an LDAP server implementation, to update its 
tables of recognized objects or recognized operations? No different. Can 
update the software or can update the configs that drive the software.)

> Some

Not a real issue. (Hint: compare alpha-op with OID-op.)

> executable description language, not just a keyword registration

If the output is going to be URIs/URLs, the "executable description 
language" can just be:
Regular Expressions
SQL or SQL-like queries
JavaScript

Happy Thanksgiving, everybody! (for those who celebrate it/the US version)

Sean

---

From: Sean Leonard <[email-masked]>
Date: Thu, 26 Nov 2015 04:35:16 -0800

> because I think it will be hard to get agreement, but it is a

New year, new model, new participants, new thinking.

> for all namespaces.

I am okay with "namespace-independent".

(Note: for the following text, I read Juha's response but I am going to 
split my response into two responses: this response handles the generic 
feedback, and the second will respond specifically to the interesting 
nuances or points that Juha raised.)

> That sameness can actually exist at any of
> (i) Component Lexical: [...]
>
> (ii) Element Lexical: [...]
>
> (iii) Semantic: [...]
> [even the third case implies a dynamic registry that every "resolver" can check in real time]
>
> to be meaningful for every assigned-name [...]

> [Defining] (i) requires only that we define the internal syntax of
> r-components.
>
> (a) Reserving [and standardizing] some set of elements (or element keywords or
> equivalent)
>
> in any sense "experimental", [...]
>     query       to
> [Another variation...ROP...]

I am getting the sense that either the -14-seantek text was not read, 
or, the text was not clear enough about how this r-component stuff works 
(perhaps my fault).

Juha mentioned Alfred Hoenes' suggestion, a syntax like:

The problem with that is that you have to deal with the (i), (ii), 
(iii), and (iv) stuff (and (a) and (b) sub-stuff) that John just wrote 
up. Namely, you have to reserve or standardize some set of elements and 
element keywords. On top of that, you have to reserve or standardize 
service names.

The proposal on the table with -14-seantek is:

rop ["@" r-authority ] ":" r-params

where r-params is comprised of *arbitrary* name/values parameters 
defined by the rop.

Therefore, no keywords for parameters need to be defined. There is no 
special parameter named "contentType", "s", "p", or any such thing. 
Therefore no registry for such keywords is necessary. They are all 
defined by the rop.

Yes, this is a lexical distinction (b). The rop is in a standardized 
position, before ":", and the name/values parameters are after the ":".

The question then is how to structure the <rop> production. In 
-14-seantek, I have made specific proposals. That text discusses the 
proposals; here I will only highlight advantages and disadvantages:

alpha-op: <keyword starting with ALPHA>

+ looks nice
- only in US-English
- new registry required
- contentious assignment
- bikeshedding

oid-op: <Object Identifier, starting with 0. 1. or 2.>

+ based on a great, functioning standard
+ widely deployed
+ subject to friendly user interface substitution
+ not US-English-specific
+ non-contentious assignment
+ informal universal directory exists (oid-info.com)

urn-op: <URN-based op, starting with urn*NID*>

+ namespace-defined but doesn't make rops namespace-specific 
(<urn:isbn:foo> can use <urn*ietf*op> operation)
- new registries
- registries may not be public
- more hassle for namespace maintainers to maintain
- not actually a URN. Looks like it but different and very restricted 
semantics. Therefore, confusing

ns-op: <URN namespace-based-specific op, starting with * (which is 
syntactic sugar for urn*NID* where NID is the same as the urn:NID)>

+ syntactic sugar
+ see urn-op
- see urn-op

Sean

---

From: Sean Leonard <[email-masked]>
Date: Thu, 26 Nov 2015 05:10:12 -0800

I wanted to pull out, and respond to, a specific paragraph at the bottom 
of this message:

> bad for long-term stability (and maybe "persistence").

In some ways, r-component functions like a URI.

However, I tried going down that path when deriving the syntax in 
-14-seantek, and it failed miserably. Syntactically, it is not elegant 
to stuff a complete URI in an r-component (or in general, in the query 
production of a URI). Restricting ourselves to absolute-URI (i.e., no 
fragment):

1. It can contain "?". That needs to be escaped. So, "?" must turn into %3F.
2. %3F needs to be distinguished. So, percent-encoding for "?" has to be 
double-encoded: %253F.
3. Now all % has to be distinguished. So all % must become %25xx.
4. Characters like [ ] (for IP literals) must also be percent-encoded.
5. You may as well percent-encode all of the non-reserved characters.

By the time you do all those things, the URI looks extremely ugly. 
Elegance lost.

Furthermore, it significantly messes up the parsing since a parser now 
has to recursively parse a URI. And what if the URI is a urn: URI? 
Endless recursion. (Let's not even get into fragment identifiers.)

At the end of the day, yes, a URI also involves a choice of identifier, 
an authority, and a set of "parameters". That is where the similarity ends.

There is a good point, however: the URI's analogous identifier to the 
<rop> is the scheme, such as http, mailto, magnet, ldap, file, etc. URIs 
have benefited from, and have widely proliferated because, the 
procedural cost to create a new URI scheme is extremely low. You just 
write some URI handler software, and optionally register that string 
("svn", "callto", etc.) with your local operating system/operating 
environment. The difficult hurdles that the IETF set up for permanent 
URI scheme registrations? Forget about them: just define and go. That is 
why the IETF transitioned to provisional registrations [BCP35].

We need that same agility ("generativity") with rop identifiers.

I am sensitive to the need (as you + Juha pointed out) for long-term 
stability. I believe that the <rop> proposal addresses those needs. 
(Hint: OID-op...)

Best,

Sean

---

From: Sean Leonard <[email-masked]>
Date: Thu, 26 Nov 2015 05:30:13 -0800

One other thing:

On 11/24/2015 12:26 PM, John C Klensin wrote:
>> r-component. [...]
> [...] What is specified is where they are
> interpreted, not what they do, [...]
>

Maybe this is just my hard-nosed, pragmatic American utilitarianism 
speaking. But I am a pragmatic American, and an engineer, so I like 
making things that maximize utility. :)

A standard can be phrased only to impose requirements on parsers. In 
that case, parsers can interoperate. A standard can also be phrased to 
impose requirements on resolvers (things that turn the parsed data items 
into something else). In that case, the resolvers can interoperate, and 
the things before (parsers) and after (URI-dereferencers, media type 
handlers) the resolvers can interoperate with the resolvers, because 
resolvers have at least some well-defined interfaces in and out of them.

When we (and by "we", I mean, the text of the URNBIS draft/standard) 
separate the query and fragment productions from the assigned-name, 
syntactically, we are saying that they are different, semantically, and 
are to be processed with different implementations (or parts of 
implementations) that follow other standard conventions.

By making f-component act like fragment, in at least some instances, we 
are saying that at least some implementations are supposed to take the 
other components, and return one or more instances of Internet content 
(Internet content = Internet media type and data stream, aka 
Content-Type: header with parameters). Because fragment is defined by 
the media type.

By making q-component act like query, in at least some instances, we are 
saying that at least some implementations are supposed to take the other 
components, and return one of more instances of URIs (URLs), so that the 
q-component can be smushed into the URI (URL), so that the URI (URL) can 
be dereferenced.

Therefore, if we want <urn:...?loginid=seantek#section2> to work, 
interoperably, we need to make it so that, /regardless of the particular 
resolver implementation/, ?loginid=seantek and #section2 operate on the 
same results (or sufficiently similar results, given a similar state of 
the network).

I do not find imposing standards on resolvers to be outside of 2141bis' 
scope. Maybe it is feature creep, but the motivation for imposing 
resolver standards (i.e., interoperable resolver behavior) is implied by 
how we are defining f-component and q-component actually to make some sense.

Sean

---

From: Sean Leonard <[email-masked]>
Date: Thu, 26 Nov 2015 06:55:47 -0800

Hello,

On 11/26/2015 12:42 AM, Hakala, Juha E wrote:
> [...]
> This might work in an environment where there is just one system which holds relevant information about the thing. But for instance my library has four systems: national bibliography for descriptive metadata, digital asset management system for the documents themselves, license database for rights metadata and digital archive for long term preservation purposes (and administrative metadata). So it is essential to define what kind of information the user wants, [...]
> [s & p discussion]
>
> So in the community I represent, just asking for "info" will not do. Our resolvers must know more precisely what the client/user wants.

This is a good illustration of the limitations of generic keywords such 
as "info". Not only is "info" pretty generic, different constituencies 
will vie for using the generic keyword "info" as their One True Keyword, 
and will attempt to foist complexity into (industry-specific) params.

The first-order solution (per -14-seantek) is to make rop registration 
extremely easy, so that different groups can define different rop 
identifiers that work for them. If you want the results from the 
national bibliography for descriptive metadata, use a rop identifier 
that specifically asks to get the national (presumably, Finnish) 
bibliography for descriptive metadata!

The second-order solution is to make rop identifiers non-contentious. 
Everybody wants "info" (or "license" or "buy" or ...etc.) because it 
looks nice. Just like domain names: everybody wants hotels.com because 
it is mnemonic. (Almost) nobody is going to fight over 
0.9.2342.19200300.100.1.5 (the OID for favouriteDrink), except, perhaps, 
that they wish it were shorter. So use OIDs, and make the OIDs as short 
as you can get away with.

The third-order solution is to make rop definitions easy to implement.
We need to avoid the "commonName" problem that plagued/plagues 
X.509/PKIX. The nice thing is that 2.5.4.3 (the OID for commonName) is 
that it is short and well-known. The bad thing is that defining new 
attributes in X.509/PKIX implementations has been traditionally very 
hard. Among other things, you have to define the ASN.1 attribute format, 
and even the default DirectoryString was no walk in the park. So, 
everybody shoves semantic junk into "commonName", such as the name of a 
web server, or worse, "the only thing that a user ever sees", so if you 
put in "Bill Gates" all of a sudden you enable spoofing attacks.

In general, I am in favor of very basic regular expressions, and things 
like HTTP redirects. For example, the rop for "national (Finnish) 
bibliography for descriptive metadata" can be defined as: rewrite the 
URN to http://bibliography.fi/urn/{URN-assigned-name}. The end.

Basically this makes the URN resolver pretty dumb, requiring more smarts 
during URI dereferencing (e.g., an HTTP server). Problem solved by 
indirection! As long as it gets out of the URN resolver step, I am 
happy, and the HTTP industry (which is huge and deals with this sort of 
thing all the time) will be happy too.

> [...] deciding exactly how much (and what) we need to say about r-components in rfc2141bis is not going to be easy. [...] Trying to incorporate that too into 2141bis would delay us a lot. Besides, it would be good if other persistent identifier systems use the same resolution service names and parameters as URN when possible, but it will be harder to convince them to do that if all r-component related information is  "hidden" in 2141bis.

We have a concrete proposal on the table: -14-seantek. So take a look at 
that. If it says too much, we can trim. If it says too little, we can add.

In this feedback, I am hearing a call for URNs to reuse another 
persistent identifier system. In a prior feedback, John K expressed 
concern about "normative dependency on X.660" 
<mid:[email-masked]>.

Let's get it out in the open: if you want to reuse another persistent 
identifier system (with or without parameters), you're going to have a 
normative dependency on it. So: "work it out." :-)

Obviously my thumb is on the scale for Object Identifiers [X.660], 
because they actually work, and are deployed. But I have proposed an 
extensible syntax in -14-seantek that permits other schemes.

(Note: In -14-seantek, the use of percent-encoding in alpha-op is 
prohibited. I did not want to get into the hazards of UTF-8 encoding and 
Unicode comparison. It's just not worth it. You already have to deal 
with that stuff in the NID / NSS component anyway.)

> [...]
>
> This level can be reached by r-components like "info&dc" or "s=I2C&p=dc" which specify exactly what is being requested. It is possible to add even more details; for instance, r-component like "s=I2C&p=MARC21&level=5" means that the client wants a complete MARC 21 record, not an abridged one.

Not much to say here beyond what I have already said...all that can be 
encoded in r-params, or in different rops, or both.

I did want to point out, however, that r-params (in -14-seantek) 
specifies a very specific data model:
unordered map of name/values pairs
each name has an ordered list of 0 or more values
each value is a well-formed Unicode string, plus possible $ or + 
characters, with reserved purposes

This data model can be represented straightforwardly in the common data 
formats:
JSON: object (key = name string) -> array of strings
CBOR: map (key = name utf8 string) -> array of utf8 strings
ASN.1: SET OF SEQUENCE {name UTF8String, SEQUENCE OF UTF8String}
XML: fragment of 
<name1><value>TEXT</value><value>TEXT</value>...</name1>...<nameN>...</nameN>
etc.

>> that every "resolver" can check in real time.   [...]

I do not think that a global dynamic registry is a goal of this spec. 
But, I also do not think it is too difficult to build.

If we have a global namespace of rops (whether OIDs, or alpha-ops, or 
other things), then we can use the DNS:

<rop-id>._rop.urn.arpa

which has a resource like a regular expression. For example, the rop for 
"get the national (presumably, Finnish) bibliography for descriptive 
metadata" can be 1.3.6.1.4.1.11.9999, in which case, a regular 
expression would be stored at the DNS entry:

"1.3.6.1.4.1.11.9999"._rop.urn.arpa

that would rewrite the assigned-name part to 
http://bibliography.fi/urn/{URN-assigned-name}.

That is not proposed for all rops...but it will work for a significant 
fraction of them, and is consistent with the overall philosophy of DDDS 
[RFC3404]. Notably (unlike DDDS), it is namespace-independent, which is 
(I think) what Keith Moore wants. For more complicated rops, an 
implementation can do correspondingly more complicated things than DNS 
lookups.

>  From rfc2141bis point of view, specifying internal syntax might be within scope. But I am not sure if we are ready yet to do that.  Trying to specify keywords (elements) and their parameters a'la RFC 2483 is not practical.

As illustrated above. And hopefully solved too.

>   After URN syntax has been modernized, we should outline a mechanism with which the registry is maintained.

Maintaining registries is...something I am not in favor of. For reasons 
outlined above. Mainly, it's too contentious. That is why in 
-14-seantek, I proposed "RFC Required" (but not IETF Consensus) to be 
the standard for new alpha-ops. Something to think about. (Obviously a 
registry can still be maintained.)

> Service mnemonics from RFC 2483 could and should be a starting point 
> for creation of this set. In addition to that we need parameters, such 
> as metadata formats for I2C and I2CS. 

Good things to include in another RFC, hence RFC Required.

> Experimental services may eventually become commonplace, so we need a mechanism for upgrading the status of a service or a service parameter.

An excellent set of reasons to use OIDs. Anybody can mint a OID, and if 
a OID catches on, the OID gets baked into other 
products/configurations/standards.

> When it comes to creation of smart URN resolvers in general and r-components in particular, nobody "has gotten there" yet. Other PIDs, most notably ARK and Handle, do provide some resolution servic…

Correct me if I am wrong, but <proprietary identifier> -> <HTTP 
redirect> -> resources  seems to be a common trend in this space, no? I 
am thinking of https://dx.doi.org in particular.

> There are no existing URN implementations with features discussed here. But there is an increasingly sophisticated technical environment with which electronic resources are managed, and URNs must f…

I think we are getting there.

Best regards,

Sean