Subject: [urn] review of rfc2141bis-12

From: Keith Moore <[email-masked]>
Date: Wed, 01 Jul 2015 18:03:47 -0400

In an effort to make my input very concrete, I have suggested text to 
fix most of the issues I've identified.  I'm sorry that this makes this 
review longer and more difficult to read.

Summary of significant problems (no particular significance to the order):

1. The URN syntax grammar is ambiguous, and is inconsistent with the text.
2. The current grammar either requires encoding of queries represented 
as q-components (without specifying any such encoding) or disallows 
queries containing '??'.
3. For the sake of interoperability between clients, interpretation of 
q-component, r-component, and f-component by client software needs to be 
consistent across all URNs that can resolve to URLs. The current text 
explicitly permits variant uses of these components, without setting any 
restrictions on such usage.

There are several minor problems also, and a few nits.

Comments and suggestion for changes in document order:

*1.*  Section 3, first paragraph:

The rationale for the syntax changes made in this document needs some work.

IMO, "Semantic flexibility in the use of URNs within particular 
applications" is not a desirable goal for URNs, or maybe I just 
interpret those words differently than intended.  We need for URNs to be 
interpreted uniformly from one (client) application to the next.   
Perhaps this is just an ambiguity that needs to be cleared up, but the 
need to have client implementations interpret URN components uniformly 
from one client to the next (i.e. interoperability) should inherently 
constrain the semantics of URN components to some degree.

(I'll have separate comments for urnbis-semantics.)

CURRENT:

3.  URN Syntax

    The syntax of URNs as provided in [RFC2141] was defined before the
    updated specification of URIs in [RFC3986].  In order to ensure
    consistency with the URI syntax as well as semantic flexibility in
    the use of URNs within particular applications (see
    [I-D.ietf-urnbis-semantics-clarif] for further discussion), this
    specification does the following:

RECOMMENDED:

3.  URN Syntax

    The syntax of URNs as provided in [RFC2141] was defined before the
    updated specification of URIs in [RFC3986].*In order to facilitate**
**    the use of URNs with parameters similar to URI queries and fragments,**
**    permit parameters influencing URN resolution to be included with**
**    URNs, and ease use of URNs with identifier spaces that**include**
**    the '/' character,*  this specification does the following:

(see also my note below about including "~" and "&" in NSSs)

*2.*  Problems with grammar in section 3

a) The grammar is both ambiguous (in that it permits the same string to 
be parsed in multiple/conflicting ways) and inconsistent with the 
text.   In particular, if "??" separates a q-string from r-string, then 
q-string should not permit "?" to appear twice in succession, and the 
currently specified grammar does.

b) The grammar refers to "q-string" etc. but the text refers to 
"q-component" etc.  I could find no reason for the discrepancy.

c) Also, the encoding problem for q-component still remains.    See below.

CURRENT:

       namestring    = assigned-name
                       [ "?" q-string ]
                       [ "??" r-string ]
                       [ "#" f-string ]
       assigned-name = "urn" ":" NID ":" NSS
       NID           = (alphanum) 0*30(ldh) (alphanum)
       ldh           = alphanum / "-"
       NSS           = pchar *(pchar / "/")
       q-string      = pchar *( pchar / "/" / "?" )
       r-string      = pchar *( pchar / "/" / "?" )
       f-string      = fragment

    Note: The character "?" can be used without percent-encoding inside
    q-components, r-components, and f-components.

ALTERNATIVE 1 (keep order and separators the same but fix ambiguity):

       namestring    = assigned-name
                       [ "?" q-*component*  ]
                       [ "??" r-*component*  ]
                       [ "#" f-*component*  ]
       assigned-name = "urn" ":" NID ":" NSS
       NID           = (alphanum) 0*30(ldh) (alphanum)
       ldh           = alphanum / "-"
       NSS           = pchar *(pchar / "/")
       q-*component*    =*pchar *( pchar / "/" / ( "?" ( pchar / "/")))*
       r-*component*    = pchar *( pchar / "/" / "?" )
       f-*component*    = fragment

    Note: The character "?" can be used without percent-encoding inside
    q-components, r-components, and f-components, but two adjacent "?"
    characters MUST NOT appear inside a q-component.

Basically, the change specifies that any "?" in a q-component must be 
followed by either a pchar or a "/".

However, this suggested grammar still has the encoding problem that I 
mentioned earlier:   If q-component is interpreted as a query to be 
attached to a URL (which is what I thought we had agreed to, at least 
for URNs that resolve to URLs), there need to be clear rules for copying 
or converting that q-component text to the URL query text.   The above 
grammar does not permit multiple "?"s to appear adjacent in a 
q-component.   This implies that if there's ever some need to bundle a 
URN with a query that contains "??", the "??" must somehow be encoded so 
that (a) it doesn't violate the URN syntax grammar, and (b) the original 
query string is recovered when bundled with the URL.

Simply assuming percent-encoding doesn't fully specify a solution, 
because while a query that contains "??" could be encoded as %3F%3F, a 
resource might reasonably interpret %3F%3F differently from "??"

This is one reason that I believe it's much simpler, and more likely to 
be implemented correctly, to not constrain the syntax of q-components 
beyond the syntax of RFC 3986 queries.   And that implies either (a) 
putting r-component somewhere else than following q-component [e.g. 
ALTERNATIVE 2 below], or (b) using a different character sequence to 
separate r-component from q-component [e.g. ALTERNATIVE 3].   Then we 
can specify that if a URN with a q-component resolves to a URL, the 
q-component is simply copied to the URL query.

RECOMMENDED ALTERNATIVE 2 (switch order of r-component and q-component):

       namestring    = assigned-name
                       *[ "??" r-component ]
*                       *[ "?" q-component ]
*                       [ "#" f-*component*  ]
       assigned-name = "urn" ":" NID ":" NSS
       NID           = (alphanum) 0*30(ldh) (alphanum)
       ldh           = alphanum / "-"
       NSS           = pchar *(pchar / "/")
       q-*component*    *=  *( pchar / "/" / "?" )*
       r-*component*    *= pchar *( pchar / "/" )*
       f-*component*    = fragment

    Note: The character "?" can be used without percent-encoding inside
    q-components, r-components, and f-components.

ALTERNATIVE 3 (use "##" as the introducer for r-component)

OOPS:  I noticed after I had most of this review complete that this 
grammar isn't compatible with RFC 3986, since 3986 doesn't permit a 
fragment to contain "#".   I'm leaving this example here anyway to 
illustrate that there might be more than one way to tweak the grammar to 
make q-component have identical syntax to query and thus fix the 
encoding problem.  For instance, if we don't really need to permit '&' 
in an NSS, maybe '&' could be used to introduce r-component.

       namestring    = assigned-name
                       [ "?" q-*component*  ]
                       [*"##"*  r-*component*  ]
                       [ "#" f-*component*  ]
       assigned-name = "urn" ":" NID ":" NSS
       NID           = (alphanum) 0*30(ldh) (alphanum)
       ldh           = alphanum / "-"
       NSS           = pchar *(pchar / "/")
       q-*component*    *= *( pchar / "/" / "?" )*
       r-*component*    = pchar *( pchar / "/" / "?" )
       f-*component*    = fragment

    Note: The character "?" can be used without percent-encoding inside
    q-components, r-components, and f-components.

Either alternative 2 or alternative 3 fixes the encoding problem. The 
order of components in alternative 2 seems more logical to me. Also, 
both alternative 2 and alternative 3 permit the first character of a 
q-component to be "?".

Note: My suggestions for changes to the remainder of the document assume 
that alternative 2 replaces the current grammar.   Some of the language 
would need tweaking if a different grammar were chosen.

*3.*  Section 3.2, 2nd paragraph:

    This document modifies the syntax of the NSS to allow the following
    characters: "/", "~", and "&".

Are there reasons to permit "~" and "&", and have we sufficiently 
investigated the implications of doing so?   I am wondering whether this 
will break parsers for no good reason.  I have seen URLs which "~" was 
encoded to "%7E" (even though this shouldn't be necessary) and "&" to 
"%26" (which also shouldn't be necessary except in a query that uses 
HTML FORM encoding for HTTP GET).

*4. * This document needs to clarify (somewhere) that the change to the 
URN grammar DOES NOT change encoding rules for existing URN namespaces, 
if there are existing namespaces that are used outside of the URN 
context, and in which use any of "/", "~", or "&" are allowed to appear 
in those names in native form.

*5.*  Also, something that I think could possibly use a bit more 
explicit attention in the registration section and perhaps elsewhere: In 
some sense there are two kinds of URN namespaces: those that exist 
solely as URNs, and identifier spaces that exist independently of 
URNs.   The latter kind of identifier has a "native" form and a URN 
form, and its URN namespace registration should specify an algorithm for 
converting from native form to URN form.   Ideally this algorithm 
produces a distinguished form of its input when converting it to URN 
format - this makes it easier to build generic services that answer 
queries for arbitrary URNs.   If the algorithm is just to use 
percent-encoding, that's fine, but IMO this should be explicitly 
specified in the namespace registration. I don't think we should 
unilaterally impose this algorithm for all identifier spaces.

Client software that just handles URNs doesn't need to worry about 
this.   Such software should always see the URN form of such identifiers 
and should be able to handle all URNs in a uniform manner.   However, 
software that allows such identifiers to be input directly (e.g. without 
the urn:xxx: prefix) and converts those identifiers to URNs, needs to be 
aware of the conversion rules for each URN namespace that it accepts.

Anyway, I think it would help to have some background text early in the 
document that explains this, and one or two specific items in the 
registration template

(I can try to come up with text if desired, but I didn't want to delay 
this review any longer.)

(Also, much of this item is also reflected in the next item.  Sorry for 
not merging them; I wanted to get this out the door.)

*6.*  Section 3.2, 3rd paragraph:

I'm concerned that telling implementors to use %-encoding of disallowed 
characters that appear in NSS might conflict with an existing URN 
namespace's encoding rule.  Also, %-encoding is a mess, because it's 
used by so many different layers of software without any clear guidance 
about which layers undo which portions of the encoding.  So I doubt that 
it's a good idea to specify a uniform rule for how every namespace 
should encode its native names to URN format; instead, I think that 
encoding considerations should be part of namespace-specific 
"canonicalization" rules.

As for interoperability: if a URN appears with its urn:nid prefix, 
software should be able to assume that it's already in appropriate 
format, with all necessary encoding applied, and apply uniform parsing 
rules, uniform resolution code, etc. to that URN.   But if an identifier 
appears without a urn:nid prefix, software shouldn't be trying to 
generate a URN from it, or otherwise treat it as a URN, unless that 
software knows the namespace-specific rules for encoding those 
identifiers as URNs.   And somewhere, probably in this document, there 
should be a discussion about the difference between URN format and 
namespace-specific "native" formats of identifiers.

CURRENT:

    Depending on the rules governing a namespace, names that are valid in
    a namespace might contain characters that are not allowed by the
    "pchar" production referenced above (e.g., characters outside the
    ASCII range or characters that are reserved in URIs, such as "?", and
    "#").  While such a string might be a valid name, it is not a valid
    URN until it has been translated into a conformant NSS.  Translation
    is done by percent-encoding each disallowed character using the
    method defined in Section 2.1 of the generic URI specification
    [RFC3986].  Note that the "%" character is allowed in the NSS only
    for the purpose of percent-encoding.

RECOMMENDED:

    Depending on the rules governing a namespace, names that are valid in
    a namespace might contain characters that are not allowed by the
    "pchar" production referenced above (e.g., characters outside the
    ASCII range or characters that are reserved in URIs, such as "?", and
    "#").  While such a string might be a valid name, it is not a valid
    URN until it has been translated into a conformant NSS.*Translation**
**    of an identifier from its "native" format to URN format is
    accomplished**  by using the canonicalization and encoding methods
    defined for that**URN namespace in that namespace's registration.
    Software that is not aware of those URN canonicalization**  and
    encoding rules for the namespace used MUST NOT construct URNs
    from such identifiers.*   

*7.* Last paragraph of section 3.2:

CURRENT:

    If a namespace designates one or more characters conforming to the
    "pchar" rule as having special meaning for that namespace (e.g., "@")
    and the namespace also uses that character in a literal sense, when
    used in a literal sense the character MUST be percent-encoded (e.g.,
    "%40").  For related considerations with regard to NID registration,
    see Section 7.3.2.

RECOMMENDED:

    If a namespace designates one or more characters conforming to the
    "pchar" rule as having special meaning for that namespace (e.g., "@")
    and the namespace also uses that character in a literal sense,*that**
*    *namespace registration MUST specify a means of representing
*    *its native format*  *identifiers as URNs in a way that conforms
*    *with the URN NSS grammar*.*See Section 7.3.2.*

*8.* Section 3.3:

I have several problems with this section overall:

Defining new URN components without saying anything about semantics (a) 
invites inconsistent interpretation of those components from one URN 
namespace to the next (which thwarts the ability to write client 
software that handles URNs in a uniform way), and (b) also threatens to 
harm the ability to specify queries and/or fragments with URNs that 
resolve to URLs.

Even though the semantics of q-components and f-components might vary 
from one URN to the next, it's important for this document to make clear 
that client software can handle these in a uniform manner.  For URNs 
that resolve to URLs, q-component /always/ maps to the URL query (if 
that URL scheme supports query), and f-component /always/ maps to the 
URL fragment.  At least for those resources, the semantics of the 
q-components are defined by those resources, not by the namespace.  The 
semantics of f-components are defined by the media type of the resource, 
not by its URN namespace.   (The namespace can choose to not assign 
names to resources that don't fit its criteria for interpretation of 
queries or fragments, but it's important that 2141bis not promote the 
idea that the interpretation of these components is arbitrary, or 
namespace-specific.)

It's also important to recognize that a resource can quite reasonably 
have multiple URNs, in the same and/or different namespaces.  This must 
not create a conflict or ambiguity over the interpretation of 
f-components or q-components with respect to that resource, either when 
such components appear within URNs or when those components appear in 
relative references within those resources.

If this document is going to focus "almost exclusively" on URN syntax, 
then we need to have an accompanying document that discusses semantics 
of the new components.  I think that's probably a Bad Idea.   Every time 
a reader has to switch from one document to another, there's a good 
chance that he'll lose some context, or worse, not bother to read the 
next document.   I'm fine with -semantics-clarif being in a separate 
document if it just explains why certain decisions were made, but it's 
probably not a good idea to entirely separate syntax and semantics.   So 
I'm proceeding on the assumption that at least minimum 
interpretation/semantics of these new components needs to be specified 
in 2141bis.

CURRENT:

    Because this specification focuses almost exclusively on URN syntax,
    it does not define the semantics of the q-component, r-component, and
    f-component for URNs in general.  Instead, additional specifications
    might establish these matters for URN-related services (such as URN
    resolution) or for individual URN namespaces (e.g., to handle
    extended information about the resource identified by a URN).
    Defining the details of such usage is the responsibility of
    specifications for URN resolution services, namespace registration
    requests or specifications for individual namespaces, and other
    appropriate documentation (such as policy documents governing the
    management of a given URN namespace).

    Unless specifically defined for a particular namespace after
    publication of this document, use of these components is disallowed,
    thereby maintaining strict backward compatibility with namespaces
    defined in accordance with [RFC2141] and registered in accordance
    with [RFC3406].

SUGGESTED:

    *As compared to [RFC2141], this document extends the URN syntax to**
*    *permit inclusion of three new types of components: q-component,**
*    *r-component, and f-component.**Because this specification focuses**
*    *almost exclusively on URN syntax, it does not define detailed**
*    *semantics for these components for URNs in general.  However, each of**
*    *these components has a distinct role from the others, which is**
*    *independent of the URN and its namespace, and it is intended that**
*    *clients be able to handle these components uniformly for all URNs.**
*    *In general:**
**
**        o  q-component is intended for parameters to be transmitted to**
**           the named resource and interpreted by that resource;**
**
**        o  r-component is intended for parameters to be transmitted to**
**           resolution services and interpreted by those services; and**
**
**        o  f-component is intended to be interpreted by the client as**
**           a specification for a location within or region of, the**
**           named resource.**
*
    *The above is true regardless of the URN's namespace.**
**
**    Whenever a URN resolves to a URL which may be used to access the**
**    resource, there is a more specific interpretation of q-component and**
**    f-component: the q-component is copied verbatim to the query portion**
**    of the URL (if that URL scheme supports query), and the f-component**
**    is copied verbatim to the fragment portion of the URL.   This is**
**    necessary, among other reasons, so that interpretation of q-component**
**    and f-component when associated with URNs, will be consistent with**
**    the interpretation of relative references containing query and/or**
**    fragment, respectively, within documents which are accessible via URNs.**
**    Thus, for resources which may be accessed via a URL, the semantics of**
**    q-component are identical to those for queries to that resource, and**
**    the semantics of f-components are identical to those of fragments for**
**    that resource.   Semantics of q-component and f-component for resources**
**    that inherently cannot be accessed via a URL are undefined by this**
**    document; however, they SHOULD be consistent with the above roles.**
**    It is expected that additional specifications will define the semantics**
**    of r-components.**
**
**    Note:  In general, neither syntax nor semantics of q-component or
    f-component are**  defined by, or dependent on, the namespace of the URN.
    **A particular**  namespace might, however, define use of r-components that
    are specific**to its namespace and supported by the resolution services
    which that**  namespace operates and/or recommends.**
**
    Any of these components MAY be used with URNs from existing namespaces
    whether or not the namespace explicitly supports them.  As described
    above, the interpretation of q-component and f-component is namespace-
    independent.  Interpretation of an r-component may be namespace-dependent
    to some degree if it relies on behavior of a namespace-specific
    resolution service.   If a URN contains r-components that are not
    defined by the namespace, the meaning of the URN and result of resolution
    are out of scope for this specification.
*

Note about the last paragraph of the suggested text as compared to the 
original: I think it's pointless to forbid use of a *-component with 
existing URNs, especially when the client's interpretation of such a 
component should be consistent from one URN to the next anyway.   It's 
unreasonable to expect clients to implement namespace-specific behavior 
for interpretation of *-components.
*
**9.* Section 3.3.1

Again, it is important that this document be clear about the appropriate 
roles for each of these components, and for this document to not 
encourage inconsistent use of these components from one client 
application or namespace to another.   Also, it's important to be clear 
that q-components are not passed to URN resolvers, so that they are not 
taken into account in URN resolution.

Also, the example chosen in the -12 text to illustrate use of 
q-components in this section, while valid, seems likely to cause 
confusion.  While it is certainly possible and reasonable to have a 
resource that returns metadata about other resources, and for a URN to 
be assigned to a metadata-returning resource, the example URN doesn't 
actually specify the URI of a resource for which metadata is to be 
returned.   The reader is likely to presume that the resource named is 
returning metadata about itself.  And while a resource could certainly 
do that (and it might be a useful convention), that's assuming a lot 
more than anything that's been specified to date.  Normally one does not 
obtain metadata by querying a resource, one obtains metadata by 
contacting a resolution service - and the r-component, not the 
q-component, must be used to pass paramters to the resolution service.

CURRENT:

3.3.1.  q-component

    Although the URN q-component has close to the same syntax as the URI
    query component, the URN q-component need not be semantically
    equivalent to the URI query component and therefore is called by a
    different name in this specification to emphasize that the
    relationship is about syntax only (see also
    [I-D.ietf-urnbis-semantics-clarif]).

    The q-component is indicated by the first question mark ("?")
    character and terminated by a double question mark ("??"), by a
    number sign ("#") character, or by the end of the URI.  The q-string
    MUST NOT begin with a question mark ("?") since a double question
    mark is reserved for use by an r-component (Section 3.3.2).  The
    characters slash ("/") and question mark ("?") may represent data
    within the q-string.  Note that characters outside the ASCII range
    [RFC20] MUST be percent-encoded using the method defined in
    Section 2.1 of the generic URI specification [RFC3986].

    As described under Section 4, the q-component SHALL NOT be taken into
    account when determining URN equivalence.

    The q-component is primarily intended to be used for passing
    information in requests to the resources identified by a URN or to
    applications that manage such resources.  Consider the hypothetical
    example of passing parameters to an application that returns metadata
    about a resource (say, Dublin Core [RFC5013] data about a published
    book).  This could perhaps be accomplished by specifying the desired
    metadata field (e.g., "description") in the q-component, resulting in
    URNs such as
    "urn:example:0-395-36341-1?operation=search&field=description".
    However, this primary purpose is not intended to forestall other
    potential uses for q-components.

SUGGESTED:

(note: the following text assumes the grammar alternative 2 and will need
to be changed if a different fix for the grammar is chosen)

3.3.1.  q-component

*    The URN q-component has the same syntax as the URI query component.**
**    If a URN resolves to a URL with a scheme that supports a query**
**    component, the q-component from the URN is copied verbatim to the**
**    query component of the URL.**   If the URN does not resolve to a URL,**
**    the interpretation of the q-component is undefined by this**
**    specification; however, it SHOULD be used to communicate parameters**
**    that are interpreted by a resource.  The q-component MUST NOT be
    used**  to communicate parameters to a resolution service.**
**
**    **The q-component follows the r-component if one exists.  The start of
    the**  q-component is indicated by**  a**  question mark ("?") character and**
**    terminated by a number sign ("#") character, or by the end of the URI.
   *  The characters slash ("/") and question mark ("?") may represent data
    within the q-component.  Note that characters outside the ASCII range
    [RFC20] MUST be percent-encoded using the method defined in
    Section 2.1 of the generic URI specification [RFC3986].

    As described under Section 4, the q-component SHALL NOT be taken into
    account when determining URN equivalence.*   Similarly, the q-component SHALL
    NOT be taken into account when resolving a URN and MUST**  NOT be passed
    to a resolution service when querying the service for metadata.*

    The q-component is*entirely*  intended to be used for passing
    information in requests to the resources identified by a URN or to
    applications that manage such resources.*  Consider the hypothetical
    example of passing parameters to an application that returns weather**
**    maps from different regions and/or for different time periods.  This**
**    could perhaps be accomplished by specifying lat/long coordinates and**
**    dates/times in the URN's q-component.*

*
**10. * Section 3.3.2

CURRENT:

3.3.2.  r-component

    The URN r-component has no syntactic equivalent in URIs.

    The r-component is indicated by a double question mark ("??") and
    terminated by a number sign ("#") character or by the end of the URI.
    The characters slash ("/") and question mark ("?") may represent data
    within the r-string.  Note that characters outside the ASCII range
    [RFC20] MUST be percent-encoded using the method defined in
    Section 2.1 of the generic URI specification [RFC3986].

    As described under Section 4, the r-component SHALL NOT be taken into
    account when determining URN equivalence.

    The r-component is primarily intended to be used for passing
    information in requests to URN resolution services (not, as with the
    q-component, to the resources identified by a URN or to applications
    that manage such resources).  Consider the hypothetical example of
    passing parameters to resolution service (say, an ISO alpha-2 country
    code [ISO3166-1] in order to scope down the preferred country in
    which to search for a physical copy of a book).  This could perhaps
    be accomplished by specifying the country code in the r-component,
    resulting in URNs such as "urn:example:0-395-36341-1??cc=uk".
    However, this primary purpose is not intended to forestall other
    potential uses for r-components.

    This document defines only the syntax of the r-component and reserves
    it for future use.  The exact semantics of the r-component and its
    use in URN resolution protocols are a matter for potential
    standardization in separate specifications.

SUGGESTED:

(note: the following assumes the grammar alternative 2 and will need
changes for different fixes to the grammar)

3.3.2.  r-component

    The URN r-component has no syntactic equivalent in URIs.

*    **The r-component precedes any q-component.   It is indicated by a double
    question mark ("??") and terminated by a number sign ("#") character, a
    question mark ("?"), or by the end of the URI.   The slash ("/") may
    represent data within the r-component, but the question mark ("?") MUST
    NOT appear in an r-component.   If there is a need to represent a question
    mark in data passed to a resolver, it MUST be percent-encoded in the URN.**   **
*    Note that characters outside the ASCII range
    [RFC20] MUST be percent-encoded using the method defined in
    Section 2.1 of the generic URI specification [RFC3986].

    As described under Section 4, the r-component SHALL NOT be taken into
    account when determining URN equivalence.*The r-component SHALL be**
**    supplied along with the URN when presenting requests to URN
    resolution**  **services.*

    The r-component is*entirely*  intended to be used for passing
    information in requests to URN resolution services (not, as with the
    q-component, to the resources identified by a URN or to applications
    that manage such resources).  Consider the hypothetical example of
    passing parameters to resolution service (say, an ISO alpha-2 country
    code [ISO3166-1] in order to scope down the preferred country in
    which to search for a physical copy of a book).  This could perhaps
    be accomplished by specifying the country code in the r-component,
    resulting in URNs such as "urn:example:0-395-36341-1??cc=uk".
    
    This document defines only the syntax of the r-component and reserves
    it for future use.  The exact semantics of the r-component and its
    use in URN resolution protocols are a matter for potential
    standardization in separate specifications.

*11.*  Section 3.3.3

CURRENT

3.3.3.  f-component

    Although the URN f-component has the same syntax as the URI fragment
    component, the URN f-component need not be semantically equivalent to
    the URI fragment component and therefore is called by a different
    name in this specification to emphasize that the relationship is
    about syntax only (see also [I-D.ietf-urnbis-semantics-clarif]).

    The f-component is indicated by the presence of a number sign ("#")
    character and terminated by the end of the URI.  Note that characters
    outside the ASCII range [RFC20] MUST be percent-encoded using the
    method defined in Section 2.1 of the generic URI specification
    [RFC3986].

    As described under Section 4, the f-component SHALL NOT be taken into
    account when determining URN equivalence.

    The f-component is primarily intended to distinguish the integral
    parts of resources named by URNs in particular namespaces.  Consider
    the hypothetical example of obtaining resources that are part of a
    larger entity (say, the chapters of a book).  Each part could be
    specified in the f-component, resulting in URNs such as
    "urn:example:978-952-10-7060-0#chapter1".  However, this primary
    purpose is not intended to forestall other potential uses for
    f-components.

ALTERNATIVE

3.3.3.  f-component

*    The URN f-component has the same syntax as the URI fragment
    component.   When a URN containing an f-component resolves to a URL,**
**    the f-component from the URN is copied verbatim into the fragment**
**    of that URL.  Interpretation of the f-component for URNs that do**
**    not resolve to URLs is not defined by this specification.**
*
    The f-component is indicated by the presence of a number sign ("#")
    character and terminated by the end of the URI.  Note that characters
    outside the ASCII range [RFC20] MUST be percent-encoded using the
    method defined in Section 2.1 of the generic URI specification
    [RFC3986].

    As described under Section 4, the f-component SHALL NOT be taken into
    account when determining URN equivalence.*    Similarly, the f-component**
**    MUST NOT be passed to resolution servers when querying them for
    resource**  locations or metadata.*

    The f-component is primarily intended to distinguish the integral
    parts of resources named by URNs in particular namespaces.  Consider
    the hypothetical example of obtaining resources that are part of a
    larger entity (say, the chapters of a book).  Each part could be
    specified in the f-component, resulting in URNs such as
    "urn:example:978-952-10-7060-0#chapter1".  However, this primary
    purpose is not intended to forestall other potential uses for
    f-components*  for URNs that do not resolve to network-accessible**
**    resources.*

*12.*  Section 5.3

(NIT) I think it's a bit of a stretch to say what the result of use of a 
relative
reference will be if there's no base URI.

CURRENT:

    If no such base URI exists, use of a relative reference with respect
    to a URN is an error and will yield results that might be unexpected
    or not useful.

RECOMMENDED:

    If no such base URI exists, use of a relative reference with respect
    to a URN is an error.*   Client behavior in this case is undefined.
*

*13.*  Section 6, about halfway through

Clarify that a resource MAY have more than one URN assigned to
it, either in the same namespace or in different namespaces.

CURRENT:

    With regard to global uniqueness, using different NIDs for different
    collections of identifiers ensures that no two URNs will be the same
    for different resources, since each collection is required to
    uniquely assign each identifier.  However, a single resource can have
    more than one URN assigned to it for different purposes (for example,
    if a book were published in a monograph series, it could have both an
    ISBN [RFC3187] and an ISSN [RFC3044] assigned to it, resulting in two
    URNs referring to the same book).  Subject to other constraints, such
    as those imposed by the URI syntax [RFC3986], the rules of the URN
    scheme are intended to allow preserving the normal and natural form
    of identifiers specified elsewhere when they are treated as URN
    namespaces.

RECOMMENDED:

    With regard to global uniqueness, using different NIDs for different
    collections of identifiers ensures that no two URNs will be the same
    for different resources, since each collection is required to
    uniquely assign each identifier.*However, a single resource MAY have
    more than one URN assigned to it, either in the same namespace
    (if the namespace permits it) or**  in different namespaces, and
    either for similar purposes or different**purposes.*  (for example,
    if a book were published in a monograph series, it could have both an
    ISBN [RFC3187] and an ISSN [RFC3044] assigned to it, resulting in two
    URNs referring to the same book).  Subject to other constraints, such
    as those imposed by the URI syntax [RFC3986], the rules of the URN scheme
    are intended to allow preserving the normal and natural form of
    identifiers specified  elsewhere when they are treated as URN
    namespaces.

*14.* section 6, last paragraph

If we're deprecating X- namespaces, fine, but we should reserve
namespaces beginning with X- to prevent conflicts with identifiers
that might already exist in the wild.

CURRENT:

    Note: [RFC3406] defined a third type of "experimental namespaces",
    denoted by prefixing the namespace identifier with the string "X-".
    Consistent with general IETF conclusions about similar approaches
    [RFC6648], this specification removes the experimental category and
    syntax.  Because experimental namespaces were never registered,
    removing the experimental category has no impact on the existing
    registries or future registration procedures.  Because they are not
    registered, strings that refer to existing experimental namespaces
    are not valid URNs.  Truly experimental usages can, of course, employ
    the 'example' namespace [RFC6963].

RECOMMENDED:

    Note: [RFC3406] defined a third type of "experimental namespaces",
    denoted by prefixing the namespace identifier with the string "X-".
    Consistent with general IETF conclusions about similar approaches
    [RFC6648], this specification removes the experimental category and
    syntax.  Because experimental namespaces were never registered,
    removing the experimental category has no impact on the existing
    registries.*However, to avoid the potential for conflict between**
**    previously-allowed unregistered experimental namespaces and**
**    namespaces registered in the future, no registrations will be**
**    accepted for new namespaces beginning with "X-".*Because they are not
    registered, strings that refer to experimental namespaces
    are not valid URNs.  Truly experimental usages*MAY*, of course, employ
    the 'example' namespace [RFC6963].

*15.* Section 6.1

CURRENT:

    5.  It MUST NOT start with the string "xn--" or any other string
        consisting of two letters followed by two hyphens.  Those strings
        are reserved for potential representation of DNS A-labels and
        similar strings in the future [RFC5890].

RECOMMENDED:

    5.  It MUST NOT start with the string "xn--" or any other string
        consisting of two letters followed by two hyphens.  Those strings
        are reserved for potential representation of DNS A-labels and
        similar strings in the future [RFC5890].

*    6.  It MUST NOT start with the string "X-" so that it will not
        be confused with an experimental namespace previously
        permitted by [RFC3406], nor conflict**  with such a namespace.*

---

From: John C Klensin <[email-masked]>
Date: Sat, 25 Jul 2015 15:41:37 -0400

Keith,

I'm going to try to respond to a few elements of this (I think
my comments are complementary to Juha's) but leave most of this
for Peter to finish remarks that I know are coming...

--On Wednesday, July 01, 2015 6:03 PM -0400 Keith Moore
<[email-masked]> wrote:

>...

I hate ABNF but will go back and check this.

> 2. The current grammar either requires encoding of queries
> represented as q-components (without specifying any such
> encoding) or disallows queries containing '??'.

When I wrote that, I assumed that "encoding" would reflect the
3986 definition and force us into "%"-type arrangements.  We
are, after all, following that syntax.   If you think that needs
to be made more explicit, I don't see that as a problem.

That does get us to the more substantive and troublesome comment
below.

> 3. For the sake of interoperability between clients,
> interpretation of q-component, r-component, and f-component by
> client software needs to be consistent across all URNs that
> can resolve to URLs. The current text explicitly permits
> variant uses of these components, without setting any

I think we have a real conundrum here and I don't know how to
resolve it.  I also think you are contradicting yourself.  That
perception (correct or not) is a symptom of the problem.  On the
one hand, we want and need to allow significant flexibility to
namespace-definers, especially those who are trying to overlay
URNs on existing, standardized, and extensively-deployed
namespaces.  Another, related, reason for that flexibility is
that it is fairly clear that, if we engage in a discussion with
the managers of a namestpace that starts like:

 IETF:  Don't do <foo>
 Namespace manager: But <foo> is required to embed our
	namespace in URNs.
 IETF:  Don't do <foo unless you encode it using method
	<X>
 Namespace manager: <X> is horrible to look at and will
	cause integrity, transcription, and comparison problems
	with our identifiers.
 IETF: Don't do <foo>

Jokes about the protocol police notwithstanding, that type of
conversation can, in the real world and remembering that,
ultimately, the namespace manager makes the decisions, we don't,
there are only three plausible outcomes and one less likely one:

 (i) Do <foo> anyway and believe that any resulting
	messes are Someone Else's Problem.
 (ii) Write off URNs and ask for a <foo> URI.
 (iii) Write off namespace-specific URIs and either give
	up on Internet resolution or use
	http://foo-manager.org/<whatever>, presumably playing
	DNS games if needed.
 (iv) (the unlikely one) Give in and follow our rules,
	irritating those who define and use their identifiers
	and pay their salaries.

I think the way to avoid those choices is to spread as wide an
umbrella as possible, even if it means giving up on purity.
Nothing prevents us from saying "if you do <foo> (or fail to do
<X> with it) that is likely to cause problems that you need to
understand and expect" but that is as far as I think we should
go.

My reasoning above informed some of the other text to which you
object so, at least until we hear from Peter, consider them
context.
	
>...
 
> a) The grammar is both ambiguous (in that it permits the same
> string to be parsed in multiple/conflicting ways) and
> inconsistent with the text.   In particular, if "??" separates
> a q-string from r-string, then q-string should not permit "?"
> to appear twice in succession, and the currently specified
> grammar does.

If you have an improved grammar suggestion, I'm sure we will
welcome it.  Otherwise, you get a textual discussion/
restriction (as I think you suggested).

> b) The grammar refers to "q-string" etc. but the text refers
> to "q-component" etc.  I could find no reason for the
> discrepancy.

I think I was intending that "q-component" include the leading
"?" and that q-string was without it.  But, if that confuses
things rather than clarifying them...

>...
> Basically, the change specifies that any "?" in a q-component
> must be followed by either a pchar or a "/".

Of, of course, a # because 
    urn:blather:frizzlepop?foobar?#
is presumably valid, if ugly.
 
> However, this suggested grammar still has the encoding problem
> that I mentioned earlier:   If q-component is interpreted as a
> query to be attached to a URL (which is what I thought we had
> agreed to, at least for URNs that resolve to URLs), there need
> to be clear rules for copying or converting that q-component
> text to the URL query text.   The above grammar does not
> permit multiple "?"s to appear adjacent in a q-component.
> This implies that if there's ever some need to bundle a URN
> with a query that contains "??", the "??" must somehow be
> encoded so that (a) it doesn't violate the URN syntax grammar,
> and (b) the original query string is recovered when bundled
> with the URL.
> 
> Simply assuming percent-encoding doesn't fully specify a
> solution, because while a query that contains "??" could be
> encoded as %3F%3F, a resource might reasonably interpret
> %3F%3F differently from "??"
> 
> This is one reason that I believe it's much simpler, and more
> likely to be implemented correctly, to not constrain the
> syntax of q-components beyond the syntax of RFC 3986 queries.
> And that implies either (a) putting r-component somewhere else
> than following q-component [e.g. ALTERNATIVE 2 below], or (b)
> using a different character sequence to separate r-component
> from q-component [e.g. ALTERNATIVE 3].   Then we can specify
> that if a URN with a q-component resolves to a URL, the

> ALTERNATIVE 3 (use "##" as the introducer for r-component)

Keith, I think this has been settled.  This is clearly up to
Andy and Melinda, but I think the WG has concluded that
q-component and then r-component was, on balance, the right way
to do things and that you are not introducing anything new
above.  The claim that %-encoding in complex situations
introduces confusion and/or ambiguity has been made many times
before, including by me, and consistently shot down by presumed
URI experts.  If there is a problem in that area, either
construct an erratum for RFC 3986 or convince the WG (and others
concerned) that the original "URNs are not URIs" model was
correct... a model that you argue against when you suggest that
URI processors ought to be able to handle URNs in an unambiguous
way.

> OOPS:  I noticed after I had most of this review complete that
> this grammar isn't compatible with RFC 3986, since 3986
> doesn't permit a fragment to contain "#".   I'm leaving this
> example here anyway to illustrate that there might be more
> than one way to tweak the grammar to make q-component have
> identical syntax to query and thus fix the encoding problem.
> For instance, if we don't really need to permit '&' in an NSS,
> maybe '&' could be used to introduce r-component.

At the price of making URN syntax even more complex.  I thought
we had considered introducing yet more special characters and
then conclude it wasn't a good idea.  I'd be willing to revisit
that if you are not the only one in the WG who thinks it is a
good idea.

>...

    best,
     john

---

From: Keith Moore <[email-masked]>
Date: Tue, 28 Jul 2015 02:01:35 -0400

> I hate ABNF but will go back and check this.

thanks.
> to be made more explicit, I don't see that as a problem.

I have the impression that I've still failed to adequately explain what 
I think the problem is.   But maybe I've been overestimating the 
severity of the problem.   Mostly I'm just worried about edge cases that 
might result when a URN with a q-component is resolved to a URL with a 
query, given that the syntax for a q-component (in particular the 
characters that are allowed without percent-encoding) is different than 
that of a query.

So for instance if a resource accepting a query were accessed via URL, 
the URL with the bundled query might look like

http://example.com/foo?q=who?

whereas if the same resource were accessed via URN, the URN would need 
to look like

urn:example:bar?q=who%3F

which would then resolve to

http://example.com/foo?q=who%3F

and the resource would (might) actually see a different GET command 
between the two cases, and perhaps produce different results.

But what I'm seeing is that some web browsers percent-encode "?" in a 
search string even though RFC 3986 doesn't require this.   So it might 
be that in practice, the resource isn't likely to see a difference 
between the two, and is unlikely to treat them differently even if it 
does see a difference.

A similar potential glitch would be if http://example.com/foo contained 
a link to <A HREF="?q=bar??zot">.   But if the resource were accessed 
instead via URN urn:example:bar, that relative reference (if combined 
with the base URI via RFC 3986 rules) would produce 
urn:example:bar?q=bar??zot which would have zot interpreted as an 
r-component even though it was intended to be part of a query.

I realize that we've declared that when evaluating a relative reference 
with respect to a resource named by a URN, that reference is supposed to 
actually be evaluated relative to a base URL that was either obtained 
via the resolution process or specified in the resource itself.   But 
for various reasons I suspect that we'll still have client software 
trying to make sense of URN + relative reference without an associated URL.

Since I believe that protocol designs and system should try to avoid 
obscure edge cases, I would prefer that the syntax permit a direct 
copying of URN q-component to URL query (and vice versa if this is ever 
applicable).   And that's why I was looking for ways to change the ABNF 
to permit this, including changing the order of q- and r-components.   I 
think that will produce fewer implementation bugs, less obscure and odd 
behavior, and less user frustration.

But I also realize that this is at least the third time I've tried to 
fix this, and that the glitches that this causes will probably be 
infrequent.   I also strongly suspect that this will come up again in 
IETF-wide Last Call because I expect the web types will want to go over 
this with a fine-toothed comb.   So while I still don't understand why 
you so strongly insist that r-component has to follow q-component, I 
feel like other issues are more substantive and I've done due diligence 
trying to point it out.   So maybe the thing to do is see if it gets 
raised again in IETF-wide Last Call, where I expect the whole 
interaction between URN components and URL components will get a lot of 
attention.

> perception (correct or not) is a symptom of the problem.

Ok, I have no idea what is meant by the above, and I'm not aware of my 
contradicting myself.  If you think it's important to explain, please 
explain.   If not, I won't worry about it.
> On the

To me it looks like there's a conflict between:

(a) the goal of being able to accommodate some new namespaces into the 
URN umbrella with minimal rewriting, and
(b) the goal of having URNs of arbitrary namespaces be resolvable and 
usable by clients.

If you make interpretation of q-component, r-component, or f-component 
dependent on the namespace (or worse, on the individual URN), this 
presents some awkward choices to client developers. Either

(i) clients will only resolve URNs of namespaces for which they know the 
interpretation of those components, or
(ii) clients interpret these components uniformly for all namespaces, 
this producing strange behavior for URNs that don't conform, or
(iii) some combination of the first two, with a generic default 
interpretation of these components and special-case behavior for some 
namespaces, or
(iv) clients have to ask resolution services how to interpret these 
components for a particular namespace or URN, or
(v) few if any clients implement URN resolution support at all, because 
it's just too flaky.

All of these choices seem to me to be highly undesirable.   I also think 
it's highly undesirable (and highly unlikely to get through IETF-wide 
Last Call) for q- or f-components of URNs to be interpreted in a 
substantially different way than queries and fragments are interpreted 
for URLs.

Choice (i) favors some kinds of URNs over others.   Essentially it makes 
it likely that clients will only support a small number of URN 
namespaces that happen to be in favor at the moment.   (much as clients 
tend not to support as many URL schemes as they used to)

Choice (ii) produces strange behavior when resolving URNs that don't fit 
the convention, and I have a hard time believing we would recommend it.

Choice (iii) is perhaps the most palatable and pragmatic, and I've said 
in the past that I could accept a small and limited number of namespaces 
that required variant handling.   But it would still be confusing to 
users who would have to learn that these components mean different 
things in different namespaces.

Choice (iv) is also confusing to users, adds a lot of complexity to the 
system, actually seems to encourage a lack of uniformity in use of these 
components, and creates new security vulnerabilities.

With uniform interpretation of URN components, I would say (for example) 
that a f-component should never be presented to a resolution service, 
and a q-component should only be presented to a resolution service if 
the service being requested were the actual resource.   For all other 
requests, only the assigned portion of the URN and the r-component 
should be presented.   That basically enforces uniformity of 
interpretation of components, which makes the system both simpler and 
more reliable and also more predictable to both implementors and users.

But if choice (iv) were adopted, nothing would prevent a resolution 
service from returning a completely different resource based on the 
q-component or f-component.   This strikes me as not only poor layering, 
but an architectural and security disaster - basically it's letting the 
resolution service assert control over interpretation of the resource at 
any time.   The last thing we need is an information system that 
facilitates either centralized control over resource interpretation or 
centralized monitoring of queries sent to resources.

Finally, I would regard choice (v) as a complete failure, since URNs 
were always intended to be resolvable.

So while I acknowledge the difficulties you've listed above - that some 
namespaces might not fit well into URN syntax with uniform 
interpretation of components, and that this presents some awkward 
choices for users of those namespaces - that risk needs to be balanced 
against the risk that accommodating variant interpretations of URN 
components impairs URN resolution.

And while I do want to be able to accommodate new URN namespaces, it's 
long been understood that URNs would, to the extent practicable, fit 
within both the syntax and semantic conventions that were established 
for URLs circa 1994.   RFC 2141 reserved both "?" and "#" precisely 
because it was expected that if these were used with URNs at all, they 
should be used to denote query and fragment (assuming we could figure 
out how to define it in a way that made sense with URNs), or at least 
something similar.    The notion that names from other namespaces might 
require encoding to be represented in URN form has also been established 
since at least 1999 (RFC 2611).   It was always understood that for some 
namespaces this would be awkward - either requiring percent-encoding 
which is ugly, or some mapping from one delimiter to another that might 
not be obvious.   So basically, I think the decisions regarding how much 
namespace-specific variation to permit in a URN (either syntax or 
interpretation) have been in place for around 20 years, and it seems 
late to be changing them much.   But even if that weren't a concern, the 
problems I've listed above would still need to be dealt with.

> welcome it.
The various "alternative" ABNF fragments I suggested in my previous 
message each attempted to fix this issue.

Keith