Subject: Re: [urn] 2141bis questions - (2) Section 3.3.2 q-component clarification

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 06 Jan 2015 18:32:33 +0900

On 2014/12/24 02:49, Andrew Newton wrote:
> On Tue, Dec 23, 2014 at 12:13 PM, John C Klensin <[email-masked]> wrote:
>
>>        "Note that "?" is not a special character after its
>>          first instance introduces a q-component (assuming one is
>>          allowed by the namespace).   "?" appearing later in a
>>          q-component or in an f-component does not require
>>          %-escaping except as required on a per-namespace basis."
>>
>> Is that roughly correct?  What we want?
>>
>
> <chair hat="off"/>
>
> Using my possibly faulty recollection of 3986, I thought "?" was a general
> delimiter

That's correct, see http://tools.ietf.org/html/rfc3986#section-2.2.

> and those needed to be escaped no matter what when not being used
> as a delimiter.

No. See http://tools.ietf.org/html/rfc3986#section-3.4 and
http://tools.ietf.org/html/rfc3986#section-3.5.

> From an implementation standpoint, certainly escaping "?"
> everywhere is easier than only escaping it in certain portions of the UR*.

It depends. Certainly, you don't want to escape the '?' that starts the 
q-component itself.

Regards,   Martin.

P.S.: Checking the spec beats recollection (faulty or not) every time.

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 06 Jan 2015 18:41:26 +0900

On 2014/12/24 03:41, Peter Saint-Andre - &yet wrote:
> On 12/23/14, 10:13 AM, John C Klensin wrote:
>> In section 3.3.2 on q-component, it seems reasonable to add a
>> bit of explanation, thereby providing some protection from
>> people who might assume they know what 3986 says without
>> checking it

Probably a good idea.

>> (especially given the ABNF issues with that
>> document).

[I think we had a policy of avoiding such remarks.]

>>  I propose something like:
>>
> As I read RFC 3986, what you propose is indeed roughly correct as an
> informational note.

I suggest shortening it to something like:

Note that "?" can be used without %-encoding inside q-components and 
f-components.

This removes the "except as required on a per-namespace basis", but I 
think it would be a really terribly bad idea to make escaping rules 
scheme dependent.

>> There is also another issue with that paragraph.  Given the
>> "syntax only" principle, it is not clear that unquoted "?"

[I guess by 'unquoted', you mean not %-encoded?]

must
>> be prohibited from appearing in the NSS and/or p-component
>
> I think it must be prohibited, given the ABNF definitions in RFC 3986
> that we borrow here.
>
>      NSS           = 1*(pchar)
>      p-component   = "/" path-absolute
>      path-absolute = "/" [ segment-nz *( "/" segment ) ]
>      segment       = *pchar
>      pchar         = unreserved / pct-encoded / sub-delims / ":" / "@"
>
> (no gen-delims allowed there)
>
>> (if
>> we think that is distinct).
>
> I think the p-component is not part of the NSS.
>
>      namestring    = assigned-name
>                      [ p-component ]
>                      [ q-component ]
>                      [ f-component ]
>      assigned-name = "urn" ":" NID ":" NSS
>
>> The argument for prohibiting it is
>> potential confusion.  The argument for allowing it is that it
>> might be needed for externally-defined name spaces for which
>> %-encoding would either be clutter or actually problematic.
>
> I would favor prohibiting it.

Of course it must be prohibited, because the first non-%-encoded '?' 
starts the query part (q-component). You cannot have the cake and eat it 
too.

Regards,   Martin.

---

From: Andrew Newton <[email-masked]>
Date: Tue, 6 Jan 2015 09:46:50 -0500

On Tue, Jan 6, 2015 at 4:32 AM, "Martin J. Dürst" <[email-masked]>
wrote:

> http://tools.ietf.org/html/rfc3986#section-3.5

As I said, when not being used as a delimiter... which is what is going on
in 3.4 and 3.5. Or am I missing some finer point you are trying to make?

> It depends. Certainly, you don't want to escape the '?' that starts the

Obviously! But that's not the issue given the text quoted above from the
draft.

-andy

---

From: John C Klensin <[email-masked]>
Date: Wed, 21 Jan 2015 09:11:56 -0500

(top post and summary) 

Martin,

Thanks (and apologies for the delayed response).  It was
precisely the syntax in Section 3.4 that led to my comment/
question.

Peter and Andy,

Just to review what the ABNF in 3986 actually says:

	URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

(from the top of Section 3).  Note that the "?" that introduces
the query is _not_ part of the query.

   query = *( pchar / "/" / "?" )

(from the middle of Section 3.4 and as quoted by Martin)  Unless
I'm having _really_ severe reading problems, that means that "?"
may freely appear in query parts without quoting.  

Note that the production for <query> above allows both
    foo:hier-part-stuff?#   and
    foo:hier-part-stuff?//
I wonder how many URI parsers and systems can really deal with
those forms and whether we should be as permissive?

<co-editor and all other hats off>  
I'm pushing this for a specific reason, discussed at length on
the list some months ago.  If we really need the flexibility
that having a number of different communities of URN uses seems
to imply, especially if we anticipate communities and uses that
haven't been heard from yet, there are really strong arguments
for allowing the query string to be a series of name-value
pairs.   As I read 3986, nothing prevents establishing that
principle on a per-scheme basis.   Even if we decided that, for
URN purposes, there were two types of queries -- unstructured
ones and name-value pairs, we could have ABNF of 

  URN-query = unstructured-URN-query / nv-URN-query
  unstructured-URN-query =  *( pchar / "/"  )
  nv-URN-query = nv-pair *( "?" nv-pair )
  nv-pair = name "=" value

with some reasonable definition of "name" and "value".  If we
didn't feel a need to allow both, we could eliminate
unstructured-URN-query and collapse the productions.

Note that the above would conform to 3986 syntax because the
union of the two forms conforms to the 3986 definition of
"query".

</co-editor...> 

As a final piece of this, Peter wrote:

>...

>   assigned-name = "urn" ":" NID ":" NSS

It seems to me that is our choice and that we could, as easily
and without violating anything about 3986 syntax, define the
above as:

     namestring    = assigned-name
                     [ q-component ]
                     [ f-component ]
     assigned-name = "urn" ":" NID ":" NSS [ q-component ]

It would be a funny way to overlay 3986's syntax but, AFAICT,
completely valid, especially given the issues with hier-part and
the RHS-matching for <path>, which are not allowed by ABNF.

So I think the question is still open although my preference is
more clear than it was when I asked.

    john

--On Tuesday, January 06, 2015 18:32 +0900 "\"Martin J.
Dürst\"" <[email-masked]> wrote:

>> Using my possibly faulty recollection of 3986, I thought "?"
>> was a general delimiter
> 
> That's correct, see
> http://tools.ietf.org/html/rfc3986#section-2.2.
> 
>> and those needed to be escaped no matter what when not being
>> used as a delimiter.
> 
> It depends. Certainly, you don't want to escape the '?' that
> starts the q-component itself.
> 
> Regards,   Martin.
>

---

From: Sean Leonard <[email-masked]>
Date: Wed, 21 Jan 2015 10:27:07 -0800

On 1/21/2015 6:11 AM, John C Klensin wrote:
> [query stuff]
> principle on a per-scheme basis. [...]

Hate to put a wet blanket on all these desiderata, but honestly, who 
really needs or wants to use q-components in URNs?

Let's say you can use q-components, but any common structure will only 
be considered when actual use cases and communities materialize. Until 
then, percent-encode the "?" in a NSS.

The thing is that the definition of URN is that the NSS is basically an 
opaque identifier. As I have argued, NSSes have "dividers", but not 
"hierarchy" (e.g.: urn:ietf:rfc:3986 -- the : in rfc:3986 is just a 
divider for convenience). q-components shouldn't be part of the NSS 
since they represent parameter thingies.

"Worry about stuff that matters, when it matters."

Sean