Subject: Re: [urn] 2141bis questions - (3) Section 4.1 "normalization" comment

From: John C Klensin <[email-masked]>
Date: Wed, 21 Jan 2015 09:37:22 -0500

--On Tuesday, December 23, 2014 11:48 -0700 Peter Saint-Andre -
&yet <[email-masked]> wrote:

>> Independent of the terminology it uses, the text in 3986
>> specifies that the characters that follow "%" in
>> percent-encoding must be converted to upper case and that
>> scheme names (and the host component) must be converted to
>> lower case. The text in the current version of 2141bis says
>> (construct 4) that the p-component is to have "case
>> normalization" applied, but doesn't specify upper case, lower
>> case, or case folding. We either need to get p-components out
>> of that list, making them case-sensitive, or need to choose
>> one of the three.
> 
> I lean toward removing it from the list, but I'd like to think
> about it more.

I lean that way too.  Have you thought further about this?  Does
anyone else have comments?

>>  Note that,
>> because of situations like "what happens with accent marks
>> when converting to upper case" (and its opposite of an
>> accent-free upper case character potentially mapping to two
>> or more lower case characters), arguments that
>> position-dependent characters in Arabic or other scripts
>> should really be treated as case variations, etc., this is
>> not a trivial decision.
 
> ASCII only mitigates that concern.

Indeed.

> We are using the syntax from RFC 3986, not RFC 3987.

The recent discussion on the IETF list calls that distinction
into question.  Just as the ties between RFC 2141 and RFC 1630
has caused us some confusion and problems since RFC 3986 was
adopted, it would be good to not be tightly bound to 3986 if it
is going to be replaced by one or more specifications with 3097
(or some other i18n treatment) rolled in.

>> If we care
>> about consistency with the reasoning of IDNA and probably
>> PRECIS, we should probably (but only probably) preserve
>> case-sensitivity as much as possible.
> 
> For p, q, and f components, I think that makes the most sense.

See above.  Again, anyone want to express another opinion?

    john

---

From: Mark Davis ☕️ <[email-masked]>
Date: Wed, 21 Jan 2015 16:12:45 +0100

While the situation with casing is not trivial, it is not as complicated as
sometimes portrayed.

Unicode does have a 'case normalization', called "case folding", which
erases case differences, and is widely deployed. In a few instances, it
does not always match local conventions for casing (I can go into details
if desired). Like normalization, it has stability guarantees (
http://unicode.org/policies/stability_policy.html#Case_Folding).

There are some cases where case folding does interact with NFKC, so there
is a special folding (normalization) that combines the two, called NFKC_CF.

These are described in Section 3.13 Default Case Algorithms of the Unicode
Standard (R4, R5) — http://www.unicode.org/versions/Unicode7.0.0/ch03.pdf.
There are also common libraries, such as ICU, that supply APIs for these.

​I do not wish to weigh in on whether comparison or storage should be
case-sensitive or insensitive​; just wanted to point out that there is not
a technical barrier if case-insensitivity is desired.