Subject: Re: [precis] U+19DA

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 29 Aug 2014 13:31:40 -0600

On 7/30/14, 12:48 PM, Joe Hildebrand (jhildebr) wrote:
> Draft-17 of the precis-framework doc says:
>
> "The PRECIS framework, which is defined in terms of the latest version
>     of Unicode as of the time of this writing (6.3), treats the character
>     U+19DA NEW TAI LUE THAM as DISALLOWED.  Implementers need to be aware
>     that this treatment is different from IDNA2008 (originally defined in
>     terms of Unicode 5.2), which treats U+19DA as PVALID."
>
> RFC 6452 amends IDNA to say:

Actually, RFC 6452 does not update any of the IDNA2008 specification. It 
does make note of changes to several Unicode code points, though.

> "1.3.  U+19DA NEW TAI LUE THAM DIGIT ONE
>
>     The GeneralCategory for this character changes from Nd to No.  This
>     implies that the derived property value changes from PVALID to
>     DISALLOWED."
>
> So the "PVALID" part of the precis draft likely needs to change.

Yes, that paragraph is poorly worded. I suggest the following substitution:

    Three Unicode code points underwent changes in their GeneralCategory
    between Unicode 5.2 (current at the time IDNA2008 was originally
    published) and Unicode 6.0, as described in [RFC6452].  Implementers
    might need to be aware that the treatment of these characters differs
    depending on which version of Unicode is available on the system that
    is using IDNA2008 or PRECIS, and that other such differences are
    possible between the version of Unicode current at the time of this
    writing (7.0) and future versions.

> Further, I get FREE_PVAL for U+19DA, because it now hits the
> OtherLetterDigits rule (R), since its general category is No.

Correct.

Peter

---

From: John C Klensin <[email-masked]>
Date: Fri, 29 Aug 2014 16:23:26 -0400

--On Friday, August 29, 2014 13:31 -0600 Peter Saint-Andre
<[email-masked]> wrote:

>> "The PRECIS framework, which is defined in terms of the
>> latest version of Unicode as of the time of this writing
>>     (6.3), treats the character U+19DA NEW TAI LUE THAM as
>>     DISALLOWED.  Implementers need to be aware that this
>>     treatment is different from IDNA2008 (originally defined
>>     in terms of Unicode 5.2), which treats U+19DA as PVALID."
>> 
> Actually, RFC 6452 does not update any of the IDNA2008
> specification. It does make note of changes to several Unicode
> code points, though.
> 
>>     The GeneralCategory for this character changes from Nd to
>>     No.  This implies that the derived property value changes
>>     from PVALID to DISALLOWED."
>> 
 
> Three Unicode code points underwent changes in their
> GeneralCategory between Unicode 5.2 (current at the time
> IDNA2008 was originally published) and Unicode 6.0, as
> described in [RFC6452]. Implementers might need to be aware
> that the treatment of these characters differs depending on
> which version of Unicode is available on the system that is
> using IDNA2008 or PRECIS, and that other such differences are
> possible between the version of Unicode current at the time of
> this writing (7.0) and future versions.

Peter, Patrik should check me on this, but I believe that would
be more clear and precise if it said something more like "...
Implementers of libraries and other systems that directly use
the rule sets rather than derived tables might need to be
aware...".

You are correct that RFC 6452 didn't change IDNA2008 but,
because we decided to _not_ apply a backward compatibility rule
to U+19DA, the derived property value for U+19DA in the IANA
copies of the derived tables synchronized to Unicode 6.0 (and
any other tables created directly from 6.0 _did_ change.  It is
those table that are at issue and not actually the version of
Unicode.  If the 6.0 tables were applied on a system that was
using Unicode 5.2, no harm would be done but U+19DA, a code
point that is assigned in both versions, would be DISALLOWED
even though a table calculated from 5.2 would have it as PVALID.

>...

  john

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 30 Aug 2014 08:37:36 +0200

> the rule sets rather than derived tables might need to be

Does not really matter if one include "libraries" in there, because it depends of course on what the library do. If the library implement IDNA2008, IDNA2008 on top of Unicode 6.0, IDNA2008 on Unicode 5.2, PRECIS etc.

What I think is important is that the reader do understand that IDNA2008 is _really_ a set of algorithms that when applied to a specific version of Unicode get for each codepoint a derived property value. This implies the same algorithms applied to a different version of Unicode will generate a different set of derived property values (as there are differences between the two versions of Unicode). What IETF is doing is keeping track of those differences, and draw conclusions of whether the differences are acceptable, or whether the differences are large enough to force changes to IDNA2008. So far (up until Unicode 7.0) the conclusion has always been that the differences are acceptable _ALTHOUGH_ in between 5.2 and 6.0 the derived property value for a few individual codepoints changed in an inconsistent way.

> even though a table calculated from 5.2 would have it as PVALID.

Exactly!

   Patrik