Subject: [precis] AD Evaluation of draft-ietf-precis-framework-15

From: Pete Resnick <[email-masked]>
Date: Mon, 7 Apr 2014 12:40:27 -0500

At long last, I finally completed my review of the framework document. 
My personal feeling here is that I should go ahead and do the IETF Last 
Call and consider the below equivalent to Last Call comments, and we can 
discuss them here during Last Call. If anyone has any concerns about 
that and thinks we really need to discuss something before I issue the 
Last Call, please say so now (i.e., in the next 24 hours). Otherwise, 
the Last Call will go out tomorrow.

-----
Substantive issue:
-----

4.1.5: I'm not thrilled with this section in general, but in particular 
I'm not sure what "mixed-direction strings are not supported" means. We 
do know how to process strings that contain characters with a mix of 
directionality. Such strings are sometimes a visual challenge, but not a 
processing challenge: RFC 5893 exists because IDNs want to use "." as a 
label separator yet have text display work in a context that is unaware 
of labels. Neither using 5893 nor considering any RTL character making 
the whole string RTL is a good recommendation for most cases. Not sure 
what to do about this.

-----
Editorial issues:
-----

Throughout: Change "Informational Note:" to "Note:". I don't see any of 
them for which it makes a difference.

3.1:

I would move the first paragraph down further in the section.

I would delete the parenthetical at the end of "Contextual Rule 
Required"; no need to introduce undefined terms here.

3.2.4 and 3.3.4: The SHALLs in here seem weird to me. Above, you don't 
say that a string with a Disallowed character "SHALL be rejected". If it 
were me, I'd simply say:

    Any code points that are not yet designated in the Unicode character
    set are considered Unassigned for purposes of the XXXClass, and such
    code points are to be treated as Disallowed.

4.1:

Change "MUST register" to "are registered". MUSTs for registration seem 
silly. (If you want to say, "Implementations MUST NOT use unregistered 
classes", you could, but I don't think you want to do that.)

Change "It is RECOMMENDED for profile names to be of the form" to "The 
naming convention for profile names is to use the form".

4.2: A bit of ABNF neatening:

OLD
       fullname = namepart [1*(1*SP namepart)]
       namepart = 1*(idpoint)
NEW
       fullname = namepart *(1*SP namepart)
       namepart = 1*idpoint

9.2: It might be nice to add section numbers (3.2 and 3.3) to the 
registrations for IdentifierClass and FreeformClass.

9.3: It might be nice to note the naming convention from 4.1 in the 
template.
-----

That's it.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: Marc Blanchet <[email-masked]>
Date: Mon, 7 Apr 2014 14:26:10 -0400

Le 2014-04-07 à 13:40, Pete Resnick <[email-masked]> a écrit :

> At long last, I finally completed my review of the framework document. My personal feeling here is that I should go ahead and do the IETF Last Call and consider the below equivalent to Last Call comments, and we can discuss them here during Last Call. If anyone has any concerns about that and thinks we really need to discuss something before I issue the Last Call, please say so now (i.e., in the next 24 hours). Otherwise, the Last Call will go out tomorrow.

my take from your comments below is that we can manage them during the IETF last call, so I suggest we proceed with IETF Last call. 

Regards, Marc.

> 
> -----
> Substantive issue:
> -----
> 
> 4.1.5: I'm not thrilled with this section in general, but in particular I'm not sure what "mixed-direction strings are not supported" means. We do know how to process strings that contain characters with a mix of directionality. Such strings are sometimes a visual challenge, but not a processing challenge: RFC 5893 exists because IDNs want to use "." as a label separator yet have text display work in a context that is unaware of labels. Neither using 5893 nor considering any RTL character making the whole string RTL is a good recommendation for most cases. Not sure what to do about this.
> 
> -----
> Editorial issues:
> -----
> 
> Throughout: Change "Informational Note:" to "Note:". I don't see any of them for which it makes a difference.
> 
> 3.1:
> 
> I would delete the parenthetical at the end of "Contextual Rule Required"; no need to introduce undefined terms here.
> 
> 3.2.4 and 3.3.4: The SHALLs in here seem weird to me. Above, you don't say that a string with a Disallowed character "SHALL be rejected". If it were me, I'd simply say:
> 
> Change "MUST register" to "are registered". MUSTs for registration seem silly. (If you want to say, "Implementations MUST NOT use unregistered classes", you could, but I don't think you want to do that.)
> 
> Change "It is RECOMMENDED for profile names to be of the form" to "The naming convention for profile names is to use the form".
> 
> 9.2: It might be nice to add section numbers (3.2 and 3.3) to the registrations for IdentifierClass and FreeformClass.
> 
> 9.3: It might be nice to note the naming convention from 4.1 in the template.
> -----
> 
> That's it.
> 
> pr
> 
> -- 
> _______________________________________________
> precis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/precis

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 14 Apr 2014 21:18:04 -0600

> the Last Call will go out tomorrow.

Great, thanks Pete!

> the whole string RTL is a good recommendation for most cases. Not sure

See:

http://www.ietf.org/mail-archive/web/precis/current/msg00553.html

and

http://www.ietf.org/mail-archive/web/precis/current/msg00557.html

I am not sure how helpful it is to distinguish between processing 
challenges and presentation challenges.

I will ponder this further and reply again.

> them for which it makes a difference.

Sure.

> I would move the first paragraph down further in the section.

Seems we could delete it altogether.

> Required"; no need to introduce undefined terms here.

Sure.

>     code points are to be treated as Disallowed.

Works for me.

> naming convention for profile names is to use the form".

That's better, yes.

>        fullname = namepart *(1*SP namepart)

Noted.

> 9.3: It might be nice to note the naming convention from 4.1 in the

Will do.

> That's it.

Excellent, thanks.

Peter

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 15 Apr 2014 18:32:26 -0600

> I will ponder this further and reply again.

I recall discussion at one of our in-person meetings about bidi. As I 
recall, John Klensin sagaciously pointed out that if the PRECIS WG tried 
to define a new rule for directionality we would almost certainly get it 
wrong, and that it was better to use the Bidi Rule from RFC 5893 than to 
design something new. Even though it might be true that the Bidi Rule 
might not be a good recommendation for most cases, I do not have 
confidence in our ability to come up with a better recommendation.

Would you say that the Bidi Rule works for domain names because "." is a 
label separator? In particular, my understanding of the Bidi Rule is 
that it doesn't allow mixed-direction *labels*, although it does allow 
mixed-direction domain names (where the direction changes at the point 
of the label separator).

It seems to me that not supporting mixed-direction strings in PRECIS is 
consistent with RFC 5893. I realize that's not ideal, but it might be 
the best that *we* can do *now*. Whether some other group could do 
something better in the future might be irrelevant.

>> them for which it makes a difference.

BTW, in RFC 6120/6121 I used a personal convention of three kinds of 
notes: informational notes, interoperability notes, and security 
warnings. My use of "Informational Note:" here derives from that, 
although this document doesn't have any interoperability notes so it 
might seem a bit strange to include the adjective.

Peter

---

From: Pete Resnick <[email-masked]>
Date: Fri, 18 Apr 2014 17:29:09 -0500

>> http://www.ietf.org/mail-archive/web/precis/current/msg00553.html

...wherein Martin seems to say that this is only allowing fully RTL and 
fully LTR (i.e., no mixed direction strings).

>> http://www.ietf.org/mail-archive/web/precis/current/msg00557.html

...wherein you reply that not allowing mixed direction strings was 
intended. :-(

>> challenges and presentation challenges.

There are scant few processing challenges, at least from my 
long-ago-in-a-galaxy-far-far-away memory of the last time I wrote any 
code in this area. They are all (again, as far as I remember) 
presentation challenges.

> recall, John Klensin sagaciously pointed out that if the PRECIS WG 
> tried to define a new rule for directionality we would almost 
> certainly get it wrong, and that it was better to use the Bidi Rule 
> from RFC 5893 than to design something new. Even though it might be 
> true that the Bidi Rule might not be a good recommendation for most 
> cases, I do not have confidence in our ability to come up with a 
> better recommendation.

On that we can agree. I am sure I do not want to come up with other 
rules for inclusion (because those rules are completely dependent on the 
presentation challenges you are or are not willing to contend with). I 
guess in the end I'd just like the text in 4.1.5 softened to say, "You 
can have a directionality rule for your profile anything from 'whatever 
mix of RTL and LTR characters you like' if you don't care how they're 
presented, all the way to the complicated Bidi Rule from 5893 if you 
expect users to be typing in protocol elements, and you have to deal 
with strings with separator punctuation and presentation tools that 
don't know about them."

> Would you say that the Bidi Rule works for domain names because "." is 
> a label separator? In particular, my understanding of the Bidi Rule is 

Correct. The *only* reason for the 5893 Bidi Rule is because they wanted 
domain names to display identically in both LTR and RTL contexts when 
the display engine didn't understand that the "." was a label separator. 
(Well, I guess that's two reasons combined in one.)

> It seems to me that not supporting mixed-direction strings in PRECIS 
> is consistent with RFC 5893.

It is consistent with 5893, but I am saying that there are a number of 
cases where it's the wrong thing to do. There are protocols that 
shouldn't care whether that strings display inconsistently when the user 
changes from RTL to LTR contexts. What's the problem if I want my handle 
to be:

pee e tee e dash pei samekh het

which if I type them in that order gets you:

Pete-פסח

which in a LTR context will display to you as (reading them off from 
left to right):

pee e tee e dash het samekh pei

or in an RTL context will display to you as (reading them off from left 
to right):

het samekh pei dash pee e tee e

The only problem is if you are worried that people will type them in 
wrong because they don't know what directional context they're in. If 
it's just my handle name and I'm the only one typing it, you should 
leave me be.

> I realize that's not ideal, but it might be the best that *we* can do 
> *now*. Whether some other group could do something better in the 
> future might be irrelevant.

Like I said, I'm not looking for a new rule. Just less overt pushing 
that protocols never allow mixed direction text and that they should 
lean toward 5893.

pr

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 18 Apr 2014 20:13:29 -0600

> that protocols never allow mixed direction text and that they should

I'm still worried about foot-guns. Part of what we've tried to do in 
PREICS is to actively prevent people who don't necessarily know much 
about internationalization from shooting themselves in the foot. Yes, we 
could tell those folks "it's fine to allow whatever mix of RTL and LTR 
characters you like, as long as you don't care how they're presented" - 
but we know that could cause serious confusion if more than one actor in 
said protocol might view the same string. Do we really think that's a 
good idea?

Peter

---

From: "Martin J. Dürst" <[email-masked]>
Date: Mon, 21 Apr 2014 20:12:36 +0900

[Sorry for the top-posting, but it's way easier for those who want to 
see just the next bit of the argument.]

On 2014/04/19 11:13, Peter Saint-Andre wrote:

Even strings without a mix of RTL and LTR characters can lead to 
confusing display, e.g. if they contain digits at the start or at the 
end. Don't remember to what extent they are excluded in IDNs.

As long as Precis strings are displayed only in isolation, the 
restrictions may not be that important, but the moment any of these 
strings is displayed with something else, maybe with an entervening 
punctuation, these restrictions will be very helpful, although not 
perfect (they aren't perfect for IDNs, either).

So I think the restrictions are a good thing.

Regards,  Martin.

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 21 Apr 2014 11:44:59 -0600

> So I think the restrictions are a good thing.

I still agree.

Looking back at the text, I think Pete might have a point. I suggest the 
following change.

###

OLD
    The directionality rule of a profile specifies which strings are to
    be considered left-to-right (LTR) and right-to-left (RTL), and the
    allowable sequences of characters in LTR and RTL strings (see Unicode
    Standard Annex #9 [UAX9]); note that mixed-direction strings are not
    supported, since there is currently no widely accepted and
    implemented solution for the processing and display of mixed-
    direction strings.  Possible rules include, but are not limited to,
    (a) considering any string that contains a right-to-left code point
    to be a right-to-left string, or (b) applying the "Bidi Rule" from
    [RFC5893].

NEW
    The directionality rule of a profile specifies which strings are to
    be considered left-to-right (LTR) and right-to-left (RTL), and the
    allowable sequences of characters in LTR and RTL strings (see Unicode
    Standard Annex #9 [UAX9]).  Possible rules include, but are not
    limited to, (a) considering any string that contains a right-to-left
    code point to be a right-to-left string, or (b) applying the "Bidi
    Rule" from [RFC5893].

    Mixed-direction strings are not directly supported by the PRECIS
    framework itself, since there is currently no widely accepted and
    implemented solution for the processing and safe display of mixed-
    direction strings.  An application protocol that uses the PRECIS
    framework (or an extension to the framework) could define methods for
    handling mixed-direction strings; however, such methods are outside
    the scope of the framework.

###

Peter

---

From: Pete Resnick <[email-masked]>
Date: Mon, 21 Apr 2014 18:44:59 -0500

On 4/21/14 12:44 PM, Peter Saint-Andre wrote:
>>> I'm still worried about foot-guns.

Yeah, I think the foot-gun argument is always the trump card. So I will 
relent on asking for any significant change.

>> end. Don't remember to what extent they are excluded in IDNs.

IDN Labels can't begin with a number. LTR labels can't have any RTL 
numbers. So the only strange case is RTL strings, which can end with 
number, but the rest of the rules make bad behavior pretty hard.

I'm convinced on the restrictions. On the text:

>    Standard Annex #9 [UAX9]).

This is the part I'm still having trouble with. An "LTR string" and an 
"RTL string" are not defined entities in Annex 9; AFAICT, are entirely 
5893 inventions, and 5893 only talks in terms of "labels", not 
"strings". How about this:

    The directionality rule of a profile specifies how to treat strings
    containing left-to-right (LTR) and right-to-left (RTL) characters
    (see Unicode Standard Annex #9 [UAX9]). A profile usually specifies a
    directionality rule that restricts strings to be entirely LTR strings
    or entirely RTL strings and defines the allowable sequences of
    characters in LTR and RTL strings.

The rest of this is fine:

>    Possible rules include, but are not

s/the framework/this framework ?

pr

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 21 Apr 2014 18:40:11 -0600

On 4/21/14, 5:44 PM, Pete Resnick wrote:
> "strings".

Good catch!

> How about this:

Yes, that's better.

> s/the framework/this framework ?

+1

Peter

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 22 Apr 2014 16:55:08 +0900

Hello Peter/Pete/others,

>     implemented solution for the processing and safe display of mixed-

Please remove "processing". Processing mixed-direction strings isn't a 
problem at all. Display is the problem.

>     the scope of the framework.

I'm not sure "handling" and "methods" are the right word, unless these 
words are used in other parts of the text. They sound a bit too 
procedural for simple restrictions. In theory, it's possible to define 
new ways of displaying things, but good luck for getting these deployed :-(.

I'd prefer if the text here ended with some clear caution. Maybe 
something like changing the last part of the last sentence to

"however, such methods are outside the scope of the framework and should 
only be introduced after carefully studying bidirectional display problems."

Regards,   Martin.

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 22 Apr 2014 09:30:04 -0600

> problem at all. Display is the problem.

Correct.

> only be introduced after carefully studying bidirectional display

I originally had something like "such methods are fraught with 
difficulty so don't try to solve the problem unless you really know what 
you're doing". Adding the text you propose is fine with me.

Peter

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 22 Apr 2014 09:36:00 -0600

> you're doing". Adding the text you propose is fine with me.

So, how is this?

    Mixed-direction strings are not directly supported by the PRECIS
    framework itself, since there is currently no widely accepted and
    implemented solution for the safe display of mixed-direction strings.
    An application protocol that uses the PRECIS framework (or an
    extension to the framework) could define better ways to present
    mixed-direction strings; however, that work is outside the scope of
    this framework and would likely require a great deal of careful
    research into the problems of displaying bidirectional text.

Peter