Subject: [precis] Directionality rule (Was: Review of current document draft-ietf-precis-framework-20)

From: Pete Resnick <[email-masked]>
Date: Mon, 9 Feb 2015 11:20:20 -0600

Crap. Somehow I missed this discussion back in December, and didn't 
notice it before I sent out Last Call. When I went to put in my new 
ballot, I saw the change.

On 12/7/14 9:49 PM, Peter Saint-Andre - &yet wrote:
>>> 5.2.5.  Directionality Rule
>>>
>>>
>>> The directionality rule of a profile specifies how to treat
>>> strings containing left-to-right (LTR) and right-to-left (RTL)
>>> characters (see Unicode Standard Annex #9 [ UAX9 ]).  A profile
>>> usually specifies a directionality rule that restricts strings to
>>> be entirely LTR
>>> strings or entirely RTL strings and defines the allowable
>>> sequences of characters in LTR and RTL strings.  Possible rules
>>> include, but are not limited to, (a) considering any string that
>>> contains a right- to-left code point to be a right-to-left string,
>>> or (b) applying the "Bidi Rule" from [ RFC5893 ].
>>
>> One can not restrict to only LTR or RTL as some code points are
>> neutral regarding directionality.
>>
>> See RFC5893. This was one of the mistakes in IDNA2003.
>
> I think you're right that the foregoing text is not quite correct.
>
> And in any case, as John Klensin reiterated at the mic in a recent 
> PRECIS WG session, if we define something other than the Bidi Rule 
> then we'll likely get it wrong. I think that text predates John's 
> comments and hasn't been updated..

While I agree that the text that I had proposed earlier is not correct 
as it stands for the reason Patrik identifies, setting it back to:

    The directionality rule of a profile specifies how to treat strings
    that contain right-to-left (RTL) characters (see Unicode Standard
    Annex #9 [UAX9]).  In general this document recommends applying the
    "Bidi Rule" from [RFC5893] to strings that contain RTL characters.

(i.e., what was there before the AD Evaluation) was not correct. There 
was, I thought, agreement on the problem I was pointing out, and text 
inserted to address it. Encouraging the use of 5893 is a bad thing 
outside of the DNS. As we discussed at the time, the only reason that 
the 5893 Bidi Rule is so restrictive is because the (a) IDNA is already 
extremely restrictive in the characters it allows and (b) for DNS 
purposes, they wanted to make the display of labels around "."s more 
sane. Recommending 5893 it in other contexts is far too restrictive.  
Let me make another attempt to correct the above text without getting 
the bit about "only LTR or RTL" wrong:

    The directionality rule of a profile specifies how to treat strings
    containing combinations of characters with different directionality
    (e.g., left-to-right (LTR) characters, right-to-left (RTL)
    characters, and/or neutral directionality characters; see Unicode
    Standard Annex #9 [ UAX9 ]).  A profile usually specifies a
    directionality rule that restricts strings to contain combinations of
    either entirely LTR and neutral characters or entirely RTL and
    neutral characters. A directionality rule can also define the
    allowable sequences of characters in such strings.  The "Bidi Rule"
    from [RFC5893] is a particularly restrictive example of such a
    directionality rule.

I am not wedded to the above, but what I would hate to see was this 
document go out the door with 5893 as the *recommendation*. I am not 
about to hold up the document because of this, but if we can come up 
with text that does not recommend 5893, it would make me very happy.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Mon, 09 Feb 2015 11:16:47 -0700

On 2/9/15 10:20 AM, Pete Resnick wrote:
> outside of the DNS.

I think John Klensin made a good point at the mic at a semi-recent 
meeting (London?) that, if we try to define a new bidirectionality rule, 
we are likely to do it wrong.

> As we discussed at the time, the only reason that
> sane. Recommending 5893 it in other contexts is far too restrictive. Let
> me make another attempt to correct the above text without getting the
> bit about "only LTR or RTL" wrong:
>
> document go out the door with 5893 as the *recommendation*.

What I would hate to see is leaving directionality handling up to the 
profiles without much guidance about what to do.

> I am not

Agreed on the goal, I think.

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Mon, 09 Feb 2015 12:56:12 -0700

> profiles without much guidance about what to do.

Further thoughts...

If I read your proposed text correctly, you are saying three things:

1. A profile needs to define which directionality rule applies.

2. A profile can define its own directionality rule.

3. A profile can/might/may/should use the following rule:

A string MUST contain LTR characters and neutral characters only or 
contain RTL characters and neutral characters only, but MUST NOT contain 
both LTR and RTL characters.

(Does that rule have a name?)

4. The directionality rule defined by a particular profile may specify 
further restrictions, or may use a more restrictive rule (such as the 
Bidi Rule).

I agree with (1). I think (2) is a bad idea for the reasons John 
explained. I think (3) is an acceptable rule and I think we might want 
to give profiles a choice between that and the Bidi Rule (thus 
discouraging new directionality rules). I think (4) is a bad idea if it 
assumes (2).

Peter

---

From: Pete Resnick <[email-masked]>
Date: Mon, 9 Feb 2015 14:42:16 -0600

> 1. A profile needs to define which directionality rule applies.

Yep. I wish we didn't have to say that (because this is about display, 
not processing), but I am definitely in the rough about being able to 
say "mixed-direction is fun; have at it!".

> 2. A profile can define its own directionality rule.

Well, we have given that ability, but I don't think it's a good idea for 
a profile to do so. SHOULD NOT.

> contain RTL characters and neutral characters only, but MUST NOT 
> contain both LTR and RTL characters.

As Klensin points out to me out of band, I screwed that up too, since 
some digits and punctuation are LTR but would certainly violate the 
Principle of Least Astonishment if we disallowed them alongside RTL 
characters. So at the very least, I am going to chew on this with John 
for a bit and see if there *is* some simple formulation of a 
directionality rule short of 5893.

> (Does that rule have a name?)

"Pete's lousy attempt to formulate a simple rule"?

Consider this on hold.

> further restrictions, or may use a more restrictive rule (such as the 

Same as 2 above. We've given them the ability to do so, but it would be 
a really bad idea if they did. SHOULD NOT.

> discouraging new directionality rules). I think (4) is a bad idea if 
> it assumes (2).

Yep.

This might turn out that I'm convinced there is nothing better to say 
than "use 5893". That will cause me to weep into my beer. But give me a 
bit to chat with Klensin and we'll see if we can come up with sanity.

(*Sigh*)

pr

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Mon, 09 Feb 2015 15:53:43 -0700

On 2/9/15 1:42 PM, Pete Resnick wrote:

> than "use 5893".

That's where we ended up the last time we looked into the problem.

> That will cause me to weep into my beer.

Sorry.

> But give me a

I agree that the Bidi Rule from RFC 5893 is designed for use with DNS 
labels, and that it's not a perfect fit for PRECIS strings. But it might 
be the best we can do right now.

Peter

---

From: Pete Resnick <[email-masked]>
Date: Wed, 11 Feb 2015 15:33:15 -0600

On 2/9/15 4:53 PM, Peter Saint-Andre - &yet wrote:
>> But give me a bit to chat with Klensin and we'll see if we can come 
>> up with sanity.
>
> labels, and that it's not a perfect fit for PRECIS strings. But it 
> might be the best we can do right now.

OK, here is some text that I worked on with John. We think this has the 
desirable properties that it is both as close to correct as we're going 
to get *and* what the WG intended. This is meant to be a replacement for 
all of the text in 5.2.5:

    The directionality rule of a profile specifies how to treat strings
    containing what are often called "right-to-left" (RTL) characters.
    RTL characters come from scripts that are normally written from right
    to left and are considered by Unicode to, themselves, have right to
    left directionality. Strings containing RTL characters often also
    contain "left-to-right" (LTR) characters, such as numerals, as well
    as characters without directional properties.  Consequently, such
    strings are known as "bidirectional strings".  Using bidirectional
    strings in different layout contexts may yield display results that,
    while predictable to those who understand the display rules, may be
    counter-intuitive to casual users. Therefore some profiles may wish
    to restrict their use by specifying a directionality rule.

    The PRECIS framework does not directly address how to deal with
    bidirectional strings, since there is currently no widely accepted
    and implemented solution for the safe display of arbitrary
    bidirectional strings beyond the general Unicode bidirectional
    specification [UAX9]. Rules for management and display of
    bidirectional strings have been defined for domain name labels and
    similar identifiers in the "Bidi Rule" from [RFC5893]. If a profile
    really needs to specify a directionality rule, unless a great deal of
    careful research into the problems of displaying bidirectional text
    is done (which is outside of the scope of this framework), this
    document generally recommends using the  "Bidi Rule" from [RFC5893]
    for profiles based on IdentifierClass, and for profiles based on
    FreeformClass or other situations in which IdentifierClass is not
    appropriate, that consideration be given to using UAX #9 directly.

What do you think?

pr

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Wed, 11 Feb 2015 22:44:32 +0100

>    appropriate, that consideration be given to using UAX #9 directly.

I am not happy with "this document generally recommends"; it's not clear
if this is the recommendation or if the recommendation is elswhere; it's
also not clear if this a RFC 2119 RECOMMENDED or something weaker, made
worse by "generally" and also the preceding "unless". Should be easy to
rephrase, but I do not have a good idea right now.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
D-10243 Berlin · PGP Pub. KeyID: 0xA4357E78 · http://www.bjoernsworld.de
 Available for hire in Berlin (early 2015)  · http://www.websitedev.de/

---

From: John C Klensin <[email-masked]>
Date: Wed, 11 Feb 2015 19:14:16 -0500

--On Wednesday, February 11, 2015 22:44 +0100 Bjoern Hoehrmann
<[email-masked]> wrote:

> This is meant to be a replacement for 
>>    The directionality rule of a profile specifies how to
>>    treat strings containing what are often called
>>    "right-to-left" (RTL) characters. RTL characters come from
>...
>>    The PRECIS framework does not directly address how to deal
>>    with bidirectional strings, since there is currently no
>>    widely accepted and implemented solution for the safe
>>    display of arbitrary bidirectional strings beyond the
>>    general Unicode bidirectional specification [UAX9]. Rules
>...

> I am not happy with "this document generally recommends"; it's
> not clear if this is the recommendation or if the
> recommendation is elswhere; it's also not clear if this a RFC
> 2119 RECOMMENDED or something weaker, made worse by
> "generally" and also the preceding "unless". Should be easy to

Björn,

Let my try to explain this from the perspective of someone who
has been greatly concerned with multiple aspects of the PRECIS
work (I assume no one how has been following this list or the WG
will be surprised by that), but who is now trying to work with
Pete, the document authors, and the WG co-chairs to get this
wrapped up.  Let me also do so, at least in part, less from  the
perspective of the IETF's i18n efforts and more from that of
someone who was expected to be able to at least partially get
around in multiple languages from a rather tender age --
languages that were, by my great good (or bad) luck, written in
three separate scripts, one of which is written right to left.

For at least the last several centuries, no calligrapher and
probably no "ordinary" reader or writer of a given script has
been particularly troubled by its directionality properties.
One just writes it as it is written, whether that is left to
right, right to left, top to bottom, or some variation on what
is sometimes called serpentine.  However, when these things are
coded for computer systems in some set of "uniform" rules,
things get complicated.  We have to worry about order of bits or
bytes in transmission, storage order, potentially the difference
between storage order and rendering order, and all sorts of
subtle variations on characters that may result from what is
next to them along some dimension.  

The result is that the Unicode Bidi rules are very complex, to
the point that I often suspect that only a few people completely
understand them and that some of those may not understand all of
them three months in a row.  Certainly the observation that UAX
#9 has been revised several times, presumably to make it more
comprehensible and clarify fine points and edge cases,
contributes to a "I hope this time they got it right" sensation.
It would be easy to cast blame for this in Unicode's direction
but, as with a lot of other aspects of Unicode, I believe that a
different "universal" character set, constructed with a
different set of rules, would simply favor different tradeoffs
and exhibit a different set of problems/ difficulties.

Now back to PRECIS and the text Pete posted.

First, one of the fundamental problems with the WG, at least
IMO, is that there have been a lot of participants, probably a
considerable majority, who have been unwilling to dig deeply
into this and other complex issues, concentrating instead on
making sure their own favorite language or script could be
accommodated in some reasonable way or on a "please just tell me
what to do; I don't want to have to understand it or think about
it... as long as it doesn't mess up my deployed implementations"
style of requirement.  

Second, while this is up to Pete, the WG, and its leadership it
is getting late enough in the process that "I am not happy with"
--in the absence of specific suggestions -- is not very helpful.
We need to at least understand what you would like done, not
just that you don't like what is there (or proposed).  

If you (and, others if they agree with you), merely want the
statement to be very clear that it is referring to Section 5.2.5
and that is all that is going to be said as far as PRECIS is
concerned, I imagine we (particular Peter and Marc) can figure
out how to do that even though all the ways I can think of at
the moment would sound pretty pedantic.

If your problem (or desire) is a stronger and more specific
recommendation, then there are a couple of problems.  There are
families and layers of peculiar and/or edge cases.   I think it
is reasonably unlikely that a user or implementer who regularly
reads and writes a particular RTL script and has sufficient
wisdom and maturity to be sensible and conservative about
identifiers rather than trying to figure out what "weird stuff"
might work is going to encounter or generate them.  As long as
one avoids those cases, both RFC 5893 and UAX #9 do about what a
reasonable person with experience with the relevant scripts on
computer systems is going to expect.

On the other hand, if one wants to discuss the edge cases, it
isn't going to be possible to do it in a few paragraphs (RFC
5893 and UAX #9 are relatively long documents for a reason) and
I have doubts that the WG will be able to come up with informed
consensus ("rough" or otherwise) that anything that is said is
right.   So "general recommendation" covers most of the likely
and important cases.  If one wants to get into the edges, one
will quickly end up on one's own and, unless one is really an
expert with the things, with a fairly high likelihood of ending
up in a rather large mess.  

And that, IMO, is what the proposed text says.

best,
    john

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Wed, 11 Feb 2015 17:40:09 -0700

>     appropriate, that consideration be given to using UAX #9 directly.

I think it's good up until the last sentence, which is convoluted. But 
the tone is right and we might even want to borrow some text from 
Section 3 of RFC 5895. ;-)

A possible refactoring...

OLD
    If a profile

NEW
    The authors of a profile might think that they need to define a new
    directionality rule of their own.  This is not a good idea, even if
    the authors have done a great deal of careful research into the
    problems of displaying bidirectional text.  Instead, this document
    recommends the following:

    o For profiles based on the IdentifierClass, it is best to use the
      "Bidi Rule" from the IDNA2008 specification on right-to-left
      scripts [RFC5893].

    o For profiles based on FreeformClass (or in situations where
      IdentifierClass is not appropriate), it is best to follow the
      recommendations of Unicode Standard Annex #9 [UAX9] directly.

Peter

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 12 Feb 2015 09:10:18 +0800

I like this proposed change better.

A

On Wed, Feb 11, 2015 at 05:40:09PM -0700, Peter Saint-Andre - &yet wrote:
> I think it's good up until the last sentence, which is convoluted. But the
> tone is right and we might even want to borrow some text from Section 3 of
> RFC 5895. ;-)
> 
> _______________________________________________
> precis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/precis

-- 
Andrew Sullivan
[email-masked]

---

From: Pete Resnick <[email-masked]>
Date: Wed, 11 Feb 2015 19:38:42 -0600

>      recommendations of Unicode Standard Annex #9 [UAX9] directly.

This misses an essential bit: The original sentence *does not* recommend 
using 5893 or UAX9, where your refactoring does. What I presented says, 
"*If* you think you need a directionality rule (and you probably don't), 
*then* use 5893 or UAX9." And even then, it says that if you're basing 
on IdentifierClass *and* you really need a directionality rule, we 
*generally* recommend 5893. If you're using FreeformClass *and* you 
think (for some completely insane reason) that you need a special 
directionality rule, you should give *consideration* to UAX9.

If you want to refactor but still capture that, I'm OK.

pr

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Wed, 11 Feb 2015 18:40:21 -0700

> I like this proposed change better.

So, putting it all together and tweaking a few words here and there...

###

    The directionality rule of a profile specifies how to treat strings
    containing what are often called "right-to-left" (RTL) characters
    (see Unicode Standard Annex #9 [UAX9]).  RTL characters come from
    scripts that are normally written from right to left and are
    considered by Unicode to, themselves, have right to left
    directionality.  Strings containing RTL characters often also contain
    "left-to-right" (LTR) characters, such as numerals, as well as
    characters without directional properties.  Consequently, such
    strings are known as "bidirectional strings".  Using bidirectional
    strings in different layout contexts can yield display results that,
    while predictable to those who understand the display rules, are
    counter-intuitive to casual users.  Therefore some profiles might
    wish to restrict the use of bidirectional strings by specifying a
    directionality rule.

    The PRECIS framework does not directly address how to deal with
    bidirectional strings, since there is currently no widely accepted
    and implemented solution for the safe display of arbitrary
    bidirectional strings beyond the Unicode bidirectional algorithm
    [UAX9].  However, rules for management and display of bidirectional
    strings have been defined for domain name labels and similar
    identifiers in the "Bidi Rule" specified in the IDNA2008
    specification on right-to-left scripts [RFC5893].

    The authors of a profile might believe that they need to define a new
    directionality rule of their own.  This is not a good idea, even if
    the authors have done a great deal of careful research into the
    problems of displaying bidirectional strings.  Instead, this document
    recommends the following:

    o  For profiles based on the IdentifierClass, it is best to use the
       "Bidi Rule" specified in [RFC5893].

    o  For profiles based on the FreeformClass (or in situations where
       the IdentifierClass is not appropriate), it is best to follow the
       Unicode bidirectional algorithm [UAX9] directly.

###

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Wed, 11 Feb 2015 18:42:56 -0700

On 2/11/15 6:38 PM, Pete Resnick wrote:
> *then* use 5893 or UAX9."

Ah, you want the optative mood. :-)

> And even then, it says that if you're basing

OK, let me scribble here...

Peter

-- 
Peter Saint-Andre
https://andyet.com/

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Thu, 12 Feb 2015 03:00:29 +0100

>just that you don't like what is there (or proposed).  

The text "unless X this document generally recommends Y" is confusing. 

Consider:

  S

    This document recommends Y (see section P).

  P

    Y is RECOMMENDED [RFC2119].

Contrast with:

  S

    This document recommends Y.

The latter case may be the result of intentionally removing the SHOULD-
level requirement and forgetting to update a section re-stating it.

Similarily, compare

  S

    Y is RECOMMENDED.

and

  S

    This document recommends Y.

In the latter case, did the authors deliberately decide to use a lower
case non-keyword, or did they just phrase "Y is RECOMMENDED" poorly?

Compare

  S

    Unless X, Y is RECOMMENDED.

and

  S

    Unless X, Y is REQUIRED.

Some would argue the two cases are essentially equivalent because RFC
2119 essentially defines "RECOMMENDED" as "REQUIRED unless". Others
would argue "RECOMMENDED" is used to allow exceptions beyond "X".

Compare

  S

    Unless X, Y is generally RECOMMENDED.

  S

    Unless X, Y is RECOMMENDED.

Some would argue "generally" is used to allow exceptions beyond "X"
while others would disagree.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
D-10243 Berlin · PGP Pub. KeyID: 0xA4357E78 · http://www.bjoernsworld.de
 Available for hire in Berlin (early 2015)  · http://www.websitedev.de/

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Wed, 11 Feb 2015 18:59:31 -0700

> OK, let me scribble here...

How's this?

    The authors of a profile might believe that they need to define a new
    directionality rule of their own.  Because of the complexity of the
    issues involved, such a belief is almost always misguided, even if
    the authors have done a great deal of careful research into the
    problems of displaying bidirectional strings.  This document suggests
    that profile authors who are thinking about defining a new
    directionality rule think again, and instead consider using the "Bidi
    Rule" [RFC5893] (for profiles based on the IdentifierClass) or
    following the Unicode bidirectional algorithm [UAX9] directly (for
    profiles based on the FreeformClass or in situations where the
    IdentifierClass is not appropriate).

One thing that bothers me here is that we're not providing actionable 
guidance to the vast majority of non-delusional profile authors. Are we 
saying that it's not a good idea to say anything about directionality at 
all?

Peter

---

From: Pete Resnick <[email-masked]>
Date: Thu, 12 Feb 2015 09:50:10 -0600

On 2/11/15 7:59 PM, Peter Saint-Andre - &yet wrote:
> guidance to the vast majority of non-delusional profile authors. Are 
> we saying that it's not a good idea to say anything about 
> directionality at all?

In my mind there are two possibilities that a profile author ends up with:

1. My application doesn't care whether the strings I'm using appear 
differently in different contexts. If a string looks one way to a person 
on a LTR system and a different way to a person on an RTL system, that's 
fine. I don't care what order the characters appear (for instance, if 
numerals appear before or after punctuation).

or:

2. I want my strings to appear consistently. Numerals appearing on one 
side of a dot on an LTR system and on the other side of a dot on an RTL 
system would be a really bad thing.

If 1, your profile should say, "Directionality Rule: None".

If 2, then you want "Directionality Rule: Something", and unless you 
have some magic specialized knowledge (that you don't, trust us), you 
want "Directionality Rule: 5893". And, BTW, you had better be basing 
things on IdentifierClass; 5893 is *not* sane if you're basing on 
FreeformClass, and you probably meant to give answer #1 above.

Do we agree that the above is what the WG is saying? If so, does the 
text capture that?

pr

---

From: John C Klensin <[email-masked]>
Date: Thu, 12 Feb 2015 13:08:17 -0500

--On Wednesday, February 11, 2015 18:59 -0700 Peter Saint-Andre

> The authors of a profile might believe that they need to
> define a new directionality rule of their own.  Because of the
> complexity of the issues involved, such a belief is almost
> always misguided, even if the authors have done a great deal
> of careful research into the problems of displaying
> bidirectional strings.  This document suggests that profile
> authors who are thinking about defining a new directionality
> rule think again, and instead consider using the "Bidi Rule"
> [RFC5893] (for profiles based on the IdentifierClass) or
> following the Unicode bidirectional algorithm [UAX9] directly
> (for profiles based on the FreeformClass or in situations
> where the IdentifierClass is not appropriate).

Wfm.
  
> One thing that bothers me here is that we're not providing
> actionable guidance to the vast majority of non-delusional
> profile authors. Are we saying that it's not a good idea to
> say anything about directionality at all?

I think I don't understand what you are saying/ asking for.  It
seems to me that every version of the statement posted to the
list in the last 24 hours has said things that come down to:

(1) Directionality can be tricky.   (not actionable, but
important context)
(2) If you have something identifier-like, use 5893 unless you
know better.
(3) If you have something more free form-like, use UAX #9 unless
you know better.
(4) If you think you know better, you are probably wrong; see
above.

(2) and (3) are, AFAICT, about as actionable as things get,
representing clear statement of the character of "if <condition>
then do this..."  The "unless you know better" statements and
(4) is a warning about (2) and (3) to avoid our getting too
normative about situations we can't anticipate  

The only substantive difference I've seen in the various
statements is whether the equivalent to (3) suggests saying as
little as possible about FreeformClass.

I'd rather we didn't, but the above could be restated in
2119-speak as:

(2) If you have something identifier-like, you SHOULD use 5893.
(3) If you have something more free form-like, you SHOULD use
UAX #9.
(1) and (4) Directionality is tricky.  If you have specific
knowledge of the characters or script(s) you are using that the
contexts in which they are used, you MAY invent your own
techniques and specifications, as allowed by the above.
However, there is considerable experience to indicate that you
should not do so unless it is also absolutely necessary because
you are likely to get it wrong and, even if you don't, adopting
rules different from 5893 or UAX 9 is likely to confuse and
irritate users.

     john

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Thu, 12 Feb 2015 14:10:49 -0700

On 2/12/15 8:50 AM, Pete Resnick wrote:
> Do we agree that the above is what the WG is saying?

That seems reasonable.

> If so, does the text capture that?

Not quite. We capture the conclusions but not the reasoning. Let me see 
if I can formulate some text...

Peter

---

From: "Black, David" <[email-masked]>
Date: Thu, 12 Feb 2015 21:39:04 +0000

> (4) If you think you know better, you are probably wrong; see

+1 - I like that listing of principles.

Items (2) and (3) look like they could be stated with "SHOULD" although
I could live with weaker wording.  In contrast, I think the text for (4)
ought to definitely say "SHOULD NOT define a new directionality rule"
and that ought to be called out in the IANA Considerations for the
profiles registry in section 11.3 as important guidance to the
designated expert reviewer(s).

Thanks,
--David

> -----Original Message-----
> From: precis [mailto:[email-masked]] On Behalf Of John C Klensin
> Sent: Thursday, February 12, 2015 1:08 PM
> To: Peter Saint-Andre - &yet; Pete Resnick
> Cc: [email-masked]
> Subject: Re: [precis] Directionality rule (Was: Review of current document
> draft-ietf-precis-framework-20)
> 
> 
>

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Thu, 12 Feb 2015 14:48:41 -0700

> if I can formulate some text...

Here you go...

###

    strings are known as "bidirectional strings".

    Presenting bidirectional strings in different layout systems (e.g., a
    user interface that is configured to handle primarily an RTL script
    vs. an interface that is configured to handle primarily an LTR
    script) can yield display results that, while predictable to those
    who understand the display rules, are counter-intuitive to casual
    users.  In particular, the same bidirectional string (in PRECIS
    terms) might not be presented in the same way to users of those
    different layout systems, even though they are consistent within any
    particular layout system.  In some applications, these presentation
    differences might be considered problematic and thus the application
    designers might wish to restrict the use of bidirectional strings by
    specifying a directionality rule.  In other applications, these
    presentation differences might not be considered problematic (this
    especially tends to be true of more "free-form" strings) and thus no
    directionality rule is needed.

    The PRECIS framework does not directly address how to deal with
    bidirectional strings across all string classes and profiles, and
    does not define any new directionality rules, since at present there
    is no widely accepted and implemented solution for the safe display
    of arbitrary bidirectional strings beyond the Unicode bidirectional
    algorithm [UAX9].  Although rules for management and display of
    bidirectional strings have been defined for domain name labels and
    similar identifiers through the "Bidi Rule" specified in the IDNA2008
    specification on right-to-left scripts [RFC5893], those rules are
    quite restrictive and are not necessarily applicable to all
    bidirectional strings.

    The authors of a PRECIS profile might believe that they need to
    define a new directionality rule of their own.  Because of the
    complexity of the issues involved, such a belief is almost always
    misguided, even if the authors have done a great deal of careful
    research into the challenges of displaying bidirectional strings.
    This document strongly suggests that profile authors who are thinking
    about defining a new directionality rule think again, and instead
    consider using the "Bidi Rule" [RFC5893] (for profiles based on the
    IdentifierClass) or following the Unicode bidirectional algorithm
    [UAX9] directly (for profiles based on the FreeformClass or in
    situations where the IdentifierClass is not appropriate).

###

---

From: Pete Resnick <[email-masked]>
Date: Thu, 12 Feb 2015 16:22:28 -0600

I don't think I could do any better than the below. Thanks.

pr

---

From: Peter Saint-Andre - &yet <[email-masked]>
Date: Thu, 12 Feb 2015 16:05:13 -0700

Thanks. Re-reading, I think "even though they are consistent" needs to 
be "even though the presentation is consistent"...

-- 
Peter Saint-Andre
https://andyet.com/