Subject: [precis] names and usernames

From: Peter Saint-Andre <[email-masked]>
Date: Sun, 12 Feb 2017 16:29:54 -0700

John Klensin has brought to my attention that it is currently impossible 
to represent some people's names in PRECIS usernames because some of the 
relevant Unicode code points are disallowed by the IdentifierClass 
defined in RFC 7564 (and thus by the UsernameCaseMapped and 
UsernameCasePreserved profiles defined in RFC 7613).

First, RFC 7564 disallows "default ignorable" code points in the 
IdentifierClass. However, as I understand it some of these code points 
are need to represent characters in names that might be desirable to 
people living within communities that use Indic script and eastern 
Arabic script (e.g., Persian and writing systems derived from Persian). 
In particular, the Unicode Standard specifies that ZWJ and ZWNJ are 
"default ignorable" and it seems that these code points are especially 
important in this context.

Second, apparently some Chinese family names are typically written 
(especially outside the People's Republic of China) using characters 
that the Unicode Consortium assigns to non-BMP code points, or assigns 
in the BMP but as compatibility decomposable characters (and thus 
disallowed by RFC 7564 in the IdentifierClass).

I'm not sure whether we can solve these problems (internationalization 
is messy and we've never tried to guarantee that any particular name or 
preferred string could be represented in PRECIS usernames), but input 
from people with a deeper understanding of these issues would be 
appreciated. I have attempted to reach out to relevant experts, and will 
report back to this list with any findings.

In the meantime, I plan to submit revised I-Ds addressing other issues 
with the PRECIS specifications sometime this evening.

Peter

---

From: John C Klensin <[email-masked]>
Date: Sun, 12 Feb 2017 18:43:24 -0500

Peter,

Just to be clear to you and the WG, I don't have a strong
opinion about what decision the WG should make in these cases,
but I do believe the decision must be explicit and the result of
discussion, not something that happens as an accident without
knowledge of the relevant cases.

best,
   john

--On Sunday, February 12, 2017 16:29 -0700 Peter Saint-Andre
<[email-masked]> wrote:

> John Klensin has brought to my attention that it is currently
> impossible to represent some people's names in PRECIS
> usernames because some of the relevant Unicode code points are
> disallowed by the IdentifierClass defined in RFC 7564 (and
> thus by the UsernameCaseMapped and UsernameCasePreserved
> profiles defined in RFC 7613).
> 
> First, RFC 7564 disallows "default ignorable" code points in
> the IdentifierClass. However, as I understand it some of these
> code points are need to represent characters in names that
> might be desirable to people living within communities that
> use Indic script and eastern Arabic script (e.g., Persian and
> writing systems derived from Persian). In particular, the
> Unicode Standard specifies that ZWJ and ZWNJ are "default
> ignorable" and it seems that these code points are especially
> Second, apparently some Chinese family names are typically
> written (especially outside the People's Republic of China)
> using characters that the Unicode Consortium assigns to
> non-BMP code points, or assigns in the BMP but as
> compatibility decomposable characters (and thus disallowed by
> RFC 7564 in the IdentifierClass).
> 
> I'm not sure whether we can solve these problems
> (internationalization is messy and we've never tried to
> guarantee that any particular name or preferred string could
> be represented in PRECIS usernames), but input from people
> with a deeper understanding of these issues would be
> appreciated. I have attempted to reach out to relevant
> experts, and will report back to this list with any findings.
> 
> In the meantime, I plan to submit revised I-Ds addressing
> other issues with the PRECIS specifications sometime this
> evening.
> 
> Peter
> 
> _______________________________________________
> precis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/precis

---

From: Peter Saint-Andre <[email-masked]>
Date: Sun, 12 Feb 2017 17:02:30 -0700

Agreed. I'm trying to get input from relevant experts, because I 
strongly doubt that we have the relevant expertise within the WG.

Peter

---

From: Florian Zeitz <[email-masked]>
Date: Mon, 13 Feb 2017 04:15:31 +0100

> important in this context.
I'd have to look at it in more detail, but that assessment seems wrong
to me.
Algorithmically we check for JoinControl before
PrecisIgnorableProperties, making ZWJ and ZWNJ CONTEXTJ.
That allows them to occur after virama and where they break a cursive
connections. I'm not sure those are the only cases that John is
concerned about, but they are not generally disallowed as I understand it.

That said, I always found it a bit unsettling that it is virtually
impossible to determine the algorithmic result from the textual
description of what is and isn't allowed.

Regards,
Florian Zeitz

---

From: Peter Saint-Andre <[email-masked]>
Date: Sun, 26 Feb 2017 16:32:28 -0700

On 2/12/17 8:15 PM, Florian Zeitz wrote:
> connections. 

Correct.

> I'm not sure those are the only cases that John is

Because I don't have a good understanding of the relevant scripts and
languages, I am dependent on people who do, such as Nalini Elkins. I
shall ping her again to see if she can share her insights.

> description of what is and isn't allowed.

There is much that is unsettling about internationalization. We do the
best we can with what is given.

Peter

---

From: Peter Saint-Andre <[email-masked]>
Date: Sun, 26 Feb 2017 17:21:42 -0700

I'm still waiting to hear from experts with knowledge of Indic and
eastern Arabic scripts. However, I have been corresponding offlist with
someone who has knowledge of the second issue I raised...

>>> Second, apparently some Chinese family names 

According to my correspondent, the challenge is representation of some
given names, not family names (e.g., legislation in Taiwan stipulates
that a given name can include any character that has ever appeared in a
dictionary, even dictionaries published hundreds of years ago).

>>> are typically
>>> non-BMP code points 

John, forgive my ignorance, but it seems to me that the plane is
irrelevant here: in PRECIS we base decisions on code point properties.
Thus, for instance, any code point whose Unicode general category is
"Lo" (other letter) is allowed in the PRECIS IdentifierClass (per
Section 9.1 of RFC 7564), regardless of the plane. As an example, a code
point like U+2F804 (CJK COMPATIBILITY IDEOGRAPH-2F804) would be allowed,
even though it is in the Supplementary Ideographic Plane.

>>> or assigns in the BMP but as

My correspondent said it should be fine to disallow compatibility
decomposable characters such as U+328A (CIRCLED IDEOGRAPH MOON) because
according to him they would not be used in given or family names.

All of this is second-hand, so take it with a grain of salt.

Peter

---

From: Peter Saint-Andre <[email-masked]>
Date: Sun, 26 Feb 2017 17:42:55 -0700

> knowledge of the relevant cases.

As I've been pondering these issues, I've realized we can't guarantee
that a user identifier can be (or include) someone's name. Although that
might be a desirable property, a username is, after all, an account
identifier. There is long precedent (e.g., in email) for allowing a
smaller subset of characters in account identifiers than in display names.

Thus I suggest that we add a paragraph like the following to 7613bis:

   A "username" or "user identifier" is a string of characters
   designating an account on a computing device or system, often but not
   necessarily for use by a person.  Although some devices and system
   might allow a username to be part or all of a person's name, and a
   person might want their account designator to be part or all of their
   name, because of the complexities involved that outcome is not
   guaranteed for all human names on all computing devices or systems
   that follow the rules defined in this specification.  Protocol
   designers and application developers who wish to allow a wider range
   of characters are encouraged to consider a separation between more
   restrictive account identifiers and more expressive display names.

Peter

---

From: John C Klensin <[email-masked]>
Date: Sun, 26 Feb 2017 20:26:30 -0500

--On Sunday, February 26, 2017 17:21 -0700 Peter Saint-Andre
<[email-masked]> wrote:

> I'm still waiting to hear from experts with knowledge of Indic
> and eastern Arabic scripts. However, I have been corresponding
> offlist with someone who has knowledge of the second issue I
> raised...
> 
> According to my correspondent, the challenge is representation
> of some given names, not family names (e.g., legislation in
> Taiwan stipulates that a given name can include any character
> that has ever appeared in a dictionary, even dictionaries
> published hundreds of years ago).

Entirely possible that I misunderstood the precise nature of the
problem.  From the standpoint of what should be allowed in an
identifier that people would expect to reflect their names,
whether then name is a family name, a personal name, a tribal or
can name, or a patronymic or matronymic whose seem to be me to
make little difference.

> John, forgive my ignorance, but it seems to me that the plane
> is irrelevant here: in PRECIS we base decisions on code point
> properties. Thus, for instance, any code point whose Unicode
> general category is "Lo" (other letter) is allowed in the
> PRECIS IdentifierClass (per Section 9.1 of RFC 7564),
> regardless of the plane. As an example, a code point like
> U+2F804 (CJK COMPATIBILITY IDEOGRAPH-2F804) would be allowed,

If I recall, there is another issue with compatibility
ideographs (i.e., theu may or may not be a good example), but
your statement above should certainly be correct.  My reason for
mentioning the BMP is that we have recently again heard from
someone who wants to confine an I18n library to work in UTF-16,
apparently without surrogates.  While any problems of that sort
would certainly be bugs, that sort of UTF-16 discussion leaves
me with the sense that higher planes may be a bit less useful in
practice than the BMP.   That is not, however, a protocol
specification problem in any way.

>>>> compatibility decomposable characters (and thus disallowed
>>>> by RFC 7564 in the IdentifierClass).

> My correspondent said it should be fine to disallow
> compatibility decomposable characters such as U+328A (CIRCLED
> IDEOGRAPH MOON) because according to him they would not be
> used in given or family names.

Whether the rules of IDNA2008 are good enough is another matter
-- if nothing else, we had to work with the properties Unicode
gave us -- but the basic intent was certainly to disallow any
code point that, visually and in general understanding, was a
circled variation on a base character or a font variation that
was not normally considered a different base character

> All of this is second-hand, so take it with a grain of salt.

FWIW, consistent with my knowledge.

    john

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 27 Feb 2017 13:30:18 +0900

>>>>> by RFC 7564 in the IdentifierClass).

There are many different reasons for a character to have a compatibility 
decomposition. The oldest Unicode data file, 
http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt, even 
contains labels for these. But having looked again through the list of 
these labels, excluding them seems very appropriate in all cases.

>> used in given or family names.

This would be indeed about as weird as using using circled letters in a 
Western name. In Unicode, circled moon is followed by circled fire, 
water,... It's very clear that these are specific 'styles' for the days 
of the week in Japanese (not in Chinese, don't know about Korean).

Regards,   Martin.