Subject: [Idna-update] emoji and security

From: Asmus Freytag <[email-masked]>
Date: Mon, 12 Mar 2018 09:48:23 -0700

All,

there's a general consensus that emoji and secure IDNs do not go together.

This is clearly not something that's taken for granted by others.

Read down a few messages in this thread on the Unicode list:

https://www.unicode.org/mail-arch/unicode-ml/y2018-m03/0075.html

to find the suggestion of translating security hash codes into strings
of emoji ostensible for easier verification:

"So that makes
me wonder which one would be quicker for a human to verify on average?
Also, which one is more accurate for a human to verify? I have no idea. For
accuracy, it seems like a lot of thought was put into the visual uniqueness
of Unicode emojis. "

Discuss.

A./

PS: the reason I forwarded this is that many people find something
familiar and  accessible about emoji and will not understand why there
should be more security concerns about them than about a bunch
of bewilderingly similar and complex Chinese ideographs or a bunch
of "dawn-of-time-emoji" aka Hieroglyphics, both of which are PVALID.

---

From: Michel Suignard <[email-masked]>
Date: Mon, 12 Mar 2018 18:21:25 +0000

I could not resist commenting on this, because unlike emoji, Egyptian Hieroglyphs, while looking often like colorful pictures, are the expression of a fully formed writing system with ideograms and phonograms and all the usual attributes of such a system in term of sentence structure. Jokingly I have sometimes said that doom would be near if people realized that you could mix Emoji with hieroglyphs and result in a fully formed writing system.

Jokes aside, the 100s Egyptian hieroglyphs contains hundreds of semantic and phonetic variants and are totally unsuited for any identifier usage and are still fully IDN 2008 PVALID (unless of course you limit the scope using variant sets). And to make matter worse, there is a plan to add few more thousands to cover Ptolemaic era (which would all become PVALID as well if the synchronization is re-enabled). At the same time, we are quite careful at not encoding Hieroglyphs homoglyphs (still most people would not know the difference). (Saying ‘we’ because I am the author of one of these extensions proposal).

Finally, I share many of Asmus’s considerations on the status of IDNA and lack of synchronization with Unicode (we are both on the ICANN Integration Panel, so we do a lot of LGR work together). I am currently at ICANN 61 with some decent free time and willing to do something to make progress.

Best

Michel

From: IDNA-UPDATE <[email-masked]> On Behalf Of Asmus Freytag
Sent: Monday, March 12, 2018 9:48 AM
To: [email-masked]
Subject: [Idna-update] emoji and security

All,

"So that makes

Discuss.

A./

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 13 Mar 2018 10:30:35 +0100

On Mon, Mar 12, 2018 at 09:48:23AM -0700,
 Asmus Freytag <[email-masked]> wrote 

> there's a general consensus that emoji and secure IDNs do not go

Since the IDNA standard does not allow emojis, I'm not sure of the
point.

---

From: "John Levine" <[email-masked]>
Date: 13 Mar 2018 16:17:19 -0400

>Since the IDNA standard does not allow emojis, I'm not sure of the

Hi from ICANN in San Juan.  You know and I know that emoji domains are
a bad idea for all sorts of reasons.  But there are registries and
registrars that believe they will make big buck$ by selling them.

There have been some presentations on what's permissible and emoji
specifically which the ICANN leadership appears to understand.
Domains like .WS will sell whatever they can sell and there's not much
anyone can do about that.

Amusingly, .WS announced with great fanfare that they'd sell emoji and
managed to screw it up so the missold the same emoji to multiple
people.  We TOLD them they were easy to confuse.

R's,
John

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 13 Mar 2018 16:25:05 -0400

On Mon, Mar 12, 2018 at 06:21:25PM +0000, Michel Suignard wrote:

> I could not resist commenting on this, because unlike emoji, Egyptian
> Hieroglyphs, while looking often like colorful pictures, are the expression of
> a fully formed writing system with ideograms and phonograms and all the usual
> attributes of such a system in term of sentence structure.

Yes.  Which is why they have a general category that …

> Jokes aside, the 100s Egyptian hieroglyphs contains hundreds of semantic and
> phonetic variants and are totally unsuited for any identifier usage and are
> still fully IDN 2008 PVALID (unless of course you limit the scope using variant
> sets).

… causes them to be PVALID.  Which means they're _not_ totally
unsuited to any identifier usage, but certainly they are unsuited to
any general-purpose identifier usage and almost certainly they will
remain in at best extremely limited use effectively forever.  Emojis,
as you know perfectly well, do not have the same general category and
therefore are INVALID.

If the general-category approach to IDNA2008 was unsuitable in 2008
because it yielded examples of things that should never in principle
be identifiers, I wish someone would have made that argument.  I do
not recall it having been made, and I believe I participated pretty
avidly.  I do recall some people arguing that various archaic things
were "not needed" or "almost always unusable by anyone", but that
would have put us in the situation of going through everything one
code point at a time.  There were some who wanted to try to re-do
IDNA2003 only updated for then-current versions of Unicode, but I
think experience shows that wouldn't have worked out too well either.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Michel Suignard <[email-masked]>
Date: Tue, 13 Mar 2018 20:44:43 +0000

<<
If the general-category approach to IDNA2008 was unsuitable in 2008 because it yielded examples of things that should never in principle be identifiers, I wish someone would have made that argument.  I do not recall it having been made, and I believe I participated pretty avidly.  I do recall some people arguing that various archaic things were "not needed" or "almost always unusable by anyone", but that would have put us in the situation of going through everything one code point at a time.  There were some who wanted to try to re-do
IDNA2003 only updated for then-current versions of Unicode, but I think experience shows that wouldn't have worked out too well either.
No, I still think in context, general-category approach was the best approach possible, short of developing a new category which was unpalatable to most. As you correctly pointed out in your answer in my comment about Egyptian hieroglyphs, these may be used in limited-use identifier context (for example if you want to identify base Old Egyptian phonemes) but with some caveat.

Still, what needs to happen to unlock the IANA IDN table from being frozen at Unicode 6.3? Unicode 11.0 is months away. I don't think anyone wants to revise the protocol (IDNA 2008), can't we just agree that the protocol is not 100% failsafe and move on? Documenting the problematic cases outside the protocol seems to me the best way to proceed. And it should not be a gating factor in letting IANA revise their table (although it would be great if that documentation was progressing).

Best regards

Michel

---

From: "Patrik Fältström" <[email-masked]>
Date: Tue, 13 Mar 2018 17:35:41 -0400

On 13 Mar 2018, at 16:17, John Levine wrote:

> There have been some presentations on what's permissible and emoji specifically which the ICANN leadership appears to understand.
> Domains like .WS will sell whatever they can sell and there's not much anyone can do about that.

I think it is important to separate these three things from each other -- at least as much as possible:

- The IETF work on IDNA2008

- The ICANN work on LGRs and other generic policy issues

- Compliance issues

It looks to me like if either people intentionally muddy the water or comment on issues they are not plugged in on (or both :-D ).

   paf

---

From: John C Klensin <[email-masked]>
Date: Tue, 13 Mar 2018 18:22:15 -0400

--On Tuesday, March 13, 2018 16:17 -0400 John Levine
<[email-masked]> wrote:

>...
> There have been some presentations on what's permissible and
> emoji specifically which the ICANN leadership appears to
> understand. Domains like .WS will sell whatever they can sell
> and there's not much anyone can do about that.

For better or worse, the last statement is either untrue or
misleading.  A more accurate statement would be that ICANN, as
an institution, has had no will to do anything about it.

  john

---

From: "Patrik Fältström" <[email-masked]>
Date: Tue, 13 Mar 2018 18:38:46 -0400

On 13 Mar 2018, at 18:22, John C Klensin wrote:

> misleading.  A more accurate statement would be that ICANN, as an institution, has had no will to do anything about it.

I hope this will be discussed tomorrow at the ccNSO meeting.

   paf

---

From: John C Klensin <[email-masked]>
Date: Tue, 13 Mar 2018 19:01:37 -0400

> This is clearly not something that's taken for granted by

Possibly true, but your example below has nothing to do with the
issue.

> to find the suggestion of translating security hash codes into
> strings of emoji ostensible for easier verification:
 
> "So that makes me wonder which one would be quicker for a
> human to verify on average? Also, which one is more accurate
> for a human to verify? I have no idea. For accuracy, it seems
> like a lot of thought was put into the visual uniqueness of
> Unicode emojis. "

> Discuss.

First, whether emoji are a good idea for encoding hashes or not
has nothing to do with whether they are appropriate for domain
names.  My guess is that they are not as useful for encoding
hashes as the messages I look at seem to believe until and
unless (1) the presentation forms for the various emoji code
points are standardized sufficiently that they differ no more
from one platform to the next than conventional, not extremely
artistic, type styles do for modern letter or digit display
forms and (2) names of emoji and emoji sequences in popular use
are standardized sufficiently that text to speech programs
pronounce (or describe) them in ways that are consistent across
platforms.  Whether that hypothesis is correct or not, I would
encourage those who are interested in the question of how easily
one string of emoji can be compared to each other to find and
real Herman Chernoff's original "faces" paper about the
representation of multidimensional data.  

However, those are just my not-very-educated guesses.  As far as
IDNs and this discussion is concerned, the bottom line is that
the are Invalid for use in domain and that, IMO, those who are
anxious to see IANA's tables updated to Unicode 10 or 11 in the
relatively near future should probably understand that, with the
current level of activity and enthusiasm in the IETF for i18n
work generally and IDN work in particular, the last thing they
wand to do is to get the table update effort blocks behind a
substantive IDNA revision.

More important, I agree with Patrik that people are, once again,
confusing his three-way distinction among IETF work, ICANN
policy work, and compliance issues and my earlier,
not-quite-orthogonal, earlier distinction among protocol (DNS
and IDNA) constraints, general guidance about how policies
should be developed about registration in arbitrary zones, and
ICANN policies for the root.  If people want to make progress, I
think it would be helpful to have those distinctions more
strongly in mind... and to avoid the emoji and non-IDN uses of
code points issue entirely except as compliance issues that are
probably discussed in ICANN, legal, and regulatory contexts
rather than here.

best,
   john

---

From: Asmus Freytag <[email-masked]>
Date: Tue, 13 Mar 2018 17:09:29 -0700

> No, I still think in context, general-category approach was the best approach possible, short of developing a new category which was unpalatable to most. As you correctly pointed out in your answer…

The design of IDNA2008 reflects two or three conflicting thrusts:

1) it was intended to be updatable to newer versions by simply applying 
the properties, with using an exception list as a fail-safe.
2) it was intended to avoid clear-cut cases of dual encoding (via 
normalization)
3) it was intended to be usable on all levels of the tree - therefore it 
could not be maximally restrictive

It would have been easy to restrict IDNA 2008 to modern scripts only - 
the number of scripts that are in actual modern use of significance for 
network identifiers is not something that changes based on Unicode 
version (there really aren't any unencoded ones left in that category). 
They could have been enumerated and their script properties be used to 
filter the code points.

That was seen as too restrictive - and it's not worth arguing about 
whether that was the wrong or correct thing to do.

However, many non-modern scripts are really not suitable for general 
identifiers: there's not a body of working expertise on where the 
problems are with them. It's arguable that they should (have been) kept 
out of second level domains altogether, because nobody who includes them 
in a registry can be said to have done so understanding the 
consequences, therefore implicitly violating the prescription in RFC 5891.

Yet they are perfectly fine on some 3rd or 4th level, for example any 
level, where the same entity controls all labels and there therefore 
can't be a risk from spoofing, etc.

Now, given that the protocol as a whole is that permissive, and given 
that there are some dozens of outright homographs even in the modern 
scripts, and have been from 2008 (not to mention hundreds if not 
thousands of rather similar-looking code points), it's bordering on the 
absurd to stop all work on updating IDNA 2008 over the case of a single 
Arabic addition that isn't even an exact homoglyph (fonts that I have 
researched tend to bind the precomposed mark more tightly to the 
skeleton than an independent code point having the same shape as the mark).

> Still, what needs to happen to unlock the IANA IDN table from being frozen at Unicode 6.3? Unicode 11.0 is months away. I don't think anyone wants to revise the protocol (IDNA 2008), can't we just…
So, let's get back to that discussion.

A./
>
> Best regards
>
> Michel
> _______________________________________________
> IDNA-UPDATE mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idna-update
>

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 13 Mar 2018 21:18:13 -0400

On Tue, Mar 13, 2018 at 05:09:29PM -0700, Asmus Freytag wrote:

> It would have been easy to restrict IDNA 2008 to modern scripts only

Easy for _whom_?  The IETF just does not have that expertise, which
was something we had demonstrated pretty nicely using IDNA2003.  What
the IETF needed was a way of avoiding the problem of evaluating
particular code points, and so the Unicode properties were the path.

> identifiers: there's not a body of working expertise on where the problems
> are with them. It's arguable that they should (have been) kept out of second
> level domains altogether, because nobody who includes them in a registry can
> be said to have done so understanding the consequences, therefore implicitly
> violating the prescription in RFC 5891.

But the IETF does not have, and does not want, authority to "keep them
out".  That's a policy question, and one rather far from the IETF's
area of expertise.  Indeed, that division of labour is precisely why
the IANA root zone registry policy is set by someone other than the
IETF, which sets the policies for IANA protocol parameter registries.

> similar-looking code points), it's bordering on the absurd to stop all work
> on updating IDNA 2008 over the case of a single Arabic addition that isn't
> even an exact homoglyph

But that is not, as I think you know perfectly well, why in the past
the IAB asked for additional work and a pause.  The IAB made that
request because it was necessary to do a bunch of work to understand
the issues and attempt to get a grasp on what the overall implications
are.  I think that was a good idea, because proceeding as one ever did
without understanding how one was wrong in one's assumptions is pretty
foolish.  I think there has been considerable work done in the
meantime, and it might well be that conditions have changed (I am not
the IAB, so I can't speak about that).

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: "Patrik Fältström" <[email-masked]>
Date: Wed, 14 Mar 2018 07:04:33 -0400

On 13 Mar 2018, at 20:09, Asmus Freytag wrote:

> 1) it was intended to be updatable to newer versions by simply applying the properties, with using an exception list as a fail-safe.

Not really. The normative rules in the IDNA2008 standard where to be applied to the Unicode Versions so that derived property values where calculated, and this independent of the Unicode Version. There are no properties that are applied.

> 2) it was intended to avoid clear-cut cases of dual encoding (via normalization)
> 3) it was intended to be usable on all levels of the tree - therefore it could not be maximally restrictive

   Patrik

---

From: "Patrik Fältström" <[email-masked]>
Date: Wed, 14 Mar 2018 07:06:41 -0400

On 13 Mar 2018, at 21:18, Andrew Sullivan wrote:

> But the IETF does not have, and does not want, authority to "keep them out".  That's a policy question, and one rather far from the IETF's area of expertise.  Indeed, that division of labour is precisely why the IANA root zone registry policy is set by someone other than the IETF, which sets the policies for IANA protocol parameter registries.

Correct.

The IETF Standard explicitly say that a registry must implement a policy for what code points they want to include, so by definition, and by design, the IDNA2008 is not a minimum set of code points. It is a maximum set of code points to be used.

   Patrik

---

From: Michel Suignard <[email-masked]>
Date: Wed, 14 Mar 2018 14:34:26 +0000

I want to show how on a concrete level not having a post Unicode 6.3 IDN table is hurting work done in ICANN. Unicode 7.0 added support for some Myanmar minorities. For many reasons that probably do not interest people in this list, the Myanmar script coverage was a late comer in Unicode encoding, especially Myanmar minorities coverage. Therefore, Unicode 7.0 added support for Shan Pali (again a Myanmar minority). If we could use Unicode 7.0 repertoire as available in an updated IANA IDN2008 table, the root zone Integration Panel would have added 6 characters to the Maximum Starting repertoire (MSR) which is the sand box for Generation Panels (GP) when they define their own repertoire. Obviously, we have no way to know whether a Myanmar GP would adopt these characters into the RZ-LGR when/if they do that. But at this point this is not even an option.

At this point, most of the recent Unicode work on scripts (and I made the precision on purpose) is concentrated on historic ones, or disadvantaged communities. By being stuck on Unicode 6.3 we are putting these communities at disadvantage. Not a message you want to convey on these days and ages.

I think I (and most of us) have decent understanding of the tasks separation mentioned by Patrik, I just want to move on past Unicode 6.3, so we can do our work in appropriate context.

Michel

-----Original Message-----
From: IDNA-UPDATE <[email-masked]> On Behalf Of Patrik Fältström
Sent: Wednesday, March 14, 2018 4:07 AM
To: Andrew Sullivan <[email-masked]>
Cc: [email-masked]
Subject: Re: [Idna-update] emoji and security

Correct.

   Patrik

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Wed, 14 Mar 2018 15:49:39 +0100

On Tue, Mar 13, 2018 at 04:17:19PM -0400,
 John Levine <[email-masked]> wrote 
 a message of 22 lines which said:

> You know and I know that emoji domains are a bad idea for all sorts
> of reasons.

I said it was "illegal", from RFC 5892, which is not exactly the same
thing.

---

From: Kim Davies <[email-masked]>
Date: Wed, 14 Mar 2018 11:15:17 -0400

Quoting Michel Suignard on Wednesday March 14, 2018:

> I want to show how on a concrete level not having a post Unicode 6.3
> IDN table is hurting work done in ICANN. Unicode 7.0 added support
> for some Myanmar minorities. 
...
> If we could use Unicode 7.0 repertoire as available in an
> updated IANA IDN2008 table, the root zone Integration Panel would have
...
> But at this point this is not even an option.

I think a point of clarification would be useful here. The IDNA tables
published by IANA is informative, not normative. Nothing in the
IDNA 2008 specification says these tables need to be used -- a full
implementation would derive their own list of PVALID etc. from Unicode
properties and some exception tables in RFC 5892. 

However, given the IDNA tables are posted, some IDNA 2008
implementations use them as a shortcut to greatly simplify their code.
As a consequence, the landscape of implementations is likely variable
because you have some implementations using the IANA-published tables
limited to the Unicode 6.3 repertoire, and others using more recent
versions because they are deriving from Unicode properties directly.

kim

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Wed, 14 Mar 2018 08:38:35 -0700

> Not really. The normative rules in the IDNA2008 standard where to be applied to the Unicode Versions so that derived property values where calculated, and this independent of the Unicode Version. T…

Thanks for providing the details; my summary of the process was a bit 
too abridged.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Wed, 14 Mar 2018 17:11:29 +0100

On Wed, Mar 14, 2018 at 11:15:17AM -0400,
 Kim Davies <[email-masked]> wrote 

> properties and some exception tables in RFC 5892.

Let me check that I understood RFC 5892: an implementation can
calculate its own tables but, if they follow RFC 5892, they should
get, FOR THE SAME UNICODE VERSION, the same result than IANA, or
anyone else, no?

---

From: Kim Davies <[email-masked]>
Date: Wed, 14 Mar 2018 13:17:03 -0400

> get, FOR THE SAME UNICODE VERSION, the same result than IANA, or

That is correct. 

kim

---

From: "Patrik Fältström" <[email-masked]>
Date: Wed, 14 Mar 2018 13:53:14 -0400

On 14 Mar 2018, at 12:11, Stephane Bortzmeyer wrote:

> Let me check that I understood RFC 5892: an implementation can calculate its own tables but, if they follow RFC 5892, they should get, FOR THE SAME UNICODE VERSION, the same result than IANA, or anyone else, no?

Yes. The rules are normative, so two implementations of RFC5892 is to produce two equal tables of derived property values.

I have myself two implementations I have written in two different languages (Ruby and Python3 -- where the Ruby one is not using any IDN library, for example for normalization) and I compare results of both of them with the result IANA get applying their software.

That is the process we use to validate the tables IANA publish, which are non-normative.

   paf

---

From: "Patrik Fältström" <[email-masked]>
Date: Wed, 14 Mar 2018 14:00:42 -0400

On 14 Mar 2018, at 10:49, Stephane Bortzmeyer wrote:

> I said it was "illegal", from RFC 5892, which is not exactly the same thing.

The precise text is:

   DISALLOWED: Those that should clearly not be included in IDNs.
   Code points with this property value are not permitted in IDNs.

I.e. "not permitted".

   paf

---

From: Asmus Freytag <[email-masked]>
Date: Wed, 14 Mar 2018 11:24:26 -0700

We need to bring the discussion back to "what are the concrete steps 
required to get IDNA update past Unicode 6.3"?

A./

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Sat, 17 Mar 2018 06:49:49 +0100

On Wed, Mar 14, 2018 at 11:24:26AM -0700,
 Asmus Freytag <[email-masked]> wrote 

> required to get IDNA update past Unicode 6.3"?

"We have therefore asked the expert reviewer to bring the IDNA
Parameters registry up to date by June of 2018, working with other
experts as necessary."
https://www.iab.org/documents/correspondence-reports-documents/2018-2/iab-statement-on-identifiers-and-unicode/

---

From: John C Klensin <[email-masked]>
Date: Sat, 17 Mar 2018 22:57:27 -0400

Kim,

Just to clarify one other part of your comments...

--On Wednesday, March 14, 2018 11:15 -0400 Kim Davies
<[email-masked]> wrote:

> implementations use them as a shortcut to greatly simplify
> their code. As a consequence, the landscape of implementations
> is likely variable because you have some implementations using
> the IANA-published tables limited to the Unicode 6.3
> repertoire, and others using more recent versions because they
> are deriving from Unicode properties directly.

yes.  But that is inevitable, whether there are tables stored at
IANA or not.   Unless every registry in the DNS (not just TLD or
SLD registries) is either calculating code point and label
validity using the IDNA rules in real time or making that
calculation every time a new version of Unicode is announced
(and exactly the same number of minutes after the announcement)
_and_ every instance of every lookup application does one of
those things as well, there will always be some practical risk
of unsynchronized implementations.  I don't know whether having
the tables posted by/at IANA makes that problem any worse, but
it seems obvious to me that it doesn't make it much worse.

Noting that the DNS itself uses a slow synchronization model,
those properties of IDNA2008 probably don't even change the
problem in a qualitative way.

best,
    john