Subject: [idn] Re: Unicode and Security

From: Elliotte Rusty Harold <[email-masked]>
Date: Thu, 7 Feb 2002 13:21:29 -0500

>On Thu, Feb 07, 2002 at 10:34:20AM -0500, Elliotte Rusty Harold wrote:

>Unicode is a character encoding, not a glyph encoding. Furthermore,
it's
>a superset of a number of preexisting character sets, so that it was
>possible for those users to move to Unicode without problems. Since
>important preexisting character sets seperated Greek, Cyrillic and
Latin
>scripts, Unicode had to. Had Unicode not chosen to follow these
>principles, ISO 10646 would have, and it would have become the dominant
>character set, with the same problems.
>

I know why these choices were made. That has nothing to do with the
question of whether the finished product will or will not cause
security breaches.

>In any case, what is your solution? When the American Mathematical
>Society says "We need a SMALL CIRCLE for the mathematical texts", do
you
>say "no, we already have the unified LATGRKCRY SMALL O"? After they
show
>you that the two are distinct characters in their texts, do you still
>refuse because "someone might get confused"? The Universal Character
Set
>can't afford to not encode characters like that.
>

I'm not sure Unicode can be fixed at this point. The flaws may be too
deeply embedded. The real solution may involve waiting until
companies and people start losing significant amounts of money as a
result of the flaws in Unicode, and then throwing it away and
replacing it with something else. I don't like that solution, but not
liking it doesn't mean it ain't gonna happen as soon as Exxon loses a
few billion dollars because somebody spoofed them and thereby gained
access to their bidding plans for oil leases. Don't be surprised when
some large companies start issuing memos forbidding the use of
Unicode, or blocking all non-ASCII domain names at their firewall.

One possible solution at the domain name system level might be to
limit domain names to a single Unicode block or group. For instance,
Greek domain names could be allowed but not domain names that mix
Greek with Latin. Similarly, you couldn't mix Latin with Cyrillic or
Cyrillic with Greek. That would at least vastly reduce the
possibility for domain spoofing, if not eliminate it entirely.

Interesting tidbit: app1e.com (not APPLE.COM but APP1E.COM) is in
fact already registered. This attack may not be as theoretical as I
initially thought.
--

+-----------------------+------------------------+-------------------+
| Elliotte Rusty Harold | [email-masked] | Writer/Programmer |
+-----------------------+------------------------+-------------------+
|          The XML Bible, 2nd Edition (Hungry Minds, 2001)           |
|              http://www.ibiblio.org/xml/books/bible2/              |
|   http://www.amazon.com/exec/obidos/ISBN=0764547607/cafeaulaitA/   |
+----------------------------------+---------------------------------+
|  Read Cafe au Lait for Java News:  http://www.cafeaulait.org/      |
|  Read Cafe con Leche for XML News: http://www.ibiblio.org/xml/     |
+----------------------------------+---------------------------------+

---

From: [email-masked]
Date: Thu, 7 Feb 2002 13:47:35 -0500

<[email-masked]>
From: <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]
Precedence: bulk

> Unicode, or blocking all non-ASCII domain names at their firewall.

"Doom!  Doom!  Doom!  End is nigh, repent ye sinners!"

> fact already registered. This attack may not be as theoretical as I 

Interestingly enough, I find this (and whitehouse.com and whitehouse.org,
and micros0ft.com, and ...) a good example for Unicode being largely irrelevant.
Sure, Unicode gives more possibilities for abuse, but I fail to see how
a character
encoding standard can stop people from being stupid and not using public
keys or
some other means of trust in cases where it matters.  Analogously, people will
keep opening executable attachments promising sex, regardless of whether
the
's', 'e', and 'x' are Latin letters or not.

---

From: Elliotte Rusty Harold <[email-masked]>
Date: Thu, 7 Feb 2002 15:53:45 -0500

At 2:10 PM -0600 2/7/02, David Starner wrote:

>What else?

ASCII, ISO-Latin-1. Not great solutions to be sure, but if it stops
the engineers from accidentally giving away the company jewels then
IT will impose them.

>As we keep pointing out, almost every character in Unicode
>that normally has the same glyph as another is in Unicode with good
>reason. To change that to something that would fit your goals will cost
>billions right now just for the change, and then you end with a
>character set that can't round trip all the others in common use, and
>that is more painful to use for Greeks and Russian, and completely
>unusable for mathematicians. I seriously doubt the world would go to a
>massively inferior character set because of the security holes you're
>talking about.

For text files, probably not. But for the domain name system the
world very well might. Indeed, maybe it should unless this problem
can be dealt with. I suspect it can be dealt with by prohibiting
script mixing in domain names (e.g. each component of the name must
be entirely Greek or entirely Cyrillic or entirely Latin etc. Note:
something_Cyrillic.something_greek.com is OK.)  Does anybody really
need mixed Latin and Greek domain names?

Still, this would just fix the problem for domain names. This is an
use-case I suspect wasn't even considered back in the mid-80s when
Unicode got rolling. I wouldn't be surprised if in another ten years
we find similar problems in some as yet unnoticed and unconsidered
area. Technology's nasty like that.
--

---

From: [email-masked]
Date: Fri, 8 Feb 2002 01:37:10 EST

[email-masked] observed:

> Analogously, people will keep opening executable attachments promising sex,
> regardless of whether the 's', 'e', and 'x' are Latin letters or not.

They're not, of course:  U+0455 U+0435 U+0445

-Doug Ewell
 Fullerton, California
 (address will soon change to dewell at adelphia dot net)

---

From: Michael Everson <[email-masked]>
Date: Fri, 8 Feb 2002 11:30:35 +0000

>need mixed Latin and Greek domain names?

Certainly. Some years ago the European Court upheld the right of a 
Belgian man whose father was Belgian and mother was Greek to spell 
his hyphenated last name in both scripts. Why should he not be 
allowed to register a domain based on his own name?

I don't think this has anything to do with Unicode. In Unicode, we 
wish to make all the world's writing system available to everyone. 
Thieves and cheats will use it if they wish, but this detracts not 
one whit from the nobility of our enterprise.
-- 
Michael Everson *** Everson Typography *** http://www.evertype.com

---

From: Elliotte Rusty Harold <[email-masked]>
Date: Fri, 8 Feb 2002 09:02:18 -0500

>allowed to register a domain based on his own name?

Interesting. But I would content this is an edge case that need not 
be supported  in DNS if the cost of doing so is far greater than the 
benefit to be achieved. Of course under my proposal he could have 
LatinName.GreekName.com
--

---

From: Elliotte Rusty Harold <[email-masked]>
Date: Sun, 10 Feb 2002 13:05:10 -0500

At 9:52 PM +0100 2/9/02, Lars Marius Garshol wrote:

>It seems to me that this problem really needs some other fix than the
>merging of all similar-looking characters in all character sets. I
>just can't see that working.
>

That's your suggestion, not mine. I have never proposed merging all 
the similar looking characters. If I misspoke or something I wrote 
was misinterpreted as indicating that, I apologize. But that is not 
now nor ever was my position.

However, I do continue to maintain that character confusion is a real 
security risk that will have real impact on users, and that needs to 
be considered in any system that uses Unicode. In some domains the 
problem might be severe enough to eliminate Unicode from 
consideration in favor of less extensive character sets like Latin-1. 
That would be a shame, but until the Unicode consortium addresses at 
a root level the real security implications of their work, security 
conscious developers will look elsewhere. (I notice the Unicode 3.0 
book does not even have the word "security" in its index.) Many more 
developers who are at best tangentially conscious of security issues 
will go ahead and develop insecure systems because they don't realize 
the security implications of adopting Unicode.

There might be other solutions. I have proposed forbidding mixed 
block domain names; e.g. allowing Cyrillic names and Latin names but 
not mixed Cyrillic-Latin names. To me, that seems a reasonable 
trade-off. Vastly improved spoof-resistance (though still not 
perfect) while still allowing the registration of most desired 
localized names.

Another possibility is a super-normalization that does combine 
similar looking Unicode characters; e.g. in the domain name system we 
might decide that microsoft.com with Latin o's or Cyrillic o's or 
Greek o's is to resolve to the same address. No separate registration 
would be necessary or possible. This would require detailed analysis 
of the tens of thousands of Unicode characters allowed in domain 
names by fluent speakers of various languages; not easy, not cheap, 
but perhaps necessary. Besides, the security improvements, this 
proposal would also improve the system's usability. Aren't sure 
whether that URL on the bus used an o or an omicron? Doesn't matter, 
type either one.

>Similarly, the "security problems" caused by using Unicode encoding
>tricks to hide or mangle text in, say, contracts, is no different from
>using HTML or CSS (or whatever) tricks to achieve the same effect, and
>yet nobody is talking about security problems with HTML or CSS. See
>[1] for one way of dealing with it that is now being worked on.
>

Actually, people have been talking about the security problems with 
HTML for years. Search engines have gone to some effort to eliminate 
spamdexers that use these techniques. The log in HTML's eye does not, 
however, negate the existence of the log in Unicode's eye.

>So while I accept that there is a problem it does not seem to me that
>Unicode is the problem. To me the problem seems to be the complexity
>of the relationship between the bytes sent to the user and what the
>user actually sees and reacts to. That complexity is not going to
>disappear, and aspects of the same "problem" exist with just about any
>information representation, so clearly the solution must be something
>other than changing all of these syntaxes/formats/encodings.
>
>In the specific case you cite, for example, a better solution might be
>for the user's email client to keep track of all the user's contacts
>and for it to indicate in some clearly visible way whether the current
>email comes from one of them or not. Whether it uses string matching
>of email addresses or digital signatures to do that doesn't really
>matter; it solves the problem in your example either way.
>

If the internationalized domain name system comes into being without 
any resistance to spoofing, I will certainly recommend that people 
upgrade their client software to prevent this, just as I recommend 
today that people avoid Outlook Express, Microsoft Office, and other 
products that provide fertile soil for worms. I expect such efforts 
to be about equally effective; i.e. some wiser users will protect 
themselves. Many other users will be protected by accident. And many 
more users will not be protected at all.
--