Subject: [idn] quick & dirty (but not too dirty) homograph defense

From: "Adam M. Costello" <[email-masked]>
Date: Sat, 19 Feb 2005 22:17:19 +0000

Here's an idea for a quick-and-dirty enhancement to existing
applications:  Rather than disable IDNA entirely (which is quick but
too dirty), or flag all IDNs (almost as quick but still too dirty),
just flag all IDNs in .com and .net.  This would be significantly less
damaging to IDN deployment (which could proceed unhindered in the other
TLDs, particularly the ccTLDs), but is still extremely simple and could
be rolled out immediately while more sophisticated heuristics are
developed.

AMC

---

From: Erik van der Poel <[email-masked]>
Date: Sat, 19 Feb 2005 15:25:17 -0800

These ideas have been considered by Mozilla:

http://weblogs.mozillazine.org/gerv/archives/007556.html

However, Mozilla's IDN market share may not be as big as the others', so 
its decision may not affect IDN deployment much. MSIE doesn't support 
IDN to begin with. There are some plug-ins for MSIE to support IDN, so 
their decision might have more effect than Mozilla's:

http://support.microsoft.com/?kbid=842848

Let me point out that one of those plug-ins is made by VeriSign itself. 
Since VeriSign is such a big company, their IDN plug-in may have the 
largest market share. I haven't seen any numbers.

But what will VeriSign decide to do in their plug-in in response to this 
IDN spoofing issue?

I think I agree with you, however, that the more sophisticated 
heuristics can be developed later.

Erik

---

From: William Tan <[email-masked]>
Date: Sun, 20 Feb 2005 11:22:17 +1100

This sounds like a fairly good idea, but some "commercial" ccTLDs are 
actually very liberal in their IDN implementations as well. An example 
would be nic.tm, which permits any Unicode character allowed by IDNA. 
There are others, but I can't remember.

wil.

---

From: "Adam M. Costello" <[email-masked]>
Date: Sun, 20 Feb 2005 10:03:28 +0000

William Tan <[email-masked]> wrote:

> some "commercial" ccTLDs are actually very liberal in their IDN       
> implementations as well.  An example would be nic.tm, which permits   
> any Unicode character allowed by IDNA.                                

Yes, and I supposed they should be added to the blacklist as they are
discovered, but I think getting the Verisign TLDs (.com and .net) into
the blacklist will accomplish most of the benefit to be had from this
approach.  It's not like the phishers can move all their attacks to .tm
after they get shut out of .com and .net, because there aren't nearly as
many targets in .tm.  How many users have sensitive trust relationships
with sites in .tm (and .nu, etc)?

AMC

---

From: William Tan <[email-masked]>
Date: Sun, 20 Feb 2005 22:31:59 +1100

>with sites in .tm (and .nu, etc)?
True.

If Verisign drops support for languages for which they have no tables, 
and deal with the potential phishing attempts already in the registry, 
their TLD can then be removed from the blacklist. Likewise for whoever 
gets .net next.

wil.

---

From: Erik van der Poel <[email-masked]>
Date: Sun, 20 Feb 2005 06:13:45 -0800

Adam M. Costello wrote:
> many targets in .tm.

How do you propose to shut the phishers out of .com when a popular IDN 
plug-in (i-Nav) for the most popular browser (Microsoft Internet 
Explorer) is made by *VeriSign*, the very company that controls the .com 
registry?

http://www.idnnow.com/index.jsp

Does VeriSign also sell domain names to end-users under .com? I mean, 
are they also a registrar, not just a registry?

Besides, in networking, it's better to be conservative. You don't start 
with a short blacklist and then grow it when you find others. No, you 
start with a whitelist, and grow that.

Erik

---

From: Erik van der Poel <[email-masked]>
Date: Sun, 20 Feb 2005 06:22:44 -0800

> start with a whitelist, and grow that.

One could even make an argument along these lines for nameprep. Perhaps 
nameprep should not have started with the *huge* Unicode character set, 
subsequently making a feeble attempt to reduce that set to a safe one.

One could make the argument that the nameprep RFC does not adhere to the 
old rule:

"Be liberal in what you accept, and conservative in what you send"

This is from section 1.2.2 of RFC 1122, a key RFC also known as STD 3:

http://ietf.org/rfc/rfc1122.txt

I wonder if something like this would catch the attention of the IETF or 
even the IESG?

Erik

---

From: Jaap Akkerhuis <[email-masked]>
Date: Sun, 20 Feb 2005 16:29:20 +0100

The registrar business was split off and later sold. As far as I
know, it operates under the name ``network solutions''.

	jaap

---

From: Erik van der Poel <[email-masked]>
Date: Sun, 20 Feb 2005 08:34:29 -0800

> know, it operates under the name ``network solutions''.

I see. Thanks for the info. Does VeriSign have some other kind of 
financial incentive to increase the number of registrations under .com?

On the other hand, if some actual IDN phishing starts to occur under 
.com, VeriSign might have to tighten up their policy. They might 
eventually lose money on any lawsuits that are filed. So, in the long 
term, this IDN spoofing might not be in their best interests.

It's too bad they don't make any comments on this mailing list and take 
some kind of action, to put our minds at ease.

Erik

---

From: Soobok Lee <[email-masked]>
Date: Mon, 21 Feb 2005 01:42:00 +0900

Erik van der Poel wrote:

> Erik van der Poel wrote:
>
>> Besides, in networking, it's better to be conservative. You don't
>> start with a short blacklist and then grow it when you find others.
>> No, you start with a whitelist, and grow that.
>
>
> One could even make an argument along these lines for nameprep.
> Perhaps nameprep should not have started with the *huge* Unicode
> character set, subsequently making a feeble attempt to reduce that set
> to a safe one.
>
> One could make the argument that the nameprep RFC does not adhere to
> the old rule:
>
I like this "maxim" by late Jon Postel,too.

nameprep versioning had been debated 3 years ago, but AFAIK it was
dismissed.
Let's assume nameprep1 and nameprep2 have its own supported unicode char
set.
of course, nameprep1 is subset of nameprep2. IDN2 contains new unicode
char from nameprep2.

Decode&Normalize&Verify2(Normalize&Verify&Encode1(IDN2)) would fail
in the inner expression by the encoding party who uses nameprep1 ,
Decode&Normalize&Verify1(Normalize&Verify&Encode2(IDN2)) would fail
in the outer expression by the decoding party whoi uses nameprep1,
while
Decode&Normalize&Verify2(Normalize&Verify&Encode1(IDN1)) would succeed
by both parties, assuming two parties have different versions of namepreps..

nameprep versioning and applicaton-level filtering both fundamentally
change
how dns resolving works, by adding new *exception handlings* that did not
exist with ASCII only DNS. that is, dns stability issues came out at
that time.

We should have to produce perfect nameprep/stirngprep at once and use
forever
without versioning, but unicode is not designed/not prepared for
identifier use.
I think that is the source of all these homograph problems.

Please correct me if my memory about this topic is wrong, senior members.

Soobok

---

From: tedd <[email-masked]>
Date: Sun, 20 Feb 2005 12:58:25 -0500

>eventually lose money on any lawsuits that are filed. So, in the 
>long term, this IDN spoofing might not be in their best interests.
>
>It's too bad they don't make any comments on this mailing list and 
>take some kind of action, to put our minds at ease.
>
>Erik

Erik:

Well... the problem is not totally Versign's, is it?

On one hand, the list wants to solve the homograph attack (glyph 
look-alike) problem, but on the other hand when confronted with a 
clear solution to a specific problem, the list ignores it.

Case in point, the lower and upper case omega. Clearly the upper case 
omega would not be confused with any other glyph -- it's unique. 
However, the list mandates that all characters shall be lower case. 
As such, the lower case omega looks like a lower case "w" and thus 
presents a look-a-like problem.

Now, I'm not in favor of allowing both upper and lower case 
characters for each glyph into the mix, BUT, I think it wise to look 
at one -- OR-- the other to see if a solution for each glyph presents 
itself. Is this not reasonable?

As I understand it, this would only require a change in mapping. That 
should solve at least one "glyph look-alike" problem -- shouldn't it?

tedd
-- 
--------------------------------------------------------------------------------
http://sperling.com/

---

From: Erik van der Poel <[email-masked]>
Date: Sun, 20 Feb 2005 10:31:21 -0800

> Well... the problem is not totally Versign's, is it?

I agree. Part of the problem is upstream from VeriSign.

> omega would not be confused with any other glyph -- it's unique.

That may be true, but would the users like to see an upper case omega in 
the middle of a name? Don't get me wrong -- I appreciate the example -- 
but there are pros and cons to everything.

> should solve at least one "glyph look-alike" problem -- shouldn't it?

Wouldn't this require a new ACE prefix?

Erik

---

From: tedd <[email-masked]>
Date: Sun, 20 Feb 2005 14:10:12 -0500

>tedd wrote:
>>Case in point, the lower and upper case omega. Clearly the upper 
>>case omega would not be confused with any other glyph -- it's 
>>unique.

Erik replied:

>That may be true, but would the users like to see an upper case 
>omega in the middle of a name? Don't get me wrong -- I appreciate 
>the example -- but there are pros and cons to everything.

That's a good point. But, given the "look-alike" problem, and 
comparing it to the possibility that an upper-case Omega may appear 
within a name, what's the "better" solution?  You can't have both.

I'm not arguing just for the Omega glyph (that's just an example), 
but rather I believe that there are more than a single glyph where 
its uppercase may present a better solution for a "look-alike" 
problem than it's lowercase counterpart. And, I am not proposing a 
mixed case solution for the IDN, but rather a single case for each 
code point. Granted, the vast majority of code points would remain 
lowercase, but a few uppercase exceptions may alleviate the 
"look-alike" problem.

Just an idea.

tedd

---

From: Erik van der Poel <[email-masked]>
Date: Sun, 20 Feb 2005 11:19:46 -0800

I doubt that we will be able to totally solve the IDN phishing problem, 
just like we cannot totally solve the more general phishing problem. 
These phishers come up with new tricks all the time.

I'm guessing that the IDN spoofing issue will be addressed (not solved) 
at several different levels, upstream and downstream.

The Unicode Consortium has already stated that they are seriously 
considering the homograph issue, and we may eventually see tables and 
other info that others can use in heuristic filters, etc.

It will be interesting to see what happens with the IDN-related RFCs 
when they advance to the next maturity level. I'm not even going to try 
to guess what might happen there.

Many ccTLDs have already put filtering policies and deployments in 
place, and the other TLDs will probably start to filter, sooner or later.

Security-conscious apps and plug-ins will also have a layer of 
protection, with UIs ranging from the seamless to the intrusive.

I will remain on this list for the time being.

Thanks everybody,

Erik

---

From: "Martin v. Löwis" <[email-masked]>
Date: Sun, 20 Feb 2005 23:34:43 +0100

> should solve at least one "glyph look-alike" problem -- shouldn't it?

Anything involving changes in the way IDNA works (through changing
mappings, or restricting characters further on the client side) is
not a solution, whatever the problem. Any such change will take ten
years or more to implement.

Regards,
Martin

---

From: "Adam M. Costello" <[email-masked]>
Date: Sun, 20 Feb 2005 23:27:33 +0000

Erik van der Poel <[email-masked]> wrote:

> How do you propose to shut the phishers out of .com when a popular
> IDN plug-in (i-Nav) for the most popular browser (Microsoft Internet
> Explorer) is made by *VeriSign*, the very company that controls the
> .com registry?

If i-Nav leaves its users exposed to phishing attacks, and the users are
bothered by it, then I would expect them to switch to a different IDN
plug-in. isc.org released an alpha3 version of such a plug-in a year
ago, so maybe that could be whipped into shape quickly.

> No, you start with a whitelist, and grow that.

I agree, but in this case, we have already missed the start.  If
we introduce a whitelist now, after IDN deployment is already well
underway, we are effectively punishing an unknown number of innocent
early adopters, which seems like a betrayal.  If we had forseen this
problem, we could have set up the whitelist in the beginning, and
registries & registrars would have known to get themselves added to the
whitelist before making any promises to their customers.

AMC

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Mon, 21 Feb 2005 00:59:26 +0100

>whitelist before making any promises to their customers.

Dear Adam,
we known the problem, as we known the TLD problem, as we known the 
babel-name problem, as we known the IDN zone DNS management complexity, as 
we known the 3+LD problem, as we known the Verisign/ccTLD attitude, as we 
known the lack of layered analysis, etc. (I mean at least people who 
dialogued with me in private when the wg-correct was calling me a dumb 
debile).

Your punycode is a transcoding solution of interest. Tables are an attempt 
to fix part of the IDN problem, but it needs to be applied where it is 
useful and controlled by the user (in phishing: at printing and outbound 
nameprep).
Take care.
jfc

PS. I fail to see what Registrars are to do with this?

---

From: Erik van der Poel <[email-masked]>
Date: Sun, 20 Feb 2005 15:59:52 -0800

Adam M. Costello wrote:
> plug-in.

Good point.

Re: white list:
> I agree, but in this case, we have already missed the start.

mozilla.org has a pretty wide reach worldwide, partly due to the 
localization efforts. So maybe mozilla.org could get in touch with 
various people, either to get info, or to have them beta-test a version 
with a white list, etc.

Erik

---

From: John C Klensin <[email-masked]>
Date: Sun, 20 Feb 2005 21:17:16 -0500

--On Sunday, 20 February, 2005 23:34 +0100 "\"Martin v. Löwis\""
<[email-masked]> wrote:

> tedd wrote:
>> As I understand it, this would only require a change in
>> mapping. That  should solve at least one "glyph look-alike"
>> problem -- shouldn't it?
> 

Martin, 

For a change, I don't share the pessimism on this.  I think we
should be very hesitant to make changes unless it is pretty
clearly demonstrated that they are necessary, with a high
threshold on such a demonstration.  But it seems to me that:

(i) A change that would largely impact what can be registered
needs to be reflected and implemented only in 250-odd
registries.  The registry operators are mostly on their toes,
communicate with each other, and many of them are pretty early
in their implementation of IDNs and conservative about what they
are permitting.  Getting them to make changes is an entirely
different sort of problem than, e.g., trying to change
already-installed browsers or client plugins or getting people
to upgrade them.

(ii) The main things I've seen in observing and working with
registries that I didn't understand well enough a couple of
years ago to argue forcefully are things that we might be able
to change because the impact of whether someone was running an
old or new version would not be large.  For example, IDNA makes
some mappings that are dubious, not in the technical sense of
whether the characters are equivalent, but in the human factors
sense of whether treating them as equivalent leads to bad
habits.  To take a handy example from a Roman ("Latin")-based
script, I now suspect that permitting all of those font-variant
"mathematical" characters to map onto their lower-case ASCII
equivalents is a bad idea, just because it encourages users to
assume that, if something looks like a particular base
character, it is that character.  That, in turn, increases the
perceptual window for these phishing attacks.  If, instead, we
had simply banned those characters, creating an error if someone
tried to use one rather than a quiet mapping into something
else, we might have been better off.  So I now think we should
have banned them when IDNA and nameprep were defined and think I
could have made that case very strongly had I understood the
issues the way I do now.   Is it worth making that change today?
I don't know.  But I suggest that it would be possible to make
it for two reasons: (a) such a change would not change the
number of strings or characters that can be registered at all:
only the base characters can actually appear in an IDNA string
post the ToUnicode(ToASCII(char)) operation pair and (b) if I
were a browser or other application producer, I'd be seriously
considering warnings if any characters from those blocks
appeared... something IDNA certainly does not prohibit.  Changes
that increased the number of registerable characters are
problematic, but not that problematic if they don't pick up a
character that now maps and make it "real" (which is the problem
with deciding that upper case Omega is a good idea).  Reducing
the number of characters that can be registered --making a
now-valid base character invalid-- would be a much harder
problem.

(iii) Finally, and most important, there are several areas in
the world for which IDNA, as now defined, is equivalent to no
progress on IDNs at all because they need characters that are
not in Unicode 3.2.  To accommodate them, we are going to need
to upgrade to at least 4.0 and, in some cases, to sets of
characters not yet fully defined.  At some point, our upgrading
the version of Unicode is going to be an absolute necessity.
That requires changing the nameprep/stringprep tables and, no
matter how painful we think that might be, we are going to need
to figure out how to do it.

---------------

Rather than start another note, an observation about the
blacklist idea that doesn't seem to have been mentioned...  Let
me state this in personal terms:  I don't like blacklists.  The
reasons aren't technical at least in the first instance, but a
whole series of system behavior observations.  For example:

(1) It tends, in practice, to be much easier to get on them than
off of them.  One can certainly say "put .NET and .COM on the
list, and, when Verisign changes its behavior, we will take them
off".  But that hides a whole set of issues, such as whether you
believe them when they say their behavior has changed, whether
the behavior has changed enough (unfortunately, this is really
not a binary situation), and so on.

(2) There is a protocol issue with how the blacklist is
maintained.  If it is part of the code of the relevant
applications software, or a plug-in or add-on to it, we have to
face the problem that the average end user doesn't upgrade
things on the schedules we would like, resulting in a different
"harder to get off than on" case.  If the application is
designed to automatically update itself from a common source on
the network, we are suddenly faced with all of the problems
caused by various people's ideas of security: authentication of
the new tables, firewalls and system configurations that are
hostile to that sort of upgrading, and so on.

(3) As we have seen with spam, the existence of a blacklist
mechanism, no matter how well conceived, is an invitation to
abuse and/or claims that abuse is occurring.  If we create a
mechanism for putting up warnings about IDN use in particular
domains, how long will it be before there are demands to put up
warnings for domains that are, statistically, too friendly to
spammers, pornographers, phishers, music-swappers, or holders of
unpopular social or political positions?  We all understand the
difference between those issues and ones where the boundary can
be set using objective technical criteria.  But we have had long
experience with governments and policy advocates not
understanding the distinction and, indeed, being determined to
not do so.

(4) Finally, the result of too many warnings is that warnings
are ignored.  Certainly it is better to flag or color IDNs from
a few domains than it is for all domains.  And it is almost
certainly better to do something than nothing.  But, if the
domains flagged contain a large fraction of all of the
second-level registrations in the world, we should keep our
expectations of how much such measures will help very moderate.

    john

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Mon, 21 Feb 2005 12:53:05 +0100

At 03:17 21/02/2005, John C Klensin wrote:
>(a) such a change would not change the
>post the ToUnicode(ToASCII(char)) operation pair

Interesting idea, but you may lose every chance to have a RHD/LHD common 
support if you make it too simply. On the countrary it permits to have the 
information to differentiate upper and lower cases if/when needed.
jfc

---

From: Erik van der Poel <[email-masked]>
Date: Mon, 21 Feb 2005 16:13:31 -0800

John,

I'm probably missing something, but if the apps are not currently 
warning the user when characters from blocks that ought to be banned 
appear, then people and/or tools may be generating references (e.g. HTML 
documents) to those characters, without realizing that those characters 
are being mapped to some other base characters, which then work OK in 
the DNS lookup.

Maybe it is unlikely that a lot of such references would come to exist, 
and it wouldn't be such a burden to the user of a new app to see the 
occasional error e.g. when they click on such a link.

But how do you determine how many HTML documents contain bad characters 
in their links, and how do you decide that that number is low enough to 
make such a change to the spec?

So I'm wondering if you would really be able to state that such a change 
"would largely impact what can be registered". How do you know that such 
a change does not also impact the users of new clients accessing 
existing documents?

Oh wait, I know! Just get Google to do a survey in their cache?

Erik

---

From: Erik van der Poel <[email-masked]>
Date: Mon, 21 Feb 2005 16:34:25 -0800

Erik van der Poel wrote:
> "would largely impact what can be registered".

Sorry, this last part is a mistake. Banning characters that were 
previously mapped to others does *not* affect what can be registered. Duh.

But please consider the other parts of my email.

Erik

---

From: Erik van der Poel <[email-masked]>
Date: Mon, 21 Feb 2005 19:58:48 -0800

I've been wondering whether my email was clear or not. The kind of 
reference I'm talking about is the kind that started this whole IDN 
spoofing discussion:

<a href="http://www.payp&#1072;l.com/"

The above is from:

http://secunia.com/multiple_browsers_idn_spoofing_test/

As you can see, this HTML snippet is using a "numeric character 
reference" (the &#1072;) instead of the Punycode form of the name. Now, 
you may point out that HTML authors and editors should use the Punycode 
form, but the above proves that it is possible to type IDNs by hand.

HTML has a long history of allowing users to type it manually, and, 
regrettably, the browsers have an equally long history of *accepting* 
practically anything that they type, whether it be well-formed SGML or not.

This is how we ended up with such a huge mass of garbage. The Web 
browsers took the "be liberal in what you accept..." rule a bit too 
literally. The browsers were too liberal. But I digress.

My point is that if it's possible to type IDNs manually into HTML, then 
you might end up with some HTML documents that use the very characters 
that you want to ban in the next rev of nameprep. I am certainly not 
claiming that there will be very many of those. All I'm asking is, how 
do we find out whether they are going to be a problem?

Erik

Erik van der Poel wrote:
> [...] How do you know that such

---

From: John C Klensin <[email-masked]>
Date: Tue, 22 Feb 2005 00:53:29 -0500

--On Monday, 21 February, 2005 16:13 -0800 Erik van der Poel
<[email-masked]> wrote:

> John,
> 
> I'm probably missing something, but if the apps are not
> currently warning the user when characters from blocks that
> ought to be banned appear, then people and/or tools may be
> generating references (e.g. HTML documents) to those
> characters, without realizing that those characters are being
> mapped to some other base characters, which then work OK in
> the DNS lookup.

You are, of course, correct.  While warning others to think
about things from the user and content-generator side of things
rather than the registrar/registry side, I made a similar
mistake that probably leads to an incorrect conclusion.  My
assumption was that those who registered names and constructed
links would end up with the domain names in those links that
were equivalent of
ToUnicode(ToASCII(whatever-was-submitted-for-registration),
i.e., base characters only.  That would certainly be the result
of a number of the processes for obtaining those links that
crossed my mind.  But you are quite right, nothing requires it
and references could be constructed using non-base characters.
For languages with clear case distinctions, one would actually
predict that it would occur: just as the owner of
coolhotjunk.xyz might prefer to have it in the DNS and
references as CoolHotJunk.xyz, I'd expect funny mixed case
strings in IDNs which, under IDNA, would of course go into the
DNS as the punycode encoding of the lowercase-only form.

> Maybe it is unlikely that a lot of such references would come
> to exist, and it wouldn't be such a burden to the user of a
> new app to see the occasional error e.g. when they click on
> such a link.
> 
> But how do you determine how many HTML documents contain bad
> characters in their links, and how do you decide that that
> number is low enough to make such a change to the spec?
> 
> So I'm wondering if you would really be able to state that
> such a change "would largely impact what can be registered".
> How do you know that such a change does not also impact the
> users of new clients accessing existing documents?
> 

:-(

Mumble.  What a mess.
    john

---

From: John C Klensin <[email-masked]>
Date: Tue, 22 Feb 2005 01:00:40 -0500

--On Monday, 21 February, 2005 19:58 -0800 Erik van der Poel
<[email-masked]> wrote:

> I've been wondering whether my email was clear or not. The
> kind of reference I'm talking about is the kind that started
> this whole IDN spoofing discussion:
> 

Clear enough, unfortunately.  I'm still gagging.

> As you can see, this HTML snippet is using a "numeric
> character reference" (the &#1072;) instead of the Punycode
> form of the name. Now, you may point out that HTML authors and
> editors should use the Punycode form, but the above proves
> that it is possible to type IDNs by hand.
> 
> HTML has a long history of allowing users to type it manually,
> and, regrettably, the browsers have an equally long history of
> *accepting* practically anything that they type, whether it be
> well-formed SGML or not.
> 
> This is how we ended up with such a huge mass of garbage. The
> Web browsers took the "be liberal in what you accept..." rule
> a bit too literally. The browsers were too liberal. But I
> digress.
> 
> My point is that if it's possible to type IDNs manually into
> HTML, then you might end up with some HTML documents that use
> the very characters that you want to ban in the next rev of
> nameprep. I am certainly not claiming that there will be very
> many of those. All I'm asking is, how do we find out whether
> they are going to be a problem?

We don't.  Even your Google plan (which I assume was at least
partially tongue-in-cheek) would be pointless, as there is no
way to run a check over the firewall and/or database-protected
"dark web".   We ultimately need to ask ourselves, and the
implementer and, somehow, the user communities whether a more
restrictive change is important enough.  If it is, we pronounce
the words "proposed standard" and "problems that showed up in
implementation and deployment experience" and try to get it
fixed as quickly as possible, recognizing that  there will be
links out there that need to be translated forward and back
through IDNA2003 to get back links that would work with
IDNA2005.  If it isn't, we learn to cope.

   john