Subject: [idn] IDNA section 3.1 requirement 3

From: "Adam M. Costello" <[email-masked]>
Date: Wed, 16 Mar 2005 22:13:37 +0000

Consider a domain name containing a slash-homograph.

As it stands, IDNA section 3.1 requirement 3 tells applications that
they "SHOULD" display the non-ACE form.  The security considerations
section, much later, "suggests" that applications provide visual
indications of various anomalies (from which one could extrapolate that
the slash-homograph would benefit from a visual indication).

I think we've seen that these security concerns need to be less buried,
that "visual indications" are too burdensome on implementations, and
that in some cases (like this one) the recommendation to display the
non-ACE form ought to be withdrawn, or even reversed (that is, recommend
the ASCII form).

There I propose a technical change to IDNA section 3.1 requirement 3.
For reference, here it is as it stands now in RFC-3490 (with one typo
corrected):

   3) ACE labels obtained from domain name slots SHOULD be hidden from
      users when it is known that the environment can handle the non-ACE
      form, except when the ACE form is explicitly requested.  When
      it is not known whether or not the environment can handle the
      non-ACE form, the application MAY use the non-ACE form (which
      might fail, such as by not being displayed properly), or it MAY
      use the ACE form (which will look unintelligible to the user).
      Given an internationalized domain name, an equivalent domain name
      containing no ACE labels can be obtained by applying the ToUnicode
      operation (see section 4) to each label.  When requirements 2 and
      3 both apply, requirement 2 takes precedence.

Here is my proposed replacement:

--begin--

   3) When a domain label occupying or obtained from a domain name
      slot is to be shown to a user, it SHOULD NOT simply be shown in
      whatever form it was found in; before being shown it SHOULD be
      forced into either ASCII form (which can be obtained by applying
      ToASCII) or non-ACE form (which can be obtained by applying
      ToUnicode, see section 4), according to the first applicable of
      the following rules:

      a) If requirements 2 and 3 both apply, requirement 2 takes
         precedence, and the ASCII form MUST be used.

      b) When the user has explicitly requested to see one form or the
         other, that form SHOULD be shown.

      c) When it is known that the environment cannot handle the non-ACE
         form, the ASCII form SHOULD be shown.

      d) If the non-ACE form contains any character outside Unicode
         categories L (letter), N (number), and M (mark), other than
         U+002D hyphen-minus, the ACE form SHOULD be shown.

      e) If the application determines that showing the non-ACE form
         would pose too great a risk of misleading the user, the ASCII
         form MAY be shown.  Applications MAY use complex heuristics to
         estimate this risk, but SHOULD try to minimize the negative
         impact on legitimate usage of internationalized domain names.

      f) When it is not known whether the environment can handle the
         non-ACE form, the application MAY show the non-ACE form (which
         might fail, such as by not being displayed properly), or it MAY
         show the ASCII form (which will look unintelligible to the user
         if it is an ACE).

      g) In general, when rules a-f do not apply, the non-ACE form
         SHOULD be shown.

      Rules c, d, and e above apply tests to "the" non-ACE form, but
      in fact there can be many non-ACE forms that differ only in
      capitalization and/or normalization.  If a given non-ACE label
      fails some test, it MAY be converted to an equivalent non-ACE
      label by applying the map and/or normalize steps of [NAMEPREP] (or
      all the steps), and then given another chance to pass the test.

--end--

Thoughts?

AMC

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 16 Mar 2005 15:31:20 -0800

>the slash-homograph would benefit from a visual indication).

Right.

>I think we've seen that these security concerns need to be less buried,

Fair point, but it's not like that if we had put them in section 3.1, 
all browser makers would have done anything about them. From the 
testing we did a while ago, it's clear that a fair number of browser 
makers didn't even read the normative parts of the spec.

>that "visual indications" are too burdensome on implementations,

Fully disagree. We haven't seen anyone actually try visual 
indications, and some have said that they are working on them. They 
may not want to do them, but most of them didn't want to do IDNs 
either.

>  and

Question: when faced with an incomprehensible domain name with no 
visual indications, do you think that typical users will know what to 
do, or even what to be cautious of? Hint: think about what typical 
users do with SSL certificate warning dialogs.

We're trying to deal with a security issue: obscuring it won't make 
it useful to users, nor will it make them cautious, just confused.

>          U+002D hyphen-minus, the ACE form SHOULD be shown.

This is quite a reasonable attempt at picking good characters. It 
won't eliminate homographs by any matter of means, but it is a 
reasonable narrowing.

>Thoughts?

At first glance, it is a reasonable way to choose between 
unadulterated Unicode and ACE display. However, it is based on a very 
unproven assumption, namely that there isn't any other visual 
assistance that we can suggest.

Adam: could you think more about it and see how it would look if 
thought visual assistance was an option?

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: John C Klensin <[email-masked]>
Date: Wed, 16 Mar 2005 18:43:52 -0500

Adam,

I see where you are going here, but, to some considerable 
extent, I think it is a waste of time or worse.

Please take a step back from looking at coding, disclaimers, and 
this type of recommendation and put yourself into the place of 
the application developer who, other than the registrars and 
registries, is the market for the IDNA technology.  Those 
application developers, at least the ones who intend to survive, 
are quite sensitive to user reactions: users who are unhappy 
with user interfaces tend to have thoughts about going 
elsewhere.  The IETF has, for years, avoided getting involved 
with user interface design for many reasons; being the source of 
that unhappiness is one of the reasons.

You are now proposing to transform an IDNA rule that, given 
contemporary operating system and display designs, was 
essentially meaningless (but harmless) into a fairly complex set 
of rules.  If the current rule contracted the experience 
application designers wanted to deliver to their users, they 
would ignore it... and have been doing so.  If the new rule is 
at variance with the target experience, it, too, will be ignored.

Let's look at a few of the things we know about user behavior 
and reactions:

    * They do not like looking at punycode or similar "no
    obvious meaning" and/or "ugly" constructions.  If too much
    of it is displayed, they will be unhappy.

    * They don't like the unpredictable and unexpected.  If they
    are used to seeing native characters and punycode suddenly
    pops up, there had better be a good, and plausible, and
    immediately accessible explanation.

    * They get really irritated with repeated and intrusive
    warnings that they don't know how to interpret and what to
    do about.  With too many pop-up alerts, the usual response
    is to click "ok" every time and to go looking for a way to
    shut the alerts off.  Given that there are many legitimate
    cases that fall into your "display punycode" categories,
    especially in the vicinity of certain scripts and languages,
    the application would generate a lot of false positives and
    the user would learn to ignore whatever is written there.

Make whatever suggestions and recommendations you think 
appropriate, but wrapping them in conformance language about 
what SHOULD/MUST be done just brings discredit on the protocol/ 
algorithm core of IDNA.

Just my opinion.

     john

--On Wednesday, March 16, 2005 10:13 PM +0000 "Adam M. Costello" 
<[email-masked]> wrote:

> As it stands, IDNA section 3.1 requirement 3 tells
> applications that they "SHOULD" display the non-ACE form.  The
> security considerations section, much later, "suggests" that
> applications provide visual indications of various anomalies
> (from which one could extrapolate that the slash-homograph
> would benefit from a visual indication).
>
> I think we've seen that these security concerns need to be
> less buried, that "visual indications" are too burdensome on
> implementations, and that in some cases (like this one) the
> recommendation to display the non-ACE form ought to be
> withdrawn, or even reversed (that is, recommend the ASCII
> form).
>
> There I propose a technical change to IDNA section 3.1
> requirement 3. For reference, here it is as it stands now in
> RFC-3490 (with one typo corrected):
>
>    3) ACE labels obtained from domain name slots SHOULD be
> hidden from       users when it is known that the environment
> can handle the non-ACE       form, except when the ACE form is
> explicitly requested.  When       it is not known whether or
> not the environment can handle the       non-ACE form, the
> application MAY use the non-ACE form (which       might fail,
> such as by not being displayed properly), or it MAY       use
> the ACE form (which will look unintelligible to the user).
> Given an internationalized domain name, an equivalent domain
> name       containing no ACE labels can be obtained by
> applying the ToUnicode       operation (see section 4) to each
> label.  When requirements 2 and       3 both apply,
> requirement 2 takes precedence.
>
>    3) When a domain label occupying or obtained from a domain
> name       slot is to be shown to a user, it SHOULD NOT simply
> be shown in       whatever form it was found in; before being
> shown it SHOULD be       forced into either ASCII form (which
> can be obtained by applying       ToASCII) or non-ACE form
> (which can be obtained by applying       ToUnicode, see
> section 4), according to the first applicable of       the
> following rules:
>
>       a) If requirements 2 and 3 both apply, requirement 2
> takes          precedence, and the ASCII form MUST be used.
>
>       b) When the user has explicitly requested to see one
> form or the          other, that form SHOULD be shown.
>
>       c) When it is known that the environment cannot handle
> the non-ACE          form, the ASCII form SHOULD be shown.
>
>       d) If the non-ACE form contains any character outside
> Unicode          categories L (letter), N (number), and M
> (mark), other than          U+002D hyphen-minus, the ACE form
> SHOULD be shown.
>
>       e) If the application determines that showing the
> non-ACE form          would pose too great a risk of
> misleading the user, the ASCII          form MAY be shown.
> Applications MAY use complex heuristics to          estimate
> this risk, but SHOULD try to minimize the negative
>       f) When it is not known whether the environment can
> handle the          non-ACE form, the application MAY show the
> non-ACE form (which          might fail, such as by not being
> displayed properly), or it MAY          show the ASCII form
> (which will look unintelligible to the user          if it is
> an ACE).
>
>       g) In general, when rules a-f do not apply, the non-ACE
> form          SHOULD be shown.
>
>       Rules c, d, and e above apply tests to "the" non-ACE
> form, but       in fact there can be many non-ACE forms that
> differ only in       capitalization and/or normalization.  If
> a given non-ACE label       fails some test, it MAY be
> converted to an equivalent non-ACE       label by applying the
> map and/or normalize steps of [NAMEPREP] (or       all the
> steps), and then given another chance to pass the test.
>
> --end--
>
> Thoughts?
>
> AMC
>
>

---

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 17 Mar 2005 06:34:35 +0000

Paul Hoffman <[email-masked]> wrote:

> do, or even what to be cautious of?

No, but they don't need to do anything special or use any more caution
than they would with any domain name that they didn't recognize.  As
long as they don't see "paypal" or whatever, they're at least not being
misled.

> thought visual assistance was an option?

Suspicious characters could be shown in red, or highlighted, or with
squiggly underlines.  But the risk with any such scheme is that the
misleading text is still visible, so if the user doesn't take the
hint, they might still be misled.  And these approaches would be
less effective or impossible on monochrome displays or in tty-based
applications.

Another option is to show the suspicious characters using the
replacement glyph (hollow rectangle), but then cut-and-paste might
break, unless it's really clever.

I proposed the show-ACE approach because it's simpler to implement and
it's obviously effective regardless of the display technology.

>    of it is displayed, they will be unhappy.

I agree.  If enough browsers implement the proposal, no one will have a
motive to use domain labels that would show as ACE, and therefore users
will see ACE very rarely.

> Given that there are many legitimate cases that fall into your
> "display punycode" categories, especially in the vicinity of certain
> scripts and languages, the application would generate a lot of false
> positives and the user would learn to ignore whatever is written
> there.

Same comment.

> Make whatever suggestions and recommendations you think appropriate,
> but wrapping them in conformance language about what SHOULD/MUST be
> done just brings discredit on the protocol/ algorithm core of IDNA.

IDNA already makes a recommendation regarding what form to display,
but maybe you'd prefer my proposal to demote the details to cited
suggestions.  For example:

   3) When a domain label occupying or obtained from a domain name
      slot is to be shown to a user, it SHOULD NOT simply be shown in
      whatever form it was found in; before being shown it SHOULD be
      forced into either ASCII form (which can be obtained by applying
      ToASCII) or non-ACE form (which can be obtained by applying
      ToUnicode, see section 4).  Implementors are encouraged to
      develop policies that balance the conflicting goals of hiding
      unintelligible ACE strings from users and hiding potentially
      misleading Unicode strings from users.  See appendix A for
      suggestions.  When requirementst 2 and 3 both apply, requirement 2
      takes precedence.

Then the rest of the proposal would go in appendix A, without the
SHOULD/MAY language.

AMC

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 17 Mar 2005 07:37:11 -0800

I agree with John. Use of the raw punycode in place of the represented
characters will cause more user confusion, not less. User sees:

xn--tlralit-byabbe.fr versus xn--tlralit-byabb390f.fr

Presented with a collection of apparently random letters, eyes quickly glaze
over, and people really can't distinguish between two names in any sensible
fashion. Users are not going to memorize which gobbledygook is the one they
want. And this is, over course, vastly magnified once you are outside of
Latin script IDNs.

While I agree on the need to present some mechanism for distinguishing cases
(see tr36), raw punycode is not a good choice. It would even be better to
present raw IP addresses (not that I am really suggesting that).

‎Mark

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 17 Mar 2005 09:53:15 -0800

>long as they don't see "paypal" or whatever, they're at least not being

The example given at the start of your message was the 
homograph-slash. A spoof domain name that has that character and 
"paypal" will *still* display "paypal" in the Punycode. It will only 
truly work for names where the base script is not ASCII. It also 
relies heavily on browser makers ability to decide which mixed-script 
combinations are OK, and which are not, without giving them any 
guidance.

>hint, they might still be misled.

Of course. That is weighed against the user not being able to read 
what was presented at all.

For example, if a browser erroneously decided that Chinese was 
undisplayable on the device (option "c"), the user would only see 
something that didn't look like what the originator had used and 
would not be able to decide whether to accept it or not. This would 
force browser makers to check which characters were displayable each 
time it launched. Similarly, if the logic in option "e" is wrong, the 
user would never be able to view the legitimate name.

>   And these approaches would be

Highlights and squiggly underlines show up fine in monochrome. 
TTY-based applications generally can't display IDNs faithfully anyway 
due to lack of sufficient pixels; developers on those systems need to 
make other design decisions anyhow.

>break, unless it's really clever.

As you remember from a few years ago, cut-and-paste often doesn't 
work reliably with non-ASCII characters even under good conditions. 
This seems like a red herring.

>it's obviously effective regardless of the display technology.

It is effective at making some IDNs unreadable. That helps in the 
case of spoofed IDNs, and hurts in the case of legitimate IDNs.

>will see ACE very rarely.

That statement is predicated on browser makers across the world 
agreeing on how to implement option "e". That may be a tad 
optimistic. If you're wrong, it will cause IDNs in general to be 
useless due to their instability of viewing.

--Paul Hoffman, Director

---

From: John C Klensin <[email-masked]>
Date: Thu, 17 Mar 2005 14:22:14 -0500

--On Thursday, 17 March, 2005 07:37 -0800 Mark Davis
<[email-masked]> wrote:

> I agree with John. Use of the raw punycode in place of the
> represented characters will cause more user confusion, not
> less. User sees:
> 
> Presented with a collection of apparently random letters, eyes
> quickly glaze over, and people really can't distinguish
> between two names in any sensible fashion. Users are not going
> to memorize which gobbledygook is the one they want. And this
> is, over course, vastly magnified once you are outside of
> Latin script IDNs.
> 
> While I agree on the need to present some mechanism for
> distinguishing cases (see tr36), raw punycode is not a good
> choice. It would even be better to present raw IP addresses
> (not that I am really suggesting that).

I'm glad you are not, because it would cause harm and would be
unlikely to help much.  Even getting an address and
reverse-mapping wouldn't help a lot: the phishers would have
more difficulty getting hold of an appropriate reverse-mapping
delegation than the forward one, but probably not much more
difficulty.

It is probably worth noting that, in a UI context, the
punctuation problem (especially the "/" homograph version) may
not as severe as it appears.  Remembering that I don't believe
there are "solutions" to these problems, but only ways to chip
away at them, suppose that browsers and other applications that
process and present URIs (or IRIs) started parsing and coloring
them according to the parse.  No tricks about analysis of the
IDN -- just do it for everything.  And maybe color isn't the
right answer: perhaps we should borrow a note from publication
Algol and use a bold type.  So, for http://ops.ietf.org/idn, the
use would see "http://", and the "/" after "org", in bold and
the domain name and tail as normal text.  Once a user got used
to looking at that style of presentation, a homograph-"/" spoof
would stand out as highly unusual at best.  That certainly is
not a magic bullet.  It requires that users be reasonably alert
to the unusual, but that is the critical first line of defense
against phishing and similar attacks anyway.  It doesn't involve
warning messages or overwhelming the user with gibberish where
names are expected.  And maybe it is a little more in the
direction of the sort of things we should be thinking about.

best,
   john

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 17 Mar 2005 11:52:12 -0800

I agree. See also http://www.unicode.org/reports/tr36/draft/, where there
was already something along those lines.

‎Mark

----- Original Message ----- 
From: "John C Klensin" <[email-masked]>
To: "Mark Davis" <[email-masked]>; "IETF idn working group"
<[email-masked]>
Sent: Thursday, March 17, 2005 11:22
Subject: Re: [idn] IDNA section 3.1 requirement 3

---

From: James Seng <[email-masked]>
Date: Fri, 18 Mar 2005 05:45:49 +0800

I agree that that the discussion on how to represent the IDNs correctly 
to the users on the apps is very important given the recently spoofing 
attack.

I am also concern about "standardizing" how applications should flag 
out IDN or determine dangerous IDNs etc. I dont see why we need 
standardization in this area when we never really standardize the "lock 
icon" for https or etc. Instead, I think we should let market force 
work this out, and lets apps developers innovates on how to display 
these IDN properly.

OTOH, I think it is useful to document the considerations.

Does this confuse anyone?

-James Seng

On 17-Mar-05, at PM 11:37, Mark Davis wrote:

> Mark

---

From: Gervase Markham <[email-masked]>
Date: Fri, 18 Mar 2005 00:08:56 +0000

James Seng wrote:
> I am also concern about "standardizing" how applications should flag out 
> IDN or determine dangerous IDNs etc. I dont see why we need 
> icon" for https or etc. Instead, I think we should let market force work 
> this out, and lets apps developers innovates on how to display these IDN 
> properly.

I'm afraid I disagree. I was having dinner with some Microsoft security 
guys only this evening where we agreed that having the lock as a 
standard image really helped, but now we need to move to a slightly more 
fine-grained model and it would be extremely useful if everyone used the 
same notations for the new trust states.

Having said that, I believe that if we get to a situation where we need 
to "flag out IDN", we've failed.

Gerv

---

From: "Adam M. Costello" <[email-masked]>
Date: Sun, 27 Mar 2005 23:25:12 +0000

I have been persuaded that the recommendations in my proposal were too
narrow, and did not leave enough room for implementations to experiment
with alternative ways of presenting suspicious labels, like color and
decoration.  Here's a less restrictive proposal:

--begin--

[section 3.1]

     3) When a domain label occupying or obtained from a domain name
        slot is to be shown to a user, it SHOULD NOT simply be shown
        in whatever form it was found in; before being shown it SHOULD
        be forced into either ASCII form (which can be obtained by
        applying ToASCII) or non-ACE form (which can be obtained by
        applying ToUnicode, see section 4).  Implementors are encouraged
        to develop policies that balance the conflicting goals of not
        showing unintelligible ACE strings and not showing misleading
        Unicode strings.  See appendix A for suggestions.  When the user
        has explicitly requested to see one form or the other, that
        form SHOULD be shown.  When requirements 2 and 3 both apply,
        requirement 2 takes precedence.

[appendix A]

    This appendix offers suggestions only, not recommendations or
    requirements.

    For labels that are ACEs or have ACE forms, there are various
    factors that an application can consider when deciding how to
    display the label to a user.

    The ACE form is unsuitable for presentation to a user because it
    is unintelligible, unrecognizable, not very useful, and quite
    unfriendly.  Its one redeeming feature is that it is ASCII-only,
    and therefore has the best chance of being displayable and
    copy-and-pastable, and the least chance of containing misleading
    characters.

    The non-ACE form is intelligible to the user (and therefore much
    friendlier and more useful), if it is displayable.  But in an
    environment that cannot handle the characters, the non-ACE form
    could turn out to be even less useful than the ACE form.

    The non-ACE form can be misleading, by containing characters that
    look like delimiters (for example, U+2044 looks like a slash), or
    that look like characters in other scripts (for example, many Latin,
    Greek, and Cyrillic letters look alike).

    The misleading-label problem is the most complex to deal with.  A
    general approach is to identify suspicious characters, and then
    use some means to avoid displaying the suspicious characters, or
    to display them safely.  Below are a few (but certainly not all)
    possible methods.  Implementations are free to experiment and
    innovate.

    Identifying suspicious characters:

        Some characters could be considered suspicious in all contexts;
        for example, all characters outside Unicode categories L
        (letter), N (number), and M (mark), except U+002D hyphen-minus.

        Some characters could be considered suspicious in labels that
        are children of (or descendents of) certain domains.  For
        example, in a domain whose registration rules are believed to
        avoid confusion only for certain scripts, characters outside
        those scripts could be considered suspicious.  In a domain
        believed to have no restrictions on registered names, all
        non-ASCII characters could be considered suspicious.

        Some characters could be considered suspicious depending on what
        other characters appear in the same label.  For example, in a
        label containing Cyrillic, Greek, and Latin characters, one of
        those scripts could be chosen as the main script (possibly the
        one that appears first, or appears most often), and the others
        could be considered suspicious.

    Avoiding displaying suspicious characters:

        Showing the ACE form avoids displaying all non-ASCII characters,
        but see above for the disadvantages of the ACE form.

        Showing a replacement character for the suspicious characters,
        while displaying non-suspicious characters normally, will have a
        friendlier appearance than the ACE form, but is likely to break
        copy-and-paste.

        In some contexts, an escape mechanism is available that can be
        used to obscure characters.  For example, in an International
        Resource Identifier (IRI), any character can be represented by
        percent-encoded UTF-8.  This will interject some ugliness, but
        is still likely to have a friendlier appearance than the ACE
        form, and will not break copy-and-paste.

    Displaying suspicious characters safely:

        Color, highlighting, underlining, etc. could be used to flag
        suspicious characters.  One concern with this approach is
        whether the user will understand the significance of such
        markings.

--end--

The rest of this message contains a few responses to some criticisms
of my previous proposal.  The responses are probably moot, since that
proposal has now been replaced, but anyway...

Paul Hoffman <[email-masked]> wrote:

> > As long as they don't see "paypal" or whatever, they're at least not
> > being misled.
>
> "paypal" will *still* display "paypal" in the Punycode.

True.  My explanation was a bit sloppy.  Let me try again:  As long as
the user doesn't see the misleading characters, the user is not being
misled.  If the misleading character is a cyrillic "a" in "paypal", then
it's enough to avoid displaying the cyrillic "a".  If the misleading
character is a slash-homograph immediately following "paypal.com", it's
enough to avoid displaying the slash-homograph.

> As you remember from a few years ago, cut-and-paste often doesn't work
> reliably with non-ASCII characters even under good conditions.  This
> seems like a red herring.

For me, the inability to copy and paste URIs into and out of a browser's
location field would be a major inconvenience, probably enough to
make me abandon the browser in favor of another.  Once IRIs become
widespread, I expect I'll feel the same way about them.

Mark Davis <[email-masked]> wrote:

> Use of the raw punycode in place of the represented characters will
> cause more user confusion, not less.

> User sees:
> Presented with a collection of apparently random letters, eyes quickly
> glaze over, and people really can't distinguish between two names
> in any sensible fashion.  Users are not going to memorize which
> gobbledygook is the one they want.

I agree, but that's beside the point.  I don't expect anyone to
recognize an ACE, or distinguish between one ACE and another.  I
expect people to consider all ACEs as unrecognized domains.  I expect
registrants who want recognizable domains to pick domain names that
won't display as ACE to the target audience.

AMC