Subject: [xmpp] RFC 3920bis and IDNA2008

From: Bernard Aboba <[email-masked]>
Date: Tue, 15 Dec 2009 08:42:12 -0800

In looking at 3920bis there would appear to be some text that may be worth
updating for IDNA2008.

Section 17: References

You may wish to remove the [IDNA] reference and substitute references to the
IDNA2008 document set:

[IDNA2008-Defs] Klensin, J., "Internationalized Domain Names for
Applications (IDNA): Definitions and Document Framework", Internet draft
(work in progress), draft-ietf-idnabis-defs-12.txt, October 2009.

[IDNA2008-Rationale] Klensin, J., "Internationalized Domain Names for
Applications (IDNA):  Background, Explanation, and Rationale", Internet
draft (work in progress), draft-ietf-idnabis-rationale-14.txt, October 2009.

[IDNA2008-Protocol] Klensin, J., "Internationalized Domain Names for
Applications (IDNA):  Protocol", Internet draft (work in progress),
draft-ietf-idnabis-protocol-17.txt, October 2009.

[IDNA2008-Tables] Falstrom, P., "The Unicode Code Points and IDNA", Internet
draft (work in progress), draft-ietf-idnabis-tables-08.txt, November 2009.

[IDNA2008-Bidi] Alvestrand, H., Karp, C., "Right-to-left scripts for IDNA",
Internet draft (work in progress), draft-ietf-idnabis-bidi-06.txt, September
2009.

Section 13

"As specified under Section 3, a server MUST support and enforce [IDNA] for
domain identifiers..."

[BA] Suggest that [IDNA] be changed to [IDNA2008-Protocol] here.

Section 3.2

A domain identifier MUST be an "internationalized domain name" as defined in
[IDNA], that is, "a domain name in which every label is an internationalized
label".

[BA]  My suggestion is that the [IDNA] reference here be changed to
[IDNA2008-Defs].  In terms of the definition, I think you may want to be
more specific, such as using the term "U-label" instead of internationalized
label in the above sentence.

Note that [IDNA2008-Defs] Section 2.3.2.3 defines an "internationalized
domain name" a bit differently than [IDNA] did:

2.3.2.3.  Internationalized Domain Name

   An "internationalized domain name" (IDN) is a domain name that
   contains at least one A-label or U-label, but that otherwise may
   contain any mixture of NR-LDH-labels, A-labels, or U-labels.

--------------------------------------------

When preparing a text label (consisting of a sequence of Unicode code
points) for representation as an internationalized label in the process of
constructing an XMPP domain identifier or comparing two XMPP domain
identifiers, an application MUST ensure that for each text label it is
possible to apply without failing the ToASCII operation specified in [IDNA]
with the UseSTD3ASCIIRules flag set (thus forbidding ASCII code points other
than letters, digits, and hyphens). If the ToASCII operation can be applied
without failing, then the label is an internationalized label.

An internationalized domain name (and therefore an XMPP domain identifier)
is constructed from its constituent internationalized labels by following
the rules specified in [IDNA].

[BA] Suggest that this be changed to:

"When preparing a U-label (consisting of a sequence of Unicode code points)
for representation as an A-label in the process of constructing an XMPP
domain identifier or comparing two XMPP
domain identifiers, an application MUST ensure that each U-label can be
successfully converted to an A-label according to the "Punycode"
specification [RFC3492], and that other constraints on the validity
of a U-label are met, as described in [IDNA2008-Defs] Section 2.3.2.1.

An internationalized domain name (and therefore an XMPP domain identifier)
is constructed from its constitutent internationalized labels by following
the rules specified in [IDNA2008-Protocol]."

"Note: The ToASCII operation includes application of the [NAMEPREP] profile
of [STRINGPREP] and encoding using the algorithm specified in [PUNYCODE];
for details, see [IDNA]. Although the output of the ToASCII operation is not
used in XMPP, it MUST be possible to apply that operation without failing. "

[BA] As described in  draft-ietf-idnabis-rationale Section 7.3, IDNA2008 is
not dependent on Nameprep.  Therefore it is not clear to me that the Note is
still necessary.

7.3.  Character Mapping

   As discussed at length in Section 6, IDNA2003, via Nameprep (see
   Section 7.5), mapped many characters into related ones.  Those
   mappings no longer exist as requirements in IDNA2008.  These
   specifications strongly prefer that only A-labels or U-labels be used

   in protocol contexts and as much as practical more generally.
   IDNA2008 does anticipate situations in which some mapping at the time
   of user input into lookup applications is appropriate and desirable.
   The issues are discussed in Section 6 and specific recommendations

   are made in [IDNA2008-Mapping].

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 15 Dec 2009 10:17:54 -0700

On 12/15/09 9:42 AM, Bernard Aboba wrote:
> In looking at 3920bis there would appear to be some text that may be
> worth updating for IDNA2008.

Many thanks for this post (and your previous one about e2e encryption,
to which I still need to reply).

The IDNA2008 work introduces a number of interesting issues for XMPP.
And those issues are not limited to domain labels, because we also use
stringprep (as did IDNA2003) for localparts and resourceparts of XMPP
addresses. We've been dancing around these issues for quite a while,
hoping that a more general approach to "fixing i18n" would emerge within
the IETF. That has not happened yet, but I have heard rumors that there
will be more discussion about this in Anaheim. I've been meaning to
write a more general report about these issues as they impinge on XMPP
(based on my in-depth reading of the IDNA2008 specs and my own imperfect
knowledge of i18n challenges), but I haven't found the time yet. Perhaps
I'll do that in the next 10 days...

> Section 17: References
> 
> You may wish to remove the [IDNA] reference and substitute references to
> the IDNA2008 document set:
> 
> [IDNA2008-Tables] Falstrom, P., "The Unicode Code Points and IDNA",
> Internet draft (work in progress), draft-ietf-idnabis-tables-08.txt,
> November 2009.
> 
> [IDNA2008-Bidi] Alvestrand, H., Karp, C., "Right-to-left scripts for
> IDNA", Internet draft (work in progress),
> draft-ietf-idnabis-bidi-06.txt, September 2009.

If we decide that we need to "upgrade" to IDNA2008, yes. I don't see how
we could *not* do that, but it might have some nasty implications (e.g.,
perhaps existing deployments use both A-labels and U-labels).

> Section 13
> 
> "As specified under Section 3, a server MUST support and enforce [IDNA]
> for domain identifiers..."
> 

Yes, with the foregoing proviso.

> Section 3.2
> 
> A domain identifier MUST be an "internationalized domain name" as
> defined in [IDNA], that is, "a domain name in which every label is an
> internationalized label".
> 
> more specific, such as using the term "U-label" instead of
> internationalized label in the above sentence.

Yes, the IDNA2008 specs are much clearer in their terminology. Ideally
we'd recommend the use of U-labels only, but we probably have deployment
of A-labels on the XMPP network, which makes things messier.

>    contain any mixture of NR-LDH-labels, A-labels, or U-labels.

True, i18n applies only to a given label. Vint Cerf had a good post
about this a few days ago on the IDNA list.

> points) for representation as an internationalized label in the process
> of constructing an XMPP domain identifier or comparing two XMPP domain
> possible to apply without failing the ToASCII operation specified in
> [IDNA] with the UseSTD3ASCIIRules flag set (thus forbidding ASCII code
> points other than letters, digits, and hyphens). If the ToASCII
> operation can be applied without failing, then the label is an
> internationalized label.
> 
> An internationalized domain name (and therefore an XMPP domain
> identifier) is constructed from its constituent internationalized labels
> by following the rules specified in [IDNA].
> 
> "When preparing a U-label (consisting of a sequence of Unicode code
> points) for representation as an A-label in the process of constructing
> an XMPP domain identifier or comparing two XMPP
> An internationalized domain name (and therefore an XMPP domain
> identifier) is constructed from its constitutent internationalized
> labels by following the rules specified in [IDNA2008-Protocol]."
> 
> "Note: The ToASCII operation includes application of the [NAMEPREP]
> profile of [STRINGPREP] and encoding using the algorithm specified in
> [PUNYCODE]; for details, see [IDNA]. Although the output of the ToASCII
> operation is not used in XMPP, it MUST be possible to apply that
> operation without failing. "
> 
> [BA] As described in  draft-ietf-idnabis-rationale Section 7.3, IDNA2008
> is not dependent on Nameprep.  Therefore it is not clear to me that the
> Note is still necessary.

I think the note is no longer needed, and that your suggested text is
consistent with IDNA2008 (though I'll need to double-check it against my
grasp of IDNA2008, which always feels rather tenuous).

One point of clarification: I think we need to be especially careful
about statements such as "a sequence of Unicode code points" because (as
was pointed out at the IRI BoF in Hiroshima) Unicode code points are
just identifiers for characters, for example:

U+13AB U+13AA U+13F4 U+13F4 U+13AC U+13D2

Those aren't encoded at all -- I could write a sequence of Unicode code
points in pen or pencil or whatever. So we need to specify more
carefully that for XMPP the sequence of code points is encoded as UTF-8.

> ------------------------------------------------------------------------

I think we would prefer U-labels above A-levels (XMPP is pure UTF-8, so
why use A-label representations at all?). And this text feels a bit
vague to me, but I don't have clearer text to offer at the moment.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 15 Dec 2009 12:52:05 -0800

As noted in [IDNA2008-Rationale] document Section 7.5, IDNA2008 did not
modify Stringprep or Nameprep (or SASLPrep, for that matter). This enables
us to keep the issue of localparts/resourceparts separate from IDNA2008.
That's probably a good thing, since the "more general approach" (or even a
revision of Stringprep/Nameprep) does not appear on the horizon.  In the
meantime though, it may be useful to gain an understanding of how current
implementations behave.

7.5.  Stringprep Changes and Compatibility

   The Nameprep [RFC3491] specification, a key part of IDNA2003, is a
   profile of Stringprep [RFC3454].  While Nameprep is a Stringprep
   profile specific to IDNA, Stringprep is used by a number of other
   protocols.  Were Stringprep to have been modified by IDNA2008, those
   changes to improve the handling of IDNs could cause problems for non-
   DNS uses, most notably if they affected identification and
   authentication protocols.  Several elements of IDNA2008 give
   interpretations to strings prohibited under IDNA2003 or prohibit
   strings that IDNA2003 permitted.  Those elements include the proposed
   new inclusion tables [IDNA2008-Tables], the reduction in the number
   of characters permitted as input for registration or lookup
   (Section 3), and even the proposed changes in handling of right to
   left strings [IDNA2008-Bidi].  IDNA2008 does not use Nameprep or
   Stringprep at all, so there are no side-effect changes to other
   protocols.

   It is particularly important to keep IDNA processing separate from
   processing for various security protocols because some of the
   constraints that are necessary for smooth and comprehensible use of
   IDNs may be unwanted or undesirable in other contexts.  For example,
   the criteria for good passwords or passphrases are very different
   from those for desirable IDNs: passwords should be hard to guess,
   while domain names should normally be easily memorable.  Similarly,
   internationalized SCSI identifiers and other protocol components are
   likely to have different requirements than IDNs.

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 15 Dec 2009 14:05:46 -0700

On 12/15/09 1:52 PM, Bernard Aboba wrote:
> modify Stringprep or Nameprep (or SASLPrep, for that matter). This
> enables us to keep the issue of localparts/resourceparts separate from
> IDNA2008.  

Agreed. However, the fact that IDNA is moving away from the stringprep
model might give us pause. There are important differences between
domain names and usernames (localpart) or device identifiers
(resourcepart), but I think the IDNA2008 work might lead us to at least
question some of our assumptions. Perhaps we'll draw the same conclusion
we drew before, but it doesn't hurt to think about it a bit more.

> That's probably a good thing, since the "more general
> approach" (or even a revision of Stringprep/Nameprep) does not appear on
> the horizon.  

Indeed. But given that XMPP technologies are one of the main
applications of stringprep, we might have some good insights to
contribute based on our implementation experience.

> In the meantime though, it may be useful to gain an
> understanding of how current implementations behave.  

Yes.

>    likely to have different requirements than IDNs.

Yes, I would recommend that everyone on this list read all the IDNA2008
specs but it's a lot of work, which is why I hope to condense the issues
down to a mail that I can send to this list.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 15 Dec 2009 16:17:23 -0800

> we drew before, but it doesn't hurt to think about it a bit more.
Indeed, the move away from the stringprep model does raise a lot of
questions -- after all, Appendix A (Nodeprep) and Appendix B (Resourceprep)
both represent stringprep profiles.  In those Appendices we see some of the
same issues that arose with IDNA2003 (Unicode version independence, mapping,
potential bidi issues, etc.).

Unfortunately, an obvious path forward wasn't evident to me.  Even given the
pending completion of work on IDNA2008, it seems like some very substantial
effort might be required.  For example, the amount of work required to
restructure the IDNA2003 document set (even setting aside documents such as
[IDNA2008-Tables] and [IDNA2008-Bidi]) was very substantial.

Is this something that could be helped by a bar/real Stringprep-bis BOF in
Annaheim?

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 15 Dec 2009 19:44:54 -0700

On 12/15/09 5:17 PM, Bernard Aboba wrote:
> questions -- after all, Appendix A (Nodeprep) and Appendix B
> (Resourceprep) both represent stringprep profiles.  In those Appendices
> we see some of the same issues that arose with IDNA2003 (Unicode version
> independence, mapping, potential bidi issues, etc.). 
> 
> Unfortunately, an obvious path forward wasn't evident to me.  Even given
> the pending completion of work on IDNA2008, it seems like some very
> substantial effort might be required.  For example, the amount of work
> required to restructure the IDNA2003 document set (even setting aside
> documents such as [IDNA2008-Tables] and [IDNA2008-Bidi]) was very
> substantial. 

Indeed. You can see why we've been avoiding the issue. :)

> Is this something that could be helped by a bar/real Stringprep-bis BOF
> in Annaheim? 

My sense is that we'll have a real BoF about this in Anaheim, where by
"we" I mean IETF participants who know and care about this topic.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/