Subject: Re: [http-state] adopting IDNA2008 (was: Nits with references)

From: =JeffH <[email-masked]>
Date: Thu, 09 Sep 2010 16:27:33 -0700

On 06.09.2010 09:41, Adam Barth wrote:
 > On Mon, Sep 6, 2010 at 12:28 AM, Julian Reschke<[email-masked]>  wrote:
 >> RFC3490: [PROPOSED STANDARD] obsoleted by RFC5890 RFC5891
 >
 > It looks like these were just published in August 2010.  It's unclear
 > to me which is the correct document to reference.  Do you know what
 > changed between these documents?  We'd need to check which of these
 > documents better reflects reality.

Given IDNA2008 presently superseding IDNA2003 spec-wise, but over a long 
transitional period deployment-wise, here's my proposal for updating 
draft-ietf-httpstate-cookie. This also assumes that the "canonicalized string" 
paragraph (discussed in a prior msg today) is an actually separable algorithm. 
The below also adds several new normative references to -httpstate-cookie.

Note that IDNA2008 is specified strictly in terms of individual DNS labels; it 
doesn't provide a procedure for converting a domain name to/from unicode or 
ASCII, thus requiring specs referencing it to do somewhat more "work" than 
IDNA2003 required. There's also practical transitional considerations; although 
it appears the Unicode folks have nicely addressed those (I haven't studied all 
this exhaustively tho; caveat emptor at this point I'm afraid).

----8<-----

5.1.2. Canonicalized domain string

    A canonicalized domain string is a domain string converted to a series
    of NR-LDH labels (see section 2.3.2.2 of [RFC5890]) and/or A-labels
    according to the appropriate IDNA specification [RFC5891] or [RFC3490]
    (see section 6.3 of this specification), label separators are normalized
    to the U+002E (".") code point, and the labels are converted to lower case.

5.1.3. Domains and domain-match

    A string domain-matches a given domain string if at least one of the
    following conditions (1) or (2) hold:

    1.  The domain string and the string match label-by-label using normal
        DNS case-insensitive matching rules [RFC4343], regardless of whether
        the strings use the same forms of label separators.

    2.  All of the following conditions hold:

       *  The domain string is a label-by-label suffix of the string
          using normal DNS case-insensitive matching rules [RFC4343],
          regardless of whether the strings use the same forms of
          label separators.

       *  The last character of the string that is not included in the
          domain string is a U+002E (".") character.

       *  The string is a host name (i.e., not an IP address).

...

6.3 IDNA dependency and migration

    IDNA2008 [RFC5890] supersedes IDNA2003 [RFC3490] but is not
    backwards-compatible. Hence there will be a transition period (likely of
    several years). User agents SHOULD implement IDNA2008 [RFC5890], and MAY
    also implement [Unicode Technical Standard #46
    <http://unicode.org/reports/tr46/>] in order to facilitate a more smooth
    IDNA transition for users.  If IDNA2008 is not implemented, user agents
    MUST implement IDNA2003 [RFC3490].

---->8-----

HTH,

=JeffH

---

From: Adam Barth <[email-masked]>
Date: Sat, 11 Sep 2010 14:57:19 -0700

That seems fine.  I can either put this in now, or wait for more
comments (e.g., from IETF LC).

Adam

On Thu, Sep 9, 2010 at 4:27 PM, =JeffH <[email-masked]> wrote:
> draft-ietf-httpstate-cookie. This also assumes that the "canonicalized
> string" paragraph (discussed in a prior msg today) is an actually separable
> algorithm. The below also adds several new normative references to
> -httpstate-cookie.
>
> Note that IDNA2008 is specified strictly in terms of individual DNS labels;
> it doesn't provide a procedure for converting a domain name to/from unicode
> or ASCII, thus requiring specs referencing it to do somewhat more "work"
> than IDNA2003 required. There's also practical transitional considerations;
> although it appears the Unicode folks have nicely addressed those (I haven't
> studied all this exhaustively tho; caveat emptor at this point I'm afraid).
>
> ----8<-----
>
> _______________________________________________
> http-state mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/http-state
>

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Wed, 15 Sep 2010 07:41:15 +0200

I do not think it makes sense to discuss IDN processing at this point,
that should be covered by the definition of "request-host". That will
probably have to be the sequence of octets used in the DNS lookup when
dealing with DNS names (and that would be after any IDN processing), as
essentially the same is assumed for the value of the Domain attribute.

>       *  The string is a host name (i.e., not an IP address).

The `string` here is always the "canonicalized request-host" (in fact,
canonicalization is defined only for host <=> domain comparisons), so
it is a series of A-labels and NR-LDH labels, and thus cannot be any-
thing but a host name, and cannot contain "." as that is not a label.
Furthermore, a complete match is just a special case of a suffix match.

(It also seems wrong to refer to label separators other than "." here,
canonicalization as defined above normalizes them and Domain attributes
cannot contain other separators as they are octet sequences and only
0x2E is a label separating octet, as I understand RFC 1034).

With HTTP the "request-host" can be an IP address or a registered name.
Why can't this simply say a request-host "domain-matches" a "domain" if
host and domain are equivalent, or if they are registered names in a
hierarchial naming system and the host is below the domain within the
hierarchy? That would avoid discussing IDNA and canonicalization, and
work for all possible situations.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Adam Barth <[email-masked]>
Date: Thu, 16 Sep 2010 01:51:55 -0700

> essentially the same is assumed for the value of the Domain attribute.

I'm not sure your understanding is accurate.  In particular, the user
agent might never perform a DNS lookup, for example when using a proxy
server.

> Furthermore, a complete match is just a special case of a suffix match.

You're correct that we always apply this algorithm to a canonicalized
request-host.  However, canonicalization results in a sequence of
octets, not A-labels or NR-LDH labels.  The domain-match algorithm
then compares that value with the sequence of octets parsed out of an
HTTP response header.  At no time does this algorithm deal with any
DNS-level concepts.  It simply operates on sequences of octets.

> work for all possible situations.

That's insufficiently precise.  What does "equivalent" mean?  What
does "registered" or "hierarchial naming system" mean?  What does
"below" mean?  You can't avoid the discussion of IDNA or
canonicalization because those are essential elements of the correct
behavior.

Adam

---

From: Adam Barth <[email-masked]>
Date: Thu, 16 Sep 2010 02:41:32 -0700

Ok.  I've looking into this issue in some level of detail.  I've
basically done what Jeff suggests at the normative level.  At the
editorial level, I've tweaked a few things:

http://github.com/abarth/http-state/commit/4b4fb8ef90392cf089e85cc2efa53462e5d1b215

(That diff is larger than necessary because I also trimmed all the
trailing whitespace in the spec by accident.)

Details below.

> algorithm. The below also adds several new normative references to

Yeah, the canonicalization algorithm is separable.

> than IDNA2003 required.

Luckily the input to the algorithm is actual a domain name, which we
can take to be a sequence of DNS labels.  However, you're not correct
in asserting that IDNA2008 doesn't provide a way to convert to ASCII.
In particular, RFC5890 states that LHD labels are "a string consisting
of ASCII letters, digits, ...".  Therefore, we can "convert" them to
ASCII by just regarding them as the ASCII strings that they are.

> There's also practical transitional considerations;

I'm not sure whether or not the Unicode folks have done something
reasonable.  That's not really something we can deal with in this
working group.

This looks pretty good, but this algorithm outputs a sequence of
labels.  It's more convenient editorially if the algorithm outputs a
sequence of octets.  I've used your approach, but changed the output
format.

>       the strings use the same forms of label separators.

The problem here is that the second argument to this algorithm is not
a DNS name.  It's just a sequence of octets we pulled out of a domain
attribute (and converted to lower case).  For this reason, it's easier
if the canonicalized domain string is also a sequence of octets.  Once
we have both as the same type, we can just see if they're identical,
as we did before.

>         regardless of whether the strings use the same forms of

Again, by holding these quantities as octet sequences, we can just use
the usual prefix operation on sequences.

>   MUST implement IDNA2003 [RFC3490].

I took this paragraph mostly as-is, but cleaned it up slightly.  In
particular, I remove the passive voice and removed the value judgement
about the length of the transition or whether it is or is not likely.

Adam