Subject: [regext] A-label and U-label mixing - RDAP query

From: Gustavo Lozano <[email-masked]>
Date: Thu, 14 Apr 2016 19:15:47 +0000

Hello colleagues,
 
Apologies for crossposting.
 
During the public comments period of the "Registration Data Access Protocol
(RDAP) Operational Profile for gTLD Registries and Registrars",
https://www.icann.org/public-comments/rdap-profile-2015-12-03-en, the
following comment was received:
 
"Section 1.4.1 of the Operational Profile is inconsistent with the guidance
given in RFC 7482 regarding processing of RDAP queries containing a mixture
of IDN A-labels and U-labels. Per RFC 7482, ³IDNs SHOULD NOT be represented
as a mixture of A-labels and U-labels; that is, internationalized labels in
an IDN SHOULD be either all A-labels or all U-labels². This requirement is
not only inconsistent with RFC 7482, it is also counter to the consensus of
the IETF community regarding appropriate processing of IDN queries. "

Another comment on the same path says:

"Allowing A-labels and U-labels will be particularly unworkable for right to
left languages unless the RDAP server introduces arbitrary restrictions. It
is important to remember that RDAP is intended for machine-to-machine
communication. Since RFC 7482 is very clear on this guidance (do not mix the
two) any software client that generates this sort of query is broken. With
the relative youth of the RDAP standard there is unlikely to be a large
install base of software clients with said broken implementation. If ICANN
is aware of a software client that has incorrectly implemented the RDAP
standard and is now generating queries which combine A-labels and U-labels
then ICANN should take its concerns to the IETF where such challenges are
considered as part of any implementation discussion. Enshrining bad practice
within the Operational Profile will result in needless future changes to the
technology or additional service restrictions."

The comment is related to the following section of the gTLD RDAP profile:
 
"1.4.1 The RDAP server MUST support Internationalized Domain Name (IDN) RDAP
lookup queries using A-label or U-label format [RFC 5890] for domain name
and name server objects. The RDAP server MUST accept a mixture of the two
(i.e. A-label and U-label format) in the same RDAP lookup query².
 
The purpose of this message is to obtain feedback from this community
regarding this issue. We believe that the following text from RFC 7482 is a
requirement for the RDAP client: "IDNs SHOULD NOT be represented as a
mixture of A-labels and U-labels; that is, internationalized labels in an
IDN SHOULD be either all A-labels or all U-labels.".
 
RFC7482 provides an example of why a server may receive a mixture of
A-labels and U-labels in the query: "It is possible for an RDAP client to
assemble a query string from multiple independent data sources.  Such a
client might not be able to perform conversions between A-labels and
U-labels."
 
RFC7482 gives two options to the server: "An RDAP server that receives a
query string with a mixture of A-labels and U-labels MAY convert all the
U-labels to A-labels, perform IDNA processing, and proceed with exact-match
lookup.  In such cases, the response to be returned to the query source may
not match the input from the query source.  Alternatively, the server MAY
refuse to process the query".
 
We believe that the reasoning in RFC 7482 is sufficient to require gTLD RDAP
servers to accept and process a query that mixes A-labels and U-labels.

Question for this community: is the behavior specified in the gTLD RDAP
Profile (I.e. requiring processing of queries that mixes A-labels and
U-labels) consistent with RFC 7482?

Your feedback is appreciated.

Regards,
Gustavo

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Thu, 14 Apr 2016 20:13:48 +0000

From: regext [mailto:[email-masked]] On Behalf Of Gustavo Lozano
Sent: Thursday, April 14, 2016 3:16 PM
To: regext; [email-masked]
Subject: [regext] A-label and U-label mixing - RDAP query

Hello colleagues,

During the public comments period of the "Registration Data Access Protocol (RDAP) Operational Profile for gTLD Registries and Registrars", https://www.icann.org/public-comments/rdap-profile-2015-12-03-en, the following comment was received:

"Section 1.4.1 of the Operational Profile is inconsistent with the guidance given in RFC 7482 regarding processing of RDAP queries containing a mixture of IDN A-labels and U-labels. Per RFC 7482, "IDNs SHOULD NOT be represented as a mixture of A-labels and U-labels; that is, internationalized labels in an IDN SHOULD be either all A-labels or all U-labels". This requirement is not only inconsistent with RFC 7482, it is also counter to the consensus of the IETF community regarding appropriate processing of IDN queries. "

"Allowing A-labels and U-labels will be particularly unworkable for right to left languages unless the RDAP server introduces arbitrary restrictions. It is important to remember that RDAP is intended for machine-to-machine communication. Since RFC 7482 is very clear on this guidance (do not mix the two) any software client that generates this sort of query is broken. With the relative youth of the RDAP standard there is unlikely to be a large install base of software clients with said broken implementation. If ICANN is aware of a software client that has incorrectly implemented the RDAP standard and is now generating queries which combine A-labels and U-labels then ICANN should take its concerns to the IETF where such challenges are considered as part of any implementation discussion. Enshrining bad practice within the Operational Profile will result in needless future changes to the technology or additional service restrictions."

"1.4.1 The RDAP server MUST support Internationalized Domain Name (IDN) RDAP lookup queries using A-label or U-label format [RFC 5890] for domain name and name server objects. The RDAP server MUST accept a mixture of the two (i.e. A-label and U-label format) in the same RDAP lookup query".

The purpose of this message is to obtain feedback from this community regarding this issue. We believe that the following text from RFC 7482 is a requirement for the RDAP client: "IDNs SHOULD NOT be represented as a mixture of A-labels and U-labels; that is, internationalized labels in an IDN SHOULD be either all A-labels or all U-labels.".

RFC7482 provides an example of why a server may receive a mixture of A-labels and U-labels in the query: "It is possible for an RDAP client to assemble a query string from multiple independent data sources.  Such a client might not be able to perform conversions between A-labels and U-labels."

RFC7482 gives two options to the server: "An RDAP server that receives a query string with a mixture of A-labels and U-labels MAY convert all the U-labels to A-labels, perform IDNA processing, and proceed with exact-match lookup.  In such cases, the response to be returned to the query source may not match the input from the query source.  Alternatively, the server MAY refuse to process the query".

We believe that the reasoning in RFC 7482 is sufficient to require gTLD RDAP servers to accept and process a query that mixes A-labels and U-labels.

Question for this community: is the behavior specified in the gTLD RDAP Profile (I.e. requiring processing of queries that mixes A-labels and U-labels) consistent with RFC 7482?

I am the editor of the text that appears in RFC 7482. It was written to reflect guidance we received from Andrew Sullivan, John Klensin, and others - it may be worth asking them directly if they're not still following along. It was also discussed during the IESG review. This wasn't an easy topic to wade through.

Having said that, I do believe that the profile requirement that a server MUST support mixed-label lookups is inconsistent with RFC 7482. The text in Section 3.1.3 is clear on the guidance to server operators. As you quoted above:

"An RDAP server that receives a query string with a mixture of A-labels and U-labels MAY convert all the U-labels to A-labels, perform IDNA processing, and proceed with exact-match lookup.  In such cases, the response to be returned to the query source may not match the input from the query source.  Alternatively, the server MAY refuse to process the query."

The RFC says that the server operator can choose which action to perform: convert to all A-labels and process, or refuse to process. Processing a mixture of the label forms isn't one of the available options.

Scott

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 14 Apr 2016 16:35:48 -0400

Hi,

> refuse to process the query".

This means there are two acceptable paths here, and for the reasons
given in 7482 we also know that servers need to be prepared to handle
the mixed-label cases.  There is arguably an advantage of consistency
in specifying which of the two options servers under agreement with
ICANN need to be able to do, but since the two options are each
specified in the RFC it seems that clients also need to be prepared to
cope with either branch.  To me it seems like it'd be friendlier to
the ultimate consumer if the server did the IDNA processing and
proceeded.

> U-labels) consistent with RFC 7482?
 
If this requirement is "contractally required", then I think it is
consistent.  If it is "technically" or "operationally required", it is
arguably inconsistent.  I hope the distinction I'm drawing is plain.

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Gustavo Lozano <[email-masked]>
Date: Thu, 14 Apr 2016 21:18:31 +0000

Thank you Scott,

Probably my wording was not clear, the objective of section 1.4.1 of the
gTLD profile is to define which option (I.e. convert to all A-labels and
process, or refuse to process) is to be implemented by all gTLD RDAP
servers. 

Do you think that the following wording would be consistent with RFC 7482?

"1.4.1 The RDAP server MUST support Internationalized Domain Name (IDN) RDAP
lookup queries using A-label or U-label format [RFC 5890] for domain name
and name server objects. An RDAP server that receives a query string with a
mixture of A-labels and U-labels MUST convert all the U-labels to A-labels,
perform IDNA processing, and proceed with exact-match lookup."

Regards,
Gustavo

From:  "Hollenbeck, Scott" <[email-masked]>
Date:  Thursday, April 14, 2016 at 13:13
To:  Gustavo Lozano <[email-masked]>, regext <[email-masked]>,
"[email-masked]" <[email-masked]>
Subject:  RE: A-label and U-label mixing - RDAP query

> given in RFC 7482 regarding processing of RDAP queries containing a mixture of
> IDN A-labels and U-labels. Per RFC 7482, ³IDNs SHOULD NOT be represented as a
> mixture of A-labels and U-labels; that is, internationalized labels in an IDN
> SHOULD be either all A-labels or all U-labels². This requirement is not only
> inconsistent with RFC 7482, it is also counter to the consensus of the IETF
> community regarding appropriate processing of IDN queries. "
>  
> left languages unless the RDAP server introduces arbitrary restrictions. It is
> important to remember that RDAP is intended for machine-to-machine
> two) any software client that generates this sort of query is broken. With the
> relative youth of the RDAP standard there is unlikely to be a large install
> base of software clients with said broken implementation. If ICANN is aware of
> a software client that has incorrectly implemented the RDAP standard and is
> now generating queries which combine A-labels and U-labels then ICANN should
> take its concerns to the IETF where such challenges are considered as part of
> any implementation discussion. Enshrining bad practice within the Operational
> Profile will result in needless future changes to the technology or additional
> service restrictions."
>  
> lookup queries using A-label or U-label format [RFC 5890] for domain name and
> name server objects. The RDAP server MUST accept a mixture of the two (i.e.
> A-label and U-label format) in the same RDAP lookup query².
>  
> requirement for the RDAP client: "IDNs SHOULD NOT be represented as a mixture
> of A-labels and U-labels; that is, internationalized labels in an IDN SHOULD
> be either all A-labels or all U-labels.".
>  
> RFC7482 provides an example of why a server may receive a mixture of A-labels
> and U-labels in the query: "It is possible for an RDAP client to assemble a
> query string from multiple independent data sources.  Such a client might not
> be able to perform conversions between A-labels and U-labels."
>  
> RFC7482 gives two options to the server: "An RDAP server that receives a query
> string with a mixture of A-labels and U-labels MAY convert all the U-labels to
> A-labels, perform IDNA processing, and proceed with exact-match lookup.  In
> such cases, the response to be returned to the query source may not match the
> input from the query source.  Alternatively, the server MAY refuse to process
> the query".
>  
> I am the editor of the text that appears in RFC 7482. It was written to
> reflect guidance we received from Andrew Sullivan, John Klensin, and others 
> it may be worth asking them directly if they¹re not still following along. It
> was also discussed during the IESG review. This wasn¹t an easy topic to wade
> through.
>  
> Having said that, I do believe that the profile requirement that a server MUST
> support mixed-label lookups is inconsistent with RFC 7482. The text in Section
> 3.1.3 is clear on the guidance to server operators. As you quoted above:
>  
> ³An RDAP server that receives a query string with a mixture of A-labels and
> U-labels MAY convert all the U-labels to A-labels, perform IDNA processing,
> and proceed with exact-match lookup.  In such cases, the response to be
> returned to the query source may not match the input from the query source.
> Alternatively, the server MAY refuse to process the query.²
>  
> The RFC says that the server operator can choose which action to perform:
> convert to all A-labels and process, or refuse to process. Processing a
> mixture of the label forms isn¹t one of the available options.
>  
> Scott

---

From: "Patrik Fältström" <[email-masked]>
Date: Tue, 19 Apr 2016 21:22:02 +0200

All,

I think there are some confusion here and multiple issues are mixed. RFC 7482 I think is very clear, but I do on the other hand have some issues with the requirements here proposed.

First of all, in IDNA2008 an A-LABEL and U-LABEL are equivalent. One can convert from one to the other without any problems. Back and forth. Without loosing any information. Because of this, and RFC7482, I think it is correct to require from software to accept strings in either of the two formats. Including that servers might convert a query and because of that send back something that is encoded differently that what the client had done.

But please note that this is only an encoding issue. A-LABEL or U-LABEL. Just like URIs might be URL-encoded, as a form of transport encoding.

What I think people mix this with is the transformation that a string might go though before the string is IDNA2008 compliant. This includes normalization, and possibly rejection of some strings that include code points that according to IDNA2008 are not allowed.

According to IDNA2008, when it is said that a string is A-LABEL or U-LABEL it is taken for granted that regardless of which one of the encodings are used, it is a proper string with only code points that are valid for use in DNS. Code points that have the according to IDNA2008 derived property value PVALID.

My conclusion because of this is that what I think people want to have as a requirement is that a string that a client send to the server MUST be valid and ok according to IDNA2008. But this regardless of whether the string is encoded as an A-LABEL or U-LABEL.

That said, to be forward compatible with new versions of Unicode, a server MUST be prepared of receiving strings that are not valid strings as clients might use a newer version of Unicode than the server. I.e. a client MIGHT send non-valid strings encoded as A-LABEL or U-LABEL. This because one code point that is not PVALID in version N of Unicode might be PVALID in version (N+1).

And no, R2L or L2R have nothing to do with this. The order of code points is the logical order, never display order. Rendering is something completely different so that kind of reference should just be removed.

More clear?

I am happy to explain in more detail if someone have questions.

   Patrik

P.S. I am at home, after a last(?) cup of coffee for the evening, after two very stressful days in Stockholm so even if I think I know every detail of IDNA2008 (I should, right? ;-) ), I might still have some errors in *MY* thinking above, which I know some people cc:ed absolutely will notice. So some reservation of my thinking...it is though important to keep encoding from transformation separated from each other, and not talk about display order.

On 14 Apr 2016, at 23:18, Gustavo Lozano wrote:

> Thank you Scott,
>
> Probably my wording was not clear, the objective of section 1.4.1 of the gTLD profile is to define which option (I.e. convert to all A-labels and process, or refuse to process) is to be implemented by all gTLD RDAP servers.
>
> "1.4.1 The RDAP server MUST support Internationalized Domain Name (IDN) RDAP lookup queries using A-label or U-label format [RFC 5890] for domain name and name server objects. An RDAP server that receives a query string with a mixture of A-labels and U-labels MUST convert all the U-labels to A-labels, perform IDNA processing, and proceed with exact-match lookup."
>
> Regards,
> Gustavo
>
> To:  Gustavo Lozano <[email-masked]>, regext <[email-masked]>, "[email-masked]" <[email-masked]>