Subject: [weirds] Http return code 400 vs 404

From: Ernie Dainow <[email-masked]>
Date: Tue, 18 Mar 2014 10:24:14 -0400

The spec that covers this behaviour is 
http://tools.ietf.org/html/draft-ietf-weirds-using-http-08#section-5.4
5.4. Malformed Queries
If a server receives a query which it cannot interpret as an RDAP
query, it returns a 400 response code.

There are a number of Inter-Op test cases that flag a return of 400 
instead of 404 as an error and vice versa.

There are a whole set of missing Inter-Op test cases when a url path is 
incorrect, for example, instead of /domains?name=abc*
/domains?nam=abc*
/domain?name=abc*
/domai?name=abc*
and instead of domain/abc.info
/domai/abc.info

Here it seems the return codes should all be 400. This most likely 
points to an error in the client code.

In the Inter-Op test cases, the error is always a result of the user 
input, such as the domain name or nameserver name. For all these cases, 
it would be most helpful to return 404 to distinguish user input user 
error from implementation error.

So the following test cases (and variations) should return 404, not 400
ts_query_3_1_3_5U 
/domain/σειράτάξησυπουργείωνΣύνθεσηυπουργικούσυμβουλίουουουο.info
ts_query_3_1_3_7 /domain/xn--cafés.com

Perhaps the spec could be clarified to make this distinction more clear. 
As currently stated whether to return a 400 or a 404 is somewhat arbitrary.

-Ernie

---

From: Simon Perreault <[email-masked]>
Date: Tue, 18 Mar 2014 10:49:48 -0400

> points to an error in the client code.

Agreed. We have one similar test named "ts_query_3". It could be
expanded with the set you suggest.

> ts_query_3_1_3_7 /domain/xn--cafés.com

To be clear: these are two examples of invalid IDN. The first one
contains a disallowed character (capital sigma, Σ), while the second one
start with "xn--" but contains a non-LDH character. This makes the
desired return code 400 rather than 404. 404 would only be returned to a
query for a *valid* IDN.

User input has nothing to do with it: with a conforming client
implementation, it would be impossible for the user to generate such
invalid IDN queries. The client implementation would have to implement
the IDNA algorithm, which would process user input such that the client
never sends invalid IDNs to the server. Therefore, only a buggy client
(or our very devious test suite) would send invalid IDN queries. 400 is
thus the appropriate status code for the two queries above.

Simon
-- 
DTN made easy, lean, and smart --> http://postellation.viagenie.ca
NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
STUN/TURN server               --> http://numb.viagenie.ca

---

From: Simon Perreault <[email-masked]>
Date: Tue, 18 Mar 2014 11:56:11 -0400

Le 2014-03-18 11:25, Ernie Dainow a écrit :
> RDAP will be used as a backend for a lot of web application. So the user
> will be entering the domain name to lookup, and could enter
> 
> σειράτάξησυπουργείωνΣύνθεσηυπουργικούσυμβουλίουουουο.info
> 
> just as easily as you did in your test script. So the error originates
> with user input.

The current text allows A-labels and U-labels to be sent to the server.
That means that only IDNA-valid strings are allowed (see RFC 5890
section 2.3.2.1). The above example is not IDNA-valid, which means it is
not a U-label.

If we want to impose no IDNA requirements on the client, then it is not
sufficient to allow U-label queries: we need to allow raw Unicode. The
server would then run the IDNA algorithm on the raw Unicode, generating
an IDNA-valid string, and running the query on that.

Simon

---

From: Ernie Dainow <[email-masked]>
Date: Tue, 18 Mar 2014 17:03:13 -0400

The query spec specifies IDNA processing as in RFC5891:
   If the client sends the server an IDN in U-label format, servers that
   support IDNs MUST convert the IDN into A-label format and perform
   IDNA processing as specified in RFC 5891 [RFC5891]. The server
   should perform an exact match lookup using the A-label.

The reference for lookup (as opposed to name registration) is in  RFC 5891,
Section 5.
In 5.2. it says
The string is converted from the local character set into Unicode, if
it is not already in Unicode.

We have usually tried to offload the burden of difficult checking from the
client to the server. That principle applies here -- the client can submit
Unicode and the server does all the IDNA validity checking.

-Ernie

---

From: Simon Perreault <[email-masked]>
Date: Tue, 18 Mar 2014 17:05:55 -0400

Le 2014-03-18 17:03, Ernie Dainow a écrit :
> The reference for lookup (as opposed to name registration) is in  RFC
> 5891, Section 5. 
> In 5.2. it says
> We have usually tried to offload the burden of difficult checking from
> the client to the server. That principle applies here -- the client can
> submit Unicode and the server does all the IDNA validity checking.

I fully agree with that principle. That's why I suggest to allow not
only A-label and U-label requests, but any raw Unicode string.

Simon