Subject: [regext] RDAP lookup queries for reserved or unassignable domains

From: Mario Loffredo <[email-masked]>
Date: Wed, 7 Dec 2016 08:48:07 +0100

Hi all,

I have a question about how to deal with RDAP lookup queries for 
reserved or unassignable domains.

If a client submits a query about a reserved or unassignable domain, 
which response code should the server return?

In my opinion, if the server returns a 404 response, it could be 
misunderstood. The client could undestand the domain is available.

In the GET method, the server may include additional information 
regarding the negative answer, but it can't in the HEAD method.

Maybe the 410 response code could be used in this case ?

Thanks for your answers,
Mario

-- 
Dr. Mario Loffredo
Servizi Internet e Sviluppo Tecnologico
Unità Sistemi e Servizi Registro .it
CNR - Istituto di Informatica e Telematica
via G. Moruzzi 1, I-56124 PISA, Italy
E-Mail: [email-masked]
Phone: +39 050 3153497
Web: http://www.iit.cnr.it/mario.loffredo

---

From: Rubens Kuhl <[email-masked]>
Date: Wed, 7 Dec 2016 07:41:03 -0200

> On Dec 7, 2016, at 5:48 AM, Mario Loffredo <[email-masked]> wrote:
> 
> Hi all,
> 
> I have a question about how to deal with RDAP lookup queries for reserved or unassignable domains.
> 
> If a client submits a query about a reserved or unassignable domain, which response code should the server return?
> 
> In my opinion, if the server returns a 404 response, it could be misunderstood. The client could undestand the domain is available.
> 
> In the GET method, the server may include additional information regarding the negative answer, but it can't in the HEAD method.
> 

I think you are suggesting to keep a confusion that currently exists in the domain distribution chain between availability checking and registration data directory services (called WHOIS in the previous millennium). 
*The* tool for availability checking is domain:check transactions in EPP protocol; but this suffers from some issues:
- TLS session setup adds latency to transaction
- Resellers can't connect to EPP 

I think a lightweight availability checking protocol, possibly using DTLS and incorporating a predefined set of EPP extensions (like draft-brown-epp-fees) so it doesn't have to negotiate capabilities, would be a better approach than keeping this mix-up between use cases enshrined in stone. There are already in use similar implementations, see ftp://ftp.registro.br/pub/isavail/isavail-0.5.tar.gz <ftp://ftp.registro.br/pub/isavail/isavail-0.5.tar.gz> . 

Rubens

---

From: Marcos Sanz <[email-masked]>
Date: Wed, 7 Dec 2016 14:51:14 +0100

Hello Mario,

>misunderstood. The client could undestand the domain is available.

a very valid question. Even if you are able to signalize that the domain is not available, you would like to further communicate the clients that they don't need to query over and over again to find out if it's expired or become free.

RFC 7483 offers status  "renew prohibited"

Description: Renewal or reregistration of the object instance is
 forbidden.

draft-ietf-regext-epp-rdap-status-mapping refines that and introduces "server renew prohibited", which in my opinion is probably what you want to communicate (server policy).

>regarding the negative answer

See above.

>, but it can't in the HEAD method.

> Maybe the 410 response code could be used in this case ?

If we agree on the above, then the HEAD answer must be a 200.

Best,

Marcos

---

From: "John Levine" <[email-masked]>
Date: 7 Dec 2016 14:19:45 -0000

>misunderstood. The client could undestand the domain is available.

This is not a problem that RDAP is designed to solve.  

There are lots of reasons an RDAP server might not have info about a
domain, e.g., you ask it about a TLD it doesn't handle, and there's no
reason to assume that an RDAP server would know anything about an
unknown domain's avaiability.

It's true that you can't prevent clients from doing stupid things, but
there's nothing new about that.  If they keep hammering on you, rate
limit or block them.

R's,
John

---

From: Andrew Newton <[email-masked]>
Date: Wed, 7 Dec 2016 09:27:29 -0500

If the goals of a domain availability check are:

1) use a simple technology such as HTTP (and RDAP)
2) keep the payload small

One solution may be to simply put a query parameter on the GET such as
?availability=true.
Then the server would know to send back the minimum payload necessary,
thus avoiding excessive bandwidth and allowing the server to execute a
tighter code path for that specific purpose (i.e. no database fetches
of entities, etc...).

-andy

On Wed, Dec 7, 2016 at 8:51 AM, Marcos Sanz <[email-masked]> wrote:
> a very valid question. Even if you are able to signalize that the domain is
> not available, you would like to further communicate the clients that they
> don't need to query over and over again to find out if it's expired or
> become free.
>
> draft-ietf-regext-epp-rdap-status-mapping refines that and introduces
> "server renew prohibited", which in my opinion is probably what you want to
> communicate (server policy).
>
> _______________________________________________
> regext mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/regext
>

---

From: Andrew Newton <[email-masked]>
Date: Wed, 7 Dec 2016 09:32:49 -0500

> This is not a problem that RDAP is designed to solve.

In my opinion, with a few tweaks it could. We did sidestep the issue
in the original protocol design to keep the scope of the work small.

> unknown domain's availability.

I would think anybody doing reselling would have the availability
servers explicitly addressed and would not use redirects to find an
authoritative server, no matter the underlying protocol.

-andy

---

From: "John R Levine" <[email-masked]>
Date: 7 Dec 2016 09:56:11 -0500

> authoritative server, no matter the underlying protocol.

I was thinking that you're the RDAP server for .FOO and someone who's 
screwed up his bootstrap asks about BLAH.BAR, with whom you have no 
connection.  Or you're a reseller for a thin registry and someone asks you 
about a domain at another reseller, about which you have no info. 
(Perhaps it was transferred and the client thinks it can speed things up 
by cacheing old redirects forever.)  So either way you return a 404 code.

I suppose we could pick another code for "it's not assigned but if you pay 
me money it could be" but I really don't think it's a good idea to read 
anything beyond "I don't know" into a normal 404 response.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 7 Dec 2016 10:12:38 -0500

On Wed, Dec 07, 2016 at 09:56:11AM -0500, John R Levine wrote:
> connection.

I will just point out that this is the _exact_ reason some of us
thought the bootstrap mechanism should have been SRV records in the
DNS, because it would have neatly solved that exact problem.  

> anything beyond "I don't know" into a normal 404 response.

I agree with this in principle, but given the way humans actually use
the RDDS, there's going to need to be _some_ way to communicate this
difference.  In particular, for policy reasons it's important to
understand "this domain isn't available because someone has it", "this
domain isn't available because someone has something that prevents it
being registered", and "this domain isn't available to anyone for
policy reasons."  Consider people doing compliance checks, who maybe
shouldn't have access to the SRS directly and who should only have
access to the RDDS.  They still need to be able to see these
distinctions.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Andrew Newton <[email-masked]>
Date: Wed, 7 Dec 2016 10:20:20 -0500

> DNS, because it would have neatly solved that exact problem.

I'm unaware of any HTTP clients that natively use SRV records (there
may be some, but its far from the norm). To me, "neatly" is in the eye
of the beholder. :)

-andy

---

From: "Gould, James" <[email-masked]>
Date: Wed, 7 Dec 2016 15:30:24 +0000

It sounds like you’re attempting to morph RDDS into the SRS.  RDDS is a lookup service and the SRS is an OLTP system.  A lookup service either has the data or it doesn’t.  Extra business logic associated with availability (variant blocking, relationship blocking, reserved domains, etc.) should be left to the appropriate channel which is the SRS and not RDDS. 

-- 
 
JG
 
James F. Gould
Distinguished Engineer
Verisign
[email-masked]

On 12/7/16, 10:12 AM, "regext on behalf of Andrew Sullivan" <[email-masked] on behalf of [email-masked]> wrote:

    

    A
    
    -- 
    Andrew Sullivan
    [email-masked]

---

From: "Marc Blanchet" <[email-masked]>
Date: Wed, 07 Dec 2016 10:34:47 -0500

>> DNS, because it would have neatly solved that exact problem.

yeap. also NAPTR was also discussed.

> may be some, but its far from the norm). To me, "neatly" is in the eye

and Andy summarizes the issues that were raised with those DNS records, 
specially in sandbox environments like within browsers.

Marc.

---

From: "John Levine" <[email-masked]>
Date: 7 Dec 2016 16:59:24 -0000

>DNS, because it would have neatly solved that exact problem.  

Even though I was as enthusiastic about SRV as you were, in this regard
it's no better than the existing JSON kludge.  If the client is working
properly, it'll find the right server.  If it isn't, it won't, and we
all know that trying to add complexity to make stuff more fool-resistant
is counterproductive.  And in fact, the current kludge is slightly
more flexible since it can have separate entries for .uk and .ac.uk
which would be a challenge for SRV.

>...  In particular, for policy reasons it's important to

I wouldn't be opposed in principle to returning an optional JSON blob
along with the 404 that gives the client some hints along those lines.
But it really is optional, and an empty response is still 100%
compliant.

R's,
John

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 7 Dec 2016 12:01:43 -0500

> may be some, but its far from the norm). To me, "neatly" is in the eye

I'm similarly unaware of any that know how to look things up in an
IANA protocol parameters registry, so I never understood how this was
an objection.  But anyway, that ship has closed the barn door.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 7 Dec 2016 12:04:19 -0500

On Wed, Dec 07, 2016 at 03:30:24PM +0000, Gould, James wrote:
> It sounds like you’re attempting to morph RDDS into the SRS.

Nope.

>RDDS is a lookup service and the SRS is an OLTP system.  A lookup
> service either has the data or it doesn’t.  Extra business logic
> associated with availability (variant blocking, relationship
> blocking, reserved domains, etc.) should be left to the appropriate
> channel which is the SRS and not RDDS.

Also nope.  The idea that the Registration Data Directory Service
doesn't have knowledge about the Registration Data such that the
repository policies are a secret from the RDDS strikes me as at least
unintuitive.  Moreover, since one RDDS service (whois) already
actually has mechanisms for communicating such information today at
least in some implementations, we have running code that we need to
conform to.

Best regards,

A
-- 
Andrew Sullivan
[email-masked]

---

From: Rubens Kuhl <[email-masked]>
Date: Wed, 7 Dec 2016 15:16:01 -0200

> Em 7 de dez de 2016, à(s) 15:04:000, Andrew Sullivan <[email-masked]> escreveu:
> 
> unintuitive.

Unintuitive as it is, that was exactly the output of the ICANN EWG on RDDS... if Registration Data is stored at a central location as was part of that proposal, RDDS and SRS will be much further apart than it is today. 

>  Moreover, since one RDDS service (whois) already

We can do that using a local WHOIS port-43 attender that uses properly done availability checks using whatever protocol we get consensus on. Note that large registrars already do that, having a local WHOIS-like server that knows who to query next and to not query availability for domains already present at TLD zone files. 

Rubens

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 7 Dec 2016 17:13:49 -0500

On Wed, Dec 07, 2016 at 03:16:01PM -0200, Rubens Kuhl wrote:
> 
> Unintuitive as it is, that was exactly the output of the ICANN EWG on RDDS.

Perhaps fortunately, the EWG's output doesn't determine ICANN policy
on this.  Because the centralised RDS that they were proposing was,
IMO, very close to the worst scaling plan I've heard on the Internet
in a very long time.
 
> We can do that using a local WHOIS port-43 attender

Really?  Are we still talking about keeping that miserable obsolete
nightmare around even longer?

Port 43 must die.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Rubens Kuhl <[email-masked]>
Date: Wed, 7 Dec 2016 20:22:04 -0200

> Em 7 de dez de 2016, à(s) 20:13:000, Andrew Sullivan <[email-masked]> escreveu:
> 

Whether EWG centralized RDS will go live is a discussion for a different forum, but the fact that such a proposal got consensus show that assuming RDDS and SRS will always be closely tied is false. It's how it is today, it might keep being that way after the current RDS WG ends its work, but the chance (or risk) of decoupling is there. 

> nightmare around even longer?

Not as an organization to organization protocol, just to support old code that is out there, like order systems that do WHOIS lookup to determine availability. What I'm saying is that we can make RDDS not workable as an availability checking mechanism without breaking those applications by providing workarounds. 

Rubens

---

From: "Francisco Obispo" <[email-masked]>
Date: Wed, 07 Dec 2016 15:51:12 -0800

Hi *,

In my opinion, in order for RDAP to be successful, those use cases will 
have to be taken into account. Regular users do use WHOIS to query for 
availability, or if a name is unavailable they want to know why, I know 
this first hand from feedback we received when we launched Uniregistry.

for instance:

```
$ whois -h whois.nic.link thisnameisavailable.link
The queried object does not exist: Domain thisnameisavailable.link is 
available for registration.

>>> Last update of WHOIS database: 2016-12-07T23:41:41.710Z <<<

[…]

```

```
$ whois -h whois.nic.link iana.link
The queried object does not exist: See below for more information
>>> registry reserved

>>> Last update of WHOIS database: 2016-12-07T23:42:27.510Z <<<
[…]
```

```
$ whois -h whois.nic.link iana.foobarbaz
The queried object does not exist: See below for more information
>>> tld not supported by this registry interface

>>> Last update of WHOIS database: 2016-12-07T23:42:43.702Z <<<
[…]
```

```
$ whois -h whois.nic.link obispo.link
Domain Name: obispo.link
Domain ID: DO_d278b6e470d7fc1d3135e5261b65dc34-UR
WHOIS Server: whois.uniregistry.net
Referral URL: http://whois.uniregistry.net
Updated Date: 2016-03-23T23:08:14.025Z
Creation Date: 2014-04-15T16:14:20.513Z
Registry Expiry Date: 2017-04-15T16:14:20.513Z
[…]
```

Those use cases need to be preserved in the new system, we must be able 
to signal different responses based on the object situation, and as a 
registry, people expect this to be authoritative and accurate.

regards,

On 7 Dec 2016, at 7:30, Gould, James wrote:

> It sounds like you’re attempting to morph RDDS into the SRS.  RDDS 
> is a lookup service and the SRS is an OLTP system.  A lookup service 
> either has the data or it doesn’t.  Extra business logic associated 
> with availability (variant blocking, relationship blocking, reserved 
> domains, etc.) should be left to the appropriate channel which is the 
> SRS and not RDDS.
>
> -- 
>
> JG
>
> James F. Gould
> Distinguished Engineer
> Verisign
> [email-masked]
>
>
>
> On 12/7/16, 10:12 AM, "regext on behalf of Andrew Sullivan" 
> <[email-masked] on behalf of [email-masked]> wrote:
>

—
Francisco Obispo
Uniregistry Inc.

---

From: "Francisco Obispo" <[email-masked]>
Date: Wed, 07 Dec 2016 15:55:07 -0800

By the way,

I had to resend this a couple times, and finally use a .COM tld to get my mail through.

Is the IETF is blocking the .LINK tld ?

```
This is the mail system at host zimbra1.uniregistry.com.

I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.

For further assistance, please send mail to postmaster.

If you do so, please include this problem report. You can
delete your own text from the attached returned message.

                   The mail system

<[email-masked]>: host mail.ietf.org[4.31.198.44] said: 550 5.7.1
    <[email-masked]>: Sender address rejected: Blocked for spam,
    please see http://www.ietf.org/secretariat.html (in reply to RCPT TO
    command)
```

```
This is the mail system at host server.obispo.link.

I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.

For further assistance, please send mail to postmaster.

If you do so, please include this problem report. You can
delete your own text from the attached returned message.

                   The mail system

<[email-masked]>: host mail.ietf.org[4.31.198.44] said: 550 5.7.1
    <[email-masked]>: Sender address rejected: Blocked for spam, please
    see http://www.ietf.org/secretariat.html (in reply to RCPT TO command)
```

On 7 Dec 2016, at 15:51, Francisco Obispo wrote:

> Hi *,
>
> In my opinion, in order for RDAP to be successful, those use cases will have to be taken into account. Regular users do use WHOIS to query for availability, or if a name is unavailable they want to know why, I know this first hand from feedback we received when we launched Uniregistry.
>
> for instance:
>
> ```
> The queried object does not exist: Domain thisnameisavailable.link is available for registration.
>
> Those use cases need to be preserved in the new system, we must be able to signal different responses based on the object situation, and as a registry, people expect this to be authoritative and accurate.
>
>
> regards,
>
>
>

—
Francisco Obispo
Uniregistry Inc.