Subject: [weirds] FW: New Version Notification for draft-sheng-weirds-icann-rws-dnrd-01.txt

From: Steve Sheng <[email-masked]>
Date: Mon, 12 Mar 2012 21:02:07 -0700

Dear IETF colleagues,

  A few of us from ICANN, ISC and CNNIC worked together and revised the
earlier domain draft. In this draft, we tried to follow the RIR query
pattern as much as possible, and have also added considerations for
querying IDNs and displaying internationalized registration data, since
this is a major concern for domain registries.

  Here is the URL of the draft:
http://tools.ietf.org/html/draft-sheng-weirds-icann-rws-dnrd-01

  We welcome feedback on this draft. The draft also identifies serveral
outstanding issues that need further work. Our question is are there more
domain registries willing to build on this and work together on this?

Kind regards, 
Steve

-----Original Message-----
From: "[email-masked]" <[email-masked]>
Date: Mon, 12 Mar 2012 15:06:50 -0700
To: Francisco Arias <[email-masked]>
Cc: Steve Sheng <[email-masked]>, "[email-masked]"
<[email-masked]>, "[email-masked]" <[email-masked]>
Subject: New Version Notification for
draft-sheng-weirds-icann-rws-dnrd-01.txt

>A new version of I-D, draft-sheng-weirds-icann-rws-dnrd-01.txt has been
>successfully submitted by Francisco Arias and posted to the IETF
>repository.
>
>Filename:	 draft-sheng-weirds-icann-rws-dnrd
>Revision:	 01
>Title:		 A RESTful Web Service for Domain Name Registration Data
>(RWS-DNRD)
>Creation date:	 2012-03-12
>WG ID:		 Individual Submission
>Number of pages: 22
>
>Abstract:
>   This document specifies a RESTful Web Service for querying Domain
>   Name Registration Data (WHOIS data).
>
>   The purpose of this document is to facilitate discussion and serve as
>   input into a standards process in this area, currently being
>   discussed on the Worthwhile Extensible Internet Registry Data Service
>   (WEIRDS) mailing list (https://www.ietf.org/mailman/listinfo/weirds).
>                  
>

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Tue, 13 Mar 2012 11:20:49 +0000

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Steve Sheng
> Sent: Tuesday, March 13, 2012 12:02 AM
> To: [email-masked]
> Subject: [weirds] FW: New Version Notification for draft-sheng-weirds-
> icann-rws-dnrd-01.txt
> 
> Dear IETF colleagues,
> 

I am willing to work on this. I am employed by Verisign, registry operator for .com, .net, and a number of smaller TLDs.

Scott

---

From: Ray Bellis <[email-masked]>
Date: Tue, 13 Mar 2012 11:30:08 +0000

On 13 Mar 2012, at 11:20, Hollenbeck, Scott wrote:

I am willing to work on this.

Likewise, employed by .uk

That said, I'd rather see a separation between the query pattern specification and the response specifications, as Andy Newton has done for the RIR objects.

Ray

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Tue, 13 Mar 2012 11:42:00 +0000

From: [email-masked] [mailto:[email-masked]] On Behalf Of Ray Bellis
Sent: Tuesday, March 13, 2012 7:30 AM
To: [email-masked]
Subject: Re: [weirds] New Version Notification for draft-sheng-weirds-icann-rws-dnrd-01.txt

Ray

I concur. I'm also willing to help with document writing.

Scott

---

From: Gavin Brown <[email-masked]>
Date: Tue, 13 Mar 2012 12:05:03 +0000

> domain registries willing to build on this and work together on this?

CentralNic supports this effort and we are willing to work on it.

Gavin.

-- 
Gavin Brown
Chief Technology Officer
CentralNic Ltd
Innovative, Reliable and Flexible Registry Services
for ccTLD, gTLD and private domain name registries
https://www.centralnic.com/

CentralNic Ltd is a company registered in England and Wales with company
number 4985780. Registered Offices: 35-39 Moorgate, London, EC2R 6AR.

---

From: Arturo Servin <[email-masked]>
Date: Tue, 13 Mar 2012 10:25:32 -0600

Nice!

	I am willing to review and comment.

> That said, I'd rather see a separation between the query pattern specification and the response specifications, as Andy Newton has done for the RIR objects.

+1

	I think the separation is healthy.

	Also, why to complicate your life with XML? 

	I think it would be easier to define the model using json and then to export the schema to XML. Doing the other way around I think is more complex.

Regards
as

---

From: Francisco Obispo <[email-masked]>
Date: Tue, 13 Mar 2012 09:31:42 -0700

Not really,

At first JSON might seem like the best option, however, XML provides much more than just data exchange:

1) Validation: it is usually one more line of code, and it allows you to ensure that the data received matches a certain set of rules, doing so in JSON requires additional code to be written to validate data structures.

2) Namespaces: clear namespace declarations, allow to have separation between the enclosure and the data contained within.

3) Presentation: XML has XSLT and Stylesheets, that enable the browser to transform, and present an end-user with a pretty version without any additional effort. JSON requires some code within the server to translate it before presentation.

JSON is not being left out at this point, we're just proposing an XML model because it just seems more suitable for the task. 

Francisco

> 	I think it would be easier to define the model using json and then to export the schema to XML. Doing the other way around I think is more complex.

Francisco Obispo 
email: [email-masked]
Phone: +1 650 423 1374 || INOC-DBA *3557* NOC
PGP KeyID = B38DB1BE

---

From: Ray Bellis <[email-masked]>
Date: Tue, 13 Mar 2012 16:43:33 +0000

> 	Also, why to complicate your life with XML? 

I'm no fan of XML, indeed quite the opposite!

However I do see the logic in re-using the XML elements defined for EPP in this case.

I would just prefer not to see the XML schema and query pattern in the same document.

kind regards,

Ray

---

From: Warren Kumari <[email-masked]>
Date: Tue, 13 Mar 2012 15:09:16 -0600

> I am willing to work on this. I am employed by Verisign, registry operator for .com, .net, and a number of smaller TLDs.

I too am willing (and eager) to work on this. While I am not a registry / registrar, I have worked for a registrar in the past, and am involved in the security community, one of the future consumers of the data….

W

> 
> Scott
> _______________________________________________
> weirds mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/weirds
> 

--
Don't be impressed with unintelligible stuff said condescendingly.
    -- Radia Perlman.

---

From: Alan Maitland <[email-masked]>
Date: Tue, 13 Mar 2012 16:47:07 -0600

On 3/12/2012 10:02 PM, Steve Sheng wrote:
> From: "[email-masked]"<[email-masked]>
> To: Francisco Arias<[email-masked]>
> Cc: Steve Sheng<[email-masked]>, "[email-masked]"
> <[email-masked]>, "[email-masked]"<[email-masked]>

I am willing to work with the group in any way I can as regards feedback 
for the draft-sheng-weirds-icann-rws-dnrd-01.  Though like another 
poster to this list I am not a member of a Registry, I can offer some 
inputs from an implementer / administrative consumer's perspective.

Alan Maitland

---

From: Wil Tan <[email-masked]>
Date: Wed, 14 Mar 2012 10:52:46 +1100

> domain registries willing to build on this and work together on this?
I'm employed by Cloud Registry. We support the efforts and are willing to
contribute towards the cause, be it document drafting, review or
prototyping and interoperability tests.

-- 
Wil Tan
CTO
Cloud Registry <http://www.cloudregistry.net>

---

From: Hugo Salgado <[email-masked]>
Date: Tue, 13 Mar 2012 21:49:16 -0300

> outstanding issues that need further work. Our question is are there more

I think it'll be better to replace the ".tld" suffix in the examples
with the RFC2606 compliant ".example", the reserved tld for
documentation.

Regards,

Hugo Salgado
NIC Chile - .CL

---

From: Steve Sheng <[email-masked]>
Date: Wed, 14 Mar 2012 10:16:05 -0700

Thank you Hugo, changes accepted.

Steve

---

From: Francisco Arias <[email-masked]>
Date: Thu, 15 Mar 2012 16:50:18 -0700

>I would just prefer not to see the XML schema and query pattern in the

I see no problem with separating the URI structure and the reply format
(be either XML, JSON, etc.).

__

Francisco.

---

From: Arturo Servin <[email-masked]>
Date: Thu, 15 Mar 2012 18:02:10 -0600

Thanks, that would be great.

Regards,
as

---

From: James Mitchell <[email-masked]>
Date: Fri, 16 Mar 2012 18:58:58 +1100

>domain registries willing to build on this and work together on this?

Thanks for putting this draft together. We (AusRegistry) are interested in
and will contribute to any effort on a replacement for the WHOIS protocol
for domain name registration data. Comments on the draft below.

Overall:

I think this draft is on the right track, however as this is a replacement
for WHOIS I believe it should address behaviour given existing use-cases.
For example this draft should explicitly state how a server should respond
when
  - an object does not exist - is it 404 or a 200 with a message stating
'object not found'?
  - a server wants a free-text message to be returned, for example 'name
is reserved' or 'name may be accepting third level registrations' - does
the server reject requests containing only the application/xml header
until the text/plain header is allowed?
  - a client has reached a server-defined query limit - is there a
specific HTTP code or is a free text message returned as above?
- a server wants a blob of text explaining the rights of the server
operator and the client to be displayed to the client - should the server
respond with multipart messages?.

If these are not covered we will have servers that provide inconsistent
behaviour. Note that there may be some overlap here with the RIR work.

3.1 Domain:
I do not see any value in the use of path segments to get 'registration',
'contact', or 'registrar' information for a domain name. This information
can be extracted from the full response and existing whois consumers have
been parsing this information from the full response for many many years.
These additions appear to be feature creep that will contribute to
increased cost of development and lower motivation for change. Otherwise
the motivation for this new behaviour should be explicitly stated.

3.5. Signaling Response Formats:
I believe it is important that this work specify exactly ONE mandatory
format for the time being. HTTP provides mechanisms to signal preferred
media type and these mechanisms can be used to support alternate formats.
Avoiding the argument about what formats are mandatory and assuming the
response is XML I suggest replacing the text with something along the
lines of below.

-- 
Servers MUST support the XML response format as described in this
document. Servers MAY also support additional formats to meet local
requirements. A server that provides a format compatible with its
plain-text responses of the current WHOIS protocol is an example of the
use of an alternate format. To promote interoperability, servers providing
formats other than plain-text are encouraged to publish their formats
through the IETF and to update this document.

As servers may support more than one format, clients SHOULD signal the
preferred response format using the HTTP Accept request header field.
Mechanisms by which a client establishes the formats supported by a
specific server are outside the scope of this draft.

The existing supported response formats and their MIME types are:
  XML - application/xml
  plain text - text/plain
--

7.3. Considerations for Indicating Language/scripts in Responses:
HTTP supports both Accept-Language and Content-Language. Is there a reason
these fields cannot be used?

Regards,

James Mitchell
AusRegistry International

---

From: Ray Bellis <[email-masked]>
Date: Fri, 16 Mar 2012 08:09:20 +0000

>  - an object does not exist - is it 404 or a 200 with a message stating

IMHO, a 404 error.

> until the text/plain header is allowed?

no comment

> specific HTTP code or is a free text message returned as above?

429 Too Many Requests (draft-nottingham-http-new-status-04)

Ray

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 16 Mar 2012 16:45:17 +0000

> Sent: Friday, March 16, 2012 1:09 AM
> To: James Mitchell
> Cc: [email-masked]
> Subject: Re: [weirds] FW: New Version Notification for draft-sheng-weirds-icann-rws-dnrd-01.txt
> 
> > For example this draft should explicitly state how a server should
> > respond when
> >  - an object does not exist - is it 404 or a 200 with a message
> > stating 'object not found'?
> 
> IMHO, a 404 error.

Why?  I'd prefer 200 with a "No such record" message, since a 404 would otherwise be indistinguishable from a service that's currently broken in some way.

-MSK

---

From: Francisco Obispo <[email-masked]>
Date: Fri, 16 Mar 2012 09:49:20 -0700

404 is 'not found' 

if you're just looking to see if an object exists or not, a 404 response will be available right in the HTTP header, no need to parse, etc.

If you issue a HEAD request, you will find this very handy.

Francisco Obispo 
email: [email-masked]

---

From: Francisco Obispo <[email-masked]>
Date: Fri, 16 Mar 2012 09:50:56 -0700

This could be addressed by adding a 'disclaimer' to the response, this is part of the XML Schema discussion, and I'll add it to my to-do list.

Thanks

Francisco Obispo 
email: [email-masked]

---

From: Gavin Brown <[email-masked]>
Date: Fri, 16 Mar 2012 17:15:38 +0000

> Why?  I'd prefer 200 with a "No such record" message, since a 404 would otherwise be indistinguishable from a service that's currently broken in some way.

A broken service should return a 500 error. Servers can be configured to
do this (Apache does it automatically for CGI scripts).

G.

---

From: Edward Lewis <[email-masked]>
Date: Fri, 16 Mar 2012 17:24:10 -0400

>>>  http://tools.ietf.org/html/draft-sheng-weirds-icann-rws-dnrd-01

I am coming late to the party and the only document I have read is 
the one referenced above, so I'll submit some comments in the form of 
questions.

1.) In some WhoIs servers, DNSSEC information is (or has been) 
included.  Would including DNSSEC information (DS, DNSKEY, etc.) be 
beneficial?

(I'm not necessarily for adding it, but in the spirit of being at 
least as "capable" as port 43...)

2. Off and on I hear discussions of what is exposed in WhoIs - for 
example - should an address be full and complete to the street 
address or just part of a postal code.  (The usual context is IP 
addresses which I know is in another document.  I'm mentioning this 
as an example.)

Question #2 is - is it desirable to be able to answer differently to 
authenticated and authorized queries than anonymous queries?

If the answer is yes, then that means a whole authentication, 
authorization, and privacy mechansim(s?) be defined.

3. Is there a desire to add "source authentication" and "data 
integrity" information to this?  E.g., have a signature that is 
perhaps verified by a registry's KEY (as opposed to DNSKEY)?  (I 
recall old work in XMLSIG happening - no idea if it completed, 
succeeded/failed, just mentioning that there was once such a WG.)

I've seen this on a slide deck at the ICANN 43 ccNSO Tech Day, but 
missed it in the draft.
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis
NeuStar                    You can leave a voice message at +1-571-434-5468

2012...time to reuse those 1984 calendars!

---

From: Ray Bellis <[email-masked]>
Date: Sat, 17 Mar 2012 13:40:25 +0000

On 16 Mar 2012, at 16:45, Murray S. Kucherawy wrote:

Because then that error message has to accommodated within the data model.

I'm fairly sure (having just looked for some examples) that using HTTP status codes is "the RESTful way".

As others have pointed out, a service broken in some other way SHOULD (MUST?) be producing something other than a 404 error.

Ray

---

From: Wil Tan <[email-masked]>
Date: Sun, 18 Mar 2012 01:25:09 +1100

On Sun, Mar 18, 2012 at 12:40 AM, Ray Bellis <[email-masked]>wrote:

> Why?  I'd prefer 200 with a "No such record" message, since a 404 would
> otherwise be indistinguishable from a service that's currently broken in
> some way.
>
>
> As others have pointed out, a service broken in some other way SHOULD
> (MUST?) be producing something other than a 404 error.
>
>
Agree that using HTTP status codes is the way to go.

However, I can appreciate Murray's concern because if a client is
configured with the wrong base URI, it could very well run into a web
server that returns 404 because the resource at that URI is indeed not
found. If there isn't another machine understandable way to distinguish
between that and a weirds "no such record" response, the client will just
happily think that there's no such object in the registry.

One way of dealing with it is to return an empty xml document <rws><result
/></rws> with the appropriate Content-Type in the 404 response.

.wil

---

From: Francisco Obispo <[email-masked]>
Date: Sat, 17 Mar 2012 07:29:58 -0700

I wouldn't work for HEAD requests, as there is no body in the response.. If this is an issue, we can perhaps consider the use of another header to signal the client about an authoritative answer.

On Mar 17, 2012, at 7:25 AM, Wil Tan wrote:

> 
> One way of dealing with it is to return an empty xml document <rws><result /></rws> with the appropriate Content-Type in the 404 response.
> 
> .wil

Francisco Obispo 
email: [email-masked]

---

From: Ray Bellis <[email-masked]>
Date: Sat, 17 Mar 2012 15:42:15 +0000

On 17 Mar 2012, at 14:25, Wil Tan wrote:

> However, I can appreciate Murray's concern because if a client is configured with the wrong base URI, it could very well run into a web server that returns 404 because the resource at that URI is indeed not found. If there isn't another machine understandable way to distinguish between that and a weirds "no such record" response, the client will just happily think that there's no such object in the registry.

Yes, that's a potential concern.

However it's not _too_ far different to the problem of sending a query for foo.com to the wrong 'whois' server:

% whois -h whois.tucows.com foo.co.uk.
 No match for FOO.CO.UK.

% whois -h whois.tucows.com asdlkasjlda.com.
 No match for ASDLKASJLDA.COM.

that is, there's no indication here that the first query was sent to the wrong server.

> One way of dealing with it is to return an empty xml document <rws><result /></rws> with the appropriate Content-Type in the 404 response.

Which is kind of what I meant about having to deal with the data model.  If you do it like that, the error response has to vary depending on the desired output format.

FWIW, I don't think the "not found" part of a 404 error is mandated.  A response of "404 <object-type> not found" could suffice to differentiate a query for a non-existent resource sent to a WEIRDS server from a query incorrectly sent to a plain web server.

I don't know if that's considered an appropriate overloading of the status header in a RESTful service, though.

Ray

---

From: Francisco Obispo <[email-masked]>
Date: Sat, 17 Mar 2012 08:50:58 -0700

Like I mentioned in my previous email, 

HEAD requests are a very useful part of the RESTful interface. You don't want to mandate or rely on anything in the body to determine whether an object exists or not.

on an HTTP HEAD request you servers don't send a body, just the Headers, which are used by the client to determine among other things, whether an object exists/has been updated/moved, etc.

regards,

Francisco

Francisco Obispo 
email: [email-masked]

---

From: Eric Brunner-Williams <[email-masked]>
Date: Sat, 17 Mar 2012 12:58:43 -0400

On 3/17/12 11:42 AM, Ray Bellis wrote:
> ...

This occurred to me when i saw the /mumble/IANA_ID/mumble/ pattern in
a proposed URI.

I've pointed this out several times in ICANN policy meetings, a
significant number of IANA registrar identifiers are held by an order
of magnitude fewer business entities.

So I suggest that where a wildcard, or a meta-identifier is useful, it
is allowed, e.g., foo/enom-any/bar would match for any of the 29
eNomNNNN entries.

This would have an IANA dependency, one that ICANN compliance can, and
IMO should address, of creating the "aliases" file so that one can
ask, in fewer than N/2 queries (on average), which nominally distinct
registrar_id (and possibly distinct whois server) is not the "wrong
server".

Eric

---

From: "John Levine" <[email-masked]>
Date: 17 Mar 2012 17:52:51 -0000

>found.

Trying to anticipate all the ways people will screw up is hopeless.
(As has often been noted, if you make things idiot proof, you will
just select for a more ingenious level of idiot.)

On the other hand, a web server does return a document with a 404
response, so I'd suggest that we encourage (SHOULD) servers to return
a document that makes it clear that the server understood the
question, but doesn't have any data to answer it.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 17 Mar 2012 18:04:27 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of John Levine
> Sent: Saturday, March 17, 2012 10:53 AM
> To: [email-masked]
> response, so I'd suggest that we encourage (SHOULD) servers to return a
> document that makes it clear that the server understood the question,
> but doesn't have any data to answer it.

Right.  I'm concerned with "404" being ambiguous; it could mean "I have no data about that domain/IP address" or it could mean "You got the URL entirely wrong."  It would be awfully nice to be able to tell the difference.

-MSK

---

From: Stephen van Egmond <[email-masked]>
Date: Sat, 17 Mar 2012 14:04:42 -0400

> question, but doesn't have any data to answer it.

There are 99 other values in the 400-499 range that can be used to indicate specific types of client request errors.

---

From: "John R. Levine" <[email-masked]>
Date: 17 Mar 2012 19:31:13 +0100

> There are 99 other values in the 400-499 range that can be used to indicate specific types of client request errors.

Indeed, but the last time I checked, it's not easy to persuade many web 
servers to generate them.  For that matter, do you have any idea what 
browsers do with, say, a 477 error?  I think I know what they're supposed 
to do, but I'd be surprised if anyone's tested it.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

---

From: Francisco Obispo <[email-masked]>
Date: Sat, 17 Mar 2012 11:36:37 -0700

On Mar 17, 2012, at 11:31 AM, John R. Levine wrote:

> Indeed, but the last time I checked, it's not easy to persuade many web servers to generate them.  For that matter, do you have any idea what browsers do with, say, a 477 error?  I think I know what they're supposed to do, but I'd be surprised if anyone's tested it.

They don't need to, the status code should be generated by the underlying application (aka: whois service). The webserver should just pass that information along.

Browsers normally don't do anything with status codes (some exceptions like redirects, do happen at the browser level). status codes will work as signal mechanism for the protocol, nothing else.

Francisco Obispo 
email: [email-masked]

---

From: Gavin Brown <[email-masked]>
Date: Mon, 19 Mar 2012 08:14:56 +0000

410 ("gone") might be suitable. It indicates a permanent error ("the object does not exist, and never will on this server"). A server can return a 404 for non-existent objects within its bailiwick, and 410 for anything outside its bailiwick.

G.

-- 
Gavin Brown
Chief Technology Officer
CentralNic Ltd
Innovative Registry Services for ccTLD and gTLD registries
https://www.centralnic.com/

---

From: Marc Blanchet <[email-masked]>
Date: Mon, 19 Mar 2012 09:24:20 -0400

might need to think about the fact that the object (domain) does not exist as is but the reason may be useful to know: an illegal idn, a prohibited domain, exists but not that length (for an address registry), etc…. can we do this really with 404?

marc.

---

From: Andy Newton <[email-masked]>
Date: Mon, 19 Mar 2012 13:45:34 +0000

> might need to think about the fact that the object (domain) does not exist as is but the reason may be useful to know: an illegal idn, a prohibited domain, exists but not that length (for an addres…

In the 2 years we have been operating our RESTful whois service this has never been an issue. Our experience is that entities in 404s, etc… are either ignored or cause confusion. BTW, I would think an illegal IDN would be a 400.

-andy

---

From: Francisco Obispo <[email-masked]>
Date: Mon, 19 Mar 2012 10:16:58 -0700

> might need to think about the fact that the object (domain) does not exist as is but the reason may be useful to know: an illegal idn, a prohibited domain, exists but not that length (for an addres…

We could, as long as the request method is 'GET', where you can return a body with the appropriate message. It wouldn't work for HEAD as described in my previous email.

Francisco

Francisco Obispo 
email: [email-masked]

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 19 Mar 2012 15:39:29 -0400

Dear colleagues,

> At first JSON might seem like the best option, however, XML provides much more than just data exchange:

If we want a rich XML-based proposal for something resembling a WHOIS
replacement, why don't we use the one already developed?

I frankly cannot see any reason whatever to do this with XML.
Everything that we intend to return in WEIRDS is key/value data.  I
say use the form designed for that pattern, mandate it, and mandate no
alternatives.  

The claim is that IRIS didn't get widely adopted because it was too
complicated.  If that explanation is right, then a simple system that
is trivially deployed by people with my level of programming skill
(i.e. very little) is the right answer.  (To be honest, having read
the WHOIS review team's report during last week's ICANN meeting, I
have a suspicion that IRIS didn't get widely adopted partly because
the policy people don't even know it exists.  But that's a
digression.)

My experience with XML suggests that it is an excellent tool to give
incompetent programmers when you want something that mostly works but
is formally incorrect.  Many people are very good at writing XML that
works but that does not validate.  That's not the same thing as
writing good XML.

The pressure on the business and policy sides of, at least, the names
end of this problem remains that the data service is an expense.
Existing running code that doesn't need changing is by definition
cheaper to maintain than new code.  We need to make that difference as
small as logically possible, or there will be no change at all.  XML
does not minimize that expense, and therefore is (in my opinion) the
wrong choice.

The argument from EPP won't work, partly because the standard
complaint about EPP is exactly that there are too many extensions with
too many hairy little incompatibilities for the name registrar
community to be happy.  If we're supposed to scale the consumers of
the protocol from "a few hundred registrars who at least know what a
domain name is" to "2.6 billion Internet users", we're going to need
something requiring less background knowledge than EPP.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Tue, 20 Mar 2012 12:35:32 +0000

> Behalf Of Andrew Sullivan
> Sent: Monday, March 19, 2012 3:39 PM
> To: [email-masked]

[snip]

> something requiring less background knowledge than EPP.

The extensibility complexity doesn't have to be exposed to Internet users. I don't think that anonymous EPP is the right answer, but if all we're talking about is reusing core EPP data structures with a RESTful query interface the problem becomes more tractable.

Scott

---

From: Andy Newton <[email-masked]>
Date: Tue, 20 Mar 2012 13:26:23 +0000

> The extensibility complexity doesn't have to be exposed to Internet users. I don't think that anonymous EPP is the right answer, but if all we're talking about is reusing core EPP data structures w…

Internet users using Whois are a much broader set than those using EPP. They will not have received an SDK from a registry, and many will be using bash and wget and so they will be exposed to the complexities. We did an analysis of uses against our REST service and found that the overwhelming majority of queries came from custom programs, the largest percentage being browser-based Javascript but also a good deal of Java, Python, ColdFusion, Perl and PHP.

-andy

---

From: Carlos Martinez-Cagnazzo <[email-masked]>
Date: Tue, 20 Mar 2012 10:42:21 -0300

Hi,

> Internet users using Whois are a much broader set than those using EPP. They will not have received an SDK from a registry, and many will be using bash and wget and so they will be exposed to the c…
+100.  The universe of WHOIS users is a lot larger than the universe of
EPP users. This is why I keep hammering the need of being able to use
non-specialized clients. The bash+wget, or python/urllib2, or jQuery
will be common use cases and will probably represent the largest share
of queries.

C.

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Tue, 20 Mar 2012 14:14:59 +0000

> From: Carlos Martinez-Cagnazzo [mailto:[email-masked]]
> Sent: Tuesday, March 20, 2012 9:42 AM
> To: Andy Newton
> Cc: Hollenbeck, Scott; [email-masked]
> >> The extensibility complexity doesn't have to be exposed to Internet
> users. I don't think that anonymous EPP is the right answer, but if all
> we're talking about is reusing core EPP data structures with a RESTful
> query interface the problem becomes more tractable.
> > Internet users using Whois are a much broader set than those using
> EPP. They will not have received an SDK from a registry, and many will
> be using bash and wget and so they will be exposed to the complexities.
> We did an analysis of uses against our REST service and found that the
> overwhelming majority of queries came from custom programs, the largest
> percentage being browser-based Javascript but also a good deal of Java,
> Python, ColdFusion, Perl and PHP.

I'm talking about XML-formatted data blobs, *not* the query/response protocol, and thus no specialized client is required. The data will need to be returned to the client in some encoded form, and if that form happens to be XML it makes *a lot* of sense to reuse existing data structures.

Scott

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 20 Mar 2012 10:35:05 -0400

> I'm talking about XML-formatted data blobs, *not* the query/response protocol, and thus no specialized client is required. The data will need to be returned to the client in some encoded form, and…

I would agree if we were building a tool to query EPP-based
registries.  But that's not the goal: some registries aren't using EPP
at all.  Moreover, the reuse of those existing data structures
presupposes that they're the same.  Last I checked, for instance, .cz
was using EPP but had even replaced the "stock" EPP mappings for
(e.g.) contacts because they were unhappy with them.  And over on the
provreg list, we just witnessed a complaint about the DS vs. DNSKEY
interface for DNSSEC records because of the usual "too many options"
complaint.  If the goal is to replace the existing WHOIS with
something that is reliably parsable and still re-use the existing
registration data structures, then the totality of heterogeneity on
the registration side will need to be reproduced on the query side.  I
think that would be a bad thing.  So, we have to come to a common
format, and if we do there's no obvious benefit of XML to anything
else.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Andy Newton <[email-masked]>
Date: Tue, 20 Mar 2012 14:49:55 +0000

>>>> The extensibility complexity doesn't have to be exposed to Internet

[..snip..]

> I'm talking about XML-formatted data blobs, *not* the query/response protocol, and thus no specialized client is required. The data will need to be returned to the client in some encoded form, and…

For all but a few Internet users, it will not be re-use.

Speaking of complexity, I think I now understand Peter Koch's statement at the last BoF about putting IRIS over HTTP. If you look at the base namespace for the DNRD draft, it does exactly what the base namespace in IRIS does. So why not re-use that? Actually, the DNRD draft uses twice the number of XML namespaces as IRIS/DREG, so it is even more complex. Not that I'm advocating this solution, just pointing out the juxtaposition.

-andy

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Tue, 20 Mar 2012 15:28:05 +0000

> From: Andy Newton [mailto:[email-masked]]
> Sent: Tuesday, March 20, 2012 10:50 AM
> To: Hollenbeck, Scott
> Cc: [email-masked]; [email-masked]
> > I'm talking about XML-formatted data blobs, *not* the query/response
> protocol, and thus no specialized client is required. The data will
> need to be returned to the client in some encoded form, and if that
> form happens to be XML it makes *a lot* of sense to reuse existing data
> structures.
> 

If we're replacing whois the user experience is going to be completely new no matter what. If the provider experience is also completely new the (re)development cost is going to be an issue.

Anyway, back to my original point: the complexity of EPP extensions is *not* something that would need to be exposed to users of this new service.

Scott

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 20 Mar 2012 11:35:38 -0400

> Anyway, back to my original point: the complexity of EPP extensions is *not* something that would need to be exposed to users of this new service.

This is actually the claim that I either thing is wrong or else that I
don't understand.  (Whenever it's a possibility, the latter is the
most likely explanation, I grant.)  Here's the problem I see:

Suppose we have two different contact mappings, for instance, or an
extension to the standard contact mapping.

If the goal is in part to provide a predictable machine-readable
response, then there are four possibilities:

    1.  The two mappings, or the standard mapping + extension, need
    all to be mapped into a single structure so that clients can be
    prepared for it.  This is not simple re-use of EPP.

    2.  There needs to me a meta-language that allows interpretation
    of the different mappings such that they can all be understood by
    the client.  This is no different than (1) except for where and
    when the remapping is done.

    3.  We throw away anything that doesn't meet the standard EPP
    mapping, in which case we're only providing a lookup service for
    EPP and not for anything else.

    4.  Anything that we don't know about is returned as a blob, in
    which case the goal is not met.

I don't see what alternatives to the above there are, and none of them
makes simple re-use of EPP mappings without additional work and the
exposure of the grotty corners of EPP extensions.  What am I missing?

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Tue, 20 Mar 2012 15:40:15 +0000

> Sent: Tuesday, March 20, 2012 10:35 AM
> To: [email-masked]

[snip]

>  So, we have to come to a common

The perceived benefit to some number of service providers is that they already have software and systems in place to process XML.

Scott

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 20 Mar 2012 17:24:29 +0100

>     EPP and not for anything else.

This seems to me the only valid alternative, assuming that those EPP
extensions are there for a reason, and that WEIRDS' goal is a unified
format.  It has to be verified that the remaining data is sufficient.

As for not exposing XML details, actual responses don't need to be
XML.  Even the spec can contain the JSON (or text) layout.  In this
case, the only advantage of reusing EPP is the ability to trivially
compute what is the data that won't get thrown away after considering
all registrars that use EPP.  ("Trivially", in this context, means
without the need of local committees purposely created to expatiate on
the underlying semantics.)

---

From: Francisco Obispo <[email-masked]>
Date: Tue, 20 Mar 2012 09:43:00 -0700

On Mar 20, 2012, at 9:24 AM, Alessandro Vesely wrote:

> Even the spec can contain the JSON (or text) layout.  In this

Unless we go back to plain ASCII text, you will need tools to parse any of the formats in question.

Unless we incorporate some sort of signaling mechanism within JSON to simulate namespaces, otherwise, expectations will not match reality when parsing through some of those answers.

I think I've made this point in other emails, but let me reiterate it..

I'm a big fan of JSON, and I use it all the time for inter-server communication. However, the reason why I feel comfortable using it, is because I control both the sender and receiver.

When any of sender or receiver are third parties, you need to provide means for checking that the data is consistent and valid. Of course you can do this in JSON, but it requires significant more developing effort than in XML.

I'm not against it, and I think that there's room for multiple output formats associated to HTTP "content-type"s. 

Best regards,

Francisco Obispo 
email: [email-masked]

---

From: Eric Brunner-Williams <[email-masked]>
Date: Tue, 20 Mar 2012 13:52:31 -0400

> The perceived benefit to some number of service providers is that they already have software and systems in place to process XML.

Scott,

Outside of ICANN registrars and registries, no service providers
could* be using live EPP connections presently.

Processing XML is not a unique property of ICANN contractual parties,
nor of those plus the few other non-contractual registries and their
corresponding registrars, some of whom are ICANN contractual parties,
just to make things more complicated.

So the claim of benefit of using XML is less than identical with
current use of (payload in XML syntax) EPP.

Moreover, accreditation with ICANN, and therefore access to contracted
(and some other) registries using EPP, is on the order of $1k per year
in fees, and service over EPP to a registry may be obtained from one
or more of the nearly 2k entities** currently holding accreditation
for a fee.

I know there are XML consumers out there, I simply have no idea why
their syntax and access requirements exist independently of those we
captured back in the original grrp-reqs document.

Eric

* OK, two parties neither of which is a "registry" or a "registrar" in
the ICANN sense could be using EPP syntax to shuffle data back and
forth, outside of any unit test harness run by an EPP implementer.

** Most of which exist simply to sell threads to the drop pool anyway
and who's owners could just as well serve other "legitimate" interests
with no change to access policy or adverse scaling consequence.

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 20 Mar 2012 19:27:57 +0100

On 20/Mar/12 17:43, Francisco Obispo wrote:
> 
> Unless we incorporate some sort of signaling mechanism within JSON
> to simulate namespaces, otherwise, expectations will not match
> reality when parsing through some of those answers.

If results can be returned in XML, which I didn't mean to exclude, I
agree a IANA registered names space is a good idea.  However, the I-D
uses specific namespaces for registrar, contacts, hosts, and similar
items.  That seems to be meant to map, at least conceptually, a
variety of EPP-like extensions.  Doesn't that defy the purpose of
automating simple, repetitive tasks?

> When any of sender or receiver are third parties, you need to
> provide means for checking that the data is consistent and valid.
> Of course you can do this in JSON, but it requires significant more
> developing effort than in XML.

For a simple routine, unaware of registry-specific idiosyncrasies, all
the consistency that has to be checked is that variables of expected
names get the expected type of value.  Failure to comply just results
in errors or exceptions.

> I'm not against it, and I think that there's room for multiple
> output formats associated to HTTP "content-type"s.

Programs that are able to cope with multiple, run-time determined
schemata are obviously not simple.  Isn't that beyond our target?

---

From: "John Levine" <[email-masked]>
Date: 21 Mar 2012 08:47:28 -0000

>When any of sender or receiver are third parties, you need to provide
>means for checking that the data is consistent and valid. Of course you
>can do this in JSON, but it requires significant more developing effort
>than in XML.

This makes no sense.  If you're debugging your server, you check that
the responses match what's in the RFC.  Whatever syntax checking your
favorite XML library might provide would only be a small part of this.

If you're trying to look stuff up in production, you decode what you
can, and ignore what you can't.  If you think that people doing WHOIS
lookups will ignore a response because its syntax doesn't quite agree
with the spec, you must not do many WHOIS lookups.

R's,
John

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 21 Mar 2012 07:26:39 -0700

On Mar 21, 2012, at 1:47 AM, John Levine wrote:

> the responses match what's in the RFC.

This is why RFCs who use XML, have XML Schema Definitions for the protocols, so that this can be done systematically.

>  Whatever syntax checking your

Syntax != Validation

> with the spec, you must not do many WHOIS lookups.

It's not a matter on whether they will ignore it or not, it's about whether they will need to do ~400 different implementations of the parser (sadly like we have today with port 43 whois), or just one with a few extensions with the new proposal.

Francisco Obispo 
email: [email-masked]

---

From: Eric Brunner-Williams <[email-masked]>
Date: Wed, 21 Mar 2012 10:56:22 -0400

On 3/21/12 10:26 AM, Francisco Obispo wrote:
> It's not a matter on whether they will ignore it or not, it's about whether they will need to do ~400 different implementations of the parser (sadly like we have today with port 43 whois),

can the number cited ("~400") be associated with some published study
on actual implementations?

-e

---

From: Arturo Servin <[email-masked]>
Date: Wed, 21 Mar 2012 12:11:32 -0300

> This is why RFCs who use XML, have XML Schema Definitions for the protocols, so that this can be done systematically.

	Which in turn raises the barrier of entry to many developers, that´s something that we want to avoid using json in the numbers world.

Regards,
as

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 21 Mar 2012 08:18:01 -0700

Not really/necessary,

Port 43 format is free text, and with >240 ccTLDs, gTLDs, RIRs, and others, it is easy to see how different those implementations would be...

Perhaps ICANN has some details on a whois study.

Francisco Obispo 
email: [email-masked]

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 21 Mar 2012 08:30:14 -0700

The only difference with XML and JSON validation, is that for the latter, you have to do the checks by hand… 

One way to provide JSON is by doing an XLST conversion from XML->JSON,… this would allow us to get the full benefit of XML, and still provide a JSON format (validated), for those who want to avoid XML.

But the beauty of RESTful, is the fact that we can have multiple response formats coexisting.

Francisco Obispo 
email: [email-masked]

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 21 Mar 2012 11:39:31 -0400

On Wed, Mar 21, 2012 at 08:30:14AM -0700, Francisco Obispo wrote:
> 
> One way to provide JSON is by doing an XLST conversion from
> XML->JSON,… this would allow us to get the full benefit of XML, and

s/benefit/pain and I'll grant this.  But in any case,

> But the beauty of RESTful, is the fact that we can have multiple response formats coexisting.

This all seems to me to be a technical dispute about trade offs that
one can have in a working group, rather than a question that needs to
be settled as part of a charter discussion.  A key goal -- simplicity
of implementation -- is, I think, something that should go in the
charter.  How to achieve that goal (maybe it's ditching both of these
in favour of YAML.  Or reverting to SGML.  Or just IRIS over http.  Or
a dozen other bad ideas one might think of) is part of the work, I think.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Arturo Servin <[email-masked]>
Date: Wed, 21 Mar 2012 12:43:28 -0300

Or just not do validation.

	I am sceptic on how much validation is people going to do in reality.

Regards,
.as

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 21 Mar 2012 09:41:52 -0700

Agreed,

let's get the charter going first ;-)

On Mar 21, 2012, at 8:39 AM, Andrew Sullivan wrote:

> a dozen other bad ideas one might think of) is part of the work, I think

Francisco Obispo 
email: [email-masked]

---

From: Carlos Martinez-Cagnazzo <[email-masked]>
Date: Wed, 21 Mar 2012 13:54:41 -0300

+1, let's get a charter and a WG,  then we can discuss data formats over
large quantities of beer

:-)

C.

---

From: "John R. Levine" <[email-masked]>
Date: 21 Mar 2012 19:01:44 +0100

>> favorite XML library might provide would only be a small part of this.

Sigh.  Whatever validation your favorite XML library might provide would 
be only a small part of this.

> It's not a matter on whether they will ignore it or not, it's about 
> whether they will need to do ~400 different implementations of the 
> parser

The goal is certainly to get every server to implement the spec correctly, 
and I agree that it would be a good ideas to provide tools to help them 
test them.  But when some random ISP in Asia has bugs in the data they 
return, what exactly do you expect clients to do?

Regards,

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 21 Mar 2012 11:04:51 -0700

On Mar 21, 2012, at 11:01 AM, John R. Levine wrote:

> The goal is certainly to get every server to implement the spec correctly, and I agree that it would be a good ideas to provide tools to help them test them.  

+1

> But when some random ISP in Asia has bugs in the data they return, what exactly do you expect clients to do?

Prevent them from using the information as valid.. treating it as garbage, or sending it for a human for review.

Francisco Obispo 
email: [email-masked]

---

From: "John R. Levine" <[email-masked]>
Date: 21 Mar 2012 19:16:21 +0100

> The only difference with XML and JSON validation, is that for the latter, you have to do the checks by hand…

If that's the only way you know to validate results, OK, but my plan is to 
write test harnesses to debug the results, like I do for all my other 
client server stuff, regardless of what the encoding is.

> response formats coexisting.

It's hard to imagine that the IESG would ever accept something with 
multiple response formats, so there's no need to argue about this.

R's,
John

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 21 Mar 2012 11:19:31 -0700

On Mar 21, 2012, at 11:16 AM, John R. Levine wrote:

> If that's the only way you know to validate results, OK, but my plan is to write test harnesses to debug the results, like I do for all my other client server stuff, regardless of what the encoding is.

Good for you that follow good software development practices. Unfortunately that's not the case for everybody, so good luck parsing the answer from someone who has not done the same.

Francisco Obispo 
email: [email-masked]

---

From: "John R. Levine" <[email-masked]>
Date: 21 Mar 2012 19:26:16 +0100

> Good for you that follow good software development practices. Unfortunately that's not the case for everybody, so good luck parsing the answer from someone who has not done the same.

So just to make it clear, you think that the people writing the servers 
will be too incompetent to debug their code?

Regards,

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 21 Mar 2012 11:31:20 -0700

On Mar 21, 2012, at 11:26 AM, John R. Levine wrote:
>> ...
> 
> So just to make it clear, you think that the people writing the servers will be too incompetent to debug their code?
> 

I don't think that. But I can't assure you that the response will be generated without errors either… can you?

Francisco Obispo 
email: [email-masked]

---

From: "John R. Levine" <[email-masked]>
Date: 21 Mar 2012 19:36:40 +0100

> I don't think that.

Hmmn.  You sure seemed to say it two messages ago.

> But I can't assure you that the response will be 
> generated without errors either… can you?

Nope.  And neither of us can promise that the response will be correct no 
matter how much XML you stuff into it.

Regards,

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 21 Mar 2012 11:39:34 -0700

> Hmmn.  You sure seemed to say it two messages ago.

s/people/some people/ 

and I will say "yes"

>> But I can't assure you that the response will be generated without errors either… can you?
> 
> Nope.  And neither of us can promise that the response will be correct no matter how much XML you stuff into it.
> 

Exactly, and it would be nice to be able to tell when without too much effort.

Francisco Obispo 
email: [email-masked]

---

From: Andy Newton <[email-masked]>
Date: Wed, 21 Mar 2012 19:01:57 +0000

On Mar 21, 2012, at 12:54 PM, Carlos Martinez-Cagnazzo wrote:

> then we can discuss data formats over
> large quantities of beer

Are you buying? Because, my friend, you haven't enough money if this thread is any indication of how that discussion will go.

-andy

---

From: "John R. Levine" <[email-masked]>
Date: 21 Mar 2012 20:21:38 +0100

> Prevent them from using the information as valid.. treating it as 
> garbage, or sending it for a human for review.

That's pretty funny.

Regards,

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 21 Mar 2012 20:34:33 +0000

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Andy Newton
> Sent: Wednesday, March 21, 2012 12:02 PM
> To: <[email-masked]>
> Cc: <[email-masked]>
> Are you buying? Because, my friend, you haven't enough money if this
> thread is any indication of how that discussion will go.

We're certainly earning our name.  :-)

---

From: "Smith, Bill" <[email-masked]>
Date: Wed, 21 Mar 2012 22:42:12 +0000

+1

---

From: Eric Brunner-Williams <[email-masked]>
Date: Wed, 21 Mar 2012 17:13:41 -0400

> Not really/necessary,

really? see below.

> Port 43 format is free text, and with >240 ccTLDs, gTLDs, RIRs, and others, it is easy to see how different those implementations would be...

i hope most everyone subscribed to this mailing list is remotely aware
of ...

(a) that 954 et seq is delightfully specified as
c: [::printable::]\r\n
s: .*\r\n <socket close>,

and

(b) there is more than one operator of a :43 server.

however, i'm not aware of the indistinguishability of necessity or
utility for human, or subhuman, parsing of data available at every :43
server.

if you're suggesting that the necessity and utility of a means to
parse the :43 return from the .tf (terres australes et antarctiques
françaises) server is indistinguishable from the necessity and utility
of a means to parse the :43 return from the .com server, i find it
hard to agree.

to generalize slightly, there exists correlations, both of trademark
necessity and utility, and law enforcement necessity and utility, and
some whois data sources and formats, and there also exist correlations
in some whois data sources and formats and registry backend operators.

so i suggest that the 80% rule is more likely to be useful than some
100% rule, and 80% of the use and variation amounts to a much smaller
number of servers than ~400.

> Perhaps ICANN has some details on a whois study.

at least one person contributing to this charter discussion has spent
years on icann whois working groups and is aware of their various
studies. the data may skew towards some correlations i mentioned
earlier, and overlook one or more of the ~400 sources of data.

-e

---

From: Dave Piscitello <[email-masked]>
Date: Thu, 22 Mar 2012 07:34:58 -0700

When we did a study on prevalence of privacy protected registrations in 2010, we sampled ~50,000 domains that Spamhaus had placed on the DBL. We collected Whois for these and needed unique parsers to extract data from responses of 96 registrars, another 15 for gTLD registries, and 10 for ccTLDs in our sample.

I can't claim for certainty that every ccTLD Whois response pattern is unique but if ~400 is "high" it's not inordinately high. Also, the number of parsers is not a constant. Certain registrars change their responses as an anti-harvesting measure. So over time, if you were writing parsers, you may have accumulated several for certain registrars.

---

From: "Michael Young" <[email-masked]>
Date: Thu, 22 Mar 2012 12:19:37 -0400

More than a few extensions I'd say, but still better than the mess today.

>It's not a matter on whether they will ignore it or not, it's about whether
they will need to do ~400 different implementations of the parser (sadly
like we have today with port 43 whois), or just one with a few extensions
with the new proposal.

>Francisco Obispo

On another note, to John's point about repeating history - (which I do
sympathize with).

Winston Churchill said:

"Success consists of going from failure to failure without loss of
enthusiasm."

So maybe this will fail, maybe it will succeed, but right now we aren't
doing anything and that's probably the least desirable of all options.

Michael Young