Subject: [weirds] Features

From: Jay Daley <[email-masked]>
Date: Wed, 15 Jun 2011 14:12:50 +1200

Here is a list of features that have been variously discussed for replacements/enhancements to WHOIS.

1.  Authentication.  The requirement for an end user to provide credentials before accessing the system.

2.  Access control.  The ability to provide different levels of access/restrictions to different end users.

3.  Rate limiting.  A specialised case of 2 above which relies on the ability to communicate access control data to the end user

4.  Internationalisation.  Full support for internationalised registration data and internationalised domain names.

5.  Standardised, machine readable queries, responses and error messages.

I for one would prefer a substantive discussion on which of these is needed in a new protocol (and then which is mandatory/optional) before that new protocol is designed.

cheers
Jay

-- 
Jay Daley
Chief Executive
.nz Registry Services (New Zealand Domain Name Registry Limited)
desk: +64 4 931 6977
mobile: +64 21 678840

---

From: "Michele Neylon :: Blacknight" <[email-masked]>
Date: Wed, 15 Jun 2011 02:15:40 +0000

Jay

Surely all of them are needed?

Regards

Michele

Mr. Michele Neylon
Blacknight
http://Blacknight.tel

Via iPhone so excuse typos and brevity

On 15 Jun 2011, at 03:13, "Jay Daley" <[email-masked]> wrote:

> _______________________________________________
> weirds mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/weirds

---

From: Jay Daley <[email-masked]>
Date: Wed, 15 Jun 2011 14:21:12 +1200

Hi Michele

If 1, 2 or 3 (and possibly 5) are mandated then it is no longer WHOIS as we know it.  Fine if that is the intent, but we need to sort out that intent first.

best
Jay

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 15 Jun 2011 10:48:44 -0400

Hi Jay,

> I for one would prefer a substantive discussion on which of these is needed in a new protocol (and then which is mandatory/optional) before that new protocol is designed.

No time like the present!  I think your list is a good one.  The
natural place for this, I suspect, is the requirements document Murray
put together:

http://tools.ietf.org/html/draft-kucherawy-weirds-requirements-00

Do you think any of the items on your list are in fact required or not
required? 

Thanks,

A

-- 
Andrew Sullivan
[email-masked]

---

From: "Owen Smigelski" <[email-masked]>
Date: Wed, 15 Jun 2011 09:42:38 -0700

I agree with Michele that all of the list are needed, with a particular
emphasis on 4.  IDNs requiring WHOIS data in ASCII in part defeats the
purpose of having IDNs (requiring additional work for registrants who
lack access or knowledge of ASCII equivalent), and has resulted in a lot
of confusing WHOIS data (transliterated, translated, and other versions
of the WHOIS data).  
__________________________
Owen Smigelski
The Sunrider Corporation dba Sunrider International
Senior Counsel
1625 Abalone Avenue
Torrance, California 90501
Tel: 310-222-9110
Fax: 310-222-9275
Email: [email-masked]

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Michele Neylon :: Blacknight
Sent: Tuesday, June 14, 2011 19:16
To: Jay Daley
Cc: [email-masked]
Subject: Re: [weirds] Features

Jay

Regards

Michele

Mr. Michele Neylon
Blacknight
http://Blacknight.tel

On 15 Jun 2011, at 03:13, "Jay Daley" <[email-masked]> wrote:

> 2.  Access control.  The ability to provide different levels of
access/restrictions to different end users.
> 
> 3.  Rate limiting.  A specialised case of 2 above which relies on the
ability to communicate access control data to the end user
> 
> 4.  Internationalisation.  Full support for internationalised
registration data and internationalised domain names.
> I for one would prefer a substantive discussion on which of these is
needed in a new protocol (and then which is mandatory/optional) before
that new protocol is designed.

---

From: Jay Daley <[email-masked]>
Date: Thu, 16 Jun 2011 11:36:36 +1200

Hi Andrew

> http://tools.ietf.org/html/draft-kucherawy-weirds-requirements-00

I struggle with a document that gets down to the UDP level before we have agreed what the technology needs to do.

> Do you think any of the items on your list are in fact required or not

We should differentiate between technology and services.  My preference is for one technology that can be used to provide multiple services - different servers running on different ports with different feature combinations.  Care is needed in the service definition though as that might lead us to the conclusion that one technology cannot provide all these services.

The services are:

A.  Public lookup of the "public register" details for a registry object for computers to parse  (aka a replacement/fix for WHOIS)

Users might include:
- everyone and anyone through proxies that displays the data
- people we don't like / trust

We undoubtedly need to fix WHOIS but with 20/20 hindsight we can see that the major stumbling block for IRIS was that it tried to "fix" too much and so presented too much of a change for insufficient perceived benefit.  In that context I would limit the features needed for this service to:
4.  Internationalisation.  
5.  Standardised messages (or guaranteed machine-readable at least).

I would also add some technical features:
- short transactions
- short lived connections
- support for out-of-channel rate limiting (on the basis that adding in-channel rate limiting is a step too far for adopters)
- primarily aimed for computer-to-computer communications than human-to-computer (if you see what I mean!)

B.  Public lookup of the "public register" details for a registry object for someone to read  (aka a replacement/fix for WHOIS)

I include this as a separate service so that we can, if agreed, decide this is not a service that we need to support, because of course this was very much the original WHOIS service.  Some good arguments against this service are:
- It can be better provided by proxies to A above
- A command line tool that does a direct TCP connection is no longer a common way for end users to interact with services.

Users might include:
- everyone and anyone
- people we don't like / trust

The only feature needed here is 
4.  Internationalisation.  

The technical features:
- short transactions
- short lived connections
- support for out-of-channel rate limiting (on the basis that adding in-channel rate limiting is a step too far for adopters)
- primarily aimed for human-to-computer interactions

C.  Controlled access to a public register with enhanced search facilities.

Users might include:
- registrars/resellers/LIRs
- lawyers (the "intellectual property" horde)

That should have these four:
1.  Authentication.
3.  Rate limiting. 
4.  Internationalisation.
5.  Standardised messages.

though it is debatable whether 3 is needed

The technical features of this are different:
- long transactions
- long lived connections
- aimed more towards human-to-computer communications

D.  Controlled access to a public register for availability checks

Users might include
- registrars/resellers

That has the same four:
1.  Authentication.
3.  Rate limiting.  (definitely needed in this case) 
4.  Internationalisation.
5.  Standardised messages.

The technical features of this are different though:
- short transactions
- long live connections (so that they don't need to set it up again)
- low latency response
- solely for computer-to-computer communications

E.  Controlled access to public register and private register

User might include
- law enforcement
- security companies

This only needs a different four:
1.  Authentication.
2.  Access control. 
4.  Internationalisation.
5.  Standardised, machine readable queries, responses and error messages.

Again, it is debatable if 3 is needed.

The technical features are different again:
- long transactions
- short lived connections
- primarily computer-to-computer communications as these users have sophisticated systems this must integrate with
- needs stronger authentication security and greater logging/audit capabilities than either of above (this may include non-repudiation of lookup requests, signed responses, etc)

So, we clearly need new technology that does C and E.  
I used to think that A could be achieved by fixing WHOIS but I've done some tests and I'm willing to concede that is highly unlikely so the technology has to be able to do A as well.  
I don't see any need for it to support B.  
Some good arguments could be made that DCHK already does D adequately.

cheers
Jay

---

From: "Smith, Bill" <[email-masked]>
Date: Wed, 15 Jun 2011 17:41:58 -0600

While authentication and access control may have been discussed elsewhere, could we have an enumeration of the reasons behind these "requirements". Further, how can these requirements be met along with a requirement for "timely, unrestricted and public access to accurate and complete WHOIS information" as stated in the ICANN (one of the "users" of WHOIS) Affirmation of Commitments.

Other comments inline:

> 1.  Authentication.  The requirement for an end user to provide credentials before accessing the system.

What would these credentials look like? Would they be unique for each WHOIS service? Is there a single directory? How are users provisioned? Deprovisioned? Is provisioning based on individual identity or "entity" identity? What standards are used to grant access? How are appeals handled?

> 2.  Access control.  The ability to provide different levels of access/restrictions to different end users.

Who determines level of access? Are access/restriction levels determined by individual or entity? (see above).

---

From: Jay Daley <[email-masked]>
Date: Thu, 16 Jun 2011 11:44:33 +1200

Hi Bill

I think our posts have crossed each other and the answers you want are in my last post about the services that WEIRDS may support.

cheers
Jay

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 15 Jun 2011 16:45:55 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Jay Daley
> Sent: Wednesday, June 15, 2011 4:37 PM
> To: Andrew Sullivan
> Cc: [email-masked]
> I struggle with a document that gets down to the UDP level before we
> have agreed what the technology needs to do.

It's just a starting point for conversation.  We could certainly split into one document (or section) for high-level requirements and one document (or section) for a much more technology-heavy discussion of those requirements.

---

From: "Smith, Bill" <[email-masked]>
Date: Wed, 15 Jun 2011 17:56:24 -0600

Jay,

I agree with your incremental approach. However, I'm quite concerned with the leap to authentication and access control. Having lived in the identity world for a decade, I'm all too familiar with how easy it is to say authentication and authorization yet how difficult they are to implement in practice, especially at scale.

I urge caution in adding these as "requirements" without considering the practical implications both for users and service providers.

Bill

> From: Jay Daley [mailto:[email-masked]]
> Sent: Wednesday, June 15, 2011 4:45 PM
> To: Smith, Bill
> Cc: [email-masked]
> I think our posts have crossed each other and the answers you want are
> in my last post about the services that WEIRDS may support.
> 
> cheers
> Jay
> 
> > While authentication and access control may have been discussed
> elsewhere, could we have an enumeration of the reasons behind these
> "requirements". Further, how can these requirements be met along with a
> requirement for "timely, unrestricted and public access to accurate and
> complete WHOIS information" as stated in the ICANN (one of the "users"
> of WHOIS) Affirmation of Commitments.
> >
> > Other comments inline:
> >
> > What would these credentials look like? Would they be unique for each
> WHOIS service? Is there a single directory? How are users provisioned?
> Deprovisioned? Is provisioning based on individual identity or "entity"
> identity? What standards are used to grant access? How are appeals
> handled?
> >
> >>
> > Who determines level of access? Are access/restriction levels
> determined by individual or entity? (see above).
> >
> >>
> >> 3.  Rate limiting.  A specialised case of 2 above which relies on
> the ability to communicate access control data to the end user
> >>

---

From: Jay Daley <[email-masked]>
Date: Thu, 16 Jun 2011 12:10:15 +1200

> I urge caution in adding these as "requirements" without considering the practical implications both for users and service providers.

I had hoped I was clear that these belong in a very different service from WHOIS.    

My concern is exactly the same as yours.  There is the very real danger that people will attempt to add them to WHOIS (or its replacement) because WHOIS is "just over there" so surely it must be the easiest way to go, ignoring the fact that that is just not how WHOIS is used by the vast majority of users.  

Our best way to head that off is accurately split out the characteristics of the different services and design appropriate technology for each, as I've attempted to do.

Some registries have avoided the "throw it all into WHOIS trap" by developing non-WHOIS services that have these features but they are not standard between registry making the job of interacting with them hard for implementors. Some examples of these non-WHOIS services are:

- a proprietary enhanced search service that allows authenticated lawyers etc to say search by registrant name.
- a high-speed name availability check

So there is clearly a case for some sort of service(s) with a standard protocol that support authentication and access control, just not WHOIS or a direct replacement for WHOIS.  

cheers
Jay

---

From: Ray Bellis <[email-masked]>
Date: Thu, 16 Jun 2011 09:35:59 +0000

On 16 Jun 2011, at 01:10, Jay Daley wrote:

Some registries have avoided the "throw it all into WHOIS trap" by developing non-WHOIS services that have these features but they are not standard between registry making the job of interacting with them hard for implementors.

I'm pretty sure Jay is alluding to .uk here, so for reference the services he's referring to are:

Some examples of these non-WHOIS services are:

our Public Register Search Service:

  http://www.nominet.org.uk/other/prss/

<http://www.nominet.org.uk/other/prss/>which is a web-based front end onto a Solr-based index of our register.

and the Domain Availablity Checker:

  http://www.nominet.org.uk/other/dac/

which gives very fast responses over a long-lived TCP connection.  This takes a domain name on a line as input, and returns a single line of output.

Ray

---

From: Keith Gaughan <[email-masked]>
Date: Thu, 16 Jun 2011 12:35:31 +0100

On 15/06/11 03:21, Jay Daley wrote:

> If 1, 2 or 3 (and possibly 5) are mandated then it is no longer WHOIS as
> we know it.  Fine if that is the intent, but we need to sort out that
> intent first.

I agree with you on 1, 2, and 3 - frankly, they're outside the remit of
WHOIS, and 2 and 3 in particular is solely a matter of policy for the
entity running the WHOIS server.

On the other hand, I disagree on 5 - 4 and 5 are intertwined closely and
are surely at the core of what we're discussing here. I'd use the example
of GANDI here, whose WHOIS server produces results as YAML, which is both
human- and machine-readable. From my point of view as somebody whose job
revolves around developing and maintaining the domain management system
for a registrar, machine-readable responses in particular are extremely
important.

K.

-- 
Keith Gaughan, Software Developer
PGP/GPG key ID: 3E896381
Blacknight Internet Solutions Ltd. <http://blacknight.com/>
12A Barrowside Business Park, Carlow, Ireland
Registered in Ireland, Company No.: 370845

---

From: Jay Daley <[email-masked]>
Date: Fri, 17 Jun 2011 07:13:10 +1200

Hi Keith

> for a registrar, machine-readable responses in particular are extremely

I'm from the ccTLD world where each TLD sets its own WHOIS syntax and output as it sees fit.   Some make this as easy to parse as possible, we in .nz for example send KV pairs, but others like .uk send pure text.  My hesitancy on 5 is because I think we need to explicitly discuss this feature with a view to ending up at a consensus view - my own aligns with your view above that any output must be machine readable.

cheers
Jay

---

From: "John Levine" <[email-masked]>
Date: 16 Jun 2011 19:55:04 -0000

>entity running the WHOIS server.

The point of a standard wouldn't be to require or even encourage
authentication, access control, or rate limiting.  The point is to
define a common way to do it, so that if WHOIS servers want clients to
authenticate or provide various levels of access, they all do it the
same way so clients can use the same software to access all of them.

On that basis, I think they're perfectly reasonable things to include.

R's,
John

---

From: Jay Daley <[email-masked]>
Date: Fri, 17 Jun 2011 08:58:47 +1200

> On that basis, I think they're perfectly reasonable things to include.

My point is that if a WHOIS server *chose* to implement any of those features then it is no longer delivering a WHOIS service but  very different service.

Any chance this discussion could move to the five service definitions that I listed, otherwise we run the danger of disagreeing without proper context to the discussion?

cheers
Jay

---

From: "John R. Levine" <[email-masked]>
Date: 16 Jun 2011 17:07:44 -0400

> My point is that if a WHOIS server *chose* to implement any of those 
> features then it is no longer delivering a WHOIS service but  very 
> different service.

Then most of the WHOIS servers run by ICANN registrars are not delivering 
a WHOIS service.  If this were ever a battle, it was lost a long time ago.

I would also prefer a WHOIS that gave people all the data all the time. 
But it ain't gonna happen.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

---

From: Jay Daley <[email-masked]>
Date: Fri, 17 Jun 2011 09:15:14 +1200

On 17/06/2011, at 9:07 AM, John R. Levine wrote:

> Then most of the WHOIS servers run by ICANN registrars are not delivering a WHOIS service.  If this were ever a battle, it was lost a long time ago.

They are not doing authentication / access control so I guess you refer to the rate limiting that everybody running a WHOIS server does?  

So long as that is out-of-channel then there is plausible justification that this is intended to protect the systems at a macro level.  As soon as it becomes in-channel then that introduces a market in access limits.

> I would also prefer a WHOIS that gave people all the data all the time. But it ain't gonna happen.

I wouldn't - far too many bad people out there for whom this data is gold.

Anyway - please can we switch to talking about the five service definitions I provided?

Jay

---

From: "John R. Levine" <[email-masked]>
Date: 16 Jun 2011 17:28:11 -0400

> They are not doing authentication / access control so I guess you refer to the rate limiting that everybody running a WHOIS server does?

They most certainly are.  Some people get a lot more rate limited than 
others.

R's,
John

---

From: Keith Gaughan <[email-masked]>
Date: Fri, 17 Jun 2011 02:37:36 +0100

On 16/06/11 22:28, John R. Levine wrote:
>> They are not doing authentication / access control so I guess you refer to
>> the rate limiting that everybody running a WHOIS server does?
> 
> They most certainly are.  Some people get a lot more rate limited than others.

The critical difference is that it's not part of the protocol, it's a matter
of server policy.

---

From: "Michael Young" <[email-masked]>
Date: Sat, 18 Jun 2011 00:56:30 -0400

Hi Jay,

Trying to get back to your original topics,

So 1-3 I think do not need reinvention - plenty of good options to pick from
already in existence.  Now I can see a health debate over those choices, but
lets work on the hard stuff first.

4 and 5 don't really exist for RDDS (Whois) and would be of great benefit to
the community if we can provide them - this is real protocol work that needs
to get done.

I do think we need to consider the requirements list (in its entirety) from
the GNSO whois requirements report - the whole whois saga can be found here
at  http://gnso.icann.org/issues/whois/ .

Note  I am cautious about getting dragged into an overly complex solution
that will never get completed, but this list of requirements needs a little
discussion and we can use it to build a roadmap - meaning we bring the most
important items to the top and address them first. 

 For those requirements we don't want to try and solve now, we do need to
make sure we leave room in any protocol work we do - for someone else to
address those requirements if they so desire.  I see that as an exercise of
enabling others with the tools to solve their own unique use cases.

I've seen a lot of arguing back over forth over what whois should be, what
it should do, how it should be built, etc etc for 10 years now.  I think, we
should focus on ENABLING instead of debating, but doing so in a progressive
prioritized manner - just like you do when you build software in the real
world :-)  If we argue over anything, lets make it which requirement we
address first.

Oh and JSON is definitely of interest, but XML is going to be easier to deal
with in the short term (more mature) and might make addressing some of the
IDN issues quicker (again more prior work to draw on)

My two cents.

-MY

Best Regards,

Michael Young
M:+1-647-289-1220

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Jay Daley
Sent: June-14-11 10:13 PM
To: [email-masked]
Subject: [weirds] Features

Here is a list of features that have been variously discussed for
replacements/enhancements to WHOIS.

1.  Authentication.  The requirement for an end user to provide credentials
before accessing the system.

4.  Internationalisation.  Full support for internationalised registration
data and internationalised domain names.

I for one would prefer a substantive discussion on which of these is needed
in a new protocol (and then which is mandatory/optional) before that new
protocol is designed.

cheers
Jay

---

From: "John Levine" <[email-masked]>
Date: 18 Jun 2011 15:24:44 -0000

>5.  Standardised, machine readable queries, responses and error messages.

There's a standard WHOIS query and response for GTLD thick registries
that is pretty consistently implemented.

For I18N, what would be necessary beyond decreeing that text fields are UTF-8?

I'm not saying that either of these are idea, but they are existing practice
or close to it.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 18 Jun 2011 15:29:08 -0000

>I'm not saying that either of these are idea, but they are existing practice

ahem, ... either of these are ideal, ...

R's,
John

---

From: "Michael Young" <[email-masked]>
Date: Sun, 19 Jun 2011 11:10:20 -0400

Sorry did you mean the ICANN GTLD registry agreement specification?  Not
sure that's wide enough for all the use cases, there's no base schema or
extension mechanism for said schema if you want to add a field type.  I
think we could certainly start with it though.

UTF-8 ok, beyond that, for starters how about adherence to RFCs 5891, 5892,
5893 and 5894?  Or do we want to assume inheritance of compliance for the
IDN string via registry enforcement? That makes me nervous though. 

Best Regards,

Michael Young
M:+1-647-289-1220

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
John Levine
Sent: June-18-11 11:29 AM
To: [email-masked]
Subject: Re: [weirds] Features

In article <[email-masked]> you write:
>>4 and 5 don't really exist for RDDS (Whois) and would be of great 
>>benefit to the community if we can provide them - this is real 
>>protocol work that needs to get done.
>
>I'm not saying that either of these are idea, but they are existing 
>practice or close to it.

R's,
John

---

From: "John R. Levine" <[email-masked]>
Date: 19 Jun 2011 11:18:28 -0400

> think we could certainly start with it though.

It has the admirable advantage of existing and being widely implemented. 
It's just text lines in a tag:value format, so the extension process is 
obvious, if not who would manage the registry of tags.  I note that at 
least one ccTLD, .US, uses the same format in its WHOIS.

> 5893 and 5894?

Since most of the UTF-8 in a WHOIS entry would not be domain names, I 
think that would be an unusually poor idea.

R's,
John

---

From: "Michael Young" <[email-masked]>
Date: Sun, 19 Jun 2011 11:24:27 -0400

-----Original Message-----
From: John R. Levine [mailto:[email-masked]] 
Sent: June-19-11 11:18 AM
To: Michael Young
Cc: [email-masked]
Subject: RE: [weirds] Features

> Sorry did you mean the ICANN GTLD registry agreement specification?  
> Not sure that's wide enough for all the use cases, there's no base 
> schema or extension mechanism for said schema if you want to add a 
> field type.  I think we could certainly start with it though.

It has the admirable advantage of existing and being widely implemented. 
It's just text lines in a tag:value format, so the extension process is
obvious, if not who would manage the registry of tags.  I note that at least
one ccTLD, .US, uses the same format in its WHOIS.

Sure but it still needs to be written down to be a standard, even if it is
obvious

> UTF-8 ok, beyond that, for starters how about adherence to RFCs 5891, 

Since most of the UTF-8 in a WHOIS entry would not be domain names, I think
that would be an unusually poor idea.

These RFCs refer to the domain string not to general output.  I wouldn't
suggest we try and make the rest of the output conform to these :-) 

R's,
John

---

From: "John R. Levine" <[email-masked]>
Date: 19 Jun 2011 11:47:06 -0400

> Sure but it still needs to be written down to be a standard, even if it is

Hmmn. What leads you to believe that it's not written down? See Appendix 5 
of any of the recent ICANN registry agreements.

>> UTF-8 ok, beyond that, for starters how about adherence to RFCs 5891,
>> 5892, 5893 and 5894?
>

Any registered IDN domain name used as a lookup key is already an A-label. 
The RFCs soon to be published by the EAI group will strongly recommend 
that domains in e-mail addresses NOT be represented as A-labels.

In any event, this is all premature optimization.  My point was just that 
there is an existing widely used resonably well specified text WHOIS 
format. If people think the best approach is to standardize existing 
practice, it's the obvious place to start.

If, on the other hand, they want to invent an all new singing dancing XML 
JSON CORBA (add buzzwords to taste) system, I'd encourage people to start 
by stating as clearly as possible what problem they're trying to solve.

Regards,

---

From: "Michael Young" <[email-masked]>
Date: Sun, 19 Jun 2011 21:22:13 -0400

Lol, John you seriously have debate energy! 

We actually really do agree here, reuse, reinforce, and make use of whatever
already exists,.....

As for the email label, interesting, Joesph Yee is here at the ICANN meeting
with me, I will ask him to join this list - since he's really a consumate
expert there - his input could be helpful to the discussion.

-Michael 

-----Original Message-----
From: John R. Levine [mailto:[email-masked]] 
Sent: June-19-11 11:47 AM
To: Michael Young
Cc: [email-masked]
Subject: RE: [weirds] Features

> Sure but it still needs to be written down to be a standard, even if 
> it is obvious

> These RFCs refer to the domain string not to general output.  I 
> wouldn't suggest we try and make the rest of the output conform to 
> these :-)

Any registered IDN domain name used as a lookup key is already an A-label. 
The RFCs soon to be published by the EAI group will strongly recommend that
domains in e-mail addresses NOT be represented as A-labels.

In any event, this is all premature optimization.  My point was just that
there is an existing widely used resonably well specified text WHOIS format.
If people think the best approach is to standardize existing practice, it's
the obvious place to start.

If, on the other hand, they want to invent an all new singing dancing XML
JSON CORBA (add buzzwords to taste) system, I'd encourage people to start by
stating as clearly as possible what problem they're trying to solve.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for
Dummies", Please consider the environment before reading this e-mail.
http://jl.ly

---

From: Jay Daley <[email-masked]>
Date: Mon, 20 Jun 2011 13:53:10 +1200

Hi John

> the obvious place to start.

In my view it isn't, for the following reasons:

1.  It only very partially meets my criterion 5 (Standardised, machine readable queries, responses and error messages)

- Response is only standardised across a small subset of gTLD registrations.  It is not standardised for
	- Registrar WHOIS for thin registries
	- ccTLDs

- Only the basic WHOIS query is standardised but not those that include parameters where each registry does its own thing

- Error messages are not standardised at all

2.  It only very partially meets by criterion 4 (Internationalisation.  Full support for internationalised registration data and internationalised domain names.)

- There is no mechanism available for a client to signal that it wants UTF-8 encoding rather than 7-bit.  Attempting to add this through say a WHOIS parameter would mean changing almost every WHOIS server first as they don't recognise parameters.

- There is no standard mechanism for a server to signal that its response is UTF-8.  This could be added by a BOM at the beginning of the output but then every client would need to be fixed to accept that gracefully.

- Without UTF-8 (or another 8 bit) there is no support for internationalised registration data, except for Japanese which has a 7-bit encoding available (ISO-2022-JP).

- The limited standardisation above is only for English labels.  Serious i18n means that every part of the WHOIS message can be internationalised.

> stating as clearly as possible what problem they're trying to solve.

I have, pretty clearly, in this message:

http://www.ietf.org/mail-archive/web/weirds/current/msg00022.html

and I've prompted you to comment on this three times now.

cheers
Jay

---

From: "John R. Levine" <[email-masked]>
Date: 19 Jun 2011 22:16:45 -0400

> and I've prompted you to comment on this three times now.

Oh, that.  It still has a bad case of premature overoptimization.  We need 
to figure out what problem we're trying to solve before going into that 
level of detail.

Domain WHOIS?  IP range WHOIS?  Some or all of the other random junk you 
can tease out of various WHOIS servers?  Does it give the same results to 
everyone?  Can clients present credentials and get more info?  What sort 
of query rate is it supposed to handle?  Does it have to handle very high 
query rates, or would be be OK to use an alternate distribution approach 
for high volume clients, like DNSBLs do? Does it need more detailed errors 
than "I'm not going to tell you that" or "I don't know"?

The existing Appendix 5 format handles domain WHOIS reasonably well.  I 
don't find any of your objections to it compelling, e.g., thin registries 
were a mistake, and at least one ccTLD uses the existing format.  Having 
written too many WHOIs parsers, I can report from experience that the 
appendix 5 format is easy for computer clients to parse.

As far as looking up registry objects, that seems like a reasonable idea 
if there is any agreement about what the vocabulary of registry objects 
is.  At this point, I don't see much.

I agree that I18n is useful, but I don't see why it requires anything more 
than putting UTF-8 in the results.  If it needs to be backward compatible 
with some prior non-UTF-8 system, that's more complicated than a query 
flag to say no UTF-8.

Regards,

---

From: Jay Daley <[email-masked]>
Date: Mon, 20 Jun 2011 14:50:50 +1200

On 20/06/2011, at 2:16 PM, John R. Levine wrote:

> Oh, that.  It still has a bad case of premature overoptimization.  We need to figure out what problem we're trying to solve before going into that level of detail.

I'm not sure how you think a service definition can be premature - that is step 1 in any analysis process.  

Step 2 is then deciding if existing technology is adequate or can be amended to work - which is the step where your arguments sit.

> Domain WHOIS?  IP range WHOIS?  Some or all of the other random junk you can tease out of various WHOIS servers?  Does it give the same results to everyone?  Can clients present credentials and get more info?  What sort of query rate is it supposed to handle?  Does it have to handle very high query rates, or would be be OK to use an alternate distribution approach for high volume clients, like DNSBLs do? Does it need more detailed errors than "I'm not going to tell you that" or "I don't know"?

I'm also not sure what that list is.  If it is a list of questions still to be answered then I don't agree - the service definitions I provided do that fine.  The fact that you are jumbling all these things together when I have cleanly separated them out makes me wonder if you understood my service definitions?  Perhaps I did not explain them enough?

> The existing Appendix 5 format handles domain WHOIS reasonably well.  I don't find any of your objections to it compelling, e.g., thin registries were a mistake, and at least one ccTLD uses the existing format.  Having written too many WHOIs parsers, I can report from experience that the appendix 5 format is easy for computer clients to parse.

So now we are back onto step 2 - can any of the service definitions I've documented by provided by current WHOIS technology?  To answer your points:

1.  You can't dismiss the problem of multifarious registrar WHOIS servers for thin registries in one line.  They exist and are not standardised at all.

2.  Roughly the other 246 ccTLDs don't use that syntax and many of them can't because they have a different data model.

Therefore the current minimal standardisation (if it even deserves that label) cannot be cited as evidence of why we should stick with WHOIS.

> As far as looking up registry objects, that seems like a reasonable idea if there is any agreement about what the vocabulary of registry objects is.  At this point, I don't see much.

I have no idea what this means.

> I agree that I18n is useful, but I don't see why it requires anything more than putting UTF-8 in the results.  If it needs to be backward compatible with some prior non-UTF-8 system, that's more complicated than a query flag to say no UTF-8.

To be clear, you are saying that WHOIS can be comfortably internationalised provided it is accepted that it will be incompatible with many existing WHOIS clients and provided that almost every WHOIS server can be rewritten to support this?

Altogether you have not provided any compelling reason as to why we should stick with WHOIS.

Nor have you addressed the key point I made in my service definitions, which is that multiple related services exist in the wild and a new single technology could underpin all of those thereby fixing several problems at once. A compelling reason to move to a new technology.

cheers
Jay

---

From: "John R. Levine" <[email-masked]>
Date: 19 Jun 2011 23:42:40 -0400

> 1.  You can't dismiss the problem of multifarious registrar WHOIS servers for thin registries in one line.  They exist and are not standardised at all.

ICANN has said they're going away.  They're dismissed.

> 2.  Roughly the other 246 ccTLDs don't use that syntax and many of them 
> can't because they have a different data model.

Then we don't have to worry about them, since we certainly can't support 
246 incompatible and undocumented data models.

I don't understand what point(s) you're trying to make, so this is my last 
response on this thread.

R's,
John

---

From: "Michele Neylon :: Blacknight" <[email-masked]>
Date: Mon, 20 Jun 2011 03:49:41 +0000

> ICANN has said they're going away.  They're dismissed.

When ? Where?

> Then we don't have to worry about them, since we certainly can't support 246 incompatible and undocumented data models.
> 
> I don't understand what point(s) you're trying to make, so this is my last response on this thread.
> 
> R's,
> John

Mr Michele Neylon
Blacknight Solutions
Hosting & Colocation, Brand Protection
ICANN Accredited Registrar
http://www.blacknight.com/
http://invadeeurope.eu
http://blog.blacknight.com/
http://blacknight.mobi/
http://mneylon.tel
Intl. +353 (0) 59  9183072
US: 213-233-1612 
UK: 0844 484 9361
Locall: 1850 929 929
Direct Dial: +353 (0)59 9183090
Twitter: http://twitter.com/mneylon
-------------------------------
Blacknight Internet Solutions Ltd, Unit 12A,Barrowside Business Park,Sleaty
Road,Graiguecullen,Carlow,Ireland  Company No.: 370845

---

From: Keith Gaughan <[email-masked]>
Date: Mon, 20 Jun 2011 12:23:29 +0100

> 1.  It only very partially meets my criterion 5 (Standardised, machine readable queries, responses and error messages)

That can mostly be solved using pragmata.

For instance, it would be useful to allow commentary in WHOIS output, so
let's say we used '#' as the comment leader. Something like '##' could be
used as a leader for pragmata, and there could be an initial pragma mandated
to specify a schema and version for the contents. Thus:

##WHOIS domain 1.0
Domain Name:                                 EXAMPLE.COM
Domain Registration Date:                    Wed Jun 21 13:58:02 GMT 2006
Domain Expiration Date:                      Mon Jun 20 23:59:59 GMT 2011
Domain Last Updated Date:                    Sun Jun 19 04:18:53 GMT 2011
...and so on...

(Personally, I'm not a massive fan of ctime()-style dates, but there you go.)

> 	- Registrar WHOIS for thin registries

An issue for ICANN, really, though we can recommend that they require that
new registrars use it for their WHOIS servers for thin registries and that
existing registrars migrate over to it. and that ccTLDs, where providing a
full WHOIS service, use it.

> - Error messages are not standardised at all

Error messages can use a separate schema. Thus, if an error occurs, you'd get
something like this:

##WHOIS error 1.0
Error Code: 2000
Description: Cat walking on keyboard.

That's a start, anyway.

> - There is no standard mechanism for a server to signal that its response is UTF-8.  This could be added by a BOM at the beginning of the output but then every client would need to be fixed to acce…

We can mandate that the standard format is itself in UTF-8. Clients only need
to look at the first line to tell it apart for any other format. I've no
problem with including a BOM at the start too. A dumb WHOIS client isn't going
to care much about it anyway, and a smart one is going to be parsing the
response anyway, so this isn't a big deal.

> - The limited standardisation above is only for English labels.  Serious i18n means that every part of the WHOIS message can be internationalised.

Nope, the labels can be treated as opaque strings that just *happen* to be in
English. A smart WHOIS client can localise the text.

A

K.