Subject: [weirds] Client handling of known attributes/names

From: James Mitchell <[email-masked]>
Date: Wed, 6 Mar 2013 01:02:15 +1100

All,

This email introduces a concern that is subtly different to my earlier email regarding client handling of unknown attributes/names. Instead, this email focuses on the approach taken thus far of decoupling data elements from their description. Again, similar to my previous email, I would currently (depending on behaviour of WERIDS clients) hesitate to recommend deployment of a WEIRDS service for the .au ccTLD.

My existing WHOIS service allows me to describe data as I see fit. This is of great value, especially when fields take on a slightly different meaning when compared between registries. Consider the extract of a whois response for ausregistry.com.au below.

Registrant:                      AUSREGISTRY PTY LTD
Registrant ID:                   OTHER 092 242 209
…
Registrant Contact ID:           Z131819709674630
Registrant Contact Name:         <withheld from this email>
Registrant Contact Email:        Visit whois.ausregistry.com.au for Web based WhoIs

Those with (EPP) registry knowledge may recognise the last three fields as attributes of the contact object (RFC 5733). What works in this WHOIS response is that the server can describe the information being returned; information belonging to the contact object is clearly identified as the "registrant contact", additional (non-standard) information is identified as the server sees fit. auDA's published WHOIS policy clarifies the description of the information displayed above, stating that the Registrant is the "Legal name of the registrant entity (eg. company name)" whereas the Registrant Contact Name is the "Name of a contact person for the registrant".

Compare this to a whois response for a .info  or .biz domain, where the domain's registrant contact's name (Registrant Contact Name above) is identified as the "Registrant Name". My concern is that a WEIRDS client will describe the registrant contact for a .au domain as the "(legal) registrant", thus mislabelling the information.

I feel that one of the goals, to improve consumption from automated systems, is being delivered at the expense of a human consumer. Decoupling data from its meta-information is appropriate when both producers and consumers have common (exact) understanding of the information exchanged. Unfortunately I don't think this is the case with domain name registries (information will be mapped differently to the response objects).

Instead, servers could provide information to clients as it should be displayed to consumers, with meta information allowing parsers to identify the information they care about. An example is below:
{
[
{ key:"Last Modified", value:"2011-10-14Txx:xx:xxZ", meta:"lastChangedDate"},
{ key:"Registrant", value:"AUSREGISTRY PTY LTD"},
{ key:"Registrant Contact Email", value:"[email-masked]", meta:"registrant-email"}
]
}

An alternative would be for clients to query field names from the server (using HTTP caching, and header information for internationalization). An example is as follows:
GET /description/domain
Response:
{
name:"Domain Name",
lastChangedDate:"Last Modified",
…
}

Comments?

Regards,
James

---

From: Simon Perreault <[email-masked]>
Date: Tue, 05 Mar 2013 15:10:50 +0100

James,

Replying to the core idea of your last two mails...

We must absolutely keep presentation data separate from actual data.

If this was XML, I'd say "keep the actual data in XML, put the 
presentation information in an XSLT style sheet".

What's the equivalent in JSON?
Can we use that?

Simon

---

From: Andy Newton <[email-masked]>
Date: Tue, 5 Mar 2013 20:00:32 +0000

>We must absolutely keep presentation data separate from actual data.

+1

>What's the equivalent in JSON?

It's called code. :)

-andy

---

From: Chris Wright <[email-masked]>
Date: Wed, 6 Mar 2013 09:54:00 +1100

Andy,

Your response is not very helpful. We have a legitimate issue that will prevent us from deploying WEIRDS as it stands, I think we have been clear on it for a while (we tried to raise it way back at the start and even voiced the concern at the session in Vancouver). We cannot rely on clients (being external parties who write the code) knowing that the registrant contact is not the registrant in our TLD. Code is extremely different to XSLT, if we were using XML, my server could return the XML and the XSLT, unless you are suggesting we return snippets of java, C (and every other language we want to support) code as part of our WEIRDS responses code is NOT an acceptable answer.

I understand the goal of separating presentation from data, and honestly I don't see including "data labels" as 'presentation' it says nothing of the layout or anything else (and can be ignored by the clients).

Since everyone seems against it, can someone who is clearly smarter than us, please explain how we are to address our concern, to date everyone seems to say no to our suggestions, but doesn't offer viable alternatives.

Thanks

Chris

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Andy Newton
Sent: Wednesday, 6 March 2013 7:01 AM
To: Simon Perreault; [email-masked]
Subject: Re: [weirds] Client handling of known attributes/names

+1

It's called code. :)

-andy

_______________________________________________
weirds mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/weirds

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 5 Mar 2013 18:12:55 -0500

> I understand the goal of separating presentation from data, and honestly I don't see including "data labels" as 'presentation' it says nothing of the layout or anything else (and can be ignored by…

This is a poorly-thought-out suggestion, so feel free to throw rocks.
What if we created classes of "unknown" types that would allow
free-form use?  This would solve the problem that John Levine quite
correctly mentioned, where every kind of unknown field with encoded
data and so on would get dumped on clients.

The idea here is to give these unknown fields types, like (say)
free-form text.  Then a client could handle the different unknown
types differently.  A client could display only types it knows, or
fields of a given type, or whatever.

Does that seem like it might help?

A

-- 
Andrew Sullivan
[email-masked]

---

From: "John Levine" <[email-masked]>
Date: 6 Mar 2013 04:17:59 -0000

>free-form use?

I think it's time for the names registries to get to work and define a
common vocabulary adequate to handle all of the important bits of
WHOIS name queries.  This might require that some of the registries
do some adaptation to fit their data to the vocabularies.

This is the place where all previous son-of-WHOIS attempts failed, and
when the names crowd muscled in here they swore it wouldn't happen
again.  Time, as we say in the U.S., to fish or cut bait.

If it only ends up being adequate for gTLD WHOIS and some of the
ccTLDs will not or cannot use it, it's still better than nothing.

R's,
John

---

From: Byron Ellacott <[email-masked]>
Date: Wed, 6 Mar 2013 05:43:28 +0000

Hi James,

If I understand your example correctly, you have two pieces of data about the party responsible for registration, namely a descriptive label and an ID.  You also have several pieces of data about a human contact for that party.  Does it seem like I've understood your example?

If so, would this abridged JSON structure reflect it accurately:

{
}

From: James Mitchell <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, 6 March 2013 12:02 AM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: [weirds] Client handling of known attributes/names

All,

Instead, servers could provide information to clients as it should be displayed to consumers, with meta information allowing parsers to identify the information they care about. An example is below:
{
[
{ key:"Last Modified", value:"2011-10-14Txx:xx:xxZ", meta:"lastChangedDate"},
{ key:"Registrant", value:"AUSREGISTRY PTY LTD"},
{ key:"Registrant Contact Email", value:"[email-masked]<mailto:[email-masked]>", meta:"registrant-email"}
]
}

Comments?

Regards,
James

---

From: Byron Ellacott <[email-masked]>
Date: Wed, 6 Mar 2013 05:49:40 +0000

Ugh.  Don't accidentally hit the send button half way through compose.  THIS structure:

{
"registrationBy" : "AUSREGISTRY PTY LTD",
"entities" : [
{
"handle": "Z131819709674630",
"entityNames": [ "<withheld from this email, too>" ],
"emails": [ "Visit whois.ausregistry.com.au for Web based WhoIs" ],
"roles" : [ "registrant contact" ]
}
]
}

As in, the registrant contact is a contact of role "registrant contact", and the registrationBy contains information about the registrant party.  This does not include registrant ID: the registrationBy field may need some work to support the difference between the registrant name (?) and ID.

  Byron

From: Byron Ellacott <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, 6 March 2013 3:43 PM
To: James Mitchell <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [weirds] Client handling of known attributes/names

Hi James,

{
}

All,

Comments?

Regards,
James

---

From: Andy Newton <[email-masked]>
Date: Wed, 6 Mar 2013 11:36:59 +0000

Chris,

While I may have used an emoticon, that was a serious comment. The
analogous web rendering technology to XSLT for JSON is code, specifically
Javascript.

That being said, I'm not convinced that your issue lies in the
presentation layer or with known vs. unknown tags. As Byron pointed out,
your issue appears to be with the data model. If I have it right, your use
case is not that much different than a proxy and/or private registration
allowed by many domain registries. And perhaps we need to better address
those concepts.

Byron's suggestion of a "legal registrant" role appears to be the right
direction, though I'd generalize it to "proxy registrant". We may also
want to add a suggested status of "privacy protected".

-andy

On 3/5/13 5:54 PM, "Chris Wright" <[email-masked]> wrote:

>Andy,
>
>Your response is not very helpful. We have a legitimate issue that will
>prevent us from deploying WEIRDS as it stands, I think we have been clear
>on it for a while (we tried to raise it way back at the start and even
>voiced the concern at the session in Vancouver). We cannot rely on
>clients (being external parties who write the code) knowing that the
>registrant contact is not the registrant in our TLD. Code is extremely
>different to XSLT, if we were using XML, my server could return the XML
>and the XSLT, unless you are suggesting we return snippets of java, C
>(and every other language we want to support) code as part of our WEIRDS
>responses code is NOT an acceptable answer.
>
>I understand the goal of separating presentation from data, and honestly
>I don't see including "data labels" as 'presentation' it says nothing of
>the layout or anything else (and can be ignored by the clients).
>
>Since everyone seems against it, can someone who is clearly smarter than
>us, please explain how we are to address our concern, to date everyone
>seems to say no to our suggestions, but doesn't offer viable alternatives.
>
>Thanks
>
>Chris
>
>

---

From: Chris Wright <[email-masked]>
Date: Wed, 6 Mar 2013 23:59:10 +1100

Thanks Andy,

I think you are right, our problems are with the data model...

Let me side step and discuss EPP for a moment..

EPP describes a protocol, that has inherently built into it a data model that its designers believed a domain name registry would use (or at least be able to 'map to'), i.e. domain, host and contacts and the various properties and relationships of/between those objects.

That data model is effectively described in a mixture of the RFCs and XML Schemas

There are many examples of Registries who, for reasons that are kind of not important to the discussion, needed to add to the data model described in EPP, and add customer fields, or could indeed add other objects. The mechanism for doing this is very well defined in EPP, there is even an RFC which describes how to extend EPP to support these changes to the data model (and other things like new command or object mappings etc).

We are searching for the similar mechanism in WEIRDS. I am certainly not expecting the working group to design mappings that suit my data model, and that would be silly as there are other registries with other models. What I am looking for is explanations on how to 'extend' WEIRDS the way we can EPP. So we need to write a draft that explains our extension and gives meaning to the new field names that we introduce, no problem with that, but it seems extensibility isn't built in (or not as explicit as it is in EPP perhaps). 

So Id like to see a mechanism where the base 'dictionary' is defined, and then simply a 'how to' for how a 'good' implementer should describe their additional elements they add to the dictionary. I would suggest to speed things up, the base dictionary for domain names simply me based off the EPP fields.

Then we get to those that implement clients... a base client will understand the base data model defined in the WEIRDS RFCs - that's a given, then each client can add the 'extensions' that they which to support (by getting access to the documentation that follows the format etc described in the 'how to extend' part of the WEIRDS specification). So that's great :)

So then we get to the behaviour when a client encounters fields it doesn't understand - currently we say ignore them and move on.. in EPP we actually require users to tell us which extensions they can understand when they login.. so we don't have these issues.. what we do have here with WEIRDS though is a single client looking to connect to many different server implementations, all of which may have their own 'extensions'. We are looking for a way to be able to programmatically tell the client about the extensions, so that we don't have to rely on the offline process. Obviously that is probably difficult, so the compromise we are suggesting is that if you include with every data field, aside from its field-code and its value, a human readable description of the field then.. it means a client can choose to:

1. implement the extensions it wants to understand as it sees fit thus semantically understand each field and do with it as it pleases, based on its understanding
2. ignore fields it doesn't understand 'or' if in the case where It is simply querying the data to display it to a user, display it to a user using the human readable description (what WhoIs does now) - this means a single simple implementation can work (for the purpose of human uses looking up whois information) with all servers regardless of the extensions to the base data model it defines

Of course this doesn't help people wanting to programmatically use the data, as they need to understand the semantic meaning of the field and would have to be 'custom' programed anyway... so not an issue

Clients that don't care about the human readable description of the field can simply ignore that part of the response, and if we are worried about overhead, put a flag in the request to specify if you want the human readable description returned or not and default it to no..

As an aside, servers, could be given the option of honouring the request accept language header and returning the human readable description of the field in the language of the client if they want to..

This solves our issue, of not requiring every WERIDS client that needs to query our WEIRDS server having to read and implement our extension document, unless of course they want to machine parse the response to do something 'useful' with it, at which point they would have to write code anyway..

The alternative, as I have said above, is we come up with a way that the protocol allows servers to communicate 'extensions' to the data model to the client, however I'm not really sure that is possible as you couldn't send 'semantic' meaning in the context of the objects registry. So really all the extension would communicate is 'field-name', 'field human readable descriptor' anyway.

This seems to make the most sense to me, and is the most 'compatible' solution, and frankly I don't understand the arguments against doing it - it certainly isn't presentation.

Thanks

Chris

-----Original Message-----
From: Andy Newton [mailto:[email-masked]] 
Sent: Wednesday, 6 March 2013 10:37 PM
To: Chris Wright; Simon Perreault; [email-masked]
Subject: Re: [weirds] Client handling of known attributes/names

Chris,

While I may have used an emoticon, that was a serious comment. The analogous web rendering technology to XSLT for JSON is code, specifically Javascript.

That being said, I'm not convinced that your issue lies in the presentation layer or with known vs. unknown tags. As Byron pointed out, your issue appears to be with the data model. If I have it right, your use case is not that much different than a proxy and/or private registration allowed by many domain registries. And perhaps we need to better address those concepts.

Byron's suggestion of a "legal registrant" role appears to be the right direction, though I'd generalize it to "proxy registrant". We may also want to add a suggested status of "privacy protected".

-andy

On 3/5/13 5:54 PM, "Chris Wright" <[email-masked]> wrote:

>prevent us from deploying WEIRDS as it stands, I think we have been 
>clear on it for a while (we tried to raise it way back at the start and 
>even voiced the concern at the session in Vancouver). We cannot rely on 
>(and every other language we want to support) code as part of our 
>WEIRDS responses code is NOT an acceptable answer.
>
>I understand the goal of separating presentation from data, and 
>honestly I don't see including "data labels" as 'presentation' it says 
>nothing of the layout or anything else (and can be ignored by the clients).
>
>Since everyone seems against it, can someone who is clearly smarter 
>than us, please explain how we are to address our concern, to date 
>everyone seems to say no to our suggestions, but doesn't offer viable alternatives.
>
>Thanks
>
>Chris
>
>

---

From: James Mitchell <[email-masked]>
Date: Thu, 7 Mar 2013 00:20:24 +1100

Andy, Byron,

I'm feeling better about where this is going. So in our case we would use
the registrant contact as the proxy (as this contact is acting on behalf
of the legal registrant to authorise transfers etc) and our "Registrant"
field would represent the name of the (legal) registrant. I'd have to
check with our policy guys how they would feel about the change in
terminology but it should work.

Confusion of the registrant name however was only one concern. Secondary
to this is our current WHOIS output identifies how the the legal
registrant claims they are eligible for the domain name. This information
may include (depending on the registration):
{
- Registrant: Legal name of the registrant entity (eg. company name)
- Registrant ID: ID number associated with the registrant entity, if any
(eg. ACN for company)
- Eligibility Type: Registrant's eligibility type (eg. "Company")
- Eligibility Name: Name used by the registrant to establish eligibility,
if different from their own legal name (eg. registered business name or
trademark)
- Eligibility ID: ID number associated with the name used by the
registrant to establish eligibility (eg. BN for registered business name,
TM number for registered trademark)
}

However, eligibility information will probably vary from registry to
registry, from business numbers to trademarks, or professional
accreditation identifiers, or to no eligibility requirements. Obviously
the consumer of this information is no "ordinary" person; they'd need the
contextual information that identifies the (fictitious) number 123456789
as an Australian Business Number, and they'd have to know how that tied in
with the registry's eligibility policy. However this information SHOULD be
available to the layperson, and I'm concerned they won't get it through
generic WEIRDS clients, at least those without "hooks" for every registry.
I still think there is room for clients to obtain this descriptive
information from servers, not for the core fields, in the absence of
specific client side knowledge.

I'm still happy with
GET /description/<prefix>
Response:
{
	<prefix>_field:{display_name:"Eligibility Name", description:"Name used
by the registrant to establish eligibility, if different from their own
legal name (eg. registered business name or trademark)"}
}

Do people still have reservations for this approach?

Thanks,
James

---

From: Andy Newton <[email-masked]>
Date: Wed, 6 Mar 2013 13:37:29 +0000

Chris,

I strenuously disagree with your assertion that extensibility is not built
into RDAP. There is a mechanism for including custom JSON values, adding
custom URL paths, registering customizations in an IANA registry, and
noting the extensions by id in the root object of the JSON response.

What you are asking for is not extensibility of the protocol but
extensibility of the clients. While admirable, I think such an endeavor is
beyond the scope of this working group. Client understanding is not just
about a display descriptions, especially for rich data. As many have
already said, clients need to be updated.

-andy

On 3/6/13 7:59 AM, "Chris Wright" <[email-masked]> wrote:

>EPP describes a protocol, that has inherently built into it a data model
>that its designers believed a domain name registry would use (or at least
>be able to 'map to'), i.e. domain, host and contacts and the various
>properties and relationships of/between those objects.
>
>There are many examples of Registries who, for reasons that are kind of
>not important to the discussion, needed to add to the data model
>described in EPP, and add customer fields, or could indeed add other
>objects. The mechanism for doing this is very well defined in EPP, there
>is even an RFC which describes how to extend EPP to support these changes
>to the data model (and other things like new command or object mappings
>etc).
>
>We are searching for the similar mechanism in WEIRDS. I am certainly not
>expecting the working group to design mappings that suit my data model,
>and that would be silly as there are other registries with other models.
>What I am looking for is explanations on how to 'extend' WEIRDS the way
>we can EPP. So we need to write a draft that explains our extension and
>gives meaning to the new field names that we introduce, no problem with
>that, but it seems extensibility isn't built in (or not as explicit as it
>is in EPP perhaps).
>
>So Id like to see a mechanism where the base 'dictionary' is defined, and
>then simply a 'how to' for how a 'good' implementer should describe their
>additional elements they add to the dictionary. I would suggest to speed
>things up, the base dictionary for domain names simply me based off the
>EPP fields.
>
>Then we get to those that implement clients... a base client will
>understand the base data model defined in the WEIRDS RFCs - that's a
>given, then each client can add the 'extensions' that they which to
>support (by getting access to the documentation that follows the format
>etc described in the 'how to extend' part of the WEIRDS specification).
>So that's great :)
>
>So then we get to the behaviour when a client encounters fields it
>doesn't understand - currently we say ignore them and move on.. in EPP we
>actually require users to tell us which extensions they can understand
>when they login.. so we don't have these issues.. what we do have here
>with WEIRDS though is a single client looking to connect to many
>different server implementations, all of which may have their own
>'extensions'. We are looking for a way to be able to programmatically
>tell the client about the extensions, so that we don't have to rely on
>the offline process. Obviously that is probably difficult, so the
>compromise we are suggesting is that if you include with every data
>field, aside from its field-code and its value, a human readable
>description of the field then.. it means a client can choose to:
>
>1. implement the extensions it wants to understand as it sees fit thus
>semantically understand each field and do with it as it pleases, based on
>its understanding
>2. ignore fields it doesn't understand 'or' if in the case where It is
>simply querying the data to display it to a user, display it to a user
>using the human readable description (what WhoIs does now) - this means a
>single simple implementation can work (for the purpose of human uses
>looking up whois information) with all servers regardless of the
>extensions to the base data model it defines
>
>Of course this doesn't help people wanting to programmatically use the
>data, as they need to understand the semantic meaning of the field and
>would have to be 'custom' programed anyway... so not an issue
>
>Clients that don't care about the human readable description of the field
>can simply ignore that part of the response, and if we are worried about
>overhead, put a flag in the request to specify if you want the human
>readable description returned or not and default it to no..
>
>As an aside, servers, could be given the option of honouring the request
>accept language header and returning the human readable description of
>the field in the language of the client if they want to..
>
>This solves our issue, of not requiring every WERIDS client that needs to
>query our WEIRDS server having to read and implement our extension
>document, unless of course they want to machine parse the response to do
>something 'useful' with it, at which point they would have to write code
>anyway..
>
>The alternative, as I have said above, is we come up with a way that the
>protocol allows servers to communicate 'extensions' to the data model to
>the client, however I'm not really sure that is possible as you couldn't
>send 'semantic' meaning in the context of the objects registry. So really
>all the extension would communicate is 'field-name', 'field human
>readable descriptor' anyway.
>
>This seems to make the most sense to me, and is the most 'compatible'
>solution, and frankly I don't understand the arguments against doing it -
>it certainly isn't presentation.
>
>Thanks
>
>Chris

---

From: Simon Perreault <[email-masked]>
Date: Wed, 06 Mar 2013 14:42:43 +0100

James, Chris,

I feel like we're talking past each other. So I'll only ask questions 
this time.

> legal name (eg. registered business name or trademark)"}

Suppose you define "au_foobar". It gets popular and other registries 
implement it. You and other registries get together and write an RFC to 
standardize "foobar" (no prefix). The RFC gets published, everyone's happy.

Now, I suppose "foobar" would no longer have "display_name" or 
"description" since it's a standard field. Right?

In that case, what is a WEIRDS client that has not yet implemented your 
RFC supposed to do when it receives that unknown "foobar" field?

Is your proposal related to only <prefix>_something fields, or does it 
concern everything except the fields defined in the base spec?

Simon

---

From: Chris Wright <[email-masked]>
Date: Thu, 7 Mar 2013 00:51:55 +1100

Sorry Andy, I retract that it isn't built in... bad use of wording on my part for the first part of the email...

> Client understanding is not just about a display descriptions, especially for rich data. As many have already said, clients need to be updated.

I agree that clients that are trying to 'do' something programmatically with the data, of course have to be updated, but clients that simply 'display' the data shouldn't need to be updated, they don't need to be updated with WhoIs right now.. it's a backwards step from that perspective... 

It feels like we are not designing a replacement for WhoIs but rather something that achieves an entirely different purpose, something that is to be used alongside it.. maybe that was the goal from the start and I have misguided myself.. 

Thanks

Chris

-----Original Message-----
From: Andy Newton [mailto:[email-masked]] 
Sent: Thursday, 7 March 2013 12:37 AM
To: Chris Wright; Simon Perreault; [email-masked]
Subject: Re: [weirds] Client handling of known attributes/names

Chris,

I strenuously disagree with your assertion that extensibility is not built into RDAP. There is a mechanism for including custom JSON values, adding custom URL paths, registering customizations in an IANA registry, and noting the extensions by id in the root object of the JSON response.

What you are asking for is not extensibility of the protocol but extensibility of the clients. While admirable, I think such an endeavor is beyond the scope of this working group. Client understanding is not just about a display descriptions, especially for rich data. As many have already said, clients need to be updated.

-andy

On 3/6/13 7:59 AM, "Chris Wright" <[email-masked]> wrote:

>EPP describes a protocol, that has inherently built into it a data 
>model that its designers believed a domain name registry would use (or 
>at least be able to 'map to'), i.e. domain, host and contacts and the 
>various properties and relationships of/between those objects.
>
>objects. The mechanism for doing this is very well defined in EPP, 
>there is even an RFC which describes how to extend EPP to support these 
>changes to the data model (and other things like new command or object 
>mappings etc).
>
>We are searching for the similar mechanism in WEIRDS. I am certainly 
>not expecting the working group to design mappings that suit my data 
>model, and that would be silly as there are other registries with other models.
>that, but it seems extensibility isn't built in (or not as explicit as 
>it is in EPP perhaps).
>
>So Id like to see a mechanism where the base 'dictionary' is defined, 
>and then simply a 'how to' for how a 'good' implementer should describe 
>their additional elements they add to the dictionary. I would suggest 
>to speed things up, the base dictionary for domain names simply me 
>based off the EPP fields.
>
>doesn't understand - currently we say ignore them and move on.. in EPP 
>we actually require users to tell us which extensions they can 
>understand when they login.. so we don't have these issues.. what we do 
>have here with WEIRDS though is a single client looking to connect to 
>many different server implementations, all of which may have their own 
>semantically understand each field and do with it as it pleases, based 
>on its understanding 2. ignore fields it doesn't understand 'or' if in 
>the case where It is simply querying the data to display it to a user, 
>display it to a user using the human readable description (what WhoIs 
>does now) - this means a single simple implementation can work (for the 
>purpose of human uses looking up whois information) with all servers 
>regardless of the extensions to the base data model it defines
>
>Clients that don't care about the human readable description of the 
>field can simply ignore that part of the response, and if we are 
>worried about overhead, put a flag in the request to specify if you 
>want the human readable description returned or not and default it to no..
>
>As an aside, servers, could be given the option of honouring the 
>request accept language header and returning the human readable 
>description of the field in the language of the client if they want to..
>
>This solves our issue, of not requiring every WERIDS client that needs 
>to query our WEIRDS server having to read and implement our extension 
>document, unless of course they want to machine parse the response to 
>do something 'useful' with it, at which point they would have to write 
>code anyway..
>
>The alternative, as I have said above, is we come up with a way that 
>the protocol allows servers to communicate 'extensions' to the data 
>model to the client, however I'm not really sure that is possible as 
>you couldn't send 'semantic' meaning in the context of the objects 
>registry. So really all the extension would communicate is 
>'field-name', 'field human readable descriptor' anyway.
>
>solution, and frankly I don't understand the arguments against doing it 
>- it certainly isn't presentation.
>
>Thanks
>
>Chris

---

From: Chris Wright <[email-masked]>
Date: Thu, 7 Mar 2013 00:55:54 +1100

I have two possible solutions...

One is everything except the base spec...

The second is, why not everything..

What if instead of putting the information in the object response, every server had a dictionary URL... (or some other word which describes it better). Something like...

http://weirds.server/weirds/dictionary

and that returned a JSON structure something like....

{
 fields: {
   field: { code: "domain-name", name: "Domain Name", description: "The name of the domain name"},
   field: { code: "registrant-contact", name: "Registrant Contact Name", description: "the registrant contact name"},
   etc the rest of the base protocol fields that this server uses
  }
  extensions-fields: {
   field: { code:"elgibility-id", name: "Elgibility ID", description: "The ABN/CAN of the blah blah" },
   etc
  }
}

Where fields array contains the base protocol fields... and then the extension fields are the fields of extensions supported by the server?

Thanks

Chris

-----Original Message-----
From: Simon Perreault [mailto:[email-masked]] 
Sent: Thursday, 7 March 2013 12:43 AM
To: James Mitchell
Cc: Andy Newton; Chris Wright; [email-masked]
Subject: Re: [weirds] Client handling of known attributes/names

James, Chris,

I feel like we're talking past each other. So I'll only ask questions this time.

Le 2013-03-06 14:20, James Mitchell a écrit :
> 	<prefix>_field:{display_name:"Eligibility Name", description:"Name 
> used by the registrant to establish eligibility, if different from 
> their own legal name (eg. registered business name or trademark)"} }

Suppose you define "au_foobar". It gets popular and other registries implement it. You and other registries get together and write an RFC to standardize "foobar" (no prefix). The RFC gets published, everyone's happy.

Now, I suppose "foobar" would no longer have "display_name" or "description" since it's a standard field. Right?

In that case, what is a WEIRDS client that has not yet implemented your RFC supposed to do when it receives that unknown "foobar" field?

Is your proposal related to only <prefix>_something fields, or does it concern everything except the fields defined in the base spec?

Simon

---

From: Andy Newton <[email-masked]>
Date: Wed, 6 Mar 2013 14:01:21 +0000

James,

My comments are in-line:

>specific client side knowledge.

I would agree that eligibility information would be different from one
registry to another.

There are three approaches that I see: 1) implement custom values for your
registry, 2) develop a unified eligibility / rights-holding data model for
all registries, or 3) use a link relationship to link to a web page
specific to your registry.

For #2, if you did this you could develop the model and publish it as an
IETF standard that could be added on to RDAP. The mechanics of that seem
easy. The hard part would be getting everybody to agree on the data model
for expressing the holding of rights to internet resources.

For #3, you could use the link relationship of terms-of-service (see
http://tools.ietf.org/html/draft-ietf-weirds-json-response-02#section-6)
or perhaps register your own link relationship with the IANA.

>Do people still have reservations for this approach?

In my opinion, you'll end up replicating all the presentation layer
semantics of a web page if you go down this path. Its easier just to use a
web page vs. replicating those features. This fits into the link
relationship mentioned above.

-andy

---

From: Chris Wright <[email-masked]>
Date: Thu, 7 Mar 2013 01:03:20 +1100

Just to expand on this.. there may need to be a dictionary per object.. or something else.. the idea needs to be expanded more... and could be the subject of another draft (which we would be more than happy to volunteer to author)...

It could be cached and clients could use standard http mechanisms like If-modified etc to make sure they have latest versions and so forth..

Thanks

Chris

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Chris Wright
Sent: Thursday, 7 March 2013 12:56 AM
To: Simon Perreault; James Mitchell
Cc: [email-masked]
Subject: Re: [weirds] Client handling of known attributes/names

{
 fields: {

Thanks

Chris

-----Original Message-----
From: Simon Perreault [mailto:[email-masked]]
Sent: Thursday, 7 March 2013 12:43 AM
To: James Mitchell
Cc: Andy Newton; Chris Wright; [email-masked]
Subject: Re: [weirds] Client handling of known attributes/names

James, Chris,

Simon

---

From: Andy Newton <[email-masked]>
Date: Wed, 6 Mar 2013 14:24:14 +0000

On 3/6/13 8:51 AM, "Chris Wright" <[email-masked]> wrote:

>I agree that clients that are trying to 'do' something programmatically
>with the data, of course have to be updated, but clients that simply
>'display' the data shouldn't need to be updated, they don't need to be
>updated with WhoIs right now.. it's a backwards step from that
>perspective...

Display can mean a lot of things. As was mentioned earlier in the thread,
what about a base64 encoded PNG? More realistically, what about JSON
values that arrays or nested objects? Simple display descriptions don't
help there. Not to mention that your idea of a display string might not be
very useful to anybody who doesn't speak your language.

>It feels like we are not designing a replacement for WhoIs but rather
>something that achieves an entirely different purpose, something that is
>to be used alongside it.. maybe that was the goal from the start and I
>have misguided myself..

Well, we have no plans to shut down our port 43 service even though it
receives less traffic than our RESTful service (imagine how much more
RESTful traffic we'll get when we become RDAP compatible). Right now our
port 43 service is simply a presentation of our RESTful service as it is a
simple proxy. I anticipate a lot of registries will head down that path.

-andy

---

From: Chris Wright <[email-masked]>
Date: Thu, 7 Mar 2013 01:51:19 +1100

-----Original Message-----
From: Andy Newton [mailto:[email-masked]] 
Sent: Thursday, 7 March 2013 1:24 AM
To: Chris Wright; Simon Perreault; [email-masked]
Subject: Re: [weirds] Client handling of known attributes/names

>Display can mean a lot of things. As was mentioned earlier in the thread, what about a base64 encoded PNG? More realistically, what about JSON values that arrays or nested objects? Simple display descriptions don't help there. Not to mention that your idea of a display string might not be very useful to anybody who doesn't speak your language.

Wouldn't this then mean that putting the descriptor information inline with the object response is the way to go then? (of course that assumes you agree that the descriptor information is required in the first place)

If we were using XML, then the 'dictionary' could just return the XSLT and we wouldn't have a problem with arrays and nested objects... (though dictionary would be a really bad name for it at that point)

So another suggestions are allowing a text mime type to be supported where we would return text documents that provide the information in the format similar to the current WhoIs, so a 'generic' WEIRDS client can just do text mime queries - I think we have suggested this before..

Also, on languages, HTTP already has a mechanism for specifying languages you want (And languages server supports) so multi lingual isn't a problem - the only concern would be if a server didn't support the language you needed, but in that case you are no worse off than you would be if we didn't do any of this at all (And for most cases you are in a better position than you were with the current WhoIs protocol).

>>something that achieves an entirely different purpose, something that 
>>is to be used alongside it.. maybe that was the goal from the start and 
>>I have misguided myself..

>Well, we have no plans to shut down our port 43 service even though it receives less traffic than our RESTful service (imagine how much more RESTful traffic we'll get when we become RDAP compatible). Right now our port 43 service is simply a presentation of our RESTful service as it is a simple proxy. I anticipate a lot of registries will head down that path.

I expect that port 43 will be around for a while yet too, and intuitively it makes sense to me that an IP address registry has seen a huge reduction in the WhoIs queries with many people moving over to the 'machine parseable' RESTful service... Im not sure the same will be true of Domain Name Registries - Im still wondering if we are supposed to be designing a replacement for WhoIs or not though, is the ultimate goal to eventually retire WhoIs?

---

From: Andy Newton <[email-masked]>
Date: Wed, 6 Mar 2013 15:20:22 +0000

On 3/6/13 9:51 AM, "Chris Wright" <[email-masked]> wrote:

>So another suggestions are allowing a text mime type to be supported
>where we would return text documents that provide the information in the
>format similar to the current WhoIs, so a 'generic' WEIRDS client can
>just do text mime queries - I think we have suggested this before..

You are free to provide plain text output. Our RESTful service does.
However, I don't see it solving any of the issues you've brought up. Even
a generic client may want to change the presentation of the data.

>Also, on languages, HTTP already has a mechanism for specifying languages
>you want (And languages server supports) so multi lingual isn't a problem
>- the only concern would be if a server didn't support the language you
>needed, but in that case you are no worse off than you would be if we
>didn't do any of this at all (And for most cases you are in a better
>position than you were with the current WhoIs protocol).

We've been told by our I18N experts that this in fact an incorrect usage
of that header.

>I expect that port 43 will be around for a while yet too, and intuitively
>it makes sense to me that an IP address registry has seen a huge
>reduction in the WhoIs queries with many people moving over to the
>'machine parseable' RESTful service... Im not sure the same will be true
>of Domain Name Registries - Im still wondering if we are supposed to be
>designing a replacement for WhoIs or not though, is the ultimate goal to
>eventually retire WhoIs?

I've yet to meet a Whois operator who didn't believe the majority of their
queries were from automation. As to the ultimate goal, that is in the
charter and is to produce a better protocol than Whois. If Whois/port 43
gets replaced in the process, that's just a side affect.

-andy

---

From: Byron Ellacott <[email-masked]>
Date: Thu, 7 Mar 2013 00:19:16 +0000

Hi Chris,

On 7/03/13 12:51 AM, "Chris Wright" <[email-masked]> wrote:

>If we were using XML, then the 'dictionary' could just return the XSLT
>and we wouldn't have a problem with arrays and nested objects... (though
>dictionary would be a really bad name for it at that point)

You could certainly use XSLT to provide one particular transformation of
structured data into, presumably, unstructured text, but a client would be
under no obligation to use it.  If you intended that all clients MUST use
that transform, why bother with the structured data in the first place?

(Answer: because most consumers of registration data don't want
unstructured text, they want something structured so they can pull out the
information that's most relevant to their present need.  See Also: data
consumers having a desire for structured addresses :-)

If your goal is to ensure that human consumers of weirds data have access
to specific information, there's a few mechanisms already in the data
model, and of course the potential to highlight where those mechanisms are
insufficient and add more:

 - server-level notices include description and descriptive link fields
 - every object has a remarks field for unstructured text data

Do you have a rough map for how you'd translate your current data model
into the current RDAP data model?  Identifying where that translation
breaks down, specifically, might let us as a working group look at where
the RDAP data model is weak, or where you might need a registry specific
extension, or where the translation might be tweaked to avoid the problem.

  Byron

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 6 Mar 2013 16:27:04 -0800

I really wanted this protocol to be based on XML, not only because of the things you've mentioned, but because clients such as web browsers now need Javascript to fetch/format the data. 

Even though I agree that JSON is really simple to implement, it does bring some other issues that I've already highlighted several times, for which XML is very mature.

We (ISC) are implementing a RESTful whois that will use XML, XSLT for transformations, etc, and of course we will add the JSON support to make it compatible with WEIRDS spec..

Francisco

> If we were using XML, then the 'dictionary' could just return the XSLT and we wouldn't have a problem with arrays and nested objects... (though dictionary would be a really bad name for it at that…

Francisco Obispo 
Director of Applications and Services - ISC
email: [email-masked]
Phone: +1 650 423 1374 || INOC-DBA *3557* NOC
PGP KeyID = B38DB1BE

---

From: "John Levine" <[email-masked]>
Date: 7 Mar 2013 02:11:49 -0000

In article <[email-masked]> you write:
>I really wanted this protocol to be based on XML, not only because of the things you've
>mentioned, but because clients such as web browsers now need Javascript to fetch/format the
>data. 

No, they don't.  There are JSON libraries in all sorts of languages.
I do my JSON work in python, but that's a matter of taste.

By the way, how do you write a PNG or JPEG renderer in xslt?

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 6 Mar 2013 18:26:57 -0800

> I do my JSON work in python, but that's a matter of taste.

Yes, but a plain browser would have to execute a program in order to be able to parse/convert the data into something useful for the user, while XML doesn't have that issue. 

Browsers support XML and XSL natively, they don't need to execute external programs. 

> By the way, how do you write a PNG or JPEG renderer in xslt?

Why do you need that? in any case I guess you could use XSL-FO [1]

[1] http://en.wikipedia.org/wiki/XSL_Formatting_Objects

Francisco Obispo

---

From: "John Levine" <[email-masked]>
Date: 7 Mar 2013 02:58:51 -0000

>Yes, but a plain browser would have to execute a program in order to be able to
>parse/convert the data into something useful for the user, while XML doesn't have that
>issue. 

This is a total red herring.  If the client is a person sitting at a
browser, the http requests will accept text/html rather than
application/json and the server can send back something in HTML if it
wants to support direct queries from people.  This should be obvious.

Please remember that the vast majority of WHOIS queries are machine to
machine, not human to machine, and that's what WEIRDS is designed for.
In most cases, there is no human directly involved, and no human
waiting to look at random semi-formatted output.

R's,
John

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 6 Mar 2013 19:24:05 -0800

> wants to support direct queries from people.  This should be obvious.

Which assumes that you have a server that is running a script capable of looking at the accept header. (or a special setup in the web server to deliver content based on that).

But every single argument will have a counter-argument at this point, I believe XML was a better solution than JSON for RDAP, the complexities of XML as some call it, address some real life issues that will have to be addressed in JSON somehow.

That's why we decided to implement it in XML with XSL and XSD, the XML can easily be transformed in to anything we need it to, even JSON.

> waiting to look at random semi-formatted output

Whois will still be consumed by human beings, the fact that a computer can query faster than a human is no argument for making the protocol less readable. Being able to consume it by both humans and computers is something I consider necessary. Specially since non-programmers will be looking at the data.

Francisco Obispo

---

From: Michael Young <[email-masked]>
Date: Wed, 6 Mar 2013 22:35:57 -0500

Non-programmers use applications, be that a browser or otherwise, to do a Whois lookup.

That said, I think JSON is great for the simple case, and XML is better for different namespace schemas. I'm on the fence really on this one,....support for both would be ideal probably.

Michael Young

M:647-289-1220

---

From: "John R Levine" <[email-masked]>
Date: 6 Mar 2013 22:47:25 -0500

> Which assumes that you have a server that is running a script capable of 
> looking at the accept header. (or a special setup in the web server to 
> deliver content based on that).

That's content negotiation.  It has been a part of standard http since 
1996.  Is there some reason you believe that it is unusual or difficult to 
implement?

> That's why we decided to implement it in XML with XSL and XSD, the XML 
> can easily be transformed in to anything we need it to, even JSON.

Is there some reason you believe it is productive to rehash arguments that 
were settled months ago?

R's,
John

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 6 Mar 2013 19:56:35 -0800

I don't think it's productive, since it's already 'settled' and there's just too much religion into JSON to leave room for anything else. We will use JSON even if we need to make it fit with a sledgehammer, and since I have the option to implement XML, I am, and will provide an interface in JSON adjusted to the specs discussed in this mailing list.

The whole issue on Namespaces was my first argument, I wanted to define a base protocol format that could be used to 'transport' the various objects. How to treat them required the concept of namespaces. The format evolved from the first draft to incorporating 'object' responses, which I still think it's a positive change.

For the rest, we will debate if JSON is better than XML and viceversa, you missed my point on HTTP and return objects, if you don't have the data, the web server won't generate it for you, it's not a content negotiation issue, its a content generation and delivery issue.

On Mar 6, 2013, at 7:47 PM, "John R Levine" <[email-masked]> wrote:

> Is there some reason you believe it is productive to rehash arguments that were settled months ago?

Francisco Obispo