Subject: [weirds] Client handling of unknown attributes/names.

From: James Mitchell <[email-masked]>
Date: Wed, 6 Mar 2013 00:04:10 +1100

All,

I have been stewing on this one awhile and have come to the belief that the current approach will result in a protocol that I would not recommend as acceptable for deployment with the .au ccTLD. Let me explain why.

Section 4.2 of draft-ietf-weirds-json-response-02 says that "Clients processing JSON [RFC4627] responses SHOULD ignore values associated with unrecognized names".

I do not want any client ignoring .au specific whois data in the response, instead (generic) clients that are acting on behalf of (people) users SHOULD present this information to the user. The distinction is the end consumer of the information; I don't care that automated service trying to identify a spam score will ignore such information. This however introduces a problem, how would the client know how to display "au_registrant_id"? Assuming a text-based output, it could be displayed as:

-- begin rendering of response --
Name: ausregistry.com.au
Last Updated: 2011-10-14
…
Additional
au_registrant_id: OTHER 092 242 209
-- end rendering of response --

Gah that looks ugly! Either the client should have mappings from the "au_registrant_id" name into a nice presentation format, or the server has to provide this meta-information. I'm going to assume the latter since the goal is to reduce client complexity. Possible solutions could include the client asking the server for rendering information (internationalization achieved using information in HTTP headers if required – HTTP caching also if required):

GET /definition/(domain/)au (a possible use for the prefix?!)
Response:
{
au_registrant_id:{display-name:"Registrant ID", description: "ID number associated with the registrant entity, if any (eg. ACN for company)"}
…
}

Alternatively, servers could include this information in the response directly:
GET /domain/ausregistry.com.au
{
name:"ausregistry.com.au"
…
au_registrant_id:{value:"OTHER 092 242 209", display-name:"Registrant ID", description: "ID number associated with the registrant entity, if any (eg. ACN for company)"}
}

Do other registries, having custom fields in their WHOIS responses, and who intend to adopt WEIRDS also share this concern?

Regards,
James

---

From: "John Levine" <[email-masked]>
Date: 5 Mar 2013 21:26:44 -0000

>Section 4.2 of draft-ietf-weirds-json-response-02 says that "Clients processing JSON [RFC4627]
>responses SHOULD ignore values associated with unrecognized names".
>
>I do not want any client ignoring .au specific whois data in the response, instead (generic)
>clients that are acting on behalf of (people) users SHOULD present this information to the user.

You're misunderstanding what that means.  It's saying that client
software should be able to skip over fields it doesn't understand,
rather than stopping and failing.  It doesn't say anything about user
interface.

As a bit of background, the IETF is famously, awesomely bad at
anything related to user interface, and we have learned to stay away
from it.  While I agree that there are plenty of situations where it
would make sense to format unknown fields in some generic way and dump
them to the screen for the user to see, there are also plenty of
situations where it doesn't.  

Perhaps the screen is really small (an iWatch or something) and there
isn't room to show more than a small fraction of what comes back.  Or
perhaps a field is a base64 encoded PNG of a person's photo, and if a
client doesn't happen to know how to decode it, the user will see a
big block of random uninformative text, which in that case is worse
than nothing.  There's no generally useful advice we can give other
than to avoid doing stupid things, but anyone who needs that advice is
unlikely to be able to follow it.

Our job is to tell clients and servers how to interoperate, not to
tell clients what to do with the data once it's decoded it.

R's,
John

PS: If a typical WEIRDS lookup returns a lot of fields that a typical
client doesn't understand, we won't done our job very well, either.

---

From: James Mitchell <[email-masked]>
Date: Wed, 6 Mar 2013 23:16:37 +1100

Response below.

>rather than stopping and failing.  It doesn't say anything about user

I don't quite agree with your comment, as later in that section is
"Consider the following JSON response with JSON names, some of which
should be ignored by clients without knowledge of their meaning". This
bugs me, as with the current WHOIS protocol servers define the meaning of
all fields, and thus there is no (human) client that will not have
knowledge of the meaning. How, after deployment of WEIRDS, does the client
now obtain knowledge of the meaning of each field?

Perhaps the text could instead say:

"Clients processing JSON [RFC4627] responses SHOULD NOT treat the
existence of values associated with unrecognised names as an error".
And
"Consider the following JSON response with JSON names, some of which are
not specified in this document."

If I misinterpreted the existing text then someone else may do as well?

>situations where it doesn't.

The problem is that we are replacing WHOIS, which commingles data and its
presentation, with one that is only data. I don't want another IDNA2008,
where mapping was removed from the protocol only for implementers to raise
their hands and fall back to IDNA2003 compatible mappings. I agree with
your latter email regarding common terminology, and we can work together
to get a few "extended attributes" ironed out, however there will probably
be a few name registries that have something different, so what about
them? There still feels like a gap in the protocol, see comment above.

>unlikely to be able to follow it.

I'll make sure our WEIRDS server will include base64 encoded PNGs
containing an IVC that clients must decode and include in their next query
;)

>client doesn't understand, we won't done our job very well, either.

Thanks,
James

---

From: Simon Perreault <[email-masked]>
Date: Wed, 06 Mar 2013 13:27:00 +0100

Le 2013-03-06 13:16, James Mitchell a écrit :
> How, after deployment of WEIRDS, does the client

The same question could be asked about a ton of other protocols that 
allow extensibility. For example, SMTP/HTTP/SIP/... with their "X-" 
headers, or vCard with its "VND-" properties.

The answer is always: code has to be added to the client. There's just 
no other way.

Simon

---

From: James Mitchell <[email-masked]>
Date: Wed, 6 Mar 2013 23:32:40 +1100

WHOIS clients get this meaning/knowledge now writing custom code. We are
suggesting a replacement to WHOIS correct?

James

On 6/03/13 11:27 PM, "Simon Perreault" <[email-masked]> wrote:

>_______________________________________________
>weirds mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/weirds

---

From: Arturo Servin <[email-masked]>
Date: Wed, 06 Mar 2013 10:36:00 -0200

AFAIK the original idea was:

"Be liberal in what you accept, and conservative in what you send."

	So, if the text says otherwise I think we should change it as James
comments.

Regards.
as

---

From: "Carlos M. Martinez" <[email-masked]>
Date: Wed, 06 Mar 2013 10:43:46 -0200

Hi,

> suggesting a replacement to WHOIS correct?

Right, but I don't see how this can change. If you want all clients to
understand all possible fields, then all possible fields have to be
inventoried, defined, etc.

If, on the other hand, we want the data model to be extensible, we are
creating a situation where some clients might not be able to make sense
of extended fields.

This is no different than the X-* headers in several protocols. The
requirement here should be something along the lines of 'first, do no
harm'. If your software doesn't understand a given field, then at the
very least it should not crash and burn, and, going one step further, it
could provide some sort of hook where code can be inserted to deal with
the unknown fields.

I don't see much of a middle ground here. Either you create an inventory
and standardize _every_ _possible_ _someone might want_ field (bad idea
IMO), or you allow extensibility and cater for the unknown (the way to
go, IMO)

As far as I understand the drafts, this expected behaviour for dealing
with the unknown is already well stated.

regards,

~Carlos

---

From: Simon Perreault <[email-masked]>
Date: Wed, 06 Mar 2013 13:54:03 +0100

> suggesting a replacement to WHOIS correct?

You misunderstood my point.

Suppose we go with your proposal and add "field name" and "field 
description" strings to all non-standard element.

The one day somebody defines a field with semantics such as: "When this 
field is present, the client shall retry its query with the ZORGLUB flag 
set to true."

These semantics are completely new. No existing WEIRDS client could 
implement them. So they need extra code to understand the new 
extensions. There's just no other way.

Simon

---

From: James Mitchell <[email-masked]>
Date: Thu, 7 Mar 2013 00:25:58 +1100

Carlos,

>IMO), or you allow extensibility and cater for the unknown (the way to

I completely agree that we do not want to standardise every possible
field. However we have two very distinct classes of clients,
software-centric (e.g. determine spam score) and human-centric (e.g.
present to a user).

A software-centric client will parse and pick out the pieces it wants; if
it wants a field specific to my registry then it already knows its
meaning. I am confident that the approach taken to date will work for 100%
of software clients.

However humans generally don't prefer X- headers, looking at XML, or JSON
names. Sure they'll eventually get used to it but is that what we want?
Why do the human-centric clients have to write custom mappings/hooks/code
to correctly present the response to their users? Sure, some human users
may only care about the creation date, but some will care about the
information for which there is no hook defined. If we want client software
to have a hook for each registry then lets do it, however I think this is
a bad idea. I'd rather generics client are able to be more generic, and
servers are able to be more specific if possible. Refer to my email in
response to Andy/Byron.

Regards,
James

---

From: James Mitchell <[email-masked]>
Date: Thu, 7 Mar 2013 00:29:09 +1100

If that was your point then we are in agreement :)

Thanks,
James

---

From: "John Levine" <[email-masked]>
Date: 6 Mar 2013 17:42:52 -0000

>suggesting a replacement to WHOIS correct?

We are designing a replacement for WHOIS for the modern Internet,
where WHOIS is overwhelmingly a machine to machine service, not
machine to human.

Fortunately for those worried about backward compatibility, most of
the the human users are likely be easy to recognize because they use
clients that ask for html rather than json, in which case you can use
the full glory of html to return whatever you want, extensions, PNG
images, and all.  When the client asks for json, it's a machine, and
the response needs to be something that a machine can use.  I suppose
there might hypothetically be users who use clients that ask for json
and then render it, and urgently need to render every possible
extension, but I think they're few enough that they can take care of
themselves.

But to back up a little, although the various names registries vary
somewhat in the way they describe domains and contacts, they're not
THAT different.  If you can't come up with a common vocabulary that
describes 95% of what's asked for and what's returned, leaving only
exotica in extensions, you really need to go away and think about it
until you can.  Otherwise we're going to end up reinventing kludges
that pattern match all the fields looking for strings that appear to
be e-mail addresses or phone numbers, which would be an admission of
complete failure.

R's,
John

---

From: Byron Ellacott <[email-masked]>
Date: Thu, 7 Mar 2013 00:28:17 +0000

>existence of values associated with unrecognised names as an error".

I am generally speaking in favour of this approach, but wonder if it could
be strengthened slightly:

--- snip ---

Clients processing JSON [RFC4627] responses SHOULD make available to end
users any values associated with unrecognised names, and MUST NOT treat
the existence of such values as an error.

--- snip ---

If a client is presenting data to a user, I'd expect to be shown that
there were fields the client couldn't interpret, but which it can display
as unstructured text.  If a client is instead extracting specific data,
I'd expect to only see that specific data and nothing else.

And as John suggested, if the user-agent requests HTML, transform the data
to whatever visual format you prefer :-)

  Byron

---

From: SM <[email-masked]>
Date: Wed, 06 Mar 2013 16:54:43 -0800

Hi James,
>existence of values associated with unrecognised names as an error".

An issue will be raised about the above text.  If you send me data 
and I cannot process it, I can either ignore it or fail.  I guess 
that what's missing is text about what to do with unrecognised "data".

Regards,
-sm

---

From: "Carlos M. martinez" <[email-masked]>
Date: Wed, 06 Mar 2013 23:38:26 -0200

IMO, you do the same thing other servers do today with unknown data:
either ignore (if you are and endpoint in the communication flow) or
pass on (if you are in the middle)

~Carlos

On 3/6/13 10:54 PM, SM wrote:
> An issue will be raised about the above text.  If you send me data and I
> cannot process it, I can either ignore it or fail.  I guess that what's
> missing is text about what to do with unrecognised "data".
> 
> Regards,
> -sm

---

From: "John Levine" <[email-masked]>
Date: 7 Mar 2013 02:21:15 -0000

>the existence of such values as an error.

My clients don't make anything available to end users.  Does this mean
that I'm not allowed to use WEIRDS?

More to the point, this is the kind of bad user interface design for
which the IETF is infamous.  It has nothing to do with making protocol
engines interoperate, and makes many unwarranted assumptions about what
client software does.

The original language said (perhaps not as clearly as it could have) that
if a client sees fields it can't handle, it should skip them rather than
barfing.  Please leave it at that.

---

From: Arturo Servin <[email-masked]>
Date: Thu, 07 Mar 2013 10:20:20 -0200

I liked the approch that Andrew Sullivan suggested [1]

	A special field to fill up with whatever is not yet defined but the
WEIRDS operator wants to generate.

[1] http://www.ietf.org/mail-archive/web/weirds/current/msg02179.html
Regards
as

---

From: Andy Newton <[email-masked]>
Date: Thu, 7 Mar 2013 14:15:05 +0000

>WEIRDS operator wants to generate.

I'm certainly not getting this. Are you and Andrew talking about a beefed
up remarks section?

Why isn't this good enough?

{
  . . .
  "remarks" :
  [
    "Eligibility Type: Company",
    "Eligibility Name: Widgets R Us",
    "Eligibility ID:   334113"
  ]
  . . .
}

The only thing I can think of is to have multiple remarks objects so each
one can have a different 'lang' value and/or maybe a title value (e.g.
"title" : "AU Registry Eligibility Criteria").

-andy