Subject: [weirds] remarks and client handling of free form data

From: Andy Newton <[email-masked]>
Date: Wed, 13 Mar 2013 16:54:46 +0000

I was asked during the session on Tuesday to send to the list a revamped
remarks structure. After looking at the draft, I think it should follow
the same pattern as the notices structure. It should also be noted that
the remarks section can be used to solve the problem mentioned by Chris
Wright and James Mitchell. Here's an example:

{
  "remarks":
  [
    {
      "title": "Unverified Contact",
      "description":
      [
        "Lunar NIC has attempted to verify the data for this contact, but
has received no response from the POC since 2010-07-28.",
        "See Lunar NIC policy 2010-XXX."
      ]
    },
    {
      "title": "Rights of Use Claim",
      "description" :
      [
        "The following information provides the rights of use information
for this internet resource.",
        "Eligibility Type: Company",
        "Eligibility Name: Acme Widgets",
        "Eligibility ID: BN955912"
      ]
    }
  ]
}

A text based client MIGHT render this as

Unverified
----------
Lunar NIC has attempted to verify the data for this contact,
but has received no response from the POC since 2010-07-28.

See Lunar NIC policy 2010-XXX.

Rights of Use Claim

-------------------
The following information provides the rights of use information
for this internet resource.

Eligibility Type: Company

Eligibility Name: Acme Widgets

Eligibility ID: BN955912

I don't think we should encourage formatting characters such as newlines
in the text as that is a rendering issue and the client knows best how to
render the data.

-andy

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Wed, 13 Mar 2013 16:57:48 +0000

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Andy Newton
> Sent: Wednesday, March 13, 2013 12:55 PM
> To: [email-masked]
> Subject: [weirds] remarks and client handling of free form data
> 
> I was asked during the session on Tuesday to send to the list a
> revamped remarks structure. After looking at the draft, I think it
> should follow the same pattern as the notices structure. It should also
> be noted that the remarks section can be used to solve the problem
> mentioned by Chris Wright and James Mitchell. Here's an example:

I like this approach, and consistency is a good thing.

Scott

---

From: Simon Perreault <[email-masked]>
Date: Wed, 13 Mar 2013 16:11:34 -0400

> Wright and James Mitchell. Here's an example:

I'm not sure what your proposal is exactly. Are you proposing to change 
the current text in any way?

> A text based client MIGHT render this as

I absolutely agree with this part. This is a good example.

> in the text as that is a rendering issue and the client knows best how to

But this is not the actual problem. The problem is that current 
implementations do things like this:

"description" : [
"Lunar NIC has attempted",
"to verify the data for this",
"contact, but has received no",
"response from the POC since 2010-07-28." ]

Can we at least agree that this is wrong?

I agree that embedding newlines may not be appropriate because that 
would be entering the realm of presentation. But splitting sentences 
like above is just silly.

Simon
-- 
DTN made easy, lean, and smart --> http://postellation.viagenie.ca
NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
STUN/TURN server               --> http://numb.viagenie.ca

---

From: Simon Perreault <[email-masked]>
Date: Wed, 13 Mar 2013 16:20:25 -0400

> the current text in any way?

Sorry, I figured it out. No need to answer that.

Simon

---

From: Andy Newton <[email-masked]>
Date: Wed, 13 Mar 2013 20:40:16 +0000

>Can we at least agree that this is wrong?

Yes. Servers should let the clients determine line breaks.

-andy

---

From: Simon Perreault <[email-masked]>
Date: Wed, 13 Mar 2013 17:59:24 -0400

> Yes. Servers should let the clients determine line breaks.

Ok, good.

Any idea how we could translate that into text?

"The "description" element's value is an array of strings. Each string 
corresponds to a ....?"

Simon

---

From: Andy Newton <[email-masked]>
Date: Wed, 13 Mar 2013 23:37:54 +0000

>"The "description" element's value is an array of strings. Each string

Paragraph? Item in a list?

-andy

---

From: Simon Perreault <[email-masked]>
Date: Wed, 13 Mar 2013 19:49:25 -0400

> Paragraph? Item in a list?

Both delve into presentation. We don't want to reinvent HTML.

If we can't figure it out, why do we have an array? Why not a single string?

I'll go further: make it a single string of HTML.

Simon

---

From: Marc Blanchet <[email-masked]>
Date: Wed, 13 Mar 2013 19:52:30 -0400

> If we can't figure it out, why do we have an array? Why not a single string?

single string.

> I'll go further: make it a single string of HTML.

the whole HTML5 spec?  that is the problem of the idea... what would be the tags... we don't want to go there.

Marc.

> _______________________________________________
> weirds mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/weirds

---

From: "John Levine" <[email-masked]>
Date: 14 Mar 2013 00:03:48 -0000

>single string.

Unless the divisions have some semantic meaning like the lines of a
postal address (yes, I know we're planning to use jcard, it's an
example) I don't see any reason to divide it up.

---

From: Byron Ellacott <[email-masked]>
Date: Thu, 14 Mar 2013 01:25:02 +0000

>example) I don't see any reason to divide it up.

Existing data models have multiple values possible for fields like
description.  What should a server do if their port 43 data model has,
e.g.:

description: This a really great
description: network block I have here.

I can look at that and see it should be a single string, and it can be
joined with a space.  But what about:

description: Customers for which FOO-IX provides global transit
description:        ** PLEASE CHECK FOR MORE UP-TO-DATE RECORDS IN RADB **

These look like two separate strings, to me.  Would it be OK to join them
with a space?  Semantically, that might not be too bad.  But what about:

description: Generated by FOO-IX
description: Customers in the northern region

As two separate pieces of data, this tells me it was generated by FOO-IX,
and it is for customers in the northern region, but combined, it tells me
it was generated by the customers themselves.

I think descriptions need to remain as a list of strings, though I would
much prefer if that first example I gave was a single string.  I'd rather
have an end user have to piece together two strings in a list to get the
sentence than have an end user get the wrong information because all
strings have been pieced together already.  I'd also prefer to be noting
the correct behaviour in the draft, such that future data collection work
might proceed along lines more suited to this new data model.

  Byron

---

From: Byron Ellacott <[email-masked]>
Date: Thu, 14 Mar 2013 01:32:39 +0000

Hi Andy,

>        "Eligibility ID: BN955912"

I am uncomfortable with this specific example.  RDAP gives us structured
data, so clients don't have to pull strings apart to get the data they
desire.  This example puts data back into unstructured strings that
clients would have to pull apart again.  I'd rather that the eligibility
data was a local registry extension, as per the "E" in WEIRDS:

{
  "rdapConformance":
  [
    "rdap_level_0",
    "eligibility_level_0"
  ],
  "remarks":
  [
    {
      "title": "Rights of Use Claim",
      "description" :
      [
        "The information contained in the eligibilty extension of this
record provides rights of use information for this Internet resource."
      ]
    }
  ],
  "eligibility_type": "Company",
  "eligibility_name": "Acme Widgets",
  "eligibiltiy_id": "BN955912"
}

  Byron

---

From: "John R Levine" <[email-masked]>
Date: 14 Mar 2013 01:21:56 -0400

> description: network block I have here.

Looks like one string.

> description:        ** PLEASE CHECK FOR MORE UP-TO-DATE RECORDS IN RADB **

Looks like two strings.

> description: Customers in the northern region

Could be either.

Like I said, they should be separate strings if there is some semantic 
division.  I don't think we need to obsess about this, server operators 
can do whatever seems reasonable.  Just please don't break them to try to 
format them into lines of length 80 or whatever.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

---

From: Byron Ellacott <[email-masked]>
Date: Thu, 14 Mar 2013 05:40:06 +0000

Hi John,

>format them into lines of length 80 or whatever.

I think we're in agreement, and I misread your last message as suggesting
everything should always be one string, sorry! :-)

  Byron

---

From: Simon Perreault <[email-masked]>
Date: Thu, 14 Mar 2013 09:36:01 -0400

> description: network block I have here.

Please allow me to continue with my heresy for just a bit...

If what you have in your database is preformatted text, you should be 
able to return preformatted text.

"description": "<pre>This is a really great\nnetwork block I have 
here.</pre>"

Simon

---

From: Andy Newton <[email-masked]>
Date: Thu, 14 Mar 2013 14:04:13 +0000

>"description": "<pre>This is a really great\nnetwork block I have

Simon,

As a security measure, we purposefully don't allow HTML tags in our Whois
output as we don't want it being a vector for embedding malicious scripts
into web sites that redisplay our data.

And if we take into account HTML, do we also want to take into account PDF
or other formats? I don't think we should worry about the format
registries use natively.

-andy

---

From: Andy Newton <[email-masked]>
Date: Thu, 14 Mar 2013 14:04:39 +0000

>format them into lines of length 80 or whatever.

I agree with this.

-andy

---

From: Andy Newton <[email-masked]>
Date: Thu, 14 Mar 2013 14:08:49 +0000

On 3/13/13 9:32 PM, "Byron Ellacott" <[email-masked]> wrote:

>desire.

Byron,

This was in response to the notion that generic clients won't display
custom JSON attributes. If a registry fears that will happen, they can use
the remarks. But I agree with you.

-andy

---

From: Simon Perreault <[email-masked]>
Date: Thu, 14 Mar 2013 10:13:40 -0400

> into web sites that redisplay our data.

Interesting!

> or other formats? I don't think we should worry about the format

It's all about what people have in their databases.

If you have preformatted text, which I suspect many do, it's not clear 
to me what you're supposed to do with the current proposal. Join all the 
lines with a single space character? Insert newlines? Use multiple array 
items?

If what you have in your database is paragraphs and bulleted lists, I 
equally don't know what you're supposed to do to make it fit into the 
JSON. Multiple array items, I guess, but it still doesn't seem right.

The reason we saw very different results with current implementations is 
that this is underspecified, and it's going to cause interop issues if 
we don't say something.

Simon

---

From: Andy Newton <[email-masked]>
Date: Thu, 14 Mar 2013 14:30:11 +0000

>lines with a single space character? Insert newlines? Use multiple array

If a registry uses EBCDIC or encodes their structures in EDI or MIME, then
it is up to them to parse it out and put it into JSON. They will be the
best to do it, not the client. If we attempt to take into account how
every registry stores their data, we'll never get this to work.

-andy

---

From: Marc Blanchet <[email-masked]>
Date: Thu, 14 Mar 2013 10:49:06 -0400

> every registry stores their data, we'll never get this to work.

right. moreover, what is important is the wire format. I'm against any presentation tags such as HTML, because it is going to be a rathole: which tags should be used, which html attributes, etc... overdesign for a simple string record.

Marc.

---

From: "Carlos M. Martinez" <[email-masked]>
Date: Thu, 14 Mar 2013 10:53:06 -0400

>>> It's all about what people have in their databases.

... snip ...

> 
> right. moreover, what is important is the wire format. I'm against any presentation tags such as HTML, because it is going to be a rathole: which tags should be used, 
> which html attributes, etc... overdesign for a simple string record.

Agree. Fully.

---

From: James Mitchell <[email-masked]>
Date: Fri, 15 Mar 2013 02:06:13 +1100

Andy,

Thanks for putting together the suggestion of using the remarks fields, it
looks like a reasonable method of achieving the desired outcome, assuming
remarks are rendered to users behind dumb clients. But unfortunately no
matter which way you spin it, remarks or custom attributes/names, either
the human user or software parser is a potential loser.

I'd rather put my money behind a solution where human users behind dumb
client software saw all the data with parsers having to deal with
semi-structured text (if they so cared about it), than another solution
where there exists the possibility that human users behind the same dumb
clients missed the data (unknowingly) but gave parsers structured data.

Thanks,
James

---

From: Simon Perreault <[email-masked]>
Date: Thu, 14 Mar 2013 11:10:15 -0400

>> which html attributes, etc... overdesign for a simple string record.

Alright, alright, I'll listen to the voice of reason. My heresy is over. 
I'm cured, I swear. I feel much better already.

I still hope to see some constraining text about the usage of array 
items that will make interop more probable. I'll wait for the next 
revision before commenting.

Thanks,
Simon

---

From: Andy Newton <[email-masked]>
Date: Thu, 14 Mar 2013 15:12:37 +0000

>revision before commenting.

I'll try to work on some text to provide guidance on this.

-andy