Subject: Re: [weirds] [Regops] Search Engines Indexing RDAP Server Content

From: Maarten Bosteels <[email-masked]>
Date: Mon, 1 Feb 2016 14:19:12 +0000

On 29/01/16 18:31, "Hollenbeck, Scott" <[email-masked]> wrote:

>> -----Original Message-----
>> From: John Levine [mailto:[email-masked]]
>> Sent: Friday, January 29, 2016 12:24 PM
>> To: [email-masked]
>> Cc: Hollenbeck, Scott
>> Subject: Re: [weirds] [Regops] Search Engines Indexing RDAP Server
>> Content
>> 
>> >We have a difference of opinion, because I tend to think that it's a
>> *good* idea to restrict access
>> >to *some* data based on a client's identify and level of
>> authorization. I do not want *my* PII
>> >appearing in search engine search results because it's accessible via
>> RDAP.
>> 
>> Sorry, but I really don't understand what you want here.  There have
>> been vertical WHOIS search engines forever that let you search WHOIS
>> data in various ways.  Most of them charge money, but as often as not
>> some access is free.  That horse left the barn a decade ago.
>> 
>> The stuff that search engines can see is exactly the same as what
>> casual users see.  If you're saying that you want to make some stuff
>> unavailable to anyone who doesn't have a password, OK, but now aren't
>> you running into the whole ICANN mess of what's in WHOIS and what
>> isn't?
>
>Yes, definitely. I see little point in deploying RDAP if all we get is JSPON-encoded WHOIS data. We have a tool that let's us fix the whole "all data visible to everyone" problem. We should try to use it.
>
>Scott

+1

Maarten

>_______________________________________________
>weirds mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/weirds

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Wed, 3 Feb 2016 13:35:13 +0000

> From: James Galvin [mailto:[email-masked]]
> Sent: Wednesday, February 03, 2016 8:01 AM
> To: Hollenbeck, Scott
> Cc: [email-masked]; [email-masked]; [email-masked]
> Subject: Re: [weirds] Search Engines Indexing RDAP Server Content
> 
> This is not a very good example of the need for operational practices.
> 
> It represents Google UK’s view of what you can find when you start at
> “http://rdg.afilias.info/rdap/help”, which includes explicit
> examples of how to use the RDAP server.

Jim, you're probably right about it not being a good example. I'm more concerned with the information that could be returned in response to an entity query being archived.

Side note: the response received for http://rdg.afilias.info/rdap/help is encoded using HTML. Sections 7 and 4.3 of RFC 7483 say it should be JSON-encoded.

"The appropriate response to /help queries as defined by [RFC7482] is to use the notices structure as defined in Section 4.3."

Would it have been indexed at all if it wasn't HTML? There's an operational practice thing in here somewhere... ;)

Scott