Subject: [weirds] Scrap Bootstrapping

From: "Hollenbeck, Scott" <[email-masked]>
Date: Tue, 12 Nov 2013 11:59:46 +0000

The more we discuss the challenges associated with bootstrapping queries the more I'm becoming convinced that there's no way to produce a completely adequate solution. Here's another potential problem: how do we bootstrap queries for entities? There's nothing in an entity identifier that can be consistently used to identify the authoritative server.

If we can't come up with one way to bootstrap all queries we should just forget the idea and explore other ways to make it easier for a client to find servers. Maybe it would be easier to create an IANA registry of servers that includes a URL to the service, a text field that identifies the server operator, and a text field that includes a list of tokens that identify the objects for which the server contains information. Clients can retrieve and parse the list, present that information to a user or program, and let the user/program pick the server to send the query to.

Scott

---

From: Simon Perreault <[email-masked]>
Date: Tue, 12 Nov 2013 10:24:52 -0500

Le 2013-11-12 06:59, Hollenbeck, Scott a écrit :
> how do we bootstrap queries for entities?

There's no global namespace for entities, contrary to domain names and 
IP addresses. Entities are specific to a particular RDAP server. So IMHO 
bootstrapping entities is a non-issue.

> Maybe it would be easier to create an IANA
> registry of servers that includes a URL to the service, a text field
> that identifies the server operator, and a text field that includes a
> list of tokens that identify the objects for which the server
> contains information. Clients can retrieve and parse the list,
> present that information to a user or program, and let the
> user/program pick the server to send the query to.

IMHO the proposal from draft-blanchet-weirds-bootstrap-ianaregistries is 
simpler and cleaner.

Simon
-- 
DTN made easy, lean, and smart --> http://postellation.viagenie.ca
NAT64/DNS64 open-source        --> http://ecdysis.viagenie.ca
STUN/TURN server               --> http://numb.viagenie.ca

---

From: "John Levine" <[email-masked]>
Date: 12 Nov 2013 15:47:32 -0000

>bootstrapping entities is a non-issue.

Right.  It would be nice if the JSON contained a pointer back to the
server that knows about the entity, since it can be hard to figure out
what it was if it was at the end of a chain of http redirects.

(Note that starting at the beginning of the redirect chain won't work.
Consider asking the .uk server about something.ac.uk.)

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 12 Nov 2013 15:58:26 -0000

>If we can't come up with one way to bootstrap all queries we should just forget the idea and
>explore other ways to make it easier for a client to find servers.

I halfway agree.  For numbers, experience suggests that faking it will
work, starting with a coarse fixed bootstrap, e.g. for IPv6 a table
derived from
http://www.iana.org/assignments/ipv6-unicast-address-assignments/ipv6-unicast-address-assignments.xhtml
which hasn't changed since 2006.  I understand the possibility that
there might someday be a lot of top level xIRs who don't talk to each
other, but we'll have to burn that bridge when we get to it.

I'm more concerned about TLD bootstraps, in particular the ones for
the new ICANN TLDs.  For the existing ICANN TLDs, there are under 20
of them, and the servers rarely change.  For ccTLDs, most don't
provide WHOIS at all, or don't provide any details beyond the fact
that it's registered, and the few that do like .US and .CO rarely
change, so again we can fake them with a fixed table.  

For the new TLDs, though, they'll all have RDAP if ICANN tells them to
which seems likely.  But since they all are ICANN contracted, how about
if we ask ICANN to add an RDAP field to the IANA data, and we scrape
that every once in a while?

If we can do that, we should be able to fake the whole thing.

---

From: Marc Blanchet <[email-masked]>
Date: Tue, 12 Nov 2013 11:08:21 -0500

Le 2013-11-12 à 10:58, John Levine <[email-masked]> a écrit :

> of them, and the servers rarely change.  

our solution should be agnostic to the fact that a TLD is a new/old gTLD or ccTLD. it is a TLD.

> For ccTLDs, most don't

The fact that data change rarely is good, but going to the conclusion that it should be burned is not good. Because then, you can not update.  
The decision of an implementer on how it will update its own table is implementer decision. What we ought to achieve is to define a method that provides accuracy and reliability of the data, therefore our method should always return the right data. If the implementer decides to cache some data, then fine, but that is an implementation decision.

> if we ask ICANN to add an RDAP field to the IANA data,

this is what in the internet draft draft-blanchet-weirds-bootstrap-ianaregistries: add a column with rdap, similar to the whois entry already in the current table of TLDs.

> and we scrape

if we agree on the iana registries, then i have already some text to give guidelines to the client on what to do with the registry: fetch it often, not often, cache it, etc... This is the extent we can do.

Marc.

> _______________________________________________
> weirds mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/weirds

---

From: "John Levine" <[email-masked]>
Date: 12 Nov 2013 16:29:06 -0000

>our solution should be agnostic to the fact that a TLD is a new/old gTLD or ccTLD. it is a TLD.

That would be nice if it were possible, but it's looking less and less
that way.

>The fact that data change rarely is good, but going to the conclusion that it should be burned is
>not good. Because then, you can not update.  

Sure you can, just like you can update any other software.  It's not
ideal in some ways, but in many applications, it works a lot better
than trying to contact remote resources that may be slow, flaky, or
otherwise hard to use.

>if we agree on the iana registries, then i have already some text to give guidelines to the
>client on what to do with the registry: fetch it often, not often, cache it, etc... This is the
>extent we can do.

Right.  I expect something like once a month to catch updates, with an
extra scrape if a client asks for a hitherto unknown TLD, would be
reasonable.  Microoptimization: if the TLD's not in the table, do a
DNS lookup for the SOA of the TLD before the scrape to see whether it
exists or it's just a typo.

R's,
John

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Tue, 12 Nov 2013 17:49:06 +0000

(Sorry, using a UI- constrained device)

Scott

> On Nov 12, 2013, at 10:25 AM, "Simon Perreault" <[email-masked]> wrote:
> 
> There's no global namespace for entities, contrary to domain names and IP addresses. Entities are specific to a particular RDAP server. So IMHO bootstrapping entities is a non-issue.
> 
Actually, I think this is part of the whole issue. Bootstrapping as currently envisioned only works for a portion of the query space. What good is a partial solution?

> IMHO the proposal from draft-blanchet-weirds-bootstrap-ianaregistries is simpler and cleaner.

Perhaps it is. If one solution works for all query types it'll make client implementation much easier - but again, I don't see how this works for all query types.

Scott

---

From: Andy Newton <[email-masked]>
Date: Tue, 12 Nov 2013 19:38:52 +0000

On 11/12/13 11:29 AM, "John Levine" <[email-masked]> wrote:

>reasonable.

To me this is the best solution.

But getting back to Scott's original point, it might also be that
bootstrapping is beyond the scope of our charter.

-andy

---

From: Ernie Dainow <[email-masked]>
Date: Tue, 12 Nov 2013 14:46:42 -0500

If there is an RDAP redirector that is kept reasonably up-to-date, there is
a fairly simple solution for most clients. Build the software using a
built-in static copy of the IANA table to provide a lookup of RDAP server
by .tld. If a tld is not found in the table, then use the redirector. Or if
there is a connection failure because the server for a tld has changed,
then use the redirector.

Since the new ICANN tlds are generally going to be quite small initially,
there won't be many redirects. The lookup table can be updated when the
software is updated, even as infrequently as once a year, without shifting
much load to the redirector.

-Ernie

---

From: Marc Blanchet <[email-masked]>
Date: Tue, 12 Nov 2013 14:48:57 -0500

> To me this is the best solution.

ok. So IANA registries with some text about "clever guidelines for clients to not fetch the IANA registry everytime. Correct?

Marc.

---

From: John R Levine <[email-masked]>
Date: Tue, 12 Nov 2013 15:05:24 -0500 (EST)

> much load to the redirector.

If people are doing their lookups through an http redirector, they'll hit 
it on every lookup and the load will be quite large.

If you mean a DNS thing like <tld>.rdap-servers.net, then I agree that DNS 
caches should limit the load.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail.

---

From: John R Levine <[email-masked]>
Date: Tue, 12 Nov 2013 15:12:04 -0500 (EST)

> ok. So IANA registries with some text about "clever guidelines for 
> clients to not fetch the IANA registry everytime. Correct?

Right.

>> bootstrapping is beyond the scope of our charter.

This is crude enough that it's probably not a good idea to make it 
normative, just note that it's one way to find bootstrap info, and other 
better bootstraps may be standardized as we gain experience with RDAP.

Regards,

---

From: "Carlos M. Martinez" <[email-masked]>
Date: Tue, 12 Nov 2013 18:23:20 -0200

Hello,

> better bootstraps may be standardized as we gain experience with RDAP.

I like this approach. Instead of settling on "THE" way to do
bootstrapping, perhaps we can suggest different approaches, some that
could work for numbers, others for names, and leave it out to
implementations to chose which one to implement.

Once we gather more operational experience we might revisit the question
and offer more definitive guidance.

The IANA listing approach for number bootstrapping is a 'good enough'
approach. The RIRs know how to handle the corner cases and will provide
adequate redirection. Also a client could grok the NRO delegated file to
have the full entry point list for every block.

For names, I'm not so sure. Names folk can suggest one or two 'good
enough' approaches, and from then on we could have a 'guidelines for
initial querying the RDAP system'.

And... before anyone else says it, can we call the discussion on
Scrapping Bootstrap, 'bootscrapping' ?

:-) (couldn't resist, sorry)

cheers!

~Carlos

---

From: Ernie Dainow <[email-masked]>
Date: Tue, 12 Nov 2013 15:59:04 -0500

> it on every lookup and the load will be quite large.

No, if they only use the redirector for tlds that are not in the lookup
table, statistically this will be less than 1% of their queries (total new
tld domains/total "old" tld domains), unless new tlds have over 1.5 million
domains before the client code is updated.

-Ernie

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 14 Nov 2013 01:42:10 -0800

> bootstrapping is beyond the scope of our charter.
It's been added as a milestone now, so at least the co-chairs and AD think
it's either explicitly or implicitly in scope.  The working group could
decide, however, that this part of the work is ultimately intractable and
not tackle it in the end.

-MSK