Subject: [Ietf-not43] contact lookups: the elephant in my room

From: william at elan.net ([email-masked])
Date: Tue Apr 1 07:49:37 2003

I like to see unified search architecture, but as you say its not very 
practical at this point. But I do not want to see this eliminated as an 
option, what I'd like is to have minimum set of what needs to be implemented
(which would mean search only in one database being queried) by whois 
service but allow for extensions for these unified queries.

On this topic, the only easy way to do distributed search is to have 
unified index server. Potentially we could say that certain resources are 
"searchable" and provide for way have all referrals "share" their 
indices with upstream referrer, that means we have to have means to 
do something more as far as what referrer is then what is currently done, 
in particular when one server is putting referral to another it does not 
already know, it would need to "notify it" (potentially as part of "bag" 
on first referral) and they would need to "negotiate" index sharing. But 
we do also here get into some non-technical issues like "privacy" as well 
as many technical details. 
  The other way is to do distribute search is to potentially have each 
request spawn parallel queries in all servers that are known as referral 
to the one being queried. This may potentially mean quite a bit longer 
processing time and resources used for any query (resources not otherwise 
local to server originally queried). 
 If people are interested, I'd like to see if we can do it as some kind 
of extension to CRISP do distributed search (I like unified index 
architecture more), but I do not think this should be part of "Requirement".

> I'm trying to figure out exactly what kind of search-by-email needs to be
> supported in the LDAP-WHOIS spec. draft-ietf-crisp-requirements-04.txt
> requires the following:
> 
>  | 3.2.1.1 Protocol Requirement
>  |
>  |    The protocol MUST contain the following lookup functions:
>  |
>  |    1.  Contact lookup given a unique reference to a contact of a
>  |        resource.
> 
> This can mean several different things. As two examples of this, it can
> mean that the globally distributed database of "domains" must be
> searchable by some kind of contact identifier, or it can mean that some
> other [different] globally distributed database of "contacts" must be
> searchable by some kind of contact identifier. It can also mean both, and
> it can also mean a whole lot more databases will eventually need to be
> searchable as well (substitute IPV4/IPV6/ASN databases for the "domain"
> database and you see that we are talking about a lot of databases).
> 
> Both of those interpretations have different ramifications.
> 
> On the one hand, searching a distributed database (domains in this case)
> by attribute value isn't pretty. It isn't something that's done with any
> other protocol in the IETF space (the closest we came was IQUERY in DNS,
> which was deprecated last year). We've kind of bounced off this topic a
> few times, but frankly, at this point the technology is more appropriate
> for doctoral thesis papers than practical application. Exactly how is a
> server supposed to say "I don't have any matches, go try some infinitely
> and recursively large array of other servers". If this interpretation is
> scaled back some -- just query a specific server for the attribute values,
> effectively reinvinting IQUERY with all of its limitations -- that is
> certainly doable but it is kind of a metaphorical change from other
> searches that we are already doing. We have to ask for the identifier
> (presumably a name and/or an email address) along with the scope they want
> to search within (ask the .com servers only), and not do referrals. This
> is tantamount to being an ugly hack and one which doesn't actually solve
> the canonical problem of finding contacts.
> 
> The other interpretation does solve the problem, but it introduces a
> management issue, which is that we aren't claiming authority over contact
> information. If we say "okay there is also some amorphous table of contact
> data in the system, go search that", then we are all of a sudden outside
> the domain space, which is all we are "authorized" for.
> 
> I can do both of these approaches, and although I'd rather just do one
> (the latter in particular), that's not really the problem I'm asking
> about. What I'm asking about is, which interpretation exactly do we want
> to use as a group?

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Tue Apr 1 08:35:05 2003

on 4/1/2003 5:01 AM [email-masked] wrote:

> unified index server.

I agree that this is feasible, and its an approach I've advocated too. But
it's really an ad-hoc solution to the problem, and is not viable as the
standards-track solution to the problem. For example, if this were the
standards-track solution, there would have to be a universally available
catalog server, there would have to be specs on cache times, there would
have to be mandates for replication, and so forth. All of this is likely
to happen through ad-hoc free-market development, but its not practical to
force this as "the one true way".

There's a couple of other secondary factors here. One factor is that using
a catalog server may not even require any protocol specification; in the
case of the LDAP-WHOIS approach, you could use existing LDAP search
mechanisms to search *any* attribute in the cache (not just email address,
but city, phone number, whatever). Describing how to do searches of any
given attribute (or all of them) would be redundant and possibly even
harmful (if we don't specify "begins with" queries, are they legal?).

Anyway, I agree with you that this is likely to be the practical solution
in the long-term, but I don't think its something we should try to
standardize at this point.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

---

From: Peter.Gietz at daasi.de (Peter Gietz)
Date: Tue Apr 1 13:16:52 2003

Yes, index servers are a solution to the problem. Althought there had 
been an attempt to define a standard directory indexing protocol (CIP, 
RFC2651-2653) it never came beyond proposed standard and it would need 
some twinkling for that. Anyway CIP would also specify things like 
update times etc. and it can be implemented in a mesh, e.g. a 
hierarchical structure of index servers that provide better privacy 
handling but adds additional referrals before the client reaches the 
data searched for. That would be a compromise between "ask one central 
index service" and "ask all data servers you know of".

3.1.7 BTW only forbids requirement of one central service where all data 
agregation takes place. A mesh of indexes without a central superindex, 
or index of indexes would not contradict 3.1.7. But beware the less 
hierarchical the greater the number of referrals to chase.

One could even think of an LDAP implementation of such meshes without 
using CIP, which might need specification of referral configuration, 
special referral handling via controls and the like...

Instead of referencing CIP (which should be revised anyway) in the crisp 
protocol docs or building meshes without CIP, you could reword the 
requirements on distributed search to something like:

"Contact lookup given a fully-qualified domain name or IP address"

where the server with the appropriate contact data of that domain could 
be looked up via DNS SRV. That seems better to me than getting rid of a 
MUST on contact lookup.

Or would this be too easy and am I not seing the proper use cases? This 
is not the first discussion of these issues on this list, but I am not 
aware of a lot of uses case discussion on this. Sorry if I missed something.

Cheers,

Peter

-- 
_______________________________________________________________________

Peter Gietz (CEO)
DAASI International GmbH                phone: +49 7071 2970336
Wilhelmstr. 106                         Fax:   +49 7071 295114
D-72074 T?bingen                        email: [email-masked]
Germany                                 Web:   www.daasi.de

Directory Applications for Advanced Security and Information Management
_______________________________________________________________________

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Tue Apr 1 13:45:24 2003

If we were working on a global white pages project, I'd be all for this. 
  But in the scope of domain registries/registrars, this would add 
complexity that does not yield adequate returns.

-andy

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Tue Apr 1 13:52:26 2003

> be looked up via DNS SRV. That seems better to me than getting rid of a 

That's definitely the approach I favor, but like I said there are some
issues with this approach (despite these problems, I still favor this
approach however)

First of all, there is no global delegation body for contact identifiers.
This is in contrast to global delegations for resources such as DNS
domains, IP addresses, and ASNs. Since there is no global delegation body,
there is no existing delegation body which can be queried. As a result,
you either have to query a specific organization (the servers for .com or
.in-addr.arpa), or you have to query the end-nodes directly (asking the
servers for example.com for information about [email-masked]).

Both of those approaches have the potential to result in incomplete answer
sets. In the case of asking the .com servers, you won't get information
about any delegations that [email-masked] may have made under something
like .co.uk, and that database may also contain out-of-date or misleading
information (using [email-masked] for some domains, [email-masked]
for others, [email-masked] for more, and so forth). Asking the
example.com servers may fail because the admin can choose not to provide
the information, or they may not even have any servers (in the latter
case, you would end up falling back to .com servers, and facing those
problems). So no matter what path you choose you are only going to get the
information that people want you to have; asking their servers directly at
least gives you a mechanism to aks for the information that they *want*
you to have, with a fallback to limited information. A catalog server will
be able to collate information from various pools, but again, this will
only work to the point where the data is timely and accurate, and there
are too many hurdles to deploy a catalog approach as the standard.

Probably the ultimate solution here is to have some kind of global body
delegate global contact identifiers, with other delegation bodies
requiring an identifier before any delegations can be made, and with the
identifier database being updated whenever a delegation is made. It would
be easy to stick a WHOIS-like front-end on that. I don't think anybody
expects that this would happen any time soon though, and I hesitate to
even suggest this because I don't want folks to think I'm advocating it
(to be clear, I'm not).

My preference would be to use a bottom-up query against an email address
as the "standard" method, with the assumption that ad-hoc catalog servers
will emerge to provide "global" interfaces which address the actual need.

---

From: leslie at thinkingcat.com (Leslie Daigle)
Date: Tue Apr 1 17:13:59 2003

To go back to Eric's original question, it is my understanding
that "the protocol" in the requirements document refers to the
interaction with a single server and its own managed database.

We have not yet discussed the conditions under which a given
server might make assertions about potential contents of other
servers (referrals).  We would need to nail that down in order to have
a referral-based service that works.

There are various proposals on the table about how to
drive queries up & down a managed tree (e.g., following
the chain "up" the domain tree to find the CRISP server
associated with a given domain); that's not currently
in the requirements, and is one of the things that we
(for some value of "we" that might be the WG) have said
IRIS should take on from the ldap-whois proposal.

If I understand Eric's point here, the suggestion is
to say that, even if you find an "authoritative" CRISP
server for (say) thinkingcat.com, if it doesn't have contact 
info for LLD02 (the contact handle for thinkingcat.com), then you 
should find & query the CRISP server for .com.

I think that would be a service-level requirement; it's an
assertion of how we think CRISP servers should be organized
in order to provide an effective service.

Quite apart from which, I believe the discussion of using 
common indexing protocol (or not) is *independent* of the
query protocol requirements we are discussing now.  CIP is
a back-end index-creating protocol; it is not a query protocol.
And note that the actual queries you might use against an
index could be quite different than those you use to access
the specific information; there is not sufficient information
in the index to be able to answer the granularity of query
you want to ask the final leaf server; you wind up having
to pull apart the query to look for hints in it anyway.

As I said last July,

http://lists.verisignlabs.com/pipermail/ietf-not43/2002-July/000157.html

there are projects that have implemented the whole index
as a service independently of the individual information
servers themselves.  We can (and probably should) do something
like that here.

Leslie.

-- 

-------------------------------------------------------------------
"Reality:
     Yours to discover."
                                -- ThinkingCat
Leslie Daigle
[email-masked]
-------------------------------------------------------------------

---

From: Peter.Gietz at daasi.de (Peter Gietz)
Date: Wed Apr 2 06:15:21 2003

Leslie Daigle wrote:
> 
> Quite apart from which, I believe the discussion of using common 
> indexing protocol (or not) is *independent* of the
> query protocol requirements we are discussing now.  

Yes. I only put it into discussion to say that, e.g. by using CIP more 
complex requirements for distributed searches could be fulfilled.

> CIP is
> the specific information; 

The index server to which a client connects to, could and IMO should 
provide the same query interface using the same protocol than the one 
used for retrieving the actual data.

 > there is not sufficient information

This is dependant on the actual index implementation; the granularity 
could well be the same, especially in a partly hierarchical index mesh.

> servers themselves.  We can (and probably should) do something

agreed. This could well be a simpler approach than TISDAG that had the 
support of a variety of directory protocols as a requirement, which 
wouldn't be the case here.

Cheers,

Peter

>>> where the server with the appropriate contact data of that domain 
>>> could be looked up via DNS SRV. That seems better to me than getting 
>>> rid of a MUST on contact lookup.
>>
>>
>>

Peter Gietz (CEO)

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Thu Apr 3 09:21:07 2003

> interaction with a single server and its own managed database.

Andy said the same thing in another (off-list) discussion. From what I
gather, the LDAP-WHOIS work already satisfies this particular requirement,
since the DN pointer provides a way to specify an exact contact within an
exact partition of the distributed database.

I was really asking about search, which isn't a requirement.

> should find & query the CRISP server for .com.

More-or-less. I'm trying to determine the most-useful way to find
information about a known person, including all of the resources under
their control (not just domains), and doing so without specifically
requiring any server-to-server communications (equals "unfunded mandate"),
yet without denying that those back-end communications may exist on a
voluntary basis.

I agree that searching-for-contacts isn't currently in the requirements.
On the other hand, it's been asked about and discussed so often that it
seems to be a pretty important feature. I mean, we all want to be able to
find out what domains Leslie Daigle has been delegated, and some of the
actors *really* want to know. Conversely, it'd be nice to clear a work
item off my list for a change, and not having to do it smells sweet too.