Subject: [ESDS] Comparison with TISDAG

From: Mark Harrison <[email-masked]>
Date: Mon, 28 Apr 2008 19:53:22 +0100

Dear ESDS folks,

At the ESDS BOF meeting, it was suggested that it would be useful to  
compare with the TISDAG work documented in RFC 2967.
There are a number of similarities between TISDAG and what Discovery  
Services are trying to achieve but also a number of significant  
differences.  I have tried to summarize these below.  I hope that this  
comparison does not misrepresent either the TISDAG or ESDS efforts.

Best regards,

- Mark

Similarities between ESDS and TISDAG:
================================

Provide a unified client query access to discover multiple sources of  
detailed information and provide the client with referrals to those  
information providers

Like TISDAG, ESDS is a starting point for search - and it provides  
referrals to other information providers to allow clients to refine  
their search and to retrieve more detailed information.

Like TISDAG, ESDS maintains an index of providers of more detailed  
information and does not attempt to do centralized mirroring of the  
complete information.

Clients are referred to individual information providers in order to  
retrieve detailed information.  When providing referrals, TISDAG is  
providing a link and a very limited amount of context information in  
order to help the client decide whether or not to follow a particular  
link.

Operators of WhitePages Directory Services (WPDS) are responsible for  
controlling access to detailed information, just as information  
services referred to by ESDS will be responsible for controlling  
access when clients contact them directly.  The fact that a TISDAG /  
ESDS returns a referral does not mean that the information service  
referred to by the referral link 1) definitely holds the record -  
false positives are allowed   2) will grant this specific client  
access to the detailed information

TISDAG proposes a modular architecture in which the internal protocol  
and access to the referral index are insulated from both the  
information service providers and the end-user clients by means of two  
interfaces - the DAG-CAP (client access point) and the DAG-SAP  
(service access point).
Discovery Services will most likely be defined in terms of a query  
client interface for querying by end users and a publishing interface  
that allows information providers to register selected events with a  
Discovery Service for indexing purposes.
The DAG-CAP is analogous to a query client interface for Discovery  
Services
The White Pages Directory Service provider (WSDP) is analogous to the  
information service provided by an entity (e.g. an organization within  
a supply chain or even an individual)
The DAG-SAP is analogous to a standard query interface for a  
particular kind of information service - it may correspond for example  
to the EPCIS Query Interface in the EPC Network architecture
The referral index is analogous to the core data storage mechanism of  
a Discovery Service

Like TISDAG, ESDS will need to provide reasonable assurances of data  
integrity.  While it should not corrupt or distort data sourced from  
information providers, ESDS may be expected to enforce access control  
policies that result in suppression of entire records or of particular  
data fields within records, depending on the client credentials and  
trust relationship with the information provider.  This is because the  
primary key for ESDS queries is much more specific than the primary  
key for TISDAG queries.

Like TISDAG, ESDS should not give some information providers  
preferential treatment over other information providers, so long as  
information providers meet the registration requirements and can prove  
that they are a genuine provider of an information service.  For  
industrial applications, we will need to have strict controls about  
who can publish information to an ESDS (register with an ESDS as an  
information provider).

Like TISDAG, there may be a need for an ESDS to perform some pruning  
of the results retrieved from its internal Referral Index.  However,  
this pruning will often be done as a final step in the mechanism of  
enforcing access control policies, rather than primarily for  
intersecting result sets.

Differences between TISDAG and ESDS:
================================

Whereas TISDAG provides 'chaining' in which more detailed results are  
also retrieved from information providers for client protocols that  
cannot handle referrals, it is unlikely that Discovery Services will  
provide a similar 'chaining' mechanism to retrieve detailed  
information from multiple providers of information services, because  
of the additional burdens this would place on a DS in terms of network  
traffic, server load, thread management etc.

TISDAG is primarily concerned with providing referrals to multiple  
providers of current up-to-date contact information - i.e. current  
state.
ESDS is concerned with providing referrals to multiple providers of  
information services that hold not only current state (e.g. current  
location / last observation of an object) but which also maintain  
historical data, such as a detailed trace of observations made within  
each organization that encountered the specific object.  The time  
dimension of the records is much more significant in ESDS, since we  
are not just concerned with the time at which a record was last  
updated - but also an intrinsic timestamp associated with the event  
information.  A major application of ESDS is as if TISDAG were trying  
to maintain referrals to information about not only the current white  
pages contact details of an individual person - but also all of their  
previous addresses, such that an individual person is not represented  
by a single 'current state' record but by an audit trail of all of  
their previous postal addresses, previous e-mail addreses, telephone  
numbers etc. and the timestamps at which these were valid.

However, the primary key for an ESDS may be much more specific (often  
unique) than a query term (such as a first name, surname, role or  
organization) that forms the primary key to a TISDAG query.  This,  
together with the intrinsic timestamp associated with each record  
means that there is much greater commercial sensitivity about the very  
specific keys that ESDS uses, since these can reveal information about  
production volumes, shipments, flow patterns of goods.

While the TISDAG protocol makes little mention of discrimination  
between end-user clients such that different end-user clients will see  
different results for the same query performed at the same moment in  
time (or will see results that differ in their degree of  
completeness), it is highly likely that many ESDS implementations will  
need to enforce access control policies which directly result in  
different clients obtaining referral results that differ in  
completeness - and sometimes differ in content, depending on which  
identities a particular client is entitled to query/know about.

TISDAG is designed for a much lower total volume of records and  
(probably) lower retention times than some ESDS implementations may  
need to support.
TISDAG is designed for volumes of records of the order of 10 million  
individual people, with some future growth expected in the future.  In  
contrast, for the problem that ESDS is trying to solve, each ESDS  
instance may need to handle trillions of records per year for some  
industry sector applications that annually produce billions of items  
per product type - and in some situations, ESDS may need to support  
record retention periods up to 30-50 years, as well as archival and  
purging mechanisms for audit purposes.

TISDAG is particularly concerned with supporting different resource  
formats of information providers.  While ESDS should be able to  
support multiple kinds of information services, it is outside of the  
scope of ESDS to handle translation between different kinds of  
information service.  Given the progress in technologies such as web  
services and WSDL in particular, it may now be sufficient for an  
information service to be represented by a URL and a standardized  
descriptive serviceType label that indicates the kind of service to be  
expected.

ESDS protocol does not expect or require ESDS implementations to  
perform auto-indexing of information services that register with it.   
Each information service must explicitly post any event which it  
wishes to be contained in the ESDS referral index.  For this reason,  
ESDS will define methods for an explicit publishing interface, that  
will only be available to bona fide registered information providers.

Whereas the Mail DAG-CAP only supports a single query per e-mail  
message, ESDS may need to be able to support batch-mode queries and  
also batch-mode responses, in which a single message contains a query  
about multiple IDs  or responses for each of a list of multiple  
individual IDs.

Whereas TISDAG proposes to provide clients with a list of WDSP  
services that failed to respond to a chaining query, ESDS will  
probably not provide intrinsic support for chaining queries and may be  
required (by access control policies) to suppress the addresses (and  
identities) of WDSP services that decline to provide referral  
information to a specific client.  However, it may be helpful to  
nevertheless provide the client with an indication of the completeness  
of the referral list provided - and perhaps even support a mechanism  
through which a client can begin negotiations with an information  
service provider that chooses to decline access and also hide their  
address from clients they do not recognize/trust.

_______________________________________________
ESDS mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/esds