Subject: Re: draft-divilly-atompub-hierarchy-00

From: Nikunj Mehta <[email-masked]>
Date: Mon, 3 Nov 2008 08:48:00 -0800

On Oct 31, 2008, at 10:36 PM, Peter Keane wrote:

> I'm glad to see hierarchies in an I-D.  I have a few questions:
>
> 1. 3.1 states: "The kind of child Feed's metadata identifies the kind
> of new Entries that it can accept."  I am not entirely clear what that
> means -- is there a mechanism for determining the "kind" of metadata?

The metadata of the child Feed specifically includes app:collection if  
the Feed is modifiable. Moreover, if the Feed accepts new master or  
detail entries, it clearly indicates that through the app:accept  
element in its backing collection metadata. I can rework the phrasing  
you identify if that is confusing.

>
> Seems like a job for atom:category, and a did see that an example in
> 7.3 included an atom:category w/ scheme
> "http://finance.example.com/cat/kind" is that, in this case, the
> "kind" to which the sentence in 3.1 is referring?

The use of category is entirely application dependent and has no  
implications on the semantics identified in this specification

>
>
> 2. 3.1 states: "A master Feed MAY itself be a child of another master
> feed."  is that child-master also considered a "detail feed" of it's
> parent?  So would that feed be both a detail feed *and* and master
> feed? and the entry that defines it both a "master entry" and a
> "detail entry?"

The app:collection's app:accept metadata tells you whether a Feed is  
master or detail. The link@rel value does not confirm a feed to be  
master or detail. Moreover, a detail Feed is terminal, i.e., no new  
children Feeds are possible. On the other hand, a master Feed may  
contain child master Feeds. From the perspective of link@rel, that  
feed is still shown as "detail" for the master Feed.

A master entry MUST contain a link@rel="detail" feed. A detail entry  
MUST NOT contain a link@rel="detail" feed. That is really the only  
distinction between the two kinds of entries.

>
>
> 3. Can a master entry have more than one detail feed? If so, how does
> a client distinguish them?  (perhaps an atom:category child of the
> atom:link@rel=detail for each?)

A master Entry can have one detail Feed, each Entry of which can have  
their own detail Feeds. In this way, you can construct one:many  
relation between master and detail. This approach makes it possible to  
distinguish each such Feed through appropriate metadata in their  
respective master Entry. One possible approach is through the use of  
atom:category, but others are easily possible.

>
>
> I have a sense that this I-D is aimed primarily at creating a
> mechanism for managing feeds with feeds, using atompub, and perhaps
> less about creating a generic mechanism for describing sub-feeds in
> atom.

I don't quite understand the inference chain.  Can you please expand  
on it?

> Here the master entry is really a convenient atompub capable
> (since atompub focuses on entries, not feeds) "wrapper" for a feed w/
> in a feed.  Is that an accurate characterization?

AtomPub does not address the need to dynamically create new  
collections - that is a major issue for non-blog applications. This I- 
D makes minimal extensions to Atom for solving this issue.

>  A more generic
> mechanism for hierarchy in Atom might address multiple sub-feeds for
> an atom entry and would likely not require that a feed be labelled a
> "master" feed before the fact (i.e., any feed could have entries that
> contained other feeds or not, and need not declare that
> functionality/purpose up front).

This I-D allows any feed to define hierarchical relation between an  
entry and a feed using the link@rel=detail. There is no a-priori need  
for the feed containing the master entry to advertise that it is  
indeed a master feed. In fact, the only time this advertisement is  
required, is if the master feed is modifiable and it wishes to inform  
its clients that a new feed/collection is automatically created when a  
new entry is created.

>  I'm not suggesting this is good or
> bad, just want to get a clear understanding of the purpose.
>
> The implementation I work on has need for this exact sort of thing.

I am glad to hear that.

>  I
> have addressed it a fairly generic way of allowing an entry to have an
> atom:link@rel=service which points to an app:service document that
> describes the sub-feeds that this entry has available for posting to
> (typically, there is more than one).

AtomPub Service documents appear to be unsuitable for expressing large  
numbers of collections and for expressing hierarchies of feeds.  
Moreover, they are yet another abstraction for developers to deal with  
when an entry and a feed are mostly sufficient.

>
>
> --peter keane
>
>
> On Fri, Oct 31, 2008 at 5:10 PM, Nikunj Mehta  
> <[email-masked]> wrote:
>>
>> Fellow Atom and AtomPub'bers,
>>
>> We'd like to discuss the draft below that introduces extensions for  
>> Atom and
>> AtomPub to deal with hierarchies of feeds and the programmatic  
>> creation and
>> removal of AtomPub Collections.
>>
>> This draft defines a mechanism for identifying hierarchical master- 
>> detail
>> relations among data feeds so that consumer applications can perform
>> standard AtomPub operations on them. A hierarchical master-detail  
>> relation
>> of an Entry to a Feed implies the detail Feed is created when the  
>> master
>> Entry is created and the Feed is removed when the Entry is removed.  
>> The
>> Entry is called the "master entry" and the Feed is called "detail  
>> feed".
>> This relationship allows a client to use AtomPub to create a new  
>> Collection
>> by posting an Entry to an existing Collection.
>>
>> Comments?
>>
>> Colm Divilly & Nikunj Mehta
>>
>> ==============================================================
>>
>>  Title        : Hierarchy Extensions to Atom Feeds
>>  Author(s)    : C. Divlly and N. Mehta
>>  Filename    : draft-divilly-atompub-hierarchy-00.txt
>>  Pages        : 18
>>
>> This I-D is available at the following URL:
>>
>> http://www.oracle.com/technology/tech/feeds/spec/draft-divilly-atompub-hierarchy-00.html
>>
>> http://www.oracle.com/technology/tech/feeds/spec/draft-divilly-atompub-hierarchy-00.txt
>>

---

From: "Brian Smith" <[email-masked]>
Date: Mon, 3 Nov 2008 22:02:19 -0600

[ I trimmed the CC's since everybody atom-protocol is probably subscribed to
atom-syntax too. ]

Nikunj Mehta wrote:
> On Oct 31, 2008, at 9:37 PM, Brian Smith wrote:
> > What do you think about the search-based mechanism that I suggested
> > in the most recent "Hierarchical Collections" thread?
> 
> As an application developer, one has two choices in organizing items -
> like del.icio.us (with annotations) or like Unix (with structure). The
> approach you suggest in [1] sounds like the former. This approach
> requires (many times the same) metadata to be provided in every entry
> to identify its parent. However, that is not appropriate in all cases,
> especially where hierarchy is required by the application.

I don't understand why the CMIS-ish approach would not be appropriate when
hierarchy is required, since it explicitly models hierarchy. What the use
cases that you had in mind? 

> Additionally, the approach requires a pretty big leap from current
> standards to enable creation of child items since there is no means of
> identifying a collection to post to for each entry.

If the application doesn't already have a specific collection in mind, it
can usually just use the collection associated with the feeds listing the
related entries (using the same atom:feed/app:collection discovery mechanism
that you suggest).

> Moreover, the approach makes it a little cumbersome to add new
> metadata to each link, since you are limited to primarily new
> attributes. Of course, you can add elements to atom:link but then it
> becomes hard to also add an inline version of the feed.

The search + link-based approach doesn't require nesting feeds within
entries or entries within entries. Instead, atom:link is used for
everything--just like the traditional relational model, the result set is
always flat, and hierarchies and relationships between the results are
reconstructed based the explicitly defined links within the results. As soon
as you nested entries and feeds within other entries, you run into troubles
when the nested entries are edited; in order for clients to see changes to
the nested entries you have to update the outer entry's
app:edited/ETag/Last-Modified, even if the outer entry was not modified.
That destroys the cachability of the collection and blows up the cost of
resyncing with the collection.

> Search can be added in an orthogonal way to the I-D and it can include
> the closure capabilities you describe here. The I-D focuses on
> achieving a hypertext-driven solution to hierarchy and avoids template
> parsing and processing.

It seems to me that search is inevitable for all but the simplest AtomPub
servers. How searches are specified--with URI templates, HTML forms, XForms,
OpenSearch extensions, or something else--is up for debate. I chose URI
templates since most people here are familiar with them and since that it
what publically-available AtomPub(-ish) servers (like GData) are using. If
most servers are going to have a search interface, and that search interface
could be used to navigate hierarchies, then you don't really need a separate
mechanism specialized for navigating hierarchies.

The other thing I don't like about the hierarchical collections idea is that
it *requires* the server to implement a huge number of collections. Again,
that hurts the caching and the performance of collection syncing. And, that
is what creates the "what collection do I use" problem; if you only have one
collection for the application, you don't have to keep track of which one to
use. This simplicity also allows the client to resync with a large,
hierarchical collection with just a minimal number of requests.
 
And, at least as described in this draft, it results in entries that are not
self-describing. For example, using the draft's mechanism, given a link to
an entry, there is no way to find its master entry; you can only find the
link to the master from the detail feed that listed that entry. There is no
link from the entry to the detail feed either, so you have to remember what
feed(s) each entry was discovered from. That complicates the client-side
caching since you can't just reuse a normal HTTP cache. The search + link
approach allows the application to navigate up and down the hierarchy based
solely on the information encoded in each entry. At least in my experience
so far, a generic HTTP cache is highly effective with the search + link
model.

> atom:link is powerful enough to model all relations, at least for read-
> only purposes. This I-D is primarily extending Atom and AtomPub to
> populate the hierarchical relations. It would be useful to see what
> other read-write relations are of interest to this community.

OpenSocial's AtomPub-based protocol would be a good place to look for
examples.

One case so far where master/detail relationships have been important for my
application is where one entry's ACL has to be inherited by its child
objects. For example, think about a blog post. A blog post could have a lot
of images (media resources) embedded in it. But, you don't want the images
to be published until the entry has been published, and if you delete the
entry, you (usually) want the images to get deleted too. That means that
app:draft and any other kind of access control would be inherited by the
"detail" resources too. I'm sure a lot of uses of Atom outside of blog
publishing will have similar requirements for handling inherited traits.

Regards,
Brian

---

From: "Peter Keane" <[email-masked]>
Date: Tue, 4 Nov 2008 10:12:46 -0600

On Mon, Nov 3, 2008 at 10:48 AM, Nikunj Mehta <[email-masked]> wrote:
> The metadata of the child Feed specifically includes app:collection if the
> Feed is modifiable. Moreover, if the Feed accepts new master or detail
> entries, it clearly indicates that through the app:accept element in its
> backing collection metadata. I can rework the phrasing you identify if that
> is confusing.

I think it be helpful to specifically state that
app:collection/app:accept will identify the kind of new entries a
child feed can accept, which could, in fact, be detail OR master
entries (and that the absence of app:collection means this is simply a
"child" feed).

> implications on the semantics identified in this specification

I had thought/hoped so -- just wanted to make sure :-).

> The app:collection's app:accept metadata tells you whether a Feed is master
> or detail. The link@rel value does not confirm a feed to be master or
> detail. Moreover, a detail Feed is terminal, i.e., no new children Feeds are
> possible. On the other hand, a master Feed may contain child master Feeds.
> From the perspective of link@rel, that feed is still shown as "detail" for
> the master Feed.
>

I think this clears up some confusion I had -- when you state "A
detail Feed can only accept a new detail Entry," a feed's status as a
"detail feed" is determined solely by the
app:collection/app:accept@h:role value in that feed and is *not* by
the link@rel=detail of it's master entry.  In fact a child feed that
is a master feed will have a containing master entry that *does* point
to it with a link@rel=detail.  I guess my confusion stems from the
fact that the term "detail" is overloaded to mean "child" when used in
link@rel and when used in app:accept@h:role, "detail" declares a
specific type of feed w/ specific characteristics (e.g., "can only
accept a new detail Entry").

> A master entry MUST contain a link@rel="detail" feed. A detail entry MUST
> NOT contain a link@rel="detail" feed. That is really the only distinction
> between the two kinds of entries.
>
>>
>>
> A master Entry can have one detail Feed, each Entry of which can have their
> own detail Feeds. In this way, you can construct one:many relation between
> master and detail. This approach makes it possible to distinguish each such
> Feed through appropriate metadata in their respective master Entry. One
> possible approach is through the use of atom:category, but others are easily
> possible.
>

This seems to contradict "A detail entry MUST NOT contain a
link@rel="detail" feed," but I suspect that's due to the dual use of
"detail" in the sense I described above (?).

> I don't quite understand the inference chain.  Can you please expand on it?
>

The idea that link@rel=detail can point to *any* logical child feed
(master/detail/other) would suggest that it *is* actually a generic
mechanism for describing sub-feeds, I was mistaken there.

> AtomPub does not address the need to dynamically create new collections -
> that is a major issue for non-blog applications. This I-D makes minimal
> extensions to Atom for solving this issue.
>
>>  A more generic
> This I-D allows any feed to define hierarchical relation between an entry
> and a feed using the link@rel=detail. There is no a-priori need for the feed
> containing the master entry to advertise that it is indeed a master feed. In
> fact, the only time this advertisement is required, is if the master feed is
> modifiable and it wishes to inform its clients that a new feed/collection is
> automatically created when a new entry is created.
>

Excellent.

> numbers of collections and for expressing hierarchies of feeds. Moreover,
> they are yet another abstraction for developers to deal with when an entry
> and a feed are mostly sufficient.
>

Yes, I wholeheartedly agree.  I suppose (if my thinking is correct)
that it might be useful to distinguish between the term "detail" as
used in a link relation as opposed to its use in @h:role.

--peter keane

>>
>>
>> --peter keane
>>
>>
>> On Fri, Oct 31, 2008 at 5:10 PM, Nikunj Mehta <[email-masked]>
>> wrote:
>>>
>>> We'd like to discuss the draft below that introduces extensions for Atom
>>> and
>>> AtomPub to deal with hierarchies of feeds and the programmatic creation
>>> and
>>> This draft defines a mechanism for identifying hierarchical master-detail
>>> of an Entry to a Feed implies the detail Feed is created when the master
>>> Entry is created and the Feed is removed when the Entry is removed. The
>>> Entry is called the "master entry" and the Feed is called "detail feed".

---

From: Nikunj Mehta <[email-masked]>
Date: Tue, 04 Nov 2008 09:17:40 -0800

I guess the main outstanding issue from your comments below is the dual
use of the word "detail". When used in
app:collection/app:accept@h:role, it means that the feed/collection
only accepts detail entries. When used in atom:link@rel, it means that
the hyperlinked resource is a child feed. I am open to suggestions to
how you would like this addressed.

Nikunj

Peter Keane wrote:

the master Feed.

A more generic

Excellent.

I am glad to hear that.

I

--peter keane

--peter keane

Comments?

---

From: Colm Divilly <[email-masked]>
Date: Tue, 04 Nov 2008 18:41:55 +0000

Hi Brian,
 I find your approach quite elegant, but it requires clients to be able 
to understand uri-templates, our I-D enables clients to navigate 
hierarchies with requiring any knowledge of uri-templates. We wanted to 
do the absolute minimum to enable hierarchies to be modeled and 
therefore deliberately steered clear of any kind of 'search' functionality.

> cases that you had in mind? 
The use cases we have given most consideration to are leveraging 
Atom/AtomPub to provide a RESTful API for manipulating data in existing 
enterprise applications.
> resyncing with the collection.
Just to be clear, we don't require nested feeds to be inlined, generally 
I'd have a preference for keeping them out-of-line, but sometimes it can 
be more intuitive to have the detail entries nested in-line with the 
master entry.
> mechanism specialized for navigating hierarchies.
I don't agree, for existing enterprise applications the number of 
possible searches that the existing system can support is often very 
limited, and those 'searches' are better expressed with concrete urls 
rather than uri-templates.
> so far, a generic HTTP cache is highly effective with the search + link
I'm not sure I follow this, can you provide an example where it would be 
useful to discover the parent from the child, are you thinking something 
like Atom Threading Extensions ? In particular I don't follow how this 
complicates caching ?

Regards,
Colm

---

From: "Peter Keane" <[email-masked]>
Date: Tue, 4 Nov 2008 13:11:13 -0600

On Tue, Nov 4, 2008 at 11:17 AM, Nikunj Mehta <[email-masked]> wrote:
> I guess the main outstanding issue from your comments below is the dual use
> of the word "detail". When used in app:collection/app:accept@h:role, it
> means that the feed/collection only accepts detail entries. When used in
> atom:link@rel, it means that the hyperlinked resource is a child feed. I am
> open to suggestions to how you would like this addressed.
>

Yes, that's the only real issue.  Even having it stated clearly as
such is a help. I suppose I might opt for a more general link
relation, such as "child." I don't see the same issue with "master"
(works in h:role and link@rel just fine).

best-
peter keane