Subject: PaceFeedRecursive

From: "Roy T. Fielding" <[email-masked]>
Date: Sun, 9 Jan 2005 16:36:04 -0800

I just created a starting point for a proposal on making the
feed element recursive, eliminating the need for RDF syntax,
atom:head, and a bunch of things proposed in other paces regarding
aggregators and digital signatures.

   http://www.intertwingly.net/wiki/pie/PaceFeedRecursive

Unfortunately, I have a paper deadline on Tuesday and can't
procrastinate any longer, so someone else can finish the details
or I'll finish it later in the week.

Cheers,

Roy T. Fielding                            <http://roy.gbiv.com/>
Chief Scientist, Day Software              <http://www.day.com/>

---

From: Graham <[email-masked]>
Date: Mon, 10 Jan 2005 01:38:08 +0000

-1; Conceptual masturbation.

Graham

---

From: "Roy T. Fielding" <[email-masked]>
Date: Sun, 9 Jan 2005 20:10:51 -0800

> -1; Conceptual masturbation.

Wow, you like it that much?  It must be a really good idea, then,
since we are all just doing this to keep you stimulated.

....Roy

---

From: Danny Ayers <[email-masked]>
Date: Mon, 10 Jan 2005 12:43:48 +0100

>    http://www.intertwingly.net/wiki/pie/PaceFeedRecursive

I like it, but seem to remember a similar proposal being rejected - anyone..?

This approach should be pretty straightforward to program against, and
what's more should offer easy processing/data access through
XSLT/XQuery and simple mapping to RDF (essentially using striped
RDF/XML).

The only issue that springs to mind is that a feed could end up with a
huge amount of redundant chars if the same feeds/entries appeared
multiple times in different branches of the hierarchy. A simple
mechanism for substituting a feed/entry with its URI might do, or
alternately an in-document #id kind of cross reference to the first
occurence of the feed entry.

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Graham <[email-masked]>
Date: Mon, 10 Jan 2005 14:54:20 +0000

> since we are all just doing this to keep you stimulated.

Sarcasm and everything. Wow. Seriously, though, all you've done is not 
bothered to flatten the data at any point. While this may please you, 
all it means is the software at the far end has to wade through a few 
more sets of angle brackets and implement a couple of stacks. Don't 
think for a second that your format contains more data that the current 
flat format, because since it's all implicit, it's worthless. If say, 
entries appear in two separate groups in a feed, it might mean that 
they came from separate sources, or it might nor it might not mean 
anything. Why not just make it explicit?

Graham

---

From: "Bob Wyman" <[email-masked]>
Date: Mon, 10 Jan 2005 11:49:35 -0500

Roy T. Fielding wrote:
> a proposal on making the feed element recursive
	Equivalent proposals have often been made in the past (sometimes by
me!). These proposals fall in the category that we've come to know as "Feed
of Feeds". Each time the proposal has been made, it has been rejected. This
explains why "head in entry" is necessary -- since we can't have a "Feed of
Feeds", we need an alternative mechanism for associating Feed metadata with
entries that have been copied from other feeds.

	The primary arguments against "Feed of Feeds" have been (my
apologies if I forgot some...)
	1. Fear that an arbitrarily deep nesting of feeds might result in
excessively large feeds. (i.e. Feed of Feeds of Feeds of Feeds...)
	2. Belief that a need to handle nested feeds would increase
complexity for client developers. With HeadInEntry, clients are not required
to understand the sourcing issues, but have the necessary data available if
they wish to use it.
	3. Desire to simplify the migration path from RSS and Atom 0.3 to
Atom V1.0. The introduction of "Feed of Feeds" would result in a drastic
change in the structure of feeds that might require significant
modifications to the design of existing clients and thus tend to retard
efforts to move to the new format. While we should be completely open to
such drastic structural changes, we should only accept such changes if the
benefits of doing so are clear, compelling and overweigh the costs of
migration.
	4. Questions concerning attribution in interfaces. If an entry is
extracted from a feed nested within another feed, then to which feed should
the entry be associated in an interface? Or, should the interface attempt to
show the full "path" of the nesting hierarchy that contains the entry? With
"Head in Entry" it is likely that people will say that an entry "belongs" to
whatever feed it was found in. However, with "Feed of Feeds" it is more
likely that client developers and users will seek to associate the entry
with its source feed. (Note: This is a psychological or social issue, not a
technical one.) 
	5. Issues concerning partially nested feeds. For instance, one may
wish to incorporate a single entry from a feed into another feed. If the
approach is to simply nest the source feed, can a partial feed be nested? If
so, what feed metadata should be maintained in the nested feed? 
	6. Concern that "Feed of Feeds" really doesn't buy you much that
Head In Entry doesn't already provide in the normal case. One advantage
provided by Feed of Feeds is that feed metadata would not be repeated in the
case of multiple entries from a single feed being incorporated into another.
However, this may not be the normal case. The requirement for "feed of
feeds" is primarily driven by searching and matching engines and other entry
aggregators that tend not to experience enough locality of reference in
their results to have large numbers of results from a single feed. In this
case, the occasional repetition of feed meta data in <head/> elements is
probably acceptable.

	I may have forgotten some of the arguments, but this should provide
a good refresher and/or catch-up on the basic issues...

		bob wyman

---

From: Antone Roundy <[email-masked]>
Date: Mon, 10 Jan 2005 11:34:02 -0700

>   http://www.intertwingly.net/wiki/pie/PaceFeedRecursive
Perhaps instead of making feed recursive, it would be better to have 
one more kind of document element to be used for aggregate feeds.  For 
example:

<aggregation xmlns="...(atom)...">
	<feed>
		<head>...</head>
		<entry>...</entry>
	</feed>
	<feed>
		<head>...</head>
		<entry>...</entry>
	</feed>
	<feed>
		<head>...</head>
		<entry>...</entry>
	</feed>
</aggregation>

This would get rid of the problem of feeds of feeds of feeds..., but 
would raise a few questions:

* Does <aggregation> need a <head> and/or the kinds of things it 
contains>?  I'd say yes, and it's role would be just like the feed/head 
in an aggregate feed based on PaceHeadInEntry.

* If one aggregates <aggregation>s, does one need to carry forward data 
from aggregation/head, and if so, how does one do that?  I'd be willing 
to punt on this and say that it just can't be done.  Allowing it would 
be similar to reintroducing feeds of feeds of feeds...

On the question of eliminating atom:head, if we go this route, we could 
eliminate entry:head, but feed:head was created for reasons other than 
enabling it to be embedded in entries.

The rationale for the pace states:

> Simplifies description of the Atom syndication format to containers 
> (feed, entry, content) and metadata that targets its own parent (feed 
> or entry).
>
I've been mulling over the question of what the hierarchical placement 
of an element means, and came up with the following possibilities for 
what a child element might be:

1) metadata about its parent (eg. modification date of an entry).
2) metadata about its parent's content??? For example, does <author> 
name the author of an entry, or of it's content?  If we decide that 
metadata about the content itself must be an attribute in the content 
element itself, then we can eliminate this one.
3) the parent's content (eg. entries in a feed, content in an entry).
4) a container for other child elements just for convenience (eg. head 
in a feed)--which would mean that the container's children could be 
metadata about their grandparents, perhaps metadata about the 
grandparent's content, or the grandparent's content itself.  
Eliminating feed/head would eliminate this one.  But remember that 
feed/head predates PaceHeadInEntry--it was created for a different 
purpose, so we may not want to be too quick to toss it out.

That's a slightly different view than what you describe, but not a 
whole lot--assuming 2 and 4 are gone, elements are either metadata 
about their parent, or their parent's content.  Without understanding 
the format, one can't tell which a particular element is unless we did 
something like the following (which I am NOT advocating--I'm happy with 
applications having to know a little about the format if they care to 
distinguish):

<feed>
	...
	<entry isContent=1">
		<title>...</title>
		<link ... />
		...
		<content isContent="1">...</content>
	</entry>
</feed>

If we explicitly state that extension elements may only interact with 
other elements in their own namespace and with their immediate parents 
(ie. they may be metadata about or the content of their immediate 
parents, or may interact as defined in the extension with other 
elements from the extension), we eliminate problems with extension 
elements trying to influence the interpretation or handling of their 
siblings, and eliminate questions of inheritance from feed to entry 
(except with elements defined in the Atom core, a la <author>).

---

From: Isofarro <[email-masked]>
Date: Mon, 10 Jan 2005 19:12:14 +0000

>>a proposal on making the feed element recursive

(Speaking for myself, I'm not clued-up enough to understand the 
problem / use-case Roy's pace solves)

> of Feeds".

Agreed. I'm still none the wiser as to alternatives of handling 
Feed of Feed type scenarios. Using an entry element to describe 
a feed inside a feels rather hackish. Using an OPML/OML document 
feels like using two distinct vocabularies where one should do.

My intention is to create an Atom-enabled "space" that is 
navigatable. Resources are Atom Entries, Collections of 
resources are Atom Feeds. From a starting point of an Atom 
Entry, I would like to be able to follow a series of links to 
find other related or neighbourly Atom entries - something like 
going to the parent feed and find a sibling feed.

I find Roy's pace useful for what I want to do - have a way of 
expressing a collection of collections - feeds of feeds. For my 
purposes I can definitely live with one level of recursion or 
nesting:

<feed>
   <entry>...</entry>
   <feed>...</feed>
   <entry>...</entry>
</feed>

I can also live with the internal feed not having any entry 
elements - it will just be a pointer and metadata to another 
feed (presumably via a link element).

 From my perspective there was a related/relevant discussion 
going on in the Atom Protocol mailing list a few months ago 
about Collections and Resources. Its easy enough to see a 
collection as an Atom feed, and a Resource as an Atom entry, but 
I can't get my head around the correct way of representing a 
Collection of collections without looking at either a feed 
inside a feed, or an OPML/OML representation for each Collection 
of Collections hierachy I'm expecting.

If we are going to use feeds and entries to represent 
collections and resources, we should at least be able to express 
collections of collections with the same vocabulary.

> excessively large feeds. (i.e. Feed of Feeds of Feeds of Feeds...)

Limit the nesting to one level. (Feed inside a feed).

> to understand the sourcing issues, but have the necessary data available if

With the above limitation the complexity would be identical to 
handling an entry, except the link element definitively refers 
to an Atom Feed rather than an Atom Entry.

> efforts to move to the new format.

The additional work required is to handle a feed element at the 
same point as expecting an entry element. The metadata expected 
in the feed is roughly the same as a summary-only <entry> 
without the summary.

> the entry be associated in an interface?

The limitation above I guess disallows this particular situation.

> wish to incorporate a single entry from a feed into another feed.

Same as 4.)

> Head In Entry doesn't already provide in the normal case.

Just the ability to list feeds that are "filters" of the current 
feed. Taking del.icio.us as an example, the collection at
http://del.icio.us/tag/css would give me recent links with the 
css tag. I'd like to have the ability to refer to the collection
http://del.icio.us/tag/css+tutorial within that feed since it is 
a related feed - a more refined filter than the current feed. 
I'd also like to be able to link the other way - in the 
/tag/css+tutorial collection be able to refer to the "parent" 
collection /tag/css and /tag/tutorial.

For me this would be a nice to have. I don't demand it. 
(Although recommendations of alternative approaches would be 
very welcome)

Just my ha'penny.
Mike

(P.S. Does Roy Fielding's email domain name - gbiv.com have 
anything to do with the colours of the Rainbow, as in roygbiv 
[Red Orange Yellow Green Blue Indigo Violet])?

---

From: Danny Ayers <[email-masked]>
Date: Mon, 10 Jan 2005 20:59:37 +0100

> apologies if I forgot some...)

Thanks Bob. 

> Head In Entry doesn't already provide in the normal case. 

Ok, I don't think there is any benefit in the normal case (feed ::=
meta, (meta, content)*), but the recursive nesting method makes it
possible to convey information about relationships between feeds. 
Which begs the questions:

How useful are the semantics of one feed containing another?
How desirable is it to convey such information in Atom format?

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Robert Sayre <[email-masked]>
Date: Mon, 10 Jan 2005 15:17:29 -0500

Danny Ayers wrote:

>meta, (meta, content)*), 

It won't screw up DSig as badly. "Collectable Signatures"[0] will still 
require XPath, though.

Robert Sayre

[0] 
http://cvs.apache.org/viewcvs.cgi/xml-security/src_samples/org/apache/xml/security/samples/signature/CreateCollectableSignature.java?rev=1.9&view=markup

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 10 Jan 2005 13:39:02 -0800

On Jan 10, 2005, at 6:54 AM, Graham wrote:
> Seriously, though, all you've done is not bothered to flatten the
> data at any point.

Why is that a goal?  We know all of the data is in the leaves, so
"flattening" them is what happens when the entries are processed
depth-first.

> While this may please you, all it means is the software at the far end 
> has to wade through a few more sets of angle brackets and implement a 
> couple of stacks.

Actually, a few less angle brackets (count them), and any client
that tries to process XML without some awareness of hierarchy is
not my concern.  Design for the clients that matter.

> Don't think for a second that your format contains more data that
> the current flat format,

I would hope not -- the point was to make that data available such
that it would not have to be replicated in every entry.

> because since it's all implicit, it's worthless.

Implicit?  Did you mean inherited?  Hierarchical containment is
explicit in XML.

> If say, entries appear in two separate groups in a feed, it might
> mean that they came from separate sources, or it might nor it might
> not mean anything. Why not just make it explicit?

You are not talking about my proposal, or at least it was
fundamentally misunderstood.  Wait until it gets filled-out.

....Roy

---

From: Eric Scheid <[email-masked]>
Date: Tue, 11 Jan 2005 11:55:50 +1100

> How desirable is it to convey such information in Atom format?

Would not the element set which describes a resource be an entry, regardless
of whether that resource is an html page, image/jpg, rss 0.91 feed, atom
feed, etc?

With our <link> element, we are able to clearly communicate that a resource
we link to is of type application/atom+xml.

Surely, if someone wanted to build the equivalent of an OPML file in Atom
format, they could do so simply by creating an entry for each feed ... where
any of those feeds could also be of the same OPML equivalent class, thus
giving you deep nesting if you really want it. Alternatively, shallow
nesting is possible by the presence of multiple <link> elements within an
entry, possibly with @rel="DC.refined" or whatever the DC term is.

I note that the Nature Publishing Group already have something similar:

    http://npg.nature.com/pdf/newsfeeds.rdf

e.

---

From: Sam Ruby <[email-masked]>
Date: Thu, 13 Jan 2005 18:55:53 -0500

> or I'll finish it later in the week.

I realize that the proposal isn't flushed out, but a question nevertheless.

The proposal apparently is for feeds to contain other feeds by 
containment.  My question is whether it would make sense to also support 
feeds containing other feeds by reference, perhaps via a link element or 
via a src= attribute (analogous to content/@src defined in 5.12.2).

A use case can be found here:

http://www.decafbad.com/blog/2005/01/13/feed_playlists_versus_feed_urls

The only additional requirement I can see is that it might make sense to 
have a separate mime type for feeds which only reference other feeds, 
and feeds which contain content.

- Sam Ruby

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 13 Jan 2005 16:44:52 -0800

On Jan 13, 2005, at 3:55 PM, Sam Ruby wrote:
> containment.  My question is whether it would make sense to also 
> support feeds containing other feeds by reference, perhaps via a link 
> element or via a src= attribute (analogous to content/@src defined in 
> 5.12.2).

Yes, I was thinking the same would apply to entry, would it not?

> http://www.decafbad.com/blog/2005/01/13/feed_playlists_versus_feed_urls

Also, the Nature feed of feeds that has been referenced a couple times.

> The only additional requirement I can see is that it might make sense 
> to have a separate mime type for feeds which only reference other 
> feeds, and feeds which contain content.

Hmm, my employer is fond of saying "everything is content" -- feeds
are just another name for collections, and most systems agree that
collections create an association between members which is content
in itself.

....Roy

---

From: Walter Underwood <[email-masked]>
Date: Thu, 13 Jan 2005 17:58:40 -0800

--On Thursday, January 13, 2005 06:55:53 PM -0500 Sam Ruby <[email-masked]> wrote:
>
> The proposal apparently is for feeds to contain other feeds by containment.
> My question is whether it would make sense to also support feeds containing
> other feeds by reference, perhaps via a link element or via a src= attribute
> (analogous to content/@src defined in 5.12.2).

As someone maintaining a spider and search engine, I'm -1 on inclusion
by reference. The additional complexity is just not worth it.

If a referenced doc is a 404, is the feed state OK? Should I index it
or not? Do I always need to re-visit referred docs when I revisit the
main one? What do I do with loops? Do I index all the content as
belonging to the main doc? What if that is included in something else
by reference, can it still be a standalone item?

For the same reasons, I'm not very hot on any sort of content by reference.
But recursive feeds by reference opens up lots more issues.

Basically, if you want your content in a search engine, it had better
be accessible with a single GET.

wunder
--
Walter Underwood
Principal Architect
Verity Ultraseek

---

From: Henry Story <[email-masked]>
Date: Wed, 19 Jan 2005 10:38:04 +0100

How is this coming along?

I think the current spec really would do with some good simplification. 
We have
RSS1.1 to look at now, at it does look a hell of a lot simpler. (Don't 
want to
start a RSS1.1 thread here please)

I think the easiest way to get what you want is a 2 step procedure:

   1. Merge the Head with the Entry constructs. They are not different 
enough
     for the difference to be important.
   2. Make a Feed a subclass of Entry, with the extra property of being 
able
      to point to a Entry (Since a Feed is a Entry, it also points to 
other Feeds)

I am  +1 on such a refactoring. Nothing will be lost doing this, and a 
lot gained
in simplicity.

Henry