Subject: Feed History -02

From: Mark Nottingham <[email-masked]>
Date: Sat, 16 Jul 2005 08:57:50 -0700

The Feed History draft has been updated to -02;

   http://ftp.ietf.org/internet-drafts/draft-nottingham-atompub-feed- 
history-02.txt

The most noticeable change in this version is the inclusion of a  
namespace URI, to allow implementation.

I don't intend to update it for a while, so as to gather  
implementation feedback.

Cheers,

--
Mark Nottingham     http://www.mnot.net/

---

From: Stefan Eissing <[email-masked]>
Date: Mon, 18 Jul 2005 09:59:56 +0200

> I don't intend to update it for a while, so as to gather  

Just a couple of thoughts on reading the document:

Ch 3. fh:stateful seems to be only needed for a newborn stateful feed.  
As an alternative one could drop fh:stateful and define that an empty  
fh:prev (refering to itself) is the last document in a stateful feed.  
That would eliminate the cases of wrong mixes of fh:stateful and  
fh:prev.

Ch 5. inserting pseudo-entries into an incomplete feed: would it make  
sense to have a general way to indicate such pseudo entries? A feed  
entry can also get lost at the publisher and the publisher might want  
to indicate that there once was a feed entry, but that he no longer has  
the (complete) document.

//Stefan

Random thoughts:
The example of a "top 10" feed (Ch 1) needs some thinking: there are  
quite some people interested in the history of "top 10" when it comes  
to music charts. One _could_ make this an atom feed and use the feed  
history to go back in time. But the underlying model is different from  
the one atom has, so maybe its not such a good idea after all. (Is  
there any ordering in a feed, btw.? I know a client can sort by date,  
but does someone rely on document order of xml elements?)

---

From: James M Snell <[email-masked]>
Date: Mon, 18 Jul 2005 08:10:14 -0700

Stefan Eissing wrote:

> Ch 3. fh:stateful seems to be only needed for a newborn stateful 
> feed.  As an alternative one could drop fh:stateful and define that an 
> empty  fh:prev (refering to itself) is the last document in a stateful 
> feed.  That would eliminate the cases of wrong mixes of fh:stateful 
> and  fh:prev.
>
+1. After going through this, fh:stateful really doesn't seem to be 
necessary.  the presence of fh:prev would be sufficient to indicate that 
the feed has a history and a blank fh:prev would work fine to indicate 
the end of the history.

- James

---

From: Henry Story <[email-masked]>
Date: Mon, 18 Jul 2005 18:32:45 +0200

Sorry I did not participate in the previous discussion for format 00.  
I only just
realized this was going on. What is clear is that this is really needed!

I agree with Stefan Eissing's random thought that it may not be a  
good idea to use
Atom for a "top 10" feed. Atom entries are not ordered in a feed for  
one. Also as I
understand it an entry in a feed is best thought of as a state of an  
external resource
at a time. Making a feed of the top x entries is to use the feed as a  
closed collection
whereas I think it is correctly interpreted as an open one.

If that is right, and so fh:stateful is not needed, then would it not  
be simpler to
extend the link element in the following really simple way:

<link rel="http://purl.org/syndication/history/1.0" type="application/ 
atom+xml"
       href="http://example.org/2003/11/index.atom" />

Just a thought.

In any case I really look forward to having this functionality.  
Thanks a lot for the
huge effort you have put into presenting this idea so clearly and  
with such patience.

Henry Story

On 18 Jul 2005, at 09:59, Stefan Eissing wrote:
>>   http://ftp.ietf.org/internet-drafts/draft-nottingham-atompub- 
>> feed-history-02.txt
>>
> feed. As an alternative one could drop fh:stateful and define that  
> an empty fh:prev (refering to itself) is the last document in a  
> stateful feed. That would eliminate the cases of wrong mixes of  
> fh:stateful and fh:prev.
>
> Ch 5. inserting pseudo-entries into an incomplete feed: would it  
> make sense to have a general way to indicate such pseudo entries? A  
> feed entry can also get lost at the publisher and the publisher  
> might want to indicate that there once was a feed entry, but that  
> he no longer has the (complete) document.
>
> //Stefan
>
> Random thoughts:
> The example of a "top 10" feed (Ch 1) needs some thinking: there  
> are quite some people interested in the history of "top 10" when it  
> comes to music charts. One _could_ make this an atom feed and use  
> the feed history to go back in time. But the underlying model is  
> different from the one atom has, so maybe its not such a good idea  
> after all. (Is there any ordering in a feed, btw.? I know a client  
> can sort by date, but does someone rely on document order of xml  
> elements?)
>

---

From: James M Snell <[email-masked]>
Date: Mon, 18 Jul 2005 09:46:53 -0700

> whereas I think it is correctly interpreted as an open one.
I disagree. Atom could be used very easily for a top 10 feed.  What is 
needed is a simple extension that provides rank orderings for entries.  
Something as simple as the following would work...

<feed ...>
  ...
  <entry>
    ...
    <r:index>1</r:index>
  </entry>
  <entry>
    <r:index>2</r:index>
  </entry>
  <entry>
    <r:index>3</r:index>
  </entry>
</feed>

>       href="http://example.org/2003/11/index.atom" />
I actually can't what my opinion on this used to be :-( but right now 
I'm thinking that a custom link relation is the right approach.

- James

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 18 Jul 2005 12:55:57 -0400

On 18/07/2005, at 11:10 AM, James M Snell wrote:
>> feed.  As an alternative one could drop fh:stateful and define  
>> that an empty  fh:prev (refering to itself) is the last document  
>> in a stateful feed.  That would eliminate the cases of wrong mixes  
>> of fh:stateful and  fh:prev.
>>
>>
> necessary.  the presence of fh:prev would be sufficient to indicate  
> that the feed has a history and a blank fh:prev would work fine to  
> indicate the end of the history.

I thought about the comments on the plane yesterday, and I agree.  
However, I'm wary of special URI values; also, I want to preserve  
stateful=false.

So, what about saying that you can omit fh:stateful *if* fh:prev is  
in the feed?

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 18 Jul 2005 12:58:12 -0400

That's what I originally did, but I have a rather strong preference  
to make a single syntax work in RSS and Atom. atom:link is  
(naturally) specific to Atom, and people will balk at using the atom  
namespace in RSS feeds.

That's not to say that every Atom extension should be usable in RSS,  
but I think this one is simple -- and valuable -- enough to do it;  
and, I don't see any technical benefit to using the link relation.

Cheers,

On 18/07/2005, at 12:32 PM, Henry Story wrote:

> If that is right, and so fh:stateful is not needed, then would it  
> not be simpler to
> <link rel="http://purl.org/syndication/history/1.0"  
> type="application/atom+xml"

---

From: James M Snell <[email-masked]>
Date: Mon, 18 Jul 2005 09:59:29 -0700

> So, what about saying that you can omit fh:stateful *if* fh:prev is  
I would say that if fh:prev is present, stateful=true is assumed.  if 
fh:prev is not present, stateful=false is assumed.  Omit fh:prev in the 
final feed in the chain and you know you've reached the end.

- James

---

From: James M Snell <[email-masked]>
Date: Mon, 18 Jul 2005 10:03:41 -0700

There is precedence for using atom:link in RSS feeds.  see 
http://feeds.feedburner.com/ITConversations-EverythingMP3.  I really 
don't think it's a problem.

---

From: Stefan Eissing <[email-masked]>
Date: Mon, 18 Jul 2005 19:29:39 +0200

Am 18.07.2005 um 18:59 schrieb James M Snell:

> fh:prev is not present, stateful=false is assumed.  Omit fh:prev in  
> the final feed in the chain and you know you've reached the end.

stateful gives a hint to a client about caching entries and maybe their  
representation in a user interface. It may be desired to see that a new  
feed is stateful(or not) even if it has no history yet. That is why I  
came up with the empty prev link as a suggestion.

I agree that "special" URIs are not that great either. Another idea  
might be nested elements:

stateful feed:   
<fh:history><fh:prev>http://example.org/thingie1.1</fh:prev></fh: 
history>
stateful initial feed: <fh:history/>
stateless feed: <fh:history><fh:none/></fh:history>

So much for the syntactic sugar...

//Stefan

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 18 Jul 2005 13:33:51 -0400

On 18/07/2005, at 1:29 PM, Stefan Eissing wrote:
> stateful feed:  <fh:history><fh:prev>http://example.org/thingie1.1</ 
> fh:prev></fh:history>

Hmm. My thinking was that allowing stateful to be omitted would be  
concise and unambiguous; to compare,

stateful feed: <fh:prev>http://example.org/thingie1.1</fh:prev>
stateful initial feed: <fh:stateful>true</fh:stateful>
stateless feed: <fh:stateful>false</fh:stateful>

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 18 Jul 2005 13:35:55 -0400

Not a problem, as such, but I don't see any benefit to reuse. It also  
begs the question of what an atom element in a non-Atom document  
means, how it's processed, etc.

On 18/07/2005, at 1:03 PM, James M Snell wrote:

> There is precedence for using atom:link in RSS feeds.  see http:// 
> feeds.feedburner.com/ITConversations-EverythingMP3.  I really don't  
> think it's a problem.

---

From: Stefan Eissing <[email-masked]>
Date: Mon, 18 Jul 2005 19:43:37 +0200

> stateless feed: <fh:stateful>false</fh:stateful>

Fine with me. As I said the discussion has reached the "syntactic  
sugar" level and your proposal has the same semantics and no nested  
elements. To be clear I would advise clients that fh:prev takes  
precedence over any fh:stateful information and then any ambiguity is  
resolved.

//Stefan

---

From: James M Snell <[email-masked]>
Date: Mon, 18 Jul 2005 10:42:54 -0700

Heh... the same questions could be asked about a lot of stuff embedded 
in RSS but that's not the issue ;-) ... fh:prev works fine.  There 
really isn't a strong argument in favor of link.  I have my own personal 
preferences but those are actually quite irrelevant :-)    I'll still 
maintain that fh:stateful is quite unnecessary but doesn't hurt anything 
if it's used -- you'll just need to be explicit about such things as 
what if a fh:prev is used without a fh:stateful being present.

---

From: James M Snell <[email-masked]>
Date: Mon, 18 Jul 2005 11:04:40 -0700

FYI: http://www.snellspace.com/wp/?p=193

Abstract: Proposes two new elements that allow entries to be ordered 
within a feed.  Full details in my blog post.

<feed xmlns="http://www.w3.org/2005/Atom"
         xmlns:o="http://www.snellspace.com/atom/extensions/proposed">
  ...
  <o:indexed />
  <entry>
    ...
    <o:index>1</o:index>
  </entry>
  <entry>
    ...
    <o:index>2</o:index>
  </entry>
  <entry>
    ...
    <o:index>3</o:index>
  </entry>
</feed>

- James

---

From: Stefan Eissing <[email-masked]>
Date: Mon, 18 Jul 2005 20:17:19 +0200

On a more semantic issue:

The described sync algorithm will work. In most scenarios the abort 
condition (e.g. all items on a historical feed are known) will also do 
the job. However this still means that clients need to check the first 
fh:prev document if they know all entries there - if my understanding 
is correct.

Is it worthy to think of something to spare clients and servers this 
lookup? Are the HTTP caching and If-* header mechanisms good enough to 
save network bandwidth?

An alternate stragety would be to require that fh:prev documents never 
change once created. Then a client can terminate the sync once it sees 
a URI it already knows. And most clients would not do more lookups than 
they are doing now...

//Stefan

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 18 Jul 2005 17:21:41 -0400

On 18/07/2005, at 2:17 PM, Stefan Eissing wrote:

> condition (e.g. all items on a historical feed are known) will also  
> do the job. However this still means that clients need to check the  
> first fh:prev document if they know all entries there - if my  
> understanding is correct.

This is one of the unanswered questions that I left out of scope. The  
consumer can examine the previous archive's URI and decide as to  
whether it's seen it or not before, and therefore avoid fetching it  
if it already has seen it. However, in this approach, it won't see  
changes that are made in the archive (e.g., if a revision -- even a  
spelling correction -- is made to an old entry); to do that it either  
has to walk back the *entire* archive each time, or the feed has to  
publish all changes -- even to old entries -- at the head of the feed.

I left it out because it has more to do with questions about entry  
deleting and ordering than with recovering state. it's an arbitrary  
decision (I had language about this in the original Pace I made), but  
it seemed like a good trade-off between complexity and capability.

Does that make sense, or am I way off-base?

> Is it worthy to think of something to spare clients and servers  
> this lookup? Are the HTTP caching and If-* header mechanisms good  
> enough to save network bandwidth?
>
> An alternate stragety would be to require that fh:prev documents  
> never change once created. Then a client can terminate the sync  
> once it sees a URI it already knows. And most clients would not do  
> more lookups than they are doing now...

---

From: "A. Pagaltzis" <[email-masked]>
Date: Tue, 19 Jul 2005 01:52:35 +0200

* Mark Nottingham <[email-masked]> [2005-07-18 23:30]:
> This is one of the unanswered questions that I left out of
> scope. The  consumer can examine the previous archive's URI and
> decide as to  whether it's seen it or not before, and therefore
> avoid fetching it  if it already has seen it. However, in this
> approach, it won't see  changes that are made in the archive
> (e.g., if a revision -- even a  spelling correction -- is made
> to an old entry); to do that it either  has to walk back the
> *entire* archive each time, or the feed has to  publish all
> changes -- even to old entries -- at the head of the feed.

These are the kinds of things my “hub archive feed” situation was
supposed to address. Because the links are all in one place, the
consumer only has to suck down one document in order to be
informed of all archive feeds and being able to decide which ones
he wants to re-/get.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>

---

From: Stefan Eissing <[email-masked]>
Date: Tue, 19 Jul 2005 10:48:34 +0200

Am 18.07.2005 um 23:21 schrieb Mark Nottingham:
> whether it's seen it or not before, and therefore avoid fetching it if 
> it already has seen it. However, in this approach, it won't see 

It is a valid starting point. I am just wondering what consequences it 
has on client implementations. Let's say CNN goes stateful, how would a 
client handle a history which soon consists of thousands of entries. 
How would a server best offer such a history to avoid clients 
retrieving it over and over again. Probably nobody has a good idea on 
that one, or?

I have the feeling that clients will need to protect themselves from 
servers with almost infinite histories. So a client will probably offer 
a XX days into the past, max NN entries setting in its UI. Maybe that 
is all that's needed.

How about:
"In case feeds are served via HTTP, server implemenations SHOULD offer 
ETag and Last-Modified headers on history documents (see RFC 2616 xxx). 
Clients SHOULD persist ETag and Last-Modified information and use If-* 
headers to ease server load on history synchronization."

//Stefan

---

From: Henry Story <[email-masked]>
Date: Tue, 19 Jul 2005 11:04:02 +0200

On 18 Jul 2005, at 23:21, Mark Nottingham wrote:
>> The described sync algorithm will work. In most scenarios the  
>> abort condition (e.g. all items on a historical feed are known)  
>> will also do the job. However this still means that clients need  
>> to check the first fh:prev document if they know all entries there  
>> - if my understanding is correct.
>>
>
> This is one of the unanswered questions that I left out of scope.  
> The consumer can examine the previous archive's URI and decide as  
> to whether it's seen it or not before, and therefore avoid fetching  
> it if it already has seen it. However, in this approach, it won't  
> see changes that are made in the archive (e.g., if a revision --  
> even a spelling correction -- is made to an old entry); to do that  
> it either has to walk back the *entire* archive each time, or the  
> feed has to publish all changes -- even to old entries -- at the  
> head of the feed.

Clearly the archive feed will work best if archive documents, once  
completed (containing a
given number of entries) never change. Readers of the archive will  
have a simple way to know when
to stop reading: there should never be a need to re-read an archive  
page - they just never change.

The archive provides a history of the feed's evolution. Earlier  
changes to the resources
described by the feed will be found in older archive documents and  
newer changes in the later
ones. One should expect some entries to be referenced in multiple  
archive feed documents. These
will be entries that have been changed over time.

Archives *should not* change. I think any librarian will agree with  
that.

> decision (I had language about this in the original Pace I made),  
> but it seemed like a good trade-off between complexity and capability.
>
>> An alternate stratgey would be to require that fh:prev documents  

I think this would be the correct strategy.

Henry Story

---

From: Henry Story <[email-masked]>
Date: Tue, 19 Jul 2005 11:24:04 +0200

> informed of all archive feeds and being able to decide which ones

I wonder if what you are trying to describe here is not a different
concept altogether from an archive feed. I guess that both are  
completely
orthogonal concepts.

Feeds tend to specialize in a number of resources they track. What would
also be useful would be a document that  described the resources   
tracked by a
feed. This would be closer to a directory listing. It would help  
point to the
current state of the resources tracked by the feed.

So when one subscribed to a feed one could then quickly get a list of  
all
the resources that the feed had responsibility for. As this could be  
quite large
some form of navigation may be necessary. Perhaps this is the type of  
thing
that the protocol group is working on.

Henry Story

---

From: Antone Roundy <[email-masked]>
Date: Tue, 19 Jul 2005 12:29:35 -0600

> That would eliminate the cases of wrong mixes of fh:stateful and 
The problem is that an empty @href in fh:prev is subject to xml:base 
processing, and who knows what the current xml:base is going to be when 
you get to it.  Is there a way to explicitly make xml:base undefined?  
If I'm not mistaken xml:base="" doesn't do it--it just adds nothing to 
the existing xml:base.  If there is a way, you could say <link 
rel="fh"prev" href="" xml:base="[whatever value sets it to 
"undefined"]" />, but otherwise, using an empty @href is probably 
overloading the wrong attribute.  A different @rel value like 
"fh:noprev" (with an empty link, since it doesn't matter what it 
actually points to) might be a step up, but using any kind of link to 
indicate the lack of a link is a little odd.

---

From: Antone Roundy <[email-masked]>
Date: Tue, 19 Jul 2005 13:01:11 -0600

On Tuesday, July 19, 2005, at 12:29  PM, Antone Roundy wrote:
> processing, and who knows what the current xml:base is going to be 
> when you get to it.  Is there a way to explicitly make xml:base 
> undefined?  If I'm not mistaken xml:base="" doesn't do it--it just 
> adds nothing to the existing xml:base.  If there is a way, you could 
> say <link rel="fh"prev" href="" xml:base="[whatever value sets it to 
Yikes, I should have caught up on the xml:base thread first!  Looks 
like the jury's out, or at least hung, on this issue.

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 21 Jul 2005 07:13:22 -0700

On 19/07/2005, at 1:48 AM, Stefan Eissing wrote:

>  Let's say CNN goes stateful, how would a client handle a history  
> which soon consists of thousands of entries. How would a server  
> best offer such a history to avoid clients retrieving it over and  
> over again. Probably nobody has a good idea on that one, or?
>
> I have the feeling that clients will need to protect themselves  
> from servers with almost infinite histories. So a client will  
> probably offer a XX days into the past, max NN entries setting in  
> its UI. Maybe that is all that's needed.

Good question. I was thinking roughly along these lines in the  
Security Considerations, but didn't want to lead people too much.

> How about:
> "In case feeds are served via HTTP, server implemenations SHOULD  
> offer ETag and Last-Modified headers on history documents (see RFC  
> 2616 xxx). Clients SHOULD persist ETag and Last-Modified  
> information and use If-* headers to ease server load on history  
> synchronization."

Hmm. Maybe not SHOULDs, but some prose might lead people in the right  
direction, perhaps an 'Implementer Notes' section?

---

From: Stefan Eissing <[email-masked]>
Date: Fri, 22 Jul 2005 13:54:45 +0200

Am 21.07.2005 um 16:13 schrieb Mark Nottingham:

> [...]

> servers with almost infinite histories. So a client will probably 
> offer a XX days into the past, max NN entries setting in its UI. Maybe 
> that is all that's needed.
>
>> 2616 xxx). Clients SHOULD persist ETag and Last-Modified information 
>> and use If-* headers to ease server load on history synchronization."
>

Ok, a SHOULD is not really required. Let common sense prevail.

The remaining question for me is if the fh:prev links are pointing to 
history records or if they are just a way to split a large feed into 
chunks.

The more I think about it, the more I lean towards leaving the spec as 
is. Since clients do not trust servers very much, a clever client will 
most likely implement a synch strategy which frequently checks if the 
prev uris change and once a day/after a week absence traverse the prev 
chain and retrieve all documents.

//Stefan

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 23 Jul 2005 09:14:23 -0700

> Archives *should not* change. I think any librarian will agree with  

I very much agree that this is the ideal that should be striven for.

However, there are some practical problems with doing it in this  
proposal.

First of all, I'm very keen to make it possible to implement history  
with currently-deployed feed-generating software; e.g., Moveable  
Type. MT isn't capable of generating a feed where changed entries are  
repeated at the top out of the box, AFAIK.

Even if it (and other software) were, it would be very annoying to  
people whose feed software doesn't understand this extension; their  
"show me the latest entries in the blog" feed would become "show me  
the latest changed entries in the blog", and every time an entry was  
modified or spell-checked, it would show up at the top.

So, it's a matter of enabling graceful deployment. Most of the reason  
I have the fh:stateful flag in there is to allow people to explicitly  
say "I don't want you to do history on this feed" because so many  
aggregators are already doing history in their own way.

The underlying problem, I think, is that different feeds have  
different semantics. Some will want every change to be included,  
others won't; for example, a blog probably doesn't need every single  
spelling correction propagated. There are some fundamental questions  
about the nature of a feed that need to be answered (and, more  
importantly, agreed upon) before we get there; for example, we now  
say that the ordering isn't significant by default; while that's  
nice, most software is going to infer something from it, so we need  
an extension to say 'sort by this', *and* have that extension widely  
deployed.

I tried to approach these problems when I wrote the original proposal  
for this in Pace form; I got strong pushback on defining a single  
model for a feed's state. Given that, as well as the deployment  
issues, I intentionally de-coupled the state reconstruction (this  
proposal) from the state model (e.g., ordering, deletion, exact  
semantics of an archive feed, etc.), so that they could be separately  
defined.

Cheers,

---

From: Dave Pawson <[email-masked]>
Date: Sun, 24 Jul 2005 09:17:33 +0100

> different semantics.

I think that's the bottom line Mark. No matter what, people
are going to do different things with 'history'.

Trying to pin down how it should all work seems to increase
the complexity by an order, never a good thing IMHO.

How about, you want my history, you make of it what you will.
I only guarantee its valid atom?
 At least that way, processing older feed material can 
develop based on a sound (and clearly understood) foundation.

regards DaveP

---

From: Henry Story <[email-masked]>
Date: Fri, 29 Jul 2005 11:57:23 +0200

I get the feeling that we should perhaps first list the main types of  
history archives, and
then deal with each one separately. I can see 3:

     1- The top 20 list: here one wants to move to the previous top  
20 list and think of them
        as one thing. The link to the next feed is not meant to be  
additive. Each feed is to
        be seen as a whole. I have a little trouble still thinking of  
these as feeds, but ...
     2- The history of changes link: the idea here is that a feed is  
a list of state changes
       to web resources. The link to the history just increases the  
length of the feed by
       pointing to the next feed page. These two pages could have  
been merged to create one
       feed page with the same end result. By moving back through the  
feed history one gets
       to see the complete history of changes to the resources  
described by the feed.
     3- A listing of the current states of the resources: here the  
end user is searching for
       all the blog entries as they currently are. No feed id is ever  
duplicated across the chain.

I think each of these has its uses.
  1. this use case is clear and well understood.
  2. is very useful for 'static' blog feeds written from a remote  
server, as once a feed has
     been archived it no longer needs to be rewritten. It is also  
very useful in helping a
     client synchronize with the changes to the feed. If a client is  
off line for a few months
     or if there are a lot of changes in a feed then the client just  
needs to follow this type
     of link to catch up with all the changes. If the entries are  
describing state changes to
     resources such as stock market valuations, then this would allow  
one to get all the
     valuations for a company going back in time.
  3. It is easier for this to be generated dynamically upon request.  
It is not really a history
     of changes, but rather a list of resources. In a feed concerning  
a blog these two can of
     course easily be confused as we think of blogs as being created  
sequentially. We then think
     of the feed history as the history of the blog entry publication  
dates. If an old blog entry
     gets changed then the page on which that blog entry originally  
appeared in the feed would
     get changed just as the permalink to the blog post itself gets  
changed.
     This type of list is of course not very useful in keeping  
someone updated as to the changes
     in a feed, as the whole linked feeds would need to be stepped  
through to see if anything has
     changed.
     I think it is with this version that the proposals of having a  
separate document that points
     to all the entries makes most sense. And this case also seems to  
be worked upon in the API
     I think.

more below...

On 23 Jul 2005, at 18:14, Mark Nottingham wrote:
>> to stop reading: there should never be a need to re-read an  
>> archive page - they just never change.
>>
>> Archives *should not* change. I think any librarian will agree  
>> with that.
>>
>
> First of all, I'm very keen to make it possible to implement  
> history with currently-deployed feed-generating software; e.g.,  
> Moveable Type. MT isn't capable of generating a feed where changed  
> entries are repeated at the top out of the box, AFAIK.

So is it that MT can only generate feed links such as describe in 3  
above?

> the latest changed entries in the blog", and every time an entry  
> was modified or spell-checked, it would show up at the top.

Well in many cases this is exactly what I want. If someone makes a  
big change to a resource that
was created one year ago (say your first blog post) then I think this  
is well worth knowing about.
Especially if I have written a comment about it. Of course if the  
change is not *significant* then
the change to the old feed archive would not be significant either.  
So perhaps I should have said
that archives should not change in any *significant* way.

> So, it's a matter of enabling graceful deployment. Most of the  
> reason I have the fh:stateful flag in there is to allow people to  
> explicitly say "I don't want you to do history on this feed"  
> because so many aggregators are already doing history in their own  
> way.

And I agree. The way you dealt with that problem was by adding a  
special flag for that case.
We can create a special flag for the other cases too...

> others won't; for example, a blog probably doesn't need every  
> single spelling correction propagated. There are some fundamental  
> questions about the nature of a feed that need to be answered (and,  
> more importantly, agreed upon) before we get there; for example, we  
> now say that the ordering isn't significant by default; while  
> that's nice, most software is going to infer something from it, so  
> we need an extension to say 'sort by this', *and* have that  
> extension widely deployed.

yes. But we don't have to answer all questions. We can just decide on  
some problem we want
to solve, delimit it clearly and solve it. No?

> I tried to approach these problems when I wrote the original  
> proposal for this in Pace form; I got strong pushback on defining a  
> single model for a feed's state. Given that, as well as the  
> deployment issues, I intentionally de-coupled the state  
> reconstruction (this proposal) from the state model (e.g.,  
> ordering, deletion, exact semantics of an archive feed, etc.), so  
> that they could be separately defined.

It there a place where these problems are clearly layed out so that I  
can get a comprehensive
view of them?

Thanks,

     Henry Story

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Jul 2005 21:12:27 +1000

On 29/7/05 7:57 PM, "Henry Story" <[email-masked]> wrote:

> 1- The top 20 list: here one wants to move to the previous top  20 list and
> think of them as one thing. The link to the next feed is not meant to be
> additive. Each feed is to be seen as a whole. I have a little trouble still
> thinking of  these as feeds, but ...

What happens if the publisher realises they have a typo and need to emit an
update to an entry? Would the set of 20 entries (with one entry updated) be
seen as a complete replacement set?

The way I see it, maybe a better way would be to have a sliding window feed
where each entry points to another Atom Feed Document with it's own URI, and
it is that second Feed Document which contains the individual items (the top
20 list).

Someone could subscribe to that second feed and poll for updates, and all
they'll ever see are updates to the 20 items there, not the 20 items from
the next week/whatever.

The idea of feeds linked to feeds has lots of utility -- feeds of comments
for one, and even a feed of feeds available on the site.

For example: this HTML page <http://www.nature.com/rss/> has an equivalent
feed document <http://npg.nature.com/pdf/newsfeeds.rdf>, where each item
links to the individual feeds for each publication. That feed doesn't update
often, mostly because NPG doesn't add many new feeds to their site all that
often. The URIs the entries of that feed link to are redirected to the
permanent URIs of the current issue (each issue has it's own feed which is
the table of contents for that issue, with a distinct and separate URI for
each issue).

It's conceivable they could also provide a feed for each publication
pointing to the table of contents feeds of each issue. That is, a feed with
an entry for each issue.

Of the above, the mechanism of a single URI which redirects to the current
issue is a situation which would still need a flag indicating that the
appropriate thing to do is to not persist older entries.

The other structure of feeds linking to feeds would require the aggregator
be able to do something useful with such links, but this can be generalised
and thus be useful for many purposes. As it is, right now with NNW I can do
something useful with such a feed: drag & drop the item headline link to my
subscriptions pane to subscribe to that feed and view the entries therein.

Both require coding effort.

e.

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Jul 2005 21:36:05 +1000

> pointing to the table of contents feeds of each issue. That is, a feed with

There is another issue with the idea individual Atom Feed Documents
describing a snapshot-in-time collection of entries ... those snapshot
documents shouldn't really be polled for updates every hour or whatever.
Thus, it would be useful for a feed to be able to indicate how often it
should be polled (or not).

Might have to write an extension for that.

e.

---

From: Henry Story <[email-masked]>
Date: Fri, 29 Jul 2005 15:39:47 +0200

Below I think I have worked out how one can in fact have a top20  
feed, and
I show how this can be combined without trouble with the  
<history:next ...>
link...

> seen as a complete replacement set?

Well if it is a typo and this is considered to be an insignificant  
change then
they can change the typo in the feed document and not need to change  
any updated
time stamps.

> it is that second Feed Document which contains the individual items  

This is certainly closer to my intuitions too.  A top 20 something is  
*not* a feed.
Feed entries  are not ordered, and are not  meant to be thought of as  
a closed
collection. At least this is my initial intuition. BUT....

I can think of a solution like the following: Let us imagine a top 20  
feed where
the resources being described by the entries are the position in the  
top list. So
we have entries with ids such as

     http://TopOfThePops.co.uk/top20/Number1
     http://TopOfThePops.co.uk/top20/Number2
     http://TopOfThePops.co.uk/top20/Number3
     ...

Each of these resources describes the songs that is at a certain rank  
in the top of
the pops chart. Each week the song in that rank may change. When a  
change occurs in the
song at a certain rank the top 20 feed with id

     http://TopOfThePops.co.uk/top20/Feed

will contain a new entry such as

    <entry>
     <title>Top of the pops entry number 1</title>
     <link href="http://TopOfThePops.co.uk/top20/Number1/"/>
     <id>http://TopOfThePops.co.uk/top20/Number1</id>
     <updated>2005-07-05T18:30:00Z</updated>
     <summary>Top of the pops winner for the week starting 5 July  
2005</summary>
   </entry>

A client that would subscribe to such a feed would automatically get  
updates every
week for each of the top 20 resources. But the feed could be  
structured exactly like
I suggest in 2.

So for a top 2 feed (20 is a bit to long for me)

<feed>
    <title type="text">My top 2 Software Books</title>
    <id>http://bblfish.net/blog/top2</id>
    <history:prev>http://bblfish.net/blog/top2/ 
Archive-2005-07-18.atom</history:prev>
    ...
    <entry>
     <title>My Top 1 book</title>
     <link href="http://bblfish.net/blog/top2/Number1/"/>
     <id>http://bblfish.net/blog/top2/Number1</id>
     <updated>2005-07-25T18:30:00Z</updated>
     <summary>My top 1 book is Service Oriented Computing by Wiley</ 
summary>
    </entry>
    <entry>
     <title>My Top 2 book</title>
     <link href="http://bblfish.net/blog/top2/Number2/"/>
     <id>http://bblfish.net/blog/top2/Number2</id>
     <updated>2005-07-25T18:30:00Z</updated>
     <summary>My second top book is "xml in a Nutshell"</summary>
   </entry>
</feed>

The above representation of the http://bblfish.net/blog/top2 feed  
points to the
archive feed http://bblfish.net/blog/top2/Archive-2005-07-18.atom

<feed>
    <title type="text">My top 2 Software Books</title>
    <id>http://bblfish.net/blog/top2</id>
    ...
    <entry>
     <title>My Top 1 book</title>
     <link href="http://bblfish.net/blog/top2/Number1/"/>
     <id>http://bblfish.net/blog/top2/Number1</id>
     <updated>2005-07-18T18:30:00Z</updated>
     <summary>My top 1 book is Java 2D Graphics</summary>
    </entry>
    <entry>
     <title>My Top 2 book</title>
     <link href="http://bblfish.net/blog/top2/Number2/"/>
     <id>http://bblfish.net/blog/top2/Number2</id>
     <updated>2005-07-18T18:30:00Z</updated>
     <summary>My second top book is "xml in a Nutshell"</summary>
   </entry>
</feed>

  As you notice only the top1 has changed from the first week but not  
the
second, and yet in both feeds there is an entry for both. A non  
change can
sometimes be an important event. But this is really up to the feed  
creator
to choose how to present his feeds. He could easily have had only had  
the
entry for http://bblfish.net/blog/top2/Number1 in the first feed.

Looking at it this way, there really seems to be no incompatibility  
between
a top 20 feed and the <history:next ...> link. My talk about archives  
not
changing should be more precisely about archives not changing in any
significant way. And this advice could be moved to an implementors  
section
and be encoded in HTTP by simply giving archive pages an infinitely long
expiry date.

> for one, and even a feed of feeds available on the site.

I completely agree. And remember for any two things there is at least  
one way they
are related. And there are many different ways feeds can be related  
to each other.
A feed may be an archival continuation of one - which is what the  
<history:next ...>
link in my opinion addresses, but there are many other ways one can  
relate feeds.

> the table of contents for that issue, with a distinct and separate  

Yes a feed can itself be an entry in another feed.

> pointing to the table of contents feeds of each issue. That is, a  

yes.

> appropriate thing to do is to not persist older entries.

I am starting to wonder whether this is really needed now that I have  
looked
at the top20 example I gave above.

> subscriptions pane to subscribe to that feed and view the entries  

I myself have no problem with feeds being entries, feeds pointing to  
other
feeds, or anything like that. A feed is a resource. It can change. A  
feed
is simply a set of state changes to resources. It is that general.

---

From: Eric Scheid <[email-masked]>
Date: Sat, 30 Jul 2005 01:01:50 +1000

On 29/7/05 11:39 PM, "Henry Story" <[email-masked]> wrote:

> Below I think I have worked out how one can in fact have a top20  feed, and I
> show how this can be combined without trouble with the  <history:next ...>
> link...
> 
> 
> Well if it is a typo and this is considered to be an insignificant  change
> then they can change the typo in the feed document and not need to change  any
> updated time stamps.

Misspelling the name of the artist for the top 20 songs list is not
insignificant. Even worse fubars are possible too -- such as attributing the
wrong artist/author to the #1 song/book (and even worse: leaving off a
co-author).

> This is certainly closer to my intuitions too.  A top 20 something is  *not* a
> feed. Feed entries are not ordered, and are not meant to be thought of as a
> closed collection. At least this is my initial intuition. BUT....

Not all Atom Feed Documents are feeds, some are static collections of
entries. We keep tripping over this :-(

> I can think of a solution like the following: Let us imagine a top 20  feed
> where the resources being described by the entries are the position in the
> top list. So we have entries with ids such as
> 
> http://TopOfThePops.co.uk/top20/Number3 ...

>    <summary>Top of the pops winner for the week starting 5 July

The problem here is that this doesn't describe the referent, it only
describes the reference. I want to see top 20 feeds where each entry links
to the referent in question. For example, the Amazon Top 10 Selling Books
feed would link to the book specific page at Amazon, not to some page saying
"the #3 selling book is at the other end of this link".

> I completely agree. And remember for any two things there is at least one way
> they are related. And there are many different ways feeds can be related  to
> each other. A feed may be an archival continuation of one - which is what the
> <history:next ...> link in my opinion addresses, but there are many other ways
> one can relate feeds.

I did a grep of an archive of atom-syntax messages .. lots of interesting
possibilities in there. James has set a nice example to copy from, so expect
a few I-D from me.

> I am starting to wonder whether this is really needed now that I have  looked

Consider the Nature case study. They have separate feed documents for each
issue, but just one public URI published. The things which are entries are
not Top N things but entries in a Table of Contents, and it's useful to be
able to aggregate that list of articles over time.

> I myself have no problem with feeds being entries, feeds pointing to  other
> feeds, or anything like that. A feed is a resource. It can change. A  feed is
> simply a set of state changes to resources. It is that general.

What's needed is a sea change in the concept model aggregators have
regarding feed URIs.

Right now it's exactly as if you have to Bookmark a web page before you can
view it -- whereas I'd like to see feed browser where I can clickity click
to wherever without having to first clutter my subscriptions list, but if I
happen to like the look of a feed I can click one button to
s/subscribe/bookmark/ it.

A related interface design concept involves nesting - right now all my
subscriptions lie flat in my subscriptions pane. I can organise them into
folders, and those folders inside more folders, but the subscriptions lie
flat inside there.

I'd love to see my subscription list look like this;

  - Fave Feeds
        Fred's Blog (12)
      + Jim's Blog (23)
      - Mike's Blog (7)
          - Categories
                Food
                Cat Pix
          - Archives
                Jan 2000
                Feb 2000
                etc
      + Tom's Blog (234)

I'm not aware of any feed readers that do that just yet, probably because
not many feeds actually contain links to other feeds as a matter of course.
Those would be feeds related to feeds there ... haven't quite thought
through how I'd like to see sub-feeds of entries as yet.

e.

---

From: [email-masked] (Peter Robinson)
Date: Sat, 30 Jul 2005 18:32:20 +0100

James M Snell <[email-masked]> wrote:

> within a feed.

[xml condensed:]

>   <entry> ... <o:index>1</o:index></entry>
>   <entry> ... <o:index>2</o:index></entry>
>   <entry> ... <o:index>3</o:index></entry>
> </feed>

Certainly a very interesting idea.  Having implemented openSearch [1] in
my product feeds [2], I'd probably implement this, because it makes
sense and would be very easy to do.  My feeds are actually ordered (and
paged) already, either by onsale date or by top seller, even though
there's no way to indicate this in the XML.

[Actually, it *should* already be in openSearch since that implicitly
expects the feed entries to be ordered corresponding to the feed-level
<openSearch:totalResults> and <openSearch:startIndex> elements.]

Just my 2c.

Regards,

Peter

[1] <http://opensearch.a9.com/spec/opensearchrss/1.0/>
[2] <http://www.ticketswitch.com/cgi-bin/atom_feed.exe?page_no=2>
(or <http://www.ticketswitch.com/cgi-bin/rss_feed.exe?page_no=2>)