Subject: Feed History / Protocol overlap

From: Robert Sayre <[email-masked]>
Date: Tue, 18 Oct 2005 12:40:22 -0400

I think the navigation elements of
draft-nottingham-atompub-feed-history-04.txt overlap with Atom
protocol navigation and deployed APP beta implementations. In fact, I
pointed this out way back in April 2005. I don't think anything has
changed.

In <http://www.mnot.net/blog/2005/04/12/feed_state> Mark Nottingham wrote:
> Way back when I put the first Atom drafts together, I included a placeholder for
> a section that I hoped would allow reconstruction of feed state. Presently, this often
> isn't necessary, because you have to be away for a seriously long time
> (e.g, on vacation) before you actually miss anything.
...
> In other words, if you happen to look away for too long you miss information,
> essentially making the channel leaky. To that end, I put together a proposal and a
> demonstration feed (in fact this very blog's feed, dear reader), in the hopes of
> convincing people that this is a real issue. Silence ensued, and the
> ATOMPUB WG declined my proposal.

to which Robert Sayre replied:
> Hmm. Your proposal concerned a couple link relations, right? Those would be
> easy to add to the format at anytime, and... Blogger and 6A have both asked for
> similar functionality on  the protocol side. Seems like more of a server layout
> and protocol problem, anyway.

to which Mark Nottingham replied:
> Robert —
> The only problem I have with that is that AFAIK so far, I don't need to know about the
> protocol document to consume an Atom document; it's only when you want to
> manipulate a feed that you have to work on that side of the house.
>
> That said, it's good to hear that others want this too.

to which Robert Sayre replied:
> Well, wouldn't it be nice if you didn't need to know about the protocol document to
> perform any of the protocol's read operations? That's my thinking. View-source on a
> couple of link relations should be enough to pick it up.

Robert Sayre

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 18 Oct 2005 10:29:55 -0700

Can you substantiate that with links to the appropriate portions of  
the current protocol draft?

--
Mark Nottingham   Principal Technologist
Office of the CTO   BEA Systems

________________________________________________________________________________
BEAWorld 2005: coming to a city near you.  Everything you need for SOA and enterprise infrastructure success.

Register now at http://www.bea.com/4beaworld

London 11-12 Oct| Paris13-14 Oct| Prague18-19 Oct |Tokyo 25-26 Oct| Beijing 7-8 Dec

---

From: Robert Sayre <[email-masked]>
Date: Tue, 18 Oct 2005 14:22:19 -0400

> the current protocol draft?

Oh that's odd. They've gone and deleted them. I can tell you that my
general impression of the current atom-protocol list was that we would
align with Amazon OpenSearch. In any case, I think it would be unwise
for the IETF to duplicate APP navigation.

Robert Sayre

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 18 Oct 2005 11:25:30 -0700

OK, well, I'm not terribly fussed by who registers them, but they  
need to be carefully defined, and it wasn't at all clear that the  
OpenSearch document did that.

Considering that there's a need for them sooner rather than later,  
would you have a problem with registering the link relations (as  
discussed in a separate thread) separately, before APP is done? If  
so, why?

--
Mark Nottingham     http://www.mnot.net/

---

From: Robert Sayre <[email-masked]>
Date: Tue, 18 Oct 2005 14:38:11 -0400

> OpenSearch document did that.

I think maybe we have a difference of opinion on what's needed here.

> discussed in a separate thread) separately, before APP is done? If

No objection.

Robert Sayre

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 18 Oct 2005 11:54:15 -0700

> I think maybe we have a difference of opinion on what's needed here.

Could you elaborate?

Thanks,

---

From: Robert Sayre <[email-masked]>
Date: Tue, 18 Oct 2005 15:14:41 -0400

> > I think maybe we have a difference of opinion on what's needed here.

I vastly prefer your first definition of next/prev. The "should not
change over time" stuff is not testable. For example, a template
change in Movable Type followed by a "Rebuild All" makes every single
archive change. As a client implementor, I can't see how the the
follwing text helps me:

"The set of entries in this document should not change over time;
i.e., this link points to a stable snapshot of entries, or an archive
of feed entries."

I think what you're describing is a sensible server implementation
strategy, because once the URI is visited, it should return 304 in the
future.

Robert Sayre

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 18 Oct 2005 12:25:48 -0700

I'm confused; the current proposal (below) doesn't have that text in  
it; for example, the definition of previous is:

> A stable URI that, when dereferenced, returns a feed document  
> containing a set of entries that sequentially precede those in the  
> current document. This can be thought of as specific to those  
> entries; in other words, it represents a fixed section of the feed,  
> rather than a sliding window over it. Note that the exact nature of  
> the ordering between the entries and documents containing them is  
> not defined by this relation; i.e., this relation is only relative.

---

From: Robert Sayre <[email-masked]>
Date: Tue, 18 Oct 2005 15:38:26 -0400

> it; for example, the definition of previous is:

OK, then I am confused.

> > current document.

I already have code that uses "next" for this. Why do we want to change it?

> > This can be thought of as specific to those

1.) I don't understand why one feed is making assertions about the
stability of another, when your draft provides explicit signals for
this.

2.) I still don't see how this helps me write a client.

3.) I don't think the notion of "fixed section" is helpful.
<fh:archive> is good, that means "don't subscribe"... I get that.

Robert Sayre

---

From: James M Snell <[email-masked]>
Date: Tue, 18 Oct 2005 12:49:36 -0700

+1 on all of Roberts comments.  While I'm ok with the current version, I 
was much happier with the original.

---

From: Thomas Broyer <[email-masked]>
Date: Tue, 18 Oct 2005 22:45:56 +0200

Robert Sayre wrote:

[about the "previous" link relation]
> I already have code that uses "next" for this. Why do we want to change it?
Er, I'm not a native English speaker, so my apologies in advance if I'm 
wrong…

If a feed is sorted by the entries' atom:updated value in descending 
order (which AtomPP currently requires for "Lists"), isn't "next" 
meaning "entries with an atom:updated representing an earlier date-time" 
(remember, entries are sorted in descending order!) and "previous" 
meaning "entries with an atom:updated representing a later date-time"?

If entries are sorted in descending order, the "first" feed is the one 
with the greatest values (atom:updated representing now, a second ago, a 
minute ago, …) and the "last" one the one with the lowest values 
(atom:updated representing the "oldest" date-times). Hence, in the 
"first" feed you'll find an atom:link[@rel="next"] and in the "last" one 
an atom:link[@rel="previous"].

Conclusion: I think you, Mark and the guys behind OpenSearch all agree…

Now I think we should ask the OpenSearch people if they already have 
requested IANA registration for their start/next/prev/end link relations 
(hasn't James already asked them?) and if they haven't, then we (the WG) 
should first agree on the terms to be used (start/end vs. first/last), 
request registration and tell the OpenSearch people that these relations 
are pending registration.
Any thoughts?

-- 
Thomas Broyer

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 18 Oct 2005 14:08:53 -0700

> I already have code that uses "next" for this. Why do we want to  

Why would your code have to change?

> stability of another, when your draft provides explicit signals for

If we go down this road, my draft will be, at the most, a users'  
guide to the link relations. It probably won't be necessary at all,  
except perhaps for <fh:incremental>false</fh:incremental>.

> 2.) I still don't see how this helps me write a client.

What are you looking for? People said they wanted to use atom:link,  
so I'm trying to accommodate that. People said they wanted the  
relations to be generic, so I'm trying to accommodate that.

> <fh:archive> is good, that means "don't subscribe"... I get that.

It characterises the nature of the feed that's being linked to.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 18 Oct 2005 14:09:20 -0700

Please disambiguate "original."

On 18/10/2005, at 12:49 PM, James M Snell wrote:

> +1 on all of Roberts comments.  While I'm ok with the current  
> version, I was much happier with the original.
> Robert Sayre wrote:
>
>

---

From: Robert Sayre <[email-masked]>
Date: Tue, 18 Oct 2005 19:13:57 -0400

> relations to be generic, so I'm trying to accommodate that.

I see a specific definition in a generic relation, but I don't
understand what the specific definition is good for and I want the
definition to be more general.

rel: next
definition: A URI that points to the next feed in a series of feeds.
For example, in a reverse-choronological series of feeds, the 'next'
URI would point deeper into the past.

> It characterises the nature of the feed that's being linked to.

I understand that. I don't understand how that enables a new feature
or easier interoperation. In otherwords, the rationale for the
definition seems circular to me.

Robert Sayre

---

From: Antone Roundy <[email-masked]>
Date: Tue, 18 Oct 2005 17:29:59 -0600

> URI would point deeper into the past.

Ohh, nice readability.  Perhaps a few refinements:

A URI that points to the next in a series of Feed documents, each  
representing a segment of the same feed.  For example, in a reverse- 
chronologically ordered series of Feed documents, the 'next' URI  
would point to the document next further in the past.

---

From: James M Snell <[email-masked]>
Date: Tue, 18 Oct 2005 16:58:20 -0700

> would point to the document next further in the past.
-1 because each of the feed documents may not represent a segment of the 
same feed.  That's one potential use case, but it's not the only one. 

+1 to Robert's version.

---

From: Antone Roundy <[email-masked]>
Date: Tue, 18 Oct 2005 18:04:03 -0600

On Oct 18, 2005, at 5:58 PM, James M Snell wrote:
>> representing a segment of the same feed.  For example, in a  
>> reverse- chronologically ordered series of Feed documents, the  
>> 'next' URI  would point to the document next further in the past.
>>
> -1 because each of the feed documents may not represent a segment  
> of the same feed.  That's one potential use case, but it's not the  
> only one.
> +1 to Robert's version.
>
-3 to being that generic.  Surely, even if we can only imagine one  
method of paging through a single feed and thus aren't concerned  
about conflict for use of the "next" link, "paging", if you can even  
call it that (okay, so maybe you're not calling it that), between  
feeds using the same link relation as paging within each of the feeds  
is not just allowing for the possibility of conflicts, it's actively  
inviting conflicts.

---

From: Robert Sayre <[email-masked]>
Date: Tue, 18 Oct 2005 20:10:55 -0400

On 10/18/05, Antone Roundy <[email-masked]> wrote:
> -3 to being that generic.

That's a very large negative number. Can you explain how your version
will me write software I otherwise couldn't?

Robert Sayre

---

From: Eric Scheid <[email-masked]>
Date: Wed, 19 Oct 2005 10:53:51 +1000

> I already have code that uses "next" for this. Why do we want to change it?

Why did you choose "next"?

e.

---

From: Robert Sayre <[email-masked]>
Date: Tue, 18 Oct 2005 20:58:58 -0400

> Why did you choose "next"?

Because that is what's already been deployed and what my software uses.

Robert Sayre

---

From: Eric Scheid <[email-masked]>
Date: Wed, 19 Oct 2005 11:08:00 +1000

> URI would point deeper into the past.

How will your code cope with a forward-chronological series of feeds?

e.

---

From: Antone Roundy <[email-masked]>
Date: Tue, 18 Oct 2005 19:56:57 -0600

> will me write software I otherwise couldn't?

Anything larger than -2 is bogomips--the point I was trying to make  
is that I think the idea of using the same link relation for paging  
within a feed and for navigating between feeds is absolutely absurd-- 
completely lacking in foresight--almost looks like an attempt to  
create for future problems.  People were complaining that trying to  
avoid problems with the hypothetical "top 100 DVDs" scenario (not  
trying to solve it--just trying to avoid problems if it comes about)  
was wandering too far off into hypotheticals, but now people want to  
make sure they can use the "next" relation for the arguably even more  
hypothetical idea of building a chain of otherwise independent  
feeds?  This boggles my mind.

Here's what my version will let you do that you won't be able to do  
if the definitions of these links allows them to be used for  
interfeed navigation--it will enable you to do paging within a feed  
that is also part of a chain of feeds (because anyone wanting to  
create a chain of feeds will have to come up with a non-conflicting  
link relation to do it).  It will also enable you to know that  
(unless somebody's breaking the spec) you are navigating through a  
single feed when you follow next and prev links around--that you are  
not jumping from feed to feed.  Your software will be able to follow  
those links with a much greater degree of confidence that it won't  
result in your users complaining "what the hell are you doing showing  
me entries from a feed I didn't subscribe to?"  It will enable your  
application to take more actions automatically without having to ask  
for confirmation from the user every time you follow another next or  
prev link to avoid such complaints.

---

From: Eric Scheid <[email-masked]>
Date: Wed, 19 Oct 2005 12:05:23 +1000

> Because that is what's already been deployed and what my software uses.

-1. 

Someone else made a poor choice*, you copied that, it's confusing, why
enshrine it into official specs? Why not take this opportunity to make a
better choice?

I looked at Pilgrim's atom feed document for March 2004 ... it said the next
document was February 2004 and the previous document was April 2004. Try
explaining to anyone that isn't writing an aggregator why that makes sense.

There are entries written in March which are a continuation of a story
written in February. So no, his feed is not reverse chronological.

You might want your aggregator to process various feeds in a reverse
chronological manner ... so please use the 'prev' link to go _backwards_
through the feed, and the 'next' link to go _forward_ through the feed.

Though there may be many Atom 0.3 feeds out there now, all those Atom 0.3
feeds will need to be rewritten for Atom 1.0. Even so, all those Atom 0.3
feeds are, you must admit, just a drop in the ocean of the potential
deployment of Atom 1.0 feeds from many many content publishers. Count the
number of websites that have semi-regular news, count the number of websites
that have feeds (of any flavour) ... it's a drop in the ocean. Now count the
number of aggregator developers..

e.

* and it's not the only poor choice that we're stuck with right now.
Consider how you might link from an HTML page which is the monthly archive
of your blog to an Atom Feed Document which is the XML representation of
that same content. We know what @href and @type to use ... but the
@rel='alternate' is already taken for the semantic of "subscribe to this
feed for ongoing updates". Thankfully we're avoiding that snafu with
@rel='self' vs @rel='subscribe'.

---

From: Antone Roundy <[email-masked]>
Date: Tue, 18 Oct 2005 21:32:31 -0600

Here's what this discussion makes me think of--RSS has a link  
element.  That link was very generic, and has been variously used to  
link to what Atom calls link/@rel="alternate" and link/ 
@rel="related", and perhaps even other things.  Once we'd gained a  
little experience and discovered that the imprecision of the meaning  
of the element was limiting uses we wanted to make of feeds, we  
created more specific types of links.  Hopefully, we were specific  
enough this time that we won't run into significant use cases that  
we've rendered impossible, but who knows.

Now we're defining a method of navigating through a chain of linked  
documents.  We know of two specific use cases that we're sure we want  
to be able to do: paging through things like search results, and  
catching up on incremental feeds (or reconstructing the entire state  
of the feed, which is an extension of catching up).  It would appear  
that the same link relation can be used to do both of those things  
without the fear of conflict, because they operate within feeds that  
have a basic difference in nature, so they're unlikely to both be  
needed within one feed.  Also, from a certain point of view, they are  
really the same thing--a way to navigate through the current state of  
the feed.  The fact that incremental feeds don't have old states that  
have been discarded and replaced the way non-incremental feeds do  
(their former state gets augmented rather than being replaced)  
doesn't make a difference with respect to the issue of navigating  
through their current state.

So why don't we create a mechanism to do those two things (that are  
really one thing), and NOT make it generic enough to encompass other  
things that we might want to do someday, which might lead to the same  
sort of limitation that RSS has by only having one generic link  
element?  Sure, we COULD do all of our interdocument navigation using  
"next" and "prev" until someday when we decide that we need something  
more specific for some of the navigation use cases.  But then we'll  
be doing some of the same things multiple ways--some people sticking  
with "next" and "prev", and some using whatever new methods or link  
relations are invented, and nobody quite sure what "next" and "prev"  
mean in any particular feed.  Why not wait till we've really figured  
out what other ways we might want to navigate between documents, and  
then devise a new method for doing it?

If we're going to create some generic link relations for people to  
experiment with, let's create somethings that's explicitly for doing  
experimental things with so that the link relations we want to do  
more specific things with aren't rendered less useful by the  
experimentation.  Register "x-next" and "x-prev" or something for  
that, or register "next-page" and "prev-page" for the things we know  
we want to do.  Or don't register any such thing--just don't promote  
use of the the link relations we define for (reasonably) well  
understood use cases to do experimental things.

We'll, I've spoken my mind plenty on this issue, so unless somebody  
brings up an issue that my opinion on couldn't be understood from  
what I've written already, I think I'll leave it at that.  If we go  
with a highly-generic definition and it causes trouble down the road,  
I'll have some big ASCII art letters ready to say "I told you so".   
If not, then oops, I guess I was wrong.

---

From: "Thomas Broyer" <[email-masked]>
Date: Wed, 19 Oct 2005 08:44:01 +0200 (CEST)

> would point to the document next further in the past.

+1, *this* is "paging".

We could add another example, e.g. sorted by relevance (within a search
result) or priority…

If you want to link between different "states" of "Top 100" feeds
(October, September, August, etc), then use something like @rel="archives"
or @rel="history", or define a @rel="previous-archive" if you really want
to navigate directly to the other feed without having to go through a
"table of contents" feed.

If some people here prefers "next-chunk" or "next-page" to just "next",
why not, my mind is open…

-- 
Thomas Broyer

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 19 Oct 2005 10:12:26 -0700

"next"
"next-chunk"
"next-page"
"next-archive"
"next-entries"
are all workable for me.

I think the real question is still (unfortunately) how specific it  
should be.

I think there are merits to both sides; the relative cost of a  
specific term isn't much, and the harm of a general term is largely  
theoretical at this point. So, I'm at a point where I'm more  
interested in moving forward than in a particular solution.

Perhaps people could +1/-1 the following options:

* Reconstructing a feed should use:
    a) a specific relation, e.g., "prev-archive"
    b) a generic relation, e.g., "previous"

(these are just examples; once we figure out how specific it should  
be, we can figure out a term more easily)

I'm +1 on both.

---

From: James M Snell <[email-masked]>
Date: Wed, 19 Oct 2005 11:02:31 -0700

>    a) a specific relation, e.g., "prev-archive"

-1.  We've got more paging use cases to think about than just archives.  
OpenSearch already demonstrates that.

>    b) a generic relation, e.g., "previous"

+1.

- James

---

From: "James Holderness" <[email-masked]>
Date: Wed, 19 Oct 2005 19:26:40 +0100

>    a) a specific relation, e.g., "prev-archive"

+0

>    b) a generic relation, e.g., "previous"

+1

---

From: Thomas Broyer <[email-masked]>
Date: Wed, 19 Oct 2005 20:52:55 +0200

>    a) a specific relation, e.g., "prev-archive"
-1, (see James' comments)
+1

-- 
Thomas Broyer

---

From: Bill de hÓra <[email-masked]>
Date: Wed, 19 Oct 2005 19:56:27 +0100

> explaining to anyone that isn't writing an aggregator why that makes sense.

"No matter which direction you head in, no matter which way the chain is
sorted, the next document is always "next", so that's not a useful
distinction IMHO. "

You said that to me about next and previous for app:collection when I
reuested the value 'next' be changed to 'previous' to be  consistent
 with the notion of elements existing earlier in time. What's different
here?

cheers
Bill

---

From: Eric Scheid <[email-masked]>
Date: Thu, 20 Oct 2005 05:05:17 +1000

+1

---

From: Thomas Broyer <[email-masked]>
Date: Wed, 19 Oct 2005 21:12:20 +0200

>>    b) a generic relation, e.g., "previous"
Hmm, I might have answered a bit too fast…

"previous" might be too generic and thus be used for things other than 
paging through a feed (would it be incremental –feed reconstruction– or 
not –search result, etc.), so I'd +1 to "previous-page".
On the other hand, if we don't register "previous" now, someone might 
register it later for something else (don't know what the IESG can 
approve…), with the same potential misuses…
But if we register "archives" or "history" at the same time (or, say, 
quite early in Atom the 1.0 deployment), misuses (the most common I 
think, using previous/next for inter-"feed state" linking, e.g. linking 
from the October Top 100 non-incremental feed to the September Top 100 
non-incremental feed) could be prevented… We should then also add a note 
in the previous/next description telling people not to use them for such 
things as described earlier, maybe also using the term "page" or 
"paging" explicitly…

As a result, I'm still -1 on prev-archive and +1 on previous, but add a 
+.75 on "previous-page".

-- 
Thomas Broyer

---

From: Eric Scheid <[email-masked]>
Date: Thu, 20 Oct 2005 05:35:10 +1000

>>> Ok, somehow this slipped under the radar on me during
>>> my first reading. -1, as I prefer next, as in, the *next*
>>> document in a chain of documents.
> 
> requested the value 'next' be changed to 'previous' to be  consistent
> with the notion of elements existing earlier in time.

> What's different here?

Nothing. 

I should have been more clear back then that I was +1 on 'previous'... my
point was directed at Joe (not you), and that his line of argument (in
favour of 'next') was not persuasive and so if he wanted 'next' then he'd
better try another line of argument.

As then as now, I'm still +1 on 'previous' for going backwards.

e.

---

From: Stefan Eissing <[email-masked]>
Date: Thu, 20 Oct 2005 12:27:35 +0200

+0

+1

---

From: Joe Gregorio <[email-masked]>
Date: Thu, 20 Oct 2005 09:39:31 -0400

>     a) a specific relation, e.g., "prev-archive"

-1

+1

   -joe
--
Joe Gregorio        http://bitworking.org