Subject: New Link Relations -- Ready to go?

From: Mark Nottingham <[email-masked]>
Date: Thu, 20 Oct 2005 16:52:39 -0700

At this point, I believe the following represent as close to  
consensus as we're able to get (although I'm happy to be proven  
wrong, as long as there are counterproposals that can gain consensus).

To move forward, please either +1 or -1 them, preferably with  
replacement text if you -1. Once there seems to be rough consensus, I  
plan to make a request to IANA and point the IESG to this discussion  
thread to illuminate their deliberations.

  -  Attribute Value: previous
  -  Description: A URI that refers to the immediately preceding  
document in a series of documents.
  -  Expected display characteristics: Undefined.
  -  Security considerations: Automated agents should take care when  
this relation crosses administrative domains (e.g., the URI has a  
different authority than the current document). Such agents should  
also take care to detect circular references.

  -  Attribute Value: next
  -  Description: A URI that refers to the immediately following  
document in a series of documents.
  -  Expected display characteristics: Undefined.
  -  Security considerations: Automated agents should take care when  
this relation crosses administrative domains (e.g., the URI has a  
different authority than the current document). Such agents should  
also take care to detect circular references.

  -  Attribute Value: first
  -  Description: A URI that refers to the furthest preceding  
document in a series of documents.
  -  Expected display characteristics: Undefined.
  -  Security considerations: Automated agents should take care when  
this relation crosses administrative domains (e.g., the URI has a  
different authority than the current document). Such agents should  
also take care to detect circular references.

  -  Attribute Value: last
  -  Description: A URI that refers to the furthest following  
document in a series of documents.
  -  Expected display characteristics: Undefined.
  -  Security considerations: Automated agents should take care when  
this relation crosses administrative domains (e.g., the URI has a  
different authority than the current document). Such agents should  
also take care to detect circular references.

  -  Attribute Value: subscribe
  -  Description: A URI that refers to a feed document containing a  
set of the most recent entries in the feed. This URI is intended to  
be subscribed to to keep abreast of recent changes in the feed. When  
different from the URI of the document where it occurs, it indicates  
that its value should be used for this purpose in place of the  
current document's URI.
  -  Expected display characteristics: Undefined.
  -  Security considerations: Users should be informed of the actual  
URI they are subscribing to, and subscription should only take place  
when it is explicitly requested.

--
Mark Nottingham     http://www.mnot.net/

---

From: James M Snell <[email-masked]>
Date: Thu, 20 Oct 2005 17:06:17 -0700

+1 to one and all.  This is perfect.

Mark Nottingham wrote:

>  -  Description: A URI that refers to the furthest preceding  document 
> in a series of documents.
>  -  Description: A URI that refers to the furthest following  document 
> in a series of documents.

---

From: Eric Scheid <[email-masked]>
Date: Fri, 21 Oct 2005 10:47:57 +1000

+1 to all

---

From: "James Holderness" <[email-masked]>
Date: Fri, 21 Oct 2005 06:38:15 +0100

+1 to all

Just a minor grammatical quibble regarding the text for "subscribe": the 
phrases ending with a preposition seem somewhat awkward to me - in 
particular the double "to". If you think I'm being overly dogmatic, though, 
feel free too ignore.

Possible replacement text:

 -  Attribute Value: subscribe
 -  Description: A URI that refers to a feed document containing a
set of the most recent entries in the feed. This is the URI to which
a UA should subscribe to keep abreast of changes in the feed. When
different from the URI of the document where it occurs, it indicates
that its value should be used for this purpose in place of the
current document's URI.
  -  Expected display characteristics: Undefined.
  -  Security considerations: Users should be informed of the actual
URI to which they are subscribing, and subscription should only take place
when it is explicitly requested.

I'm not sure about the use of UA though: should that be client? agent? Atom 
processor?

Regards
James

---

From: Henry Story <[email-masked]>
Date: Fri, 21 Oct 2005 02:42:36 -0700 (PDT)

+1 to all too.

On Fri, 21 Oct 2005, Eric Scheid wrote:

> Date: Fri, 21 Oct 2005 10:47:57 +1000
> From: Eric Scheid <[email-masked]>
> To: Atom Syntax <[email-masked]>
> Subject: Re: New Link Relations -- Ready to go?
> 
>
> +1 to all
>

---

From: Tim Bray <[email-masked]>
Date: Fri, 21 Oct 2005 06:57:23 -0700

>  -  Attribute Value: subscribe

I'm puzzled by this one.  I thought that if you wanted to subscribe  
to a feed then, well, you would subscribe to a feed.  I must have  
missed the discussion.  Could someone provide a pointer to the  
archive? -Tim

---

From: "James Holderness" <[email-masked]>
Date: Fri, 21 Oct 2005 15:38:40 +0100

Tim Bray wrote:
> I'm puzzled by this one.  I thought that if you wanted to subscribe  to a 
> feed then, well, you would subscribe to a feed.  I must have  missed the 
> discussion.  Could someone provide a pointer to the  archive? -Tim
>

The idea being that if you were to come across an archived Atom document 
(however that might happen), the presence of this link would, (a) let you 
know that it was an archive document and thus shouldn't be subscribed to, 
and (b) provide you with a URL with which you could subscribe to the actual 
feed if you so chose.

For reference, see here (end of the message):
http://www.imc.org/atom-syntax/mail-archive/msg17202.html

Also this thread (discusses the fh:archive element which served a similar 
purpose):
http://www.imc.org/atom-syntax/mail-archive/msg16724.html

Regards
James

---

From: Eric Scheid <[email-masked]>
Date: Sat, 22 Oct 2005 00:56:46 +1000

On 21/10/05 11:57 PM, "Tim Bray" <[email-masked]> wrote:

> missed the discussion.

This arose during the discussion of Atom Feed Documents as archives. It was
thought that the 'self' link in an archive would point to the archive
document itself, which meant a different relationship was needed for the
purpose of locating the URI which is the one that contains the most recent
updates/entries.

Start somewhere near here:
http://www.imc.org/atom-syntax/mail-archive/msg17208.html

e.

---

From: "Thomas Broyer" <[email-masked]>
Date: Fri, 21 Oct 2005 17:09:43 +0200 (CEST)

> document in a series of documents.

This definition doesn't prevent someone from using this link relation for
linking within a series of documents representing, say a "webring", where
"previous" and "next" will be other sites' feeds.
I really think we should make it clear that first/previous/next/last are
meant for "paging" of a single feed only.
I'll try to propose a replacement text.

>   -  Attribute Value: subscribe

+1

-- 
Thomas Broyer

---

From: Tim Bray <[email-masked]>
Date: Fri, 21 Oct 2005 08:19:32 -0700

On Oct 21, 2005, at 7:38 AM, James Holderness wrote:

> The idea being that if you were to come across an archived Atom  
> document (however that might happen), the presence of this link  
> would, (a) let you know that it was an archive document and thus  
> shouldn't be subscribed to, and (b) provide you with a URL with  
> which you could subscribe to the actual feed if you so chose.

Makes sense, but not self-evident.  I would think that the usefulness  
of this thing would be improved by a few words of explanation for  
those who come upon it without knowing the history. -Tim

---

From: "A. Pagaltzis" <[email-masked]>
Date: Fri, 21 Oct 2005 17:33:31 +0200

* Eric Scheid <[email-masked]> [2005-10-21 17:05]:
> It was thought that the 'self' link in an archive would point
> to the archive document itself, which meant a different
> relationship was needed for the purpose of locating the URI
> which is the one that contains the most recent updates/entries.

I admit I intentionally kept out of that discussion because I
don’t have a lot of personal interest in the subject, and the
thread has been so fast and furious I didn’t bother staying on
top of it just because.

But I’d like to offer some thoughts on this particular point;
apologies in advance if these points have already been raised:

First, rel="self" is going to be implemented by most everything
that groks Atom 1.0 in order to support one-click subscription,
if applicable, right? Whereas this new relationship might not
find such wide-spread support.

Second, although less important – even applications which do
support rel="subscribe" will have to implement a fallback
behaviour. Arguably, they already do in some fashion, because a
feed may omit the rel="self" link, so I don’t know if this is
such a pressing issue.

For these two (similar) reasons I think it might be wise to keep
rel="self" in the role that this new rel="subscribe" thing is
supposed to fulfill, and invent a new relationship that can point
to the canonical location of the archive feed document.

Third, and in some ways most importantly, I see overlap with Bob
Wyman’s “preferred feed” agenda from way back when:
http://www.imc.org/atom-syntax/mail-archive/msg15001.html
Does anyone remember that? Maybe it would be wise to opt for
something slightly more generic that can express semantics useful
for feed scenarios other than archives?

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>

---

From: James M Snell <[email-masked]>
Date: Fri, 21 Oct 2005 08:41:59 -0700

>find such wide-spread support.
Most aggregators supporting self-based subscription are going to use it 
to allow the UA to subscribe to the *current* document.  In other words, 
if the user is looking at an Archive feed, the self-subscription 
mechanism will lead the user to subscribe to the archive document.  With 
"subscribe", the UA is given a hint as to where the actual subscription 
document is located.

>to the canonical location of the archive feed document.
-1.  self always points to *this* document and is only usable for 
subscribing to *this* document.  You cannot use it to point to a 
separate document.

- James

---

From: James M Snell <[email-masked]>
Date: Fri, 21 Oct 2005 08:43:05 -0700

>I'll try to propose a replacement text.
-1.  I see no reason to limit this to paging of a single feed. 

- James

---

From: Eric Scheid <[email-masked]>
Date: Sat, 22 Oct 2005 02:07:20 +1000

> find such wide-spread support.

I believe we're in a moment of grace right now and we could, with a bit of
public advocacy, get 'subscribe' established and supported for one-click
subscription.

> to the canonical location of the archive feed document.

This occurred to me too, but I had my reservations about it.

    http://www.imc.org/atom-syntax/mail-archive/msg17284.html

where I wrote:

    Fortunately, the link relation 'self' was defined in such a woolly
    way we could get away with re-purposing it. A few articles here or
    there, a bit of blog chatter, and the arrival of the fabled Developers
    Guide and we'd be set.

    I'd think this would be favourable to having to come up with a
    different pair of relations, like

        'self'       = what you subscribe to,
                       may not look anything like the chunk in front of you

        'this-chunk' = link to what you are looking at,
                       not to be confused with 'self'

e.

---

From: Thomas Broyer <[email-masked]>
Date: Fri, 21 Oct 2005 18:49:36 +0200

> -1. I see no reason to limit this to paging of a single feed.
How would you use these link relations for feed state reconstruction 
(that is, automatic handling by the Atom processor, without user action 
–except probably the "please reconstruct this feed's state" action) if 
you can't know what's pointed at?

How would you navigate through a paged search result (e.g. OpenSearch 
result feed) if previous/next might point at previous/next queries (e.g. 
history of what people searched for before you) or previous/next pages 
of the result feed?

How would you disambiguate between two "next" link on pointing at the 
next chunk of a paged feed and the other pointing at the next archived 
feed? (e.g. "August Top 100, page 2" and "September Top 100")

I thought we all agreed that we need a specific definition (related to 
paging) of these relations and were previously voting about how to call 
them (previous or prev-archive)

-- 
Thomas Broyer

---

From: James M Snell <[email-masked]>
Date: Fri, 21 Oct 2005 09:55:44 -0700

Thomas Broyer wrote:

> (that is, automatic handling by the Atom processor, without user 
> action –except probably the "please reconstruct this feed's state" 
> action) if you can't know what's pointed at?
>
> result feed) if previous/next might point at previous/next queries 
> (e.g. history of what people searched for before you) or previous/next 
> pages of the result feed?
>
> paging) of these relations and were previously voting about how to 
> call them (previous or prev-archive)
>
I wouldn't disambiguate them as I don't believe it's necessary to.  
What's needed is a way of identifying the role|purpose|intent of a feed; 
not a way of disambiguating the links between them.  An "archive" feed 
should be treated differently than a "search" feed which should be 
treated differently than a "subscription" feed. 

- James

---

From: Thomas Broyer <[email-masked]>
Date: Fri, 21 Oct 2005 19:44:33 +0200

James M Snell wrote :
>> (e.g. history of what people searched for before you) or 
>> previous/next pages of the result feed?
>>
>> next chunk of a paged feed and the other pointing at the next 
>> archived feed? (e.g. "August Top 100, page 2" and "September Top 100")
>>
>> I thought we all agreed that we need a specific definition (related 
>> to paging) of these relations and were previously voting about how to 
> What's needed is a way of identifying the role|purpose|intent of a 
> feed; not a way of disambiguating the links between them.  An 
> "archive" feed should be treated differently than a "search" feed 
> which should be treated differently than a "subscription" feed.
>
So you are OK with these feeds:

GET /top50.atom HTTP/1.0
Host: music.example.net

200 OK
Content-Type: application/atom+xml
Content-Location: /top50/2005/w42/1.atom

<feed xmlns="http://www.w3.org/2005/Atom">
<title>Week #42 Top 50</title>
<subtitle>Top 50 best selling singles</subtitle>
<link rel="self" href="http://music.example.net/top50/2005/w42/1.atom" />
<link rel="subscribe" href="http://music.example.net/top50.atom" />
<link rel="previous" title="Week #41 Top 50"
    href="http://music.example.net/top50/2005/w41/1.atom" />
<link rel="next" title="Week #42 Top 50 (11th to 21st)"
    href="http://music.example.net/top50/2005/w42/2.atom" />
<fh:incremental xmlns:fh="…">false</fh:incremental>
…

GET /top50/2005/w42/2.atom HTTP/1.0
Host: music.example.net

200 OK
Content-Type: application/atom+xml

<feed xmlns="http://www.w3.org/2005/Atom">
<title>Week #42 Top 50 (11th to 20st)</title>
<subtitle>Top 50 best selling singles</subtitle>
<link rel="self" href="http://music.example.net/top50/2005/w42/2.atom" />
<link rel="subscribe" href="http://music.example.net/top50.atom" />
<link rel="previous" title="Week #41 Top 50"
    href="http://music.example.net/top50/2005/w41/1.atom" />
<link rel="previous" title="Week #42 Top 50 (1st to 10th)"
    href="http://music.example.net/top50/2005/w42/1.atom" />
<link rel="next" title="Week #42 Top 50 (21st to 31st)"
    href="http://music.example.net/top50/2005/w42/3.atom" />
<fh:incremental xmlns:fh="…">false</fh:incremental>
…

GET /top50/2005/w41/1.atom HTTP/1.0
Host: music.example.net

200 OK
Content-Type: application/atom+xml

<feed xmlns="http://www.w3.org/2005/Atom">
<title>Week #41 Top 50</title>
<subtitle>Top 50 best selling singles</subtitle>
<link rel="self" href="http://music.example.net/top50/2005/w41/1.atom" />
<link rel="subscribe" href="http://music.example.net/top50.atom" />
<link rel="previous" title="Week #40 Top 50"
    href="http://music.example.net/top50/2005/w40/1.atom" />
<link rel="next" title="Week #42 Top 50"
    href="http://music.example.net/top50/2005/w42/1.atom" />
<link rel="next" title="Week #41 Top 50 (11th to 21st)"
    href="http://music.example.net/top50/2005/w41/2.atom" />
<fh:incremental xmlns:fh="…">false</fh:incremental>
…

How do you expect a newsreader to *automatically* download this week's 
50 entries without downloading last week's entries instead?
(and show you links/buttons for you to ask download and display of 
previous/next week's Top 50)

-- 
Thomas Broyer

---

From: James M Snell <[email-masked]>
Date: Fri, 21 Oct 2005 10:50:27 -0700

> So you are OK with these feeds:
Yes, they all look good to me.

---

From: Thomas Broyer <[email-masked]>
Date: Fri, 21 Oct 2005 19:58:35 +0200

You didn't answer my last question:
>> How do you expect a newsreader to *automatically* download this 
>> week's 50 entries without downloading last week's entries instead?

-- 
Thomas Broyer

---

From: James M Snell <[email-masked]>
Date: Fri, 21 Oct 2005 11:07:54 -0700

>>> previous/next week's Top 50) 
Ah, didn't see the last question ;-)

My choice would be for it to not automatically download 'em.  Just 
present the user with the available links and let 'em follow the ones 
they want.

- James

---

From: "James Holderness" <[email-masked]>
Date: Fri, 21 Oct 2005 19:38:27 +0100

>>> previous/next week's Top 50)

I see where you're coming from, but this kind of thing is already a problem 
without even taking links into consideration.

For an aggregator to be able to do anything vaguely meaningful with a feed 
it has to be able to assume that the feed is incremental in nature. When the 
feed is updated an aggregator will by default assume that any new items can 
safely be added to the top of an inbox, any updates are updates to existing 
items, and any removed items have merely "fallen off the bottom" of the 
feed.

However, as soon as we introduce the concept of non-incremental feeds, an 
aggregator that is not aware of the concept will fail to process such a feed 
in a meaningful way. We've created a situation where an aggregator has to be 
aware of the (still to be specified) fh:incremental extension, Microsoft's 
simple list extensions for RSS, and whatever future extensions may arise; 
basically the ability to see into the future.

This problem merely repeats itself when it comes to processing archives. 
When we receive a "next" link, ideally we would like to assume it's a 
pointer to the next archive to be processed. For a regular incremental feed 
this isn't a problem. Even a search feed could be processed safely if 
ordered the right way. However, when it comes to non-incremental feeds we're 
screwed again. I agree that it sucks, but we're already stuck with that 
situation so I'm not sure that these links will make things any worse.

Regards
James

---

From: Thomas Broyer <[email-masked]>
Date: Fri, 21 Oct 2005 22:45:39 +0200

> present the user with the available links and let 'em follow the ones 

So you're indirectly telling Mark to either:
 - continue using an fh:prev extension element
 - define a more specific link relation (previous-page?)
 - stop dreaming of an automatic process for feed state reconstruction…

I'd rather say: define first/previous/next/last for "paging" only and 
use them for feed-state reconstruction. If someone wants some other 
"ordered" relationships, he'll define a new link relation (e.g. 
webring-previous).
Moreover, this is consistent with the OpenSearch start/previous/next/end 
link relations (where start=first and end=last).

-- 
Thomas Broyer

---

From: Thomas Broyer <[email-masked]>
Date: Fri, 21 Oct 2005 23:23:05 +0200

James Holderness wrote:
> I see where you're coming from, but this kind of thing is already a 
> problem without even taking links into consideration.
>
> For an aggregator to be able to do anything vaguely meaningful with a 
> feed it has to be able to assume that the feed is incremental in nature.
As I already explained, paging is orthogonal to the incremental nature 
of a feed. An incremental feed will be chunked as explained in Mark's 
current Feed History draft (just use an atom:link[@rel="previous"] 
instead of the fh:prev extension element) and a non-incremental feed as 
described in the OpenSearch result 1.1 draft. The difference is that an 
incremental feed is sorted by date so the older parts will become more 
or less "stable" over time; while a non-incremental feed is replaced as 
a whole each time it is updated, with no other relation to time.
> When the feed is updated an aggregator will by default assume that any 
> new items can safely be added to the top of an inbox, any updates are 
> updates to existing items, and any removed items have merely "fallen 
> off the bottom" of the feed.
>
> However, as soon as we introduce the concept of non-incremental feeds, 
> an aggregator that is not aware of the concept will fail to process 
> such a feed in a meaningful way. We've created a situation where an 
> aggregator has to be aware of the (still to be specified) 
> fh:incremental extension, Microsoft's simple list extensions for RSS, 
> and whatever future extensions may arise; basically the ability to see 
> into the future.
>
> This problem merely repeats itself when it comes to processing 
> archives. When we receive a "next" link, ideally we would like to 
> assume it's a pointer to the next archive to be processed. For a 
> regular incremental feed this isn't a problem. Even a search feed 
> could be processed safely if ordered the right way. However, when it 
> comes to non-incremental feeds we're screwed again. I agree that it 
> sucks, but we're already stuck with that situation so I'm not sure 
> that these links will make things any worse.
What's the difference between a search feed and a non-incremental feed? 
Aren't search feeds one facet of non-incremental feeds?

The difference between incremental and non-incremental feeds is that, 
when dealing with incremental feeds, paging can be seen as a link to 
archives, as the feed is tightly related to time. When dealing with 
non-incremental feed, the "previous page" is totally different than the 
"previous archived snapshot".
However, an incremental feed could take advantage of differentiating 
between paging and "archive linking": if linking to archives uses an 
atom:link[@rel="archives"] (or call it "history" if you prefer) to point 
at an incremental feed where each entry describes an archived set of 
entries (see [1] for a more detailed example); such a feed has the 
advantage of paging that it allows direct access to a specific point of 
time inside the feed "pages". Each "archived set of entries" could for 
example cover one or two week, so a user could navigate through the 
"feed state" or "feed history" not only by going from pages to pages but 
also by accessing archived chunks via an "index" or "table of contents".

-- 
Thomas Broyer

---

From: Thomas Broyer <[email-masked]>
Date: Fri, 21 Oct 2005 23:26:34 +0200

Thomas Broyer wrote:
> atom:link[@rel="archives"] (or call it "history" if you prefer) to 
> point at an incremental feed where each entry describes an archived 
> set of entries (see [1] for a more detailed example); such a feed has 
> the advantage of paging that it allows direct access to a specific 
> point of time inside the feed "pages". Each "archived set of entries" 
> could for example cover one or two week, so a user could navigate 
> through the "feed state" or "feed history" not only by going from 
> pages to pages but also by accessing archived chunks via an "index" or 
> "table of contents".
>

Sorry, forgot the link:

[1] http://www.imc.org/atom-syntax/mail-archive/msg17308.html

-- 
Thomas Broyer

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Oct 2005 15:13:25 -0700

That's what I was trying to do here:

> be subscribed to to keep abreast of recent changes in the feed.  
> When different from the URI of the document where it occurs, it  
> indicates that its value should be used for this purpose in place  
> of the current document's URI.

Any suggestions?

On 21/10/2005, at 8:19 AM, Tim Bray wrote:

> Makes sense, but not self-evident.  I would think that the  
> usefulness of this thing would be improved by a few words of  
> explanation for those who come upon it without knowing the history.  
> -Tim

---

From: James M Snell <[email-masked]>
Date: Fri, 21 Oct 2005 16:11:18 -0700

Mark Nottingham wrote:

>
> So, an aggregator comes across a feed in the woods. It sees it has  
> "previous" and maybe "next" link relations.
>
> The point that (I think) Thomas is making is that the people who  
> leave that feed document in the woods had better be comfortable with  
> the aggregator following the links and reconstructing the feed. After  
> all, most of them already keep feed history without any hints about  
> previous archives at all; if you give them a "previous" link, it'll  
> be a red flag to a bull, no matter what my spec says.
>
> So, a feed that used "next" and "previous" to link to different weeks  
> of the top 100 (for example) would get jumbled up pretty bad, and not  
> display too well in aggregators.
>
> My thinking is that if we go down this road, one of two things will  
> happen;
>
> a) Aggregators will start paying attention to special flags (like  
> fh:incremental) that tell them the nature of the feed, and feeds will  
> start using them real quick.
>
> b) Non-incremental uses of "previous" and "next" will die off,  
> because people who use them for those purposes will see aggregators  
> do strange things with their feeds.
>
> I think (b) is somewhat more likely. Either way, I'm OK, but I do  
> note that (a) leads to two separate extensions being required in the  
> feed, when the same purpose could be served by a more specialised  
> link relation, and (b) will lead to other, more specialised link  
> relations being created anyway.
>
> So I'm not sure why it's so important we have an effectively semantic- 
> free "previous" and "next", but if that's what happens, I'm  
> reasonably confident my use case will still be met.
>
I'm perfectly happy with leaving previous and next free of any semantics 
right now and letting the market sort things out.  If more specific link 
relations prove to be necessary, then so be it, define the more specific 
link relations.  If the market can get by with generic links + some kind 
of extra flag (e.g. incremental=true, etc) then great.  if your case (b) 
dies off... that's great too.  The point is, let's not over specify this 
thing right now; leave it open enough for the market to figure out how 
to use it.

- James

---

From: Tim Bray <[email-masked]>
Date: Fri, 21 Oct 2005 16:16:44 -0700

On Oct 21, 2005, at 3:13 PM, Mark Nottingham wrote:

>> set of the most recent entries in the feed. This URI is intended  
>> to be subscribed to to keep abreast of recent changes in the feed.  

Yes.  Acknowledge the specific case of an archival feed, an example  
is worth a thousand words.

And discuss why this exist when Atom already has <link rel="self",  
specifically designed to support auto-subscribe. -Tim

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Oct 2005 15:38:06 -0700

So, Thomas -- are you still -1 on this? I don't see how we can come  
to consensus on a more specific set of relations, given the flood of  
+1s on the generic ones.

On 21/10/2005, at 2:23 PM, Thomas Broyer wrote:

>>>>> (and show you links/buttons for you to ask download and display  
>>>>> of previous/next week's Top 50)
>>>>>
>>>>>
>>
>> I see where you're coming from, but this kind of thing is already  
>> a problem without even taking links into consideration.
>>
>> For an aggregator to be able to do anything vaguely meaningful  
>> with a feed it has to be able to assume that the feed is  
>> incremental in nature.
>>
>>
> As I already explained, paging is orthogonal to the incremental  
> nature of a feed. An incremental feed will be chunked as explained  
> in Mark's current Feed History draft (just use an atom:link 
> [@rel="previous"] instead of the fh:prev extension element) and a  
> non-incremental feed as described in the OpenSearch result 1.1  
> draft. The difference is that an incremental feed is sorted by date  
> so the older parts will become more or less "stable" over time;  
> while a non-incremental feed is replaced as a whole each time it is  
> updated, with no other relation to time.
>
>
>> When the feed is updated an aggregator will by default assume that  
>> any new items can safely be added to the top of an inbox, any  
>> updates are updates to existing items, and any removed items have  
>> merely "fallen off the bottom" of the feed.
>>
>> However, as soon as we introduce the concept of non-incremental  
>> feeds, an aggregator that is not aware of the concept will fail to  
>> process such a feed in a meaningful way. We've created a situation  
>> where an aggregator has to be aware of the (still to be specified)  
>> fh:incremental extension, Microsoft's simple list extensions for  
>> RSS, and whatever future extensions may arise; basically the  
>> ability to see into the future.
>>
>> could be processed safely if ordered the right way. However, when  
>> it comes to non-incremental feeds we're screwed again. I agree  
>> that it sucks, but we're already stuck with that situation so I'm  
>> not sure that these links will make things any worse.
>>
>>
> What's the difference between a search feed and a non-incremental  
> feed? Aren't search feeds one facet of non-incremental feeds?
>
> The difference between incremental and non-incremental feeds is  
> that, when dealing with incremental feeds, paging can be seen as a  
> link to archives, as the feed is tightly related to time. When  
> dealing with non-incremental feed, the "previous page" is totally  
> different than the "previous archived snapshot".
> However, an incremental feed could take advantage of  
> differentiating between paging and "archive linking": if linking to  
> archives uses an atom:link[@rel="archives"] (or call it "history"  
> if you prefer) to point at an incremental feed where each entry  
> describes an archived set of entries (see [1] for a more detailed  
> example); such a feed has the advantage of paging that it allows  
> direct access to a specific point of time inside the feed "pages".  
> Each "archived set of entries" could for example cover one or two  
> week, so a user could navigate through the "feed state" or "feed  
> history" not only by going from pages to pages but also by  
> accessing archived chunks via an "index" or "table of contents".
>
> -- 
> Thomas Broyer
>
>
>
>
>

--
Mark Nottingham   Principal Technologist
Office of the CTO   BEA Systems

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Oct 2005 17:03:28 -0700

How about:

  -  Description: A URI that refers to a feed document containing a  
set of the most recent entries in the feed. This URI is intended to  
be subscribed to to keep abreast of recent changes in the feed; when  
different from the URI of the document where it occurs, it indicates  
that its value should be used for this purpose in place of the  
current document's URI. For example, an archived feed document might  
contain a "subscribe" relation that points to the subscription feed's  
location, so that clients subscribe to the appropriate link. Note  
that the "self" relation was designed for a similar purpose, but is  
not suitable for that use in other feeds, whereas this relation can  
be used in those situations.

On 21/10/2005, at 4:16 PM, Tim Bray wrote:

>>>  -  Description: A URI that refers to a feed document containing  
>>> a set of the most recent entries in the feed. This URI is  
>>> intended to be subscribed to to keep abreast of recent changes in  
>>> the feed. When different from the URI of the document where it  
>>> occurs, it indicates that its value should be used for this  
>>> purpose in place of the current document's URI.
>>>
>>
>> Any suggestions?
>>
>

---

From: "James Holderness" <[email-masked]>
Date: Sat, 22 Oct 2005 02:19:28 +0100

Thomas Broyer wrote:
> As I already explained, paging is orthogonal to the incremental nature of 
> a feed. An incremental feed will be chunked as explained in Mark's current 
> Feed History draft (just use an atom:link[@rel="previous"] instead of the 
> fh:prev extension element) and a non-incremental feed as described in the 
> OpenSearch result 1.1 draft.

I beg to differ. I think the incremental state of a feed is very relevant to 
paging. If the aggregator does not know that a feed is non-incremental it 
will not be able to process the feed document in a meaningful manner. And 
when I say non-incremental I mean something like a "Top 10" list where the 
entries in the feed document are a complete replacement for any entries that 
the aggregator may have previously received from that feed.

> Aren't search feeds one facet of non-incremental feeds?

Not necessarily, no. A search feed could quite easily be implemented as an 
incremental feed. This is the most sensible approach since it would allow 
the feed to be viewed in all existing aggregators without requiring a 
special knowledge of non-incremental feeds.

The initial feed document consists of all known results at the time the 
search is initiated. As new results are discovered over time, the feed can 
be updated by adding new entries to the top of the feed in much the same way 
that new entries would be added to the top of a blogging feed. In fact, if 
you do a search with something like feedster, this is exactly the sort of 
feed you will get back.

Once you add paging support, the search results can be split over multiple 
pages in much the same way that an incremental feed splits its entries over 
multiple archives. As long as the search feed adds new entries to the top of 
the subscribed document and moves older results into archives as they fall 
out the bottom, it can be processed in exactly the same way as a regular 
incremental feed.

> example cover one or two week, so a user could navigate through the "feed 
> state" or "feed history" not only by going from pages to pages but also by 

This is all very interesting, but not possible without more link extensions 
which don't exist yet. With what we have so far we can do incremental feed 
archives; we can do at least some form of searching; we can do 
non-incremental feeds (of the "Top 10" variety) with history. I think that's 
a good start.

Regards
James

---

From: Antone Roundy <[email-masked]>
Date: Fri, 21 Oct 2005 20:10:16 -0600

On Oct 21, 2005, at 7:19 PM, James Holderness wrote:
> Not necessarily, no. A search feed could quite easily be  
> implemented as an incremental feed. This is the most sensible  
> approach since it would allow the feed to be viewed in all existing  
> aggregators without requiring a special knowledge of non- 
> incremental feeds.
If your goal is to work as well as possible with today's client  
software, then bending your data to fit their model is the most  
sensible approach, but that's not always the goal.

> The initial feed document consists of all known results at the time  
> the search is initiated. As new results are discovered over time,  
> the feed can be updated by adding new entries to the top of the  
> feed in much the same way that new entries would be added to the  
> top of a blogging feed. In fact, if you do a search with something  
> like feedster, this is exactly the sort of feed you will get back.
If creation time is relevant to the data being searched, then this  
makes sense.  But what if I want to subscribe to the top 10 Google  
results for some keywords I'm trying to optimize my site for  
(ignoring the fact that Google doesn't return search results in any  
feed format right now)?  Or what about alternative sort orders which  
are available on sites like Feedster, Google News, etc.? (You can  
sort by relevance rather than date--the date still has some weight,  
but the results aren't strictly in date order). How about Amazon.com  
affiliates who want to use an RSS parser to display affiliates links  
to "best sellers" search results?  There are a lot of search use  
cases that don't fit the incremental model.

All that said, search results are often a bit different than "top 10"  
lists and the like.  With search results, you often don't want to  
view the contents of the feed in order all at once--the first time  
you do, but after that, you may just want to see new things as they  
make it up into the top positions.  Today's clients can handle that  
just fine, unless you want to monitor more than just the first page  
of results.

---

From: Tim Bray <[email-masked]>
Date: Fri, 21 Oct 2005 23:25:09 -0700

On Oct 21, 2005, at 5:03 PM, Mark Nottingham wrote:

> be subscribed to to keep abreast of recent changes in the feed;  
> when different from the URI of the document where it occurs, it  
> of the current document's URI. For example, an archived feed  
> document might contain a "subscribe" relation that points to the  
> subscription feed's location, so that clients subscribe to the  
> appropriate link. Note that the "self" relation was designed for a  
> similar purpose, but is not suitable for that use in other feeds,  
> whereas this relation can be used in those situations.

On consideration, I am -1 to rel="subscribe".  The reason is this:  
one of the big potential value-adds Atom brings is a standards- 
compliant way to do one-click auto-subscribe, via <link rel="self" / 
 >.  You are proposing to introduce a <link rel="subscribe" /> which  
is there to support autosubscribe.  But, it turns out, only in the  
special case where the feed is static and you wouldn't actually  
subscribe to it.  I think the risk of confusing implementors and  
weakening the value proposition around <link rel="self" greatly  
exceeds the benefit of supporting this special case.

Plus, I think the thing isn't properly named.  It isn't actually  
"subscribe in the general case", it's "in the special case where the  
version you are looking at is static, point to the places where the  
changes happen because they don't happen here".  So, perhaps we  
should consider rel="current-self" or rel="dynamic-version" or some  
such.

Finally, markup design that claims to enforce a specific action is  
always questionable.  The great virtue of descriptive markup in  
general is that the tags tell you not what to do with things but what  
they are.  So on that basis, rel="current-version" or something is  
better design.  -Tim

---

From: "James Holderness" <[email-masked]>
Date: Sat, 22 Oct 2005 10:10:55 +0100

Tim Bray wrote:
> On consideration, I am -1 to rel="subscribe".  The reason is this:  one of 
> the big potential value-adds Atom brings is a standards- compliant way to 
> do one-click auto-subscribe, via <link rel="self" /
> special case where the feed is static and you wouldn't actually  subscribe 
> to it.  I think the risk of confusing implementors and  weakening the 
> value proposition around <link rel="self" greatly  exceeds the benefit of 
> supporting this special case.

At the time "subscribe" was proposed it wasn't clear that there would be a 
"first" and "last". However, since that is now the case, would it not 
short-circuit a whole lot of argument if we just threw out "subscribe" 
altogether?

Determining whether an Atom document is an archive can be achieved by 
looking for the presence of a "prev" link and/or a "first" link that is not 
equal to "self". As for finding the subscribtion URI itself - that should 
just be the "first" link shouldn't it?

I don't want to get dragged back into a long argument on this so if you 
think this is a stupid idea don't expect any defence from me. I'm just 
throwing it out there with the hope that it might be workable.

Regards
James

---

From: "James Holderness" <[email-masked]>
Date: Sat, 22 Oct 2005 10:35:58 +0100

Antone Roundy write:
> If creation time is relevant to the data being searched, then this  makes 
> sense.  But what if I want to subscribe to the top 10 Google  results for 
> some keywords I'm trying to optimize my site for  (ignoring the fact that 
> Google doesn't return search results in any  feed format right now)?

That kind of search would be best implemented as a non-incremental feed. To 
me it sounds exactly the same as a "Top 10" movie list.

> Or what about alternative sort orders which  are available on sites like 
> Feedster, Google News, etc.? (You can  sort by relevance rather than 
> date--the date still has some weight,  but the results aren't strictly in 
> date order).

Personally I would still implement this as an incremental feed (new results 
being added to the top of the feed). If you want an alternative order then 
you can make use of something like the rank extension that James was 
proposing.

I really don't see any other way that something like this could be 
implemented unless you included every single search result whenever the feed 
was refreshed. For example, let say you have a feed with a limit of 10 
items. A new search result is discovered that needs to be sorted (for 
whatever reason) to the bottom of the list (i.e. position 11). How do you go 
about informing the user of this new result short of growing the feed to 
include 11 items?

Adding a second page to your search results is not a feasible option. You 
can't expect an aggregator to retrieve the full list of pages every time it 
refreshes just on the off chance that something new was inserted way down at 
the bottom of the list.

> How about Amazon.com  affiliates who want to use an RSS parser to display 
> affiliates links  to "best sellers" search results?  There are a lot of 
> search use  cases that don't fit the incremental model.

True. I'm not disputing that. I'm just saying that a many (most?) searches 
can be implemented with the incremental model and those that can't can 
usually be implemented with a "top 10" non-incremental model.

> lists and the like.  With search results, you often don't want to  view 
> the contents of the feed in order all at once--the first time  you do, but 
> after that, you may just want to see new things as they  make it up into 
> the top positions.

This still looks like the incremental model to me. The fact that the initial 
results aren't in date order doesn't matter from a processing point of view. 
All that matters is that new results are added to the top of the list.

Regards
James

---

From: Eric Scheid <[email-masked]>
Date: Sat, 22 Oct 2005 20:28:34 +1000

On 22/10/05 4:25 PM, "Tim Bray" <[email-masked]> wrote:
> compliant way to do one-click auto-subscribe, via <link rel="self"/>
> You are proposing to introduce a <link rel="subscribe" /> which

I think you've got the special case turned around. That is, if you are
looking at a representation of the active feed then the 'self' link would
happen to match the 'subscribe' link, which is the exception. The defining
text in the spec says this about 'self'

    The value "self" signifies that the IRI in the value of
    the href attribute identifies a resource equivalent to
    the containing element.

atom:link[@rel='self'] can also appear in atom:entry, which would be used
for pointing to Atom Entry Documents, which explains why the text says "the
containing element". I don't believe anyone is thinking we should be able to
subscribe to Atom Entry Documents.

> "subscribe in the general case",

Are we yet in a situation were we have a preponderance of deployed feeds
using 'self' to mean 'subscribe'? Actually, that's the wrong question to ask
(see below)

Are we yet in a situation were we have preponderance of deployed
applications that look for 'self' for subscription purposes?

I think we're in the early days of Atom 1.0 adoption, and we can very likely
talk up via blogs, articles, etc the idea of "subscribe in the general case"
using @rel='subscribe'.

The only deployed inertia we have to over come is with aggregator
developers. We don't have to worry about any deployed feeds that only have
'self' links (I'll explain in a moment). We only need to explain things to
the various aggregator developers. The general idea would be for the
application to look for a 'subscribe' link first up, and failing that a
'self' link. Very simple.

We don't have to worry about deployed feeds with 'self' links. They won't
have 'subscribe' links unless they put effort into putting them there, in
which case they could quite simply s/self/subscribe/. If a feed does not
have 'subscribe' then 'self' would be found with the fall back.

Could there eventually be feeds out there with only 'subscribe' and not
'self' ... they would be funky but not invalid (it's a SHOULD requirement,
not a MUST) ... and the only applications that would fail to subscribe would
be outdated p.o.c.

> it's "in the special case where the

I'm thinking is that 'self' was not properly named in the first place. If
the intent of that link was for subscription purposes, for pointing to the
most recent updates for that feed, then it should have been named
'subscribe' or (as you say) a descriptive word indicating that is the sort
of thing one would want to do with it.

The idea of embedding a link to the URI to subscribe to was a very good one.

Calling it 'self' was a bone-headed one. We were so caught up with the idea
of being able to subscribe to the active feed we forgot to consider the case
of seeing that link in the context of an archive.

This is the same class of bone-headed mistake as using
/html/head/link[@rel='alternate',@type="application/atom+xml"] for feed auto
discovery. Consider for example this HTML page...

    <html>
    <head>
        <title>Archive for January 2005</title>
        <link type="application/atom+xml" rel="alternate" href="[1]">
        <link type="application/atom+xml" rel="alternate" href="[2]">
    </head>
    <body>...</body>
    </html>

Now, one of those two links points to the URI you should subscribe to if you
want to see the most recent updates, and the other link points to a URI
which will give you the Atom Feed Document for the Archive for January 2005.
Only problem is, you can't tell them apart.

We painted ourselves into a corner.

This 'self' thing is the same class of error as 'alternate'.

Can we please learn from that mistake?

> better design.

The verb/noun thing is a good point ... but how could 'self', which
generally speaking, would point to a completely different set of data, be
considered "descriptive"?

FWIW, @rel="subscription" was offered up first, but then got switched to
'subscribe'.

e.

---

From: "James Holderness" <[email-masked]>
Date: Sat, 22 Oct 2005 11:53:41 +0100

> FWIW, @rel="subscription" was offered up first, but then got switched to

I guess that was my fault. My reasoning was that it would be more in line 
with the Atom Publication Protocol [1] which uses verbs for its link 
relations when they are suggesting an action ("edit", "srcedit"). If that's 
the wrong thing to do, somebody may want to inform the WG in case they 
aren't following this.

Regards
James

[1] http://www.ietf.org/internet-drafts/draft-ietf-atompub-protocol-05.txt 
(section 12)

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 22 Oct 2005 08:40:33 -0700

You seem to be saying that because link/@rel="self" was designed for  
a specific purpose, and even though its definition is quite  
descriptive (its definition *only* says it should be used to link to  
the current document; -11 says nothing about subscription) it should  
be the only way defined to do subscription.

At the same time, you say that descriptive markup is good, and that  
relations shouldn't be proscriptive.

I'm having trouble reconciling these statements. Is "self" really a  
proscriptive relation that's poorly documented? Standardising  
mechanisms where *wink wink, nudge nudge* everybody knows what to do  
are bad practice, and I'm very against reading more than what's  
documented being read into "self". It's a link to the current  
document; that's it. If it was intended to do more, it's poorly  
documented and frankly poorly designed, because it's only useful in  
the subscription document itself.

OTOH, I'm happy to make this relation more declarative. How about:

  -  Attribute Value: current
  -  Description: A URI that, when dereferenced, returns a feed  
document containing the most recent entries in the feed.
  -  Expected display characteristics: Undefined.
  -  Security considerations: Automated agents should take care when  
this relation crosses administrative domains (e.g., the URI has a  
different authority than the current document).

?

Such a relation would be usable in both archived and "dynamic" feed  
documents (in the latter case, it would be the same as "self").

On 21/10/2005, at 11:25 PM, Tim Bray wrote:

>> to be subscribed to to keep abreast of recent changes in the feed;  
> "subscribe in the general case", it's "in the special case where  
> the version you are looking at is static, point to the places where  
> the changes happen because they don't happen here".  So, perhaps we  
> general is that the tags tell you not what to do with things but  
> what they are.  So on that basis, rel="current-version" or  
> something is better design.  -Tim

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 22 Oct 2005 08:47:01 -0700

First and Last are (or at least can be) "static"; i.e. one can read  
the relations, as currently written, as saying that they point to the  
specific set of entries ("archive") that are first and list,  
respectively, at the time that the feed is minted. Subscribing to one  
of those would be... bad.

If we had more specific relations, this would certainly be a lot  
easier. Keeping everything so loose and semantic-free seems to me  
like premature optimisation and a barrier to interoperability.

HTTP, for example, seems to work just fine, despite having concrete  
semantics that are grounded in specific use cases for almost all of  
its headers (indeed, the least-used ones are those that are more  
descriptive).

On 22/10/2005, at 2:10 AM, James Holderness wrote:

> Tim Bray wrote:
>
>> On consideration, I am -1 to rel="subscribe".  The reason is  
>> this:  one of the big potential value-adds Atom brings is a  
>> standards- compliant way to do one-click auto-subscribe, via <link  
>> rel="self" /
> At the time "subscribe" was proposed it wasn't clear that there  
> would be a "first" and "last". However, since that is now the case,  
> would it not short-circuit a whole lot of argument if we just threw  
> out "subscribe" altogether?
>
> Determining whether an Atom document is an archive can be achieved  
> by looking for the presence of a "prev" link and/or a "first" link  
> that is not equal to "self". As for finding the subscribtion URI  
> itself - that should just be the "first" link shouldn't it?
>
> I don't want to get dragged back into a long argument on this so if  
> you think this is a stupid idea don't expect any defence from me.  
> I'm just throwing it out there with the hope that it might be  
> workable.
>
> Regards
> James
>
>

---

From: Tim Bray <[email-masked]>
Date: Sat, 22 Oct 2005 09:50:10 -0700

> subscribe to Atom Entry Documents.

I'm entirely unconvinced.  There are two kinds of feed, the dynamic  
kind and the static kind.  For dynamic feeds, use rel="self" to  
subscribe.   For static feeds, use some new relationship like  
rel="dynamic" to point to the dynamic version that you might  
subscribe to.

I agree that, in retrospect, the spec should have said that one  
intended use of rel="self" is to support auto-subscribe.  I don't  
think adding on a new link relationship is the right answer.

> using 'self' to mean 'subscribe'? Actually, that's the wrong  

Well, all the Atom feeds I see use rel="self" in the way you'd expect.

> applications that look for 'self' for subscription purposes?

Well, no, because we actually don't have that many apps yet that  
really grok Atom.  When they do, and they want to do auto- 
subscription, Atom is well-equipped to support it.

> talk up via blogs, articles, etc the idea of "subscribe in the  

No.  We already have a way to do that and don't need to invent another.

-1 on rel="subscribe". -Tim

(oh, and I should have said, +1 on the rest of those relationships)

---

From: Tim Bray <[email-masked]>
Date: Sat, 22 Oct 2005 09:52:45 -0700

On Oct 22, 2005, at 8:40 AM, Mark Nottingham wrote:

> You seem to be saying that because link/@rel="self" was designed  
> for a specific purpose, and even though its definition is quite  
> descriptive (its definition *only* says it should be used to link  
> to the current document; -11 says nothing about subscription) it  
> should be the only way defined to do subscription.

Agreed that the description could be better.  What I'm actually  
saying is "since we already have a way to do subscription, we don't  
need to invent another."  Also that the problem of pointing from the  
static/archived version of a feed to the dynamic/subscribable one is  
a related but different problem, and the one that you ought to be  
solving.

> different authority than the current document).

Thank you.  +1 -Tim

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 22 Oct 2005 10:16:12 -0700

Great! I'll summarise where they are and do a last call.

On 22/10/2005, at 9:52 AM, Tim Bray wrote:

> need to invent another."  Also that the problem of pointing from  
> the static/archived version of a feed to the dynamic/subscribable  
> one is a related but different problem, and the one that you ought  
> to be solving.
>
>
>>  -  Security considerations: Automated agents should take care  
>> when this relation crosses administrative domains (e.g., the URI  
>> has a different authority than the current document).
>>
>
> Thank you.  +1 -Tim
>
>
>

---

From: [email-masked] (Peter Robinson)
Date: Sun, 23 Oct 2005 21:04:10 +0100

Having not spoken up so far, I'd like add my 2c.  My atom feeds are all
dynamic OpenSearch-style search results.  By default they also happen to
be traditional 'incremental' blog-style feeds.  Instead of static
archives, I have dynamic page=N sliding windows.

I'd like to be able to include machine-interpretable 'prev'/'next' links
so that a user agent that wants to can reconstruct a complete (logical)
feed whether that means a 'history' or just the current snapshot of
results (and at a single instant in time it will get the same set of
entries in either case).

I'd also like to be able to include a machine-interpretable
rel=subscribe link since in general the url I have to put in rel=self is
not the optimal link for subscription.

>   -  Attribute Value: previous

With regret I am -1 to those since I cannot use them consistently (with
those definitions) from 'page 1' of my feeds depending whether I have my
OpenSearch hat on or my incremental feed hat.

I need a 'pure paging' definition of those relations that can also be
used to reconstruct feed history as a side-effect in the special case
where my feeds happen to be 'incremental'.

+1

I am happy with that wording, though it may have been improved later in
the thread.  Whatever may have been the original intention, the Atom
spec is very clear that rel=self is very different from any possible
rel=subscribe.

Regards,

Peter
-- 
Peter Robinson
<http://www.ticketswitch.com/> Concerts, sport and theatre tickets

---

From: Robert Sayre <[email-masked]>
Date: Sun, 23 Oct 2005 18:49:21 -0400

> results (and at a single instant in time it will get the same set of

I don't see how Mark's definitions prevent this. +1 to all of them.
For one thing, I have hard time imagining how the proposed definitions
could ever be *wrong*.

Robert Sayre

---

From: "Thomas Broyer" <[email-masked]>
Date: Mon, 24 Oct 2005 09:31:00 +0200 (CEST)

> thing right now; leave it open enough for the market to figure out how

What are the use cases right now?

 - Mark's proposed feed state reconstruction
 - OpenSearch result feeds chunking

This is just "paging".

What is it also allowing?
 - publish any non-incremental (i.e. non time-based, like OpenSearch
results) feeds chunked in small documents (Top 50 in 5 pages of 10
entries)

What is it not covering?
 - linking between snapshots of non-incremental feeds (last week's Top 50,
OpenSearch result of previous query)
 - linking between different "sites" (i.e. webrings)

This has not yet proven to be really needed (e.g. the Top 50 web site I
saw didn't provide archives of previous rankings).
When there'll be such a need, then we'll define a new link relation (I
already proposed "archives"/"history" to link to a "table of contents"
feed allowing navigation to e.g. snapshots of non-incremental feeds;
another link relation for the "webring" use case if it proves to be needed
one day).

-- 
Thomas Broyer

---

From: "Thomas Broyer" <[email-masked]>
Date: Mon, 24 Oct 2005 10:01:46 +0200 (CEST)

James Holderness  wrote:
>> described in the OpenSearch result 1.1 draft.
>
> I beg to differ. I think the incremental state of a feed is very relevant
> to paging. If the aggregator does not know that a feed is non-incremental
> it will not be able to process the feed document in a meaningful manner.

Yes, but that's orthogonal to paging.

> And when I say non-incremental I mean something like a "Top 10" list where
> the entries in the feed document are a complete replacement for any
> entries that the aggregator may have previously received from that feed.

I have the same definition.

> be updated by adding new entries to the top of the feed in much the same
> way that new entries would be added to the top of a blogging feed. In
> fact, if you do a search with something like feedster, this is exactly the
> sort of feed you will get back.

You're describing the PubSub behaviour, which is not IMO a "search engine"
but an aggregator with filtering capabilities. PubSub filters are quite
similar to the "category feeds" you see sometimes: "I don't want the whole
blog entries, just the ones belonging to that category, or tagged with
that word". With PubSub, you say "I don't want the whole web entries, just
the one matching that filter".

You're not describing a "search feed" (or maybe more a search *result*
feed, like OpenSearch result feeds) but a "filtered aggregate feed" (as
published by PubSub).

> This is all very interesting, but not possible without more link
> extensions which don't exist yet.

Wait a bit and I'll propose them for registration. When they'll exist,
what would become your argument?

> With what we have so far we can do incremental feed archives; we can do at
> least some form of searching; we can do non-incremental feeds (of the "Top
> 10" variety) with history. I think that's a good start.

But we also want paged non-incremental feeds (OpenSearch result feeds),
while "non-incremental feeds with history" have not yet proven to be
needed.

You're trying to describe "incremental feed paging" and "navigation
through non-incremental feed snapshots" with the same link relation. When
people will want "non-incremental feed paging" (and this is already a
need), we'll have two link relations related to paging (for incremental
and non-incremental feeds) and one that can also be used to navigate into
non-incremental feeds history.

Here's a chunked incremental feed (each chunk has 10 entries):

1    11   21   31   41   51   61   71   81
|----|----|----|----|----|----|----|----|----|
^    ^
`----'
  This is a chunk.

present <------------ time -------------> past

previous/next link from chunk to chunk.

Entries are added into or before the #1 chunked above.

Here's a chunked non-incremental feed (e.g. OpenSearch result feed; each
chunk has 10 entries):
1    11   21   31   41   51   61   71   81
|----|----|----|----|----|----|----|----|----|
^    ^
`----'
  This is a chunk.

++ <------------- relevance --------------> --

previous/next link from chunk to chunk.

Here are non-chunked non-incremental feeds (e.g. Top 10 feeds with history):

1
|----|   -> Oct. 24 Top 10

1
|----|   -> Oct. 17 Top 10

1
|----|   -> Oct. 10 Top 10

previous/next would link from snapshot to snapshot ?!

And finally, here are chunked non-incremental feeds (e.g. Top 50 feeds
with history, each chunk has 10 entries):

++ <-------------- ranking ---------------> --

1    11   21   31   41   51   61   71   81
|----|----|----|----|----|----|----|----|----|  -> Sept. 2005
^    ^
`----'
  This is a chunk.

1    11   21   31   41   51   61   71   81
|----|----|----|----|----|----|----|----|----|  -> Aug. 2005

1    11   21   31   41   51   61   71   81
|----|----|----|----|----|----|----|----|----|  -> Jul. 2005

What would mean previous/next here?

I'm proposing previous/next linking from chunk to chunk inside the same
snapshot and adding a new link relation (or set of link relations) for
linking from snapshot to snapshot.

Do you now see what I'm talking about?

-- 
Thomas Broyer

---

From: Eric Scheid <[email-masked]>
Date: Mon, 24 Oct 2005 18:17:06 +1000

+1

---

From: "James Holderness" <[email-masked]>
Date: Mon, 24 Oct 2005 15:13:56 +0100

> Yes, but that's orthogonal to paging.

I think I see where the misunderstanding has arisen here. We don't agree on 
what constitutes paging in a non-incremental feed. You think it means 
splitting a single feed into multiple pages whereas I interpret the pages as 
being a history of past feeds. With your interpretation the incremental 
state is irrelevant - with mine it's very relevant.

> feed, like OpenSearch result feeds) but a "filtered aggregate feed" (as

Actually I was thinking of something somewhere in the middle. Like when you 
do a search in Google Groups it offers to notify you of new results via 
email. I would have thought that would be a standard feature for any search 
service that provided results in a syndication format. I mean what's the 
point of having something you can subscribe to if it doesn't change?

> what would become your argument?

I think what you're suggesting is overly complicated and not strictly 
necessary. However, there's nothing stopping you proposing such an 
extension. My opinion is just my opinion. If you can get support for it, 
great.

The only thing that matters at the moment is whether you're happy with 
Mark's current proposal for next/prev. If you are then we can move on. If 
you aren't, you should probably be taking up your case with him.

> while "non-incremental feeds with history" have not yet proven to be

I still don't see why OpenSearch result feeds can't be implemented as 
incremental feeds. Either they're being used as a one-off search and you 
can't subscribe to them (in which case there is no difference between 
incremental and non-incremental), or they're being updated with new results 
over time (like a filtered aggregate feed) in which case I would think they 
have to be incremental.

As for "non-incremental feeds with history", I can't point to a specific 
site that is using something like that, but I know it has been requested 
before. It's possible it was just a hypothetical "it might be useful if..." 
type of request though.

> Do you now see what I'm talking about?

I understand what you're talking about, but I just don't see the need. I 
would have expected a non-incremental feed to be a single Atom document. My 
reason for wanting paging is so that a user doesn't need to fetch data that 
he already has - this can never be a problem with a non-incremental feed 
because it doesn't grow. That leaves the next/prev links free for use as a 
history of past snapshots.

Once again, this is just my opinion. I don't expect you to agree with me. 
I'm sure the market will make its own choices about how best to use these 
extensions. If it turns out my opinion is in the minority, you may well find 
me changing my mind in the future - it wouldn't be the first time.

Regards
James

---

From: Antone Roundy <[email-masked]>
Date: Mon, 24 Oct 2005 09:42:09 -0600

On Oct 24, 2005, at 8:13 AM, James Holderness wrote:
> I still don't see why OpenSearch result feeds can't be implemented  
> as incremental feeds.
Perhaps they can, but that wouldn't always be desirable. Consider  
this scenario: Somebody writes a program that searches Google,  
scrapes the HTML results, and publishes them as an Atom feed.  My  
purpose in subscribing to the feed is not to be alerted when a new  
webpage is added to page 20 of Google's results, it's to be alerted  
whenever a new webpage makes it onto page 1.  So I don't want new  
pages added to the live end of the feed--I just want whatever is  
currently in the top 10 results, and my feed reader will tell me when  
one of them is one it hasn't seen before.

> Either they're being used as a one-off search and you can't  
> subscribe to them (in which case there is no difference between  
> incremental and non-incremental), or they're being updated with new  
> results over time (like a filtered aggregate feed) in which case I  
> would think they have to be incremental.
Given the above scenario, why wouldn't you be able to subscribe to them?

> I understand what you're talking about, but I just don't see the  
> need. I would have expected a non-incremental feed to be a single  
> Atom document.
In the case of something like a top 10 feed, I'd imagine it would  
be.  But a search results feed like what's described above may not be.

> My reason for wanting paging is so that a user doesn't need to  
> fetch data that he already has - this can never be a problem with a  
> non-incremental feed because it doesn't grow.
I'm not sure I understand--it's not as if a non-incremental feed were  
simply a static document.  They're resources whose contents are  
replaced wholesale (with the things that were in the old set possibly  
still being in the new set) rather than having their old contents  
augmented when new things are added.

---

From: "James Holderness" <[email-masked]>
Date: Mon, 24 Oct 2005 18:16:00 +0100

> Perhaps they can, but that wouldn't always be desirable. Consider  this 
> scenario: Somebody writes a program that searches Google,  scrapes the 
> HTML results, and publishes them as an Atom feed.  My  purpose in 
> subscribing to the feed is not to be alerted when a new  webpage is added 
> to page 20 of Google's results, it's to be alerted  whenever a new webpage 
> makes it onto page 1.  So I don't want new  pages added to the live end of 
> the feed--I just want whatever is  currently in the top 10 results, and my 
> feed reader will tell me when  one of them is one it hasn't seen before.

Ok. That's a valid point, but I still wouldn't be particularly interested in 
trying to accomodate that scenario. First you have someone producing a feed 
that is almost impossible to use as designed. Second you have someone using 
said feed contrary to how it was designed (although understandably so).

A more sensible approach would be a single feed document containing the top 
N results (where N is manageable in size). You could subscribe to that as a 
non-incremental feed and you would know at any point in time which were the 
top 10 results. There is no real need for paging other than as a form of 
snapshot history (i.e. what were the top 10 results last week).

> Given the above scenario, why wouldn't you be able to subscribe to them?

I can understand why you might want to subscribe to such a feed (or at least 
the first page of one). I can't understand why someone would want to create 
it though. And I wouldn't be particularly concerned if it wasn't possible 
for them to do such a thing.

>> My reason for wanting paging is so that a user doesn't need to  fetch 
>> data that he already has - this can never be a problem with a 
> simply a static document.  They're resources whose contents are  replaced 
> wholesale (with the things that were in the old set possibly  still being 
> in the new set) rather than having their old contents  augmented when new 
> things are added.

Exactly. The *entire document* is replaced. If you want to download an 
update you have to redownload the whole thing. There's no point in having a 
paging system since you can't stop after page 1. If you have to retrieve 
every single page every single time then they might as well all be in one 
page.

Once again, I have to ask the same question I asked Thomas: do you have a 
problem with Mark's next/prev proposal as it stands, or are you just arguing 
with me because you think I'm wrong? If the latter, feel free to just ignore 
me. We can agree to disagree. Unless we're discussing a particular proposal 
I don't see the point.

Regards
James

---

From: Antone Roundy <[email-masked]>
Date: Mon, 24 Oct 2005 11:53:13 -0600

On Oct 24, 2005, at 11:16 AM, James Holderness wrote:
> A more sensible approach would be a single feed document containing  
> the top N results (where N is manageable in size). You could  
> subscribe to that as a non-incremental feed and you would know at  
> any point in time which were the top 10 results. There is no real  
> need for paging other than as a form of snapshot history (i.e. what  
> were the top 10 results last week).
That is certainly a good approach--allowing the number of results to  
be determined dynamically by something in the URL, for example.   
However, it could be useful to limit the chunk size and allow paging  
for people who want more.  For example, you might allow a maximum of  
50 results per chunk, and then support ETags.  That way, if somebody  
wants to monitor the top 250, they can send 5 requests, and if most  
of the time there are no changes, they'll get a lot of 304s, but if  
occasionally something changes in the last chunk of 50 for example,  
they're only downloading 50 results each time something changes.   
There are of course other approaches, like support for just sending  
the diffs.  But that would probably more difficult for most people to  
implement, and may be less likely to be supported by a wide variety  
of clients.

Another reason for wanting to limit the number of results per query  
(and support paging for those who want more) is to avoid bandwidth  
waste if someone accidentally ads an extra digit to the desired  
number of results; or tries to waste your system resources by  
requesting huge result sets (but dropping the connection before using  
up their own bandwidth actually receiving the whole result set); or  
has a client that doesn't support paging or diffs or ETags or  
anything, and wants a huge result set (and you don't want to  
accommodate them since it would use so much bandwidth), etc.

> Once again, I have to ask the same question I asked Thomas: do you  
> have a problem with Mark's next/prev proposal as it stands, or are  
> you just arguing with me because you think I'm wrong? If the  
> latter, feel free to just ignore me. We can agree to disagree.  
> Unless we're discussing a particular proposal I don't see the point.
I have a problem with not having link relations specific to paging  
through a feed's current state.  I'm fine with having general chain  
navigation link relations, but hope that we'll get something specific  
to paging and that people will use it instead of the general link  
relations.  I've spoken my peace on that and have given up swimming  
against the tide, but am still willing to discuss specific related  
issues.