Subject: Re: Atom feed synchronization

From: James M Snell <[email-masked]>
Date: Thu, 16 Jun 2005 16:01:08 -0700

Just a bit more digging to expand the question a bit:

A third alternative to handling this is to use a Range header with a new 
token value...e.g.

Example 1: Get all entries modified after the specified HTTP-date 
(inclusive)

  GET ... HTTP/1.1
  Host: ...
  Range: modified=[HTTP-date]

Example 2: Get all entries modified between HTTP-date 1 and HTTP-date 2 
(inclusive)

  GET ... HTTP/1.1
  Host: ....
  Range: modified=[HTTP-date 1]-[HTTP-date 2]

(oh, and BTW, in the previous note 12345 is really a HTTP-date... I was 
just being lazy when typing)

- James

-------- Original Message --------
Subject: 	Atom feed synchronization
Date: 	Thu, 16 Jun 2005 15:37:14 -0700
From: 	James M Snell <[email-masked]>
To: 	[email-masked]
References: 	<p0621024ebed78efa2b25@[10.20.30.249]>

Ok, question for the group.

Scenario: A content source is updated 100 times per hour.  The Atom feed 
currently only displays the 20 most recent entries.  Users who are not 
checking the feed every 10 minutes or so are missing entries.  How do we 
address this?

Solution: Rather than using a feed with a fixed number of entries, 
provide a mechanism that allows users to specify the last time they 
retrieved the feed and have the feed return all entries added since that 
time.

Question: What is the best way to provide that mechanism: querystring 
parameter or HTTP header or some other way I'm not thinking of

   http://...?last-retrieved=12345

OR

  GET ... HTTP/1.1
  Host: ...
  Last-Retrieved: 12345

- James

---

From: James M Snell <[email-masked]>
Date: Thu, 16 Jun 2005 16:48:14 -0700

Antone Roundy wrote:

> On Thursday, June 16, 2005, at 04:37  PM, James M Snell wrote:
>
>> Scenario: A content source is updated 100 times per hour.  The Atom 
>> feed currently only displays the 20 most recent entries.  Users who 
>> are not checking the feed every 10 minutes or so are missing 
>> entries.  How do we address this?
>
>
> 1) Tell users to poll more frequently--if they don't, that's their 
> problem

Feed is updated 24 hours a day from sources all over the world.  Would 
be rather impractical for me to keep my system on 24 hours a day pinging 
the server every 10 minutes just to make sure I don't miss any content.  
Further, it just doesn't seem right to constrain Atom to a "Must Be 
Present To Win" mentality.

> 2) Make the feed longer than 20 entries

Possible, but potentially not scalable.  What if the source update rates 
double or triple over a few months time (a strong possibility).  We'd 
have to keep increasing the size of the feed to account for the 
increased publishing rates.  Even then, folks would still miss content 
unless they check regularly. Seems like it would be better to just 
include the entries that were actually modified within a given period of 
time than to keep fiddling with the size.

> 2a) If the feed is SOMETIMES updated 100 times per hour, but sometimes 
> only twice per hour, dynamically set the feed length--when 
> regenerating the feed, include all entries updated in the last hour or 
> so, down to some minimum number

Dynamically setting the feed length based on some calculation of update 
rates just seems silly to me.  But using multiple feeds each reflecting 
a different time window is a strong option.  one feed shows past hour, 
another feed shows past six hours, etc.... not perfect as it requires 
the user to do more work in selecting which feed to subscribe to.  Any 
solution that requires the user to do more work is less than optimum.

> 3) Push instead of pull

Possible but rather impractical for disconnected clients.

> 4) Create a new link @rel to point to the next X entries and keep a 
> series of static documents containing the older entries
>
Possible, but on the surface it doesn't seem to be any better or 
efficient than the header or querystring method.  In fact, it seems 
rather inefficient to me.

Thanks for the input :-)

- James