Subject: Re: Feed History -02

From: Mark Nottingham <[email-masked]>
Date: Wed, 3 Aug 2005 21:27:29 -0700

So, if I read you correctly, it sounds like you have a method whereby  
a 'top20' feed wouldn't need history:prev to give the kind of history  
that you're thinking of, right?

If that's the case, I'm tempted to just tweak the draft so that  
history:stateful is optional if history:prev is present. I was  
considering dropping stateful altogether, but I think something is  
necessary to explicitly say "don't try to keep a history of my feed."  
My latest use case for this is the RSS feed that Netflix provides to  
let you keep an eye on your queue (sort of like top20, but more  
specialised).

Sound good?

On 29/07/2005, at 6:39 AM, Henry Story wrote:

>
> Below I think I have worked out how one can in fact have a top20  
> feed, and
> I show how this can be combined without trouble with the  
> <history:next ...>
> link...
>
>
> On 29 Jul 2005, at 13:12, Eric Scheid wrote:
>
>>
>> On 29/7/05 7:57 PM, "Henry Story" <[email-masked]> wrote:
>>
>>
>>
>>> 1- The top 20 list: here one wants to move to the previous top   
>>> 20 list and
>>> think of them as one thing. The link to the next feed is not  
>>> meant to be
>>> additive. Each feed is to be seen as a whole. I have a little  
>>> trouble still
>>> thinking of  these as feeds, but ...
>>>
>>>
>>
>> What happens if the publisher realises they have a typo and need  
>> to emit an
>> update to an entry? Would the set of 20 entries (with one entry  
>> updated) be
>> seen as a complete replacement set?
>>
>
> Well if it is a typo and this is considered to be an insignificant  
> change then
> they can change the typo in the feed document and not need to  
> change any updated
> time stamps.
>
>
>> The way I see it, maybe a better way would be to have a sliding  
>> window feed
>> where each entry points to another Atom Feed Document with it's  
>> own URI, and
>> it is that second Feed Document which contains the individual  
>> items (the top
>> 20 list).
>>
>
> This is certainly closer to my intuitions too.  A top 20 something  
> is *not* a feed.
> Feed entries  are not ordered, and are not  meant to be thought of  
> as a closed
> collection. At least this is my initial intuition. BUT....
>
> I can think of a solution like the following: Let us imagine a top  
> 20 feed where
> the resources being described by the entries are the position in  
> the top list. So
> we have entries with ids such as
>
>     http://TopOfThePops.co.uk/top20/Number1
>     http://TopOfThePops.co.uk/top20/Number2
>     http://TopOfThePops.co.uk/top20/Number3
>     ...
>
> Each of these resources describes the songs that is at a certain  
> rank in the top of
> the pops chart. Each week the song in that rank may change. When a  
> change occurs in the
> song at a certain rank the top 20 feed with id
>
>     http://TopOfThePops.co.uk/top20/Feed
>
> will contain a new entry such as
>
>    <entry>
>     <title>Top of the pops entry number 1</title>
>     <link href="http://TopOfThePops.co.uk/top20/Number1/"/>
>     <id>http://TopOfThePops.co.uk/top20/Number1</id>
>     <updated>2005-07-05T18:30:00Z</updated>
>     <summary>Top of the pops winner for the week starting 5 July  
> 2005</summary>
>   </entry>
>
> A client that would subscribe to such a feed would automatically  
> get updates every
> week for each of the top 20 resources. But the feed could be  
> structured exactly like
> I suggest in 2.
>
> So for a top 2 feed (20 is a bit to long for me)
>
> <feed>
>    <title type="text">My top 2 Software Books</title>
>    <id>http://bblfish.net/blog/top2</id>
>    <history:prev>http://bblfish.net/blog/top2/ 
> Archive-2005-07-18.atom</history:prev>
>    ...
>    <entry>
>     <title>My Top 1 book</title>
>     <link href="http://bblfish.net/blog/top2/Number1/"/>
>     <id>http://bblfish.net/blog/top2/Number1</id>
>     <updated>2005-07-25T18:30:00Z</updated>
>     <summary>My top 1 book is Service Oriented Computing by Wiley</ 
> summary>
>    </entry>
>    <entry>
>     <title>My Top 2 book</title>
>     <link href="http://bblfish.net/blog/top2/Number2/"/>
>     <id>http://bblfish.net/blog/top2/Number2</id>
>     <updated>2005-07-25T18:30:00Z</updated>
>     <summary>My second top book is "xml in a Nutshell"</summary>
>   </entry>
> </feed>
>
> The above representation of the http://bblfish.net/blog/top2 feed  
> points to the
> archive feed http://bblfish.net/blog/top2/Archive-2005-07-18.atom
>
> <feed>
>    <title type="text">My top 2 Software Books</title>
>    <id>http://bblfish.net/blog/top2</id>
>    ...
>    <entry>
>     <title>My Top 1 book</title>
>     <link href="http://bblfish.net/blog/top2/Number1/"/>
>     <id>http://bblfish.net/blog/top2/Number1</id>
>     <updated>2005-07-18T18:30:00Z</updated>
>     <summary>My top 1 book is Java 2D Graphics</summary>
>    </entry>
>    <entry>
>     <title>My Top 2 book</title>
>     <link href="http://bblfish.net/blog/top2/Number2/"/>
>     <id>http://bblfish.net/blog/top2/Number2</id>
>     <updated>2005-07-18T18:30:00Z</updated>
>     <summary>My second top book is "xml in a Nutshell"</summary>
>   </entry>
> </feed>
>
>  As you notice only the top1 has changed from the first week but  
> not the
> second, and yet in both feeds there is an entry for both. A non  
> change can
> sometimes be an important event. But this is really up to the feed  
> creator
> to choose how to present his feeds. He could easily have had only  
> had the
> entry for http://bblfish.net/blog/top2/Number1 in the first feed.
>
> Looking at it this way, there really seems to be no incompatibility  
> between
> a top 20 feed and the <history:next ...> link. My talk about  
> archives not
> changing should be more precisely about archives not changing in any
> significant way. And this advice could be moved to an implementors  
> section
> and be encoded in HTTP by simply giving archive pages an infinitely  
> long
> expiry date.
>
>
>> Someone could subscribe to that second feed and poll for updates,  
>> and all
>> they'll ever see are updates to the 20 items there, not the 20  
>> items from
>> the next week/whatever.
>>
>> The idea of feeds linked to feeds has lots of utility -- feeds of  
>> comments
>> for one, and even a feed of feeds available on the site.
>>
>
> I completely agree. And remember for any two things there is at  
> least one way they
> are related. And there are many different ways feeds can be related  
> to each other.
> A feed may be an archival continuation of one - which is what the  
> <history:next ...>
> link in my opinion addresses, but there are many other ways one can  
> relate feeds.
>
>
>
>> For example: this HTML page <http://www.nature.com/rss/> has an  
>> equivalent
>> feed document <http://npg.nature.com/pdf/newsfeeds.rdf>, where  
>> each item
>> links to the individual feeds for each publication. That feed  
>> doesn't update
>> often, mostly because NPG doesn't add many new feeds to their site  
>> all that
>> often. The URIs the entries of that feed link to are redirected to  
>> the
>> permanent URIs of the current issue (each issue has it's own feed  
>> which is
>> the table of contents for that issue, with a distinct and separate  
>> URI for
>> each issue).
>>
>
> Yes a feed can itself be an entry in another feed.
>
>
>> It's conceivable they could also provide a feed for each publication
>> pointing to the table of contents feeds of each issue. That is, a  
>> feed with
>> an entry for each issue.
>>
>
> yes.
>
>
>> Of the above, the mechanism of a single URI which redirects to the  
>> current
>> issue is a situation which would still need a flag indicating that  
>> the
>> appropriate thing to do is to not persist older entries.
>>
>
> I am starting to wonder whether this is really needed now that I  
> have looked
> at the top20 example I gave above.
>
>
>>
>> The other structure of feeds linking to feeds would require the  
>> aggregator
>> be able to do something useful with such links, but this can be  
>> generalised
>> and thus be useful for many purposes. As it is, right now with NNW  
>> I can do
>> something useful with such a feed: drag & drop the item headline  
>> link to my
>> subscriptions pane to subscribe to that feed and view the entries  
>> therein.
>>
>
> I myself have no problem with feeds being entries, feeds pointing  
> to other
> feeds, or anything like that. A feed is a resource. It can change.  
> A feed
> is simply a set of state changes to resources. It is that general.
>
>
>>
>> Both require coding effort.
>>
>> e.
>>
>>
>
>
>

--
Mark Nottingham   Principal Technologist
Office of the CTO   BEA Systems

---

From: Henry Story <[email-masked]>
Date: Tue, 9 Aug 2005 13:07:29 +0200

On 4 Aug 2005, at 06:27, Mark Nottingham wrote:
> So, if I read you correctly, it sounds like you have a method  
> whereby a 'top20' feed wouldn't need history:prev to give the kind  
> of history that you're thinking of, right?
>
> necessary to explicitly say "don't try to keep a history of my  
> feed." My latest use case for this is the RSS feed that Netflix  
> provides to let you keep an eye on your queue (sort of like top20,  
> but more specialised).
>
> Sound good?

Sounds good to me.

But I would really like some way to specify that the next feed  
document is an archive (ie. won't change). This would make it easy  
for clients to know when to stop following the links, ie, when
they have cought up with the changes since they last looked at the feed.

Perhaps something like this:

<history:prev archive="yes">http://liftoff.msfc.nasa.gov/2003/04/ 
feed.rss</history>

Henry Story

---

From: Walter Underwood <[email-masked]>
Date: Tue, 09 Aug 2005 09:15:03 -0700

--On August 9, 2005 1:07:29 PM +0200 Henry Story <[email-masked]> wrote:
>
> But I would really like some way to specify that the next feed  document is an
> archive (ie. won't change). This would make it easy  for clients to know when
> to stop following the links, ie, when they have cought up with the changes
> since they last looked at the feed.

I made some proposals for cache control info (expires and max-age).
That might work for this.

wunder
--
Walter Underwood
Principal Architect, Verity

---

From: James M Snell <[email-masked]>
Date: Tue, 09 Aug 2005 09:28:52 -0700

>That might work for this.
I missed these proposals.  I've been giving some thought to an <expires 
/> and <max-age /> extension myself and was getting ready to write up a 
draft. Expires is a simple date construct specifying the exact moment 
(inclusive) that the entry/feed expires.  Max-age is a non negative 
integer specifying the number of miliseconds (inclusive) from the moment 
specified by atom:updated when then entry/feed expires.  The two cannot 
appear together within a single entry/feed and follows the same basic 
rules as atom:author elements.

- James

---

From: Walter Underwood <[email-masked]>
Date: Tue, 09 Aug 2005 09:32:59 -0700

--On August 9, 2005 9:28:52 AM -0700 James M Snell <[email-masked]> wrote:

> I missed these proposals.  I've been giving some thought to an <expires /> and <max-age /> extension myself and was getting ready to write up a draft. Expires is a simple date construct specifying the exact moment (inclusive) that the entry/feed expires.  Max-age is a non negative integer specifying the number of miliseconds (inclusive) from the moment specified by atom:updated when then entry/feed expires.  The two cannot appear together within a single entry/feed and follows the same basic

Here it is: <http://www.intertwingly.net/wiki/pie/PaceCaching>

Adding max-age also means defining IntegerConstruct and disallowing
white space around it. Formerly, it was OK as a text construct, but
the white space issues change that.

Also, we should decide whether cache information is part of the signature.
I can see arguments either way.

wunder

---

From: Henry Story <[email-masked]>
Date: Tue, 9 Aug 2005 18:46:10 +0200

On 9 Aug 2005, at 18:32, Walter Underwood wrote:
>> I missed these proposals.  I've been giving some thought to an  
>> <expires /> and <max-age /> extension myself and was getting ready  
>> to write up a draft. Expires is a simple date construct specifying  
>> the exact moment (inclusive) that the entry/feed expires.  Max-age  
>> is a non negative integer specifying the number of miliseconds  
>> (inclusive) from the moment specified by atom:updated when then  
>> entry/feed expires.  The two cannot appear together within a  
>> single entry/feed and follows the same basic
>>

Interesting... but why have a limit of one year? For archives, I  
would like a limit of
forever.

But otherwise I suppose this would do. Instead of putting the  
information in the
<history> link of the linking feed, you would put it in the archive  
feed. Which sounds
good. I suppose we end up with some duplication of information here  
with the http headers
again.

---

From: James M Snell <[email-masked]>
Date: Tue, 09 Aug 2005 10:09:53 -0700

>Here it is: <http://www.intertwingly.net/wiki/pie/PaceCaching>
The expires and max-age elements look fine. I hesitate at bringing in a 
caching discussion.  I'm much more comfortable leaving the definition of 
caching rules to the protocol level (HTTP) rather than the format 
extension level.  Namely, I don't want to have to go into defining rules 
for how HTTP headers that affect caching interact with the expires and 
max-age elements... IMHO, there is simply no value in that. 

The expires and max-age extension elements affect the feed / entry on 
the application level not the document level.  HTTP caching works on the 
document level.

>the white space issues change that.
This is easy enough.

>I can see arguments either way.
-1.  let's let caching be handled by the transport layer.

- James

---

From: Henry Story <[email-masked]>
Date: Tue, 9 Aug 2005 20:46:13 +0200

To answer my own question

[[
]]

  I found the following in the HTTP spec

[[
    To mark a response as "never expires," an origin server sends an
    Expires date approximately one year from the time the response is
    sent. HTTP/1.1 servers SHOULD NOT send Expires dates more than one
    year in the future.
]]

(though that still does not explain why.)

Now I am wondering if the http mechanism is perhaps all that is needed
for what I want with the unchanging archives. If it is then perhaps this
could be explained in the Feed History RFC. Or are there other  
reasons to
add and "expires" tag to the document itself?

Henry Story

On 9 Aug 2005, at 19:09, James M Snell wrote:
> The expires and max-age elements look fine. I hesitate at bringing  
> in a caching discussion.  I'm much more comfortable leaving the  
> definition of caching rules to the protocol level (HTTP) rather  
> than the format extension level.  Namely, I don't want to have to  
> go into defining rules for how HTTP headers that affect caching  
> interact with the expires and max-age elements... IMHO, there is  
> simply no value in that.
> The expires and max-age extension elements affect the feed / entry  
> on the application level not the document level.  HTTP caching  
> works on the document level.
>
>

---

From: James M Snell <[email-masked]>
Date: Tue, 09 Aug 2005 11:53:37 -0700

> add and "expires" tag to the document itself?

On the application level, a feed or entry may expire or age indepedently 
of whatever caching mechanisms may be applied at the transport level.  
For example, imagine a source that publishes special offers in the form 
of Atom entries that expire at a given point in time.  Now suppose that 
those entries are being distributed via XMPP and HTTP.  It is helpful to 
have a transport independent expiration/max-age mechanism whose 
semantics operate on the application layer rather than the transport layer.

- James

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 9 Aug 2005 14:44:18 -0700

On 09/08/2005, at 4:07 AM, Henry Story wrote:
> for clients to know when to stop following the links, ie, when they  
> have cought up with the changes since they last looked at the feed.
>

I'd think that would be more appropriate as an extension to the  
archive itself, wouldn't it? That way, the metadata (the fact that  
it's an archive) is part of the data (the archive feed).

E.g.,

<atom:feed>
   ...
   <archive:yes_im_an_archive/>
</atom:feed>

By (current) definition, anything that history:prev points to is an  
archive.

Cheers,

--
Mark Nottingham     http://www.mnot.net/

---

From: James M Snell <[email-masked]>
Date: Tue, 09 Aug 2005 19:46:03 -0700

This is fairly quick and off-the-cuff, but here's an initial draft to 
get the ball rolling..

  http://www.snellspace.com/public/draft-snell-atompub-feed-expires.txt

- James

Henry Story wrote:

>> definition of caching rules to the protocol level (HTTP) rather  than 
>> the format extension level.  Namely, I don't want to have to  go into 
>> defining rules for how HTTP headers that affect caching  interact 
>> with the expires and max-age elements... IMHO, there is  simply no 
>> value in that.
>> on the application level not the document level.  HTTP caching  works 
>> on the document level.
>>
>>

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 9 Aug 2005 20:31:04 -0700

HTTP isn't a transport protocol, it's a transfer protocol; i.e., the  
caching information (and other entity metadata) are *part of* the  
entity, not something that's conceptually separate.

The problem with having an "abstract" or "transport-neutral" concept  
of caching is that it leaves you with an awkward choice; you can  
either a) exactly replicate the HTTP caching model, which is  
difficult to do in other protocols, b) "dumb down" HTTP caching to a  
subset that's "neutral", or c) introduce a contradictory caching  
model and suffer the clashes between HTTP caching and it.

This is the same road that Web services sometimes tries to go down,  
and it's a painful one; coming up with the grand, protocol-neutral  
abstraction that enables all of the protocol-specific features is  
hard, and IMO not necessary. Ask yourself: are there any situations  
where you *have* to be able to seamlessly switch between protocols,  
or is it just a "convenience?"

I think we can declare victory here by simply a) using whatever  
caching mechanism is available, and b) designating a "won't change"  
flag.

On 09/08/2005, at 11:53 AM, James M Snell wrote:

> On the application level, a feed or entry may expire or age  
> indepedently of whatever caching mechanisms may be applied at the  
> transport level.  For example, imagine a source that publishes  
> special offers in the form of Atom entries that expire at a given  
> point in time.  Now suppose that those entries are being  
> distributed via XMPP and HTTP.  It is helpful to have a transport  
> independent expiration/max-age mechanism whose semantics operate on  
> the application layer rather than the transport layer.
>
> - James
>
>
>

---

From: Eric Scheid <[email-masked]>
Date: Wed, 10 Aug 2005 13:56:05 +1000

> http://www.snellspace.com/public/draft-snell-atompub-feed-expires.txt

Looks good, I think it does need a little bit of prose explaining that this
has nothing to do with caching, and should not be used in scheduling when to
revisit/refresh/expire local copies of the resource.

Similarly, if I understand correctly, when you write
   "The 'max-age' extension element is used to indicate
    the maximum age of a feed or entry."
you are referring to the max-age until the informational content of the
respective feed or entry expires. And similarly with age:expires. Yes?

Aside: a perfect example of what sense of 'expires' is in the I-D itself...

    Network Working Group
    Internet-Draft
    Expires: January 2, 2006

:-)

e.

---

From: James M Snell <[email-masked]>
Date: Tue, 09 Aug 2005 21:07:51 -0700

First off, let me stress that I am NOT talking about caching scenarios 
here...  (my use of the terms "application layer" and "transport layer" 
were an unfortunate mistake on my part that only served to confuse my point)

Let's get away from the multiprotocol question for a bit (it never leads 
anywhere constructive anyway)... Let's consider an aggregator scenario. 
Take an entry from a feed that is supposed to expire after 10 days.  The 
feed document is served up to the aggregator with the proper HTTP 
headers for expiration.  The entry is extracted from the original feed 
and dumped into an aggregated feed.  Suppose each of the entries in the 
aggregated feed are supposed to have their own distinct expirations.  
How should the aggregator communicate the appropriate expirations to the 
subscriber?  Specifying expirations on the HTTP level does not allow me 
to specify expirations for individual entries within a feed.  Use case: 
an online retailer wishes to produce a "special offers" feed.  Each 
offer in the feed is a distinct entity with it's own terms and own 
expiration:  e.g. some offers are valid for a week, other offers are 
valid for two weeks, etc.  The expiration of the offer (a business level 
construct) is independent of whether or not the feed is being cached or 
not (a protocol level construct); publishing a new version of the feed 
(e.g. by adding a new offer to the feed) should have no impact on the 
expiration of prior offers published to the feed.

Again, I am NOT attempting to reinvent an abstract or transport-neutral 
caching mechanism in the same sense that the atom:updated element is not 
attempting to reinvent Last-Modified or that the via link relation is 
not attempting to reinvent the Via header, etc.  They serve completely 
different purposes. The expires and max-age extensions I am proposing 
should NOT be used for cache control of the Atom documents in which they 
appear.

 >I think we can declare victory here by simply a) using whatever  
caching mechanism is available, and b) designating a "won't change"  flag.
Speaking *strictly* about cache control of Atom documents, +1.  No 
document level mechanisms for cache control are necessary.

- James

Mark Nottingham wrote:

>> point in time.  Now suppose that those entries are being  distributed 
>> via XMPP and HTTP.  It is helpful to have a transport  independent 
>> expiration/max-age mechanism whose semantics operate on  the 
>> application layer rather than the transport layer.
>>
>> - James
>>
>>
>>
>
>
> --

---

From: James M Snell <[email-masked]>
Date: Tue, 09 Aug 2005 21:14:39 -0700

>revisit/refresh/expire local copies of the resource.
Absolutely.  I put this together rather quickly fully knowing that I 
would need to revisit the text to fully explain the use of the 
extension.  I'll be taking a few whacks at that over the coming days as 
time allows.

>respective feed or entry expires. And similarly with age:expires. Yes?
Yes. I will be clarifying this in the draft.  The extensions are really 
only reflective of the informational content in individual entries.  As 
we do with the atom:author element, expires and max-age can be specified 
at the feed or source level, but apply to the content of individual 
entries. e.g.

<feed>
   <expires>...</expires>
   <entry></entry>
   <entry></entry>
</feed>

is equivalent to

<feed>
   <entry><expires>...</expires></entry>
   <entry><expires>...</expires></entry>
</feed>

>Aside: a perfect example of what sense of 'expires' is in the I-D itself...
Excellent point :-)

- James

---

From: Henry Story <[email-masked]>
Date: Wed, 10 Aug 2005 08:30:42 +0200

There is an interesting problem of how this interacts with the  
history tag.

If you set an a feed

<feed>
   <expires>...</expires>
   <entry></entry>
   <entry></entry>
</feed>

then what are you setting it on? Well not the document, clearly, as  
you have
pointed out since HTTP headers deal with that. So it must be on the  
feed. And
of course a feed is identified by its id.

Now we have to make sure we avoid contradictions where different feed  
documents
describing the same feed state that the feed has different expiry dates.

Eg:

----the main feed----------
<feed>
   <expires>August 25 2006</expires>
   <id>tag:example.com,2000/feed1</id>
   <entry></entry>
   <entry></entry>
   <history:previous>http://example.com/archive1</history:previous>
</feed>
---------------------------

The above is a partial description of the feed tag:example.com,2000/ 
feed1
The history:previous link points to another document that gives
us more information about that feed.

---the archive feed--------
<feed>
   <expires>August 15 2006</expires>
   <id>tag:example.com,2000/feed1</id>
   <entry></entry>
   <entry></entry>
   <history:previous>http://example.com/archive2</history:previous>
</feed>
---------------------------

Now of course we have a feed id with two expiry dates. Which one is  
correct?
In graph terms we end up with something like this:

tag:example.com,2000
    |----expires------>August 25 2006
    |----expires------>August 15 2006
    |----entry-------->...
    |----entry-------->...
    |----entry-------->...
    |----entry-------->...

One has the feeling that the expires relation should be functional,  
ie have
only one value.

This makes me think again that for what I was looking for (that the  
document
in history:previous not change, so that one can work out when to stop  
fetching
documents) can in fact be entirely be taken care of by the http  
expiry dates and
cache control mechanism. Of course if this is so, I think it should  
be noted
clearly in the history spec. ((btw. Is it easy to set expiry dates  
for documents served
by apache?))

Henry Story

On 10 Aug 2005, at 04:46, James M Snell wrote:
>
> This is fairly quick and off-the-cuff, but here's an initial draft  
> to get the ball rolling..
>
>>> The expires and max-age elements look fine. I hesitate at  
>>> bringing  in a caching discussion.  I'm much more comfortable  
>>> leaving the  definition of caching rules to the protocol level  
>>> (HTTP) rather  than the format extension level.  Namely, I don't  
>>> want to have to  go into defining rules for how HTTP headers that  
>>> affect caching  interact with the expires and max-age elements...  
>>> IMHO, there is  simply no value in that.
>>> The expires and max-age extension elements affect the feed /  
>>> entry  on the application level not the document level.  HTTP  
>>> caching  works on the document level.
>>>
>>>
>>>

---

From: James M Snell <[email-masked]>
Date: Wed, 10 Aug 2005 02:32:00 -0700

This is precisely why the expires/max-age needs to operate strictly on 
the entry level.  The expires/max-age elements may appear on the feed 
level but it actually only applies to the entries within that feed.  
Going back and reading my draft, I see that I screwed up and didn't 
indicate that properly.  So in your example, the entries contained in 
the main feed expire on August 25th, the entries contained in the 
archive feed expire August 15th.  There is no conflict.

---

From: Walter Underwood <[email-masked]>
Date: Wed, 10 Aug 2005 08:20:09 -0700

> Aside: a perfect example of what sense of 'expires' is in the I-D itself...

Especially perfect because the HTTP header does not reflect the expiration.

Honestly, another reason to put expiration inside the feed is that 
HTTP caching is just not used. Well, except to force reloads and show
you new ads. But it is extremely rare to see it per-document cache
information.

wunder

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 10 Aug 2005 10:21:06 -0700

So, you're really looking for entry-level, time-based invalidation, no?

I guess the simplest way to do this would be to dereference the link  
and see if you get a 404/410; if you do, you know it's no longer good.

That's not terribly efficient, but OTOH managing metadata in multiple  
places is tricky, and predicting the future doubly so :) Most people  
get expiration times really wrong. And clock sync becomes an issue as  
well.

I'd think that if you have reasonable control over the polling of the  
feed, and a solid enough state model (which might include an explicit  
deletion mechanism), you could have a similar effect by just removing  
the items from the feed when they expire, with the expectation that  
when they disappear from the feed, they disappear from the client.  
Would that work for your use case?

On 09/08/2005, at 9:07 PM, James M Snell wrote:

>
> First off, let me stress that I am NOT talking about caching  
> scenarios here...  (my use of the terms "application layer" and  
> "transport layer" were an unfortunate mistake on my part that only  
> served to confuse my point)
>
> Let's get away from the multiprotocol question for a bit (it never  
> leads anywhere constructive anyway)... Let's consider an aggregator  
> scenario. Take an entry from a feed that is supposed to expire  
> after 10 days.  The feed document is served up to the aggregator  
> with the proper HTTP headers for expiration.  The entry is  
> extracted from the original feed and dumped into an aggregated  
> feed.  Suppose each of the entries in the aggregated feed are  
> supposed to have their own distinct expirations.  How should the  
> aggregator communicate the appropriate expirations to the  
> subscriber?  Specifying expirations on the HTTP level does not  
> allow me to specify expirations for individual entries within a  
> feed.  Use case: an online retailer wishes to produce a "special  
> offers" feed.  Each offer in the feed is a distinct entity with  
> it's own terms and own expiration:  e.g. some offers are valid for  
> a week, other offers are valid for two weeks, etc.  The expiration  
> of the offer (a business level construct) is independent of whether  
> or not the feed is being cached or not (a protocol level  
> construct); publishing a new version of the feed (e.g. by adding a  
> new offer to the feed) should have no impact on the expiration of  
> prior offers published to the feed.
>
> Again, I am NOT attempting to reinvent an abstract or transport- 
> neutral caching mechanism in the same sense that the atom:updated  
> element is not attempting to reinvent Last-Modified or that the via  
> link relation is not attempting to reinvent the Via header, etc.   
> They serve completely different purposes. The expires and max-age  
> extensions I am proposing should NOT be used for cache control of  
> the Atom documents in which they appear.
>
>> HTTP isn't a transport protocol, it's a transfer protocol; i.e.,  
>> the  caching information (and other entity metadata) are *part of*  
>> the  entity, not something that's conceptually separate.
>>
>> The problem with having an "abstract" or "transport-neutral"  
>> concept  of caching is that it leaves you with an awkward choice;  
>> you can  either a) exactly replicate the HTTP caching model, which  
>> is  difficult to do in other protocols, b) "dumb down" HTTP  
>> caching to a  subset that's "neutral", or c) introduce a  
>> contradictory caching  model and suffer the clashes between HTTP  
>> caching and it.
>>
>> This is the same road that Web services sometimes tries to go  
>> down,  and it's a painful one; coming up with the grand, protocol- 
>> neutral  abstraction that enables all of the protocol-specific  
>> features is  hard, and IMO not necessary. Ask yourself: are there  
>> any situations  where you *have* to be able to seamlessly switch  
>> between protocols,  or is it just a "convenience?"
>>
>>> indepedently of whatever caching mechanisms may be applied at  
>>> the  transport level.  For example, imagine a source that  
>>> publishes  special offers in the form of Atom entries that expire  
>>> at a given  point in time.  Now suppose that those entries are  
>>> being  distributed via XMPP and HTTP.  It is helpful to have a  
>>> transport  independent expiration/max-age mechanism whose