Subject: RE: A date-range query mechanism for Atom and RSS (solution to RSS bloat?)

From: "Scott Watermasysk" <[email-masked]>
Date: Sun, 12 Sep 2004 22:54:32 -0400

At one point I had done something similar.

I had started to set the Etag with the date of last post requested/sent
via Rss. If the Etag was included in the request, I would compare the
dates and send a 304 if nothing changed. Otherwise, I would sent any
chances after the specified Etag date. 

I will have to dig up exactly who it was, but there was an aggregator
developer or two that pointed this was against the "Http" spec (ie, if
the document has chanced, the entire document needed to be resent), so I
removed the feature. 

I would love to get to the point where only the updated items need to be
returned. 

-Scott

Scott Watermasysk
>telligentsystems
Exceptional Service. Predictable Results.
http://www.telligentsystems.com

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Kevin A. Burton
Sent: Sunday, September 12, 2004 8:56 PM
To: [email-masked]
Subject: A date-range query mechanism for Atom and RSS (solution to RSS
bloat?)

I posted two long entries over the last two days WRT the current 
blogs.msdn.com issue...

http://www.peerfear.org/rss/permalink/2004/09/12/HTTPDeltasRSSAndDiffere
ntialContentFetching

and

http://www.peerfear.org/rss/permalink/2004/09/11/TheScalabilityOfFullCon
tentFeeds

Quick summary....

HTTP deltas are too complicated (even though I used to be a huge 
advicate of HTTP deltas)...

> A potential solution would be to add additional HTTP headers to 
> specify a date range for a query. This would be similar to 
> If-Modified-Since but would have slightly different semantics.
>
> This is obviously a strawman proposal but we could use two additional 
> HTTP headers of ModifiedAfter and ModifiedBefore. ModifiedAfter would 
> return all posts made after a given date. If there are no feeds we 
> would return HTTP 304 (not modified). (I'll address ModifiedBefore 
> below).
>
> This also supplies uplevel and downlevel compatibility. For example a 
> pre-ModifiedAfter capable client can still use the URL to fetch the 
> feed but the old semantics would be used. A post-ModifiedAfter client 
> could also integrate with existing Atom and RSS feeds that do not 
> support ModifiedAfter|Before.
>
Also this ETag proposal:

    http://www.intertwingly.net/blog/2004/09/11/Vary-ETag

Is along similar lines but I still think is too complicated and doesn't 
offer as much flexibility.  Its better than the delta approach though...

(IMO).

Kevin

-- 

Please reply using PGP.

    http://peerfear.org/pubkey.asc    
    
    NewsMonster - http://www.newsmonster.org/
    
Kevin A. Burton, Location - San Francisco, CA, Cell - 415.595.9965
       AIM/YIM - sfburtonator,  Web - http://peerfear.org/
GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412
  IRC - freenode.net #infoanarchy | #p2p-hackers | #newsmonster

---

From: Sam Ruby <[email-masked]>
Date: Mon, 13 Sep 2004 01:50:51 -0400

> the document has chanced, the entire document needed to be resent), so I

HTTP has very specific mechanism defined for partial results, and a very 
general mechanism for content negotiation.  What these aggregator 
developers were probably thinking of is the former.

Separate from any concern that they may have expressed about the HTTP 
spec, I would be very interested in whether or not this posed any 
particular problem for them.

> I would love to get to the point where only the updated items need to be

The overall theory falls under the rubric Content Negotiation[1].  The 
short version is that when people use HTTP, they don't request 
resources, they request representations of resources.  And some 
resources may have multiple representations.

Traditionally, such variations are based on content type, charset, 
encoding, language, or user agent.  But variations can be based on any 
request-header, or even on a field thereof.

To make sure that caches continue to operate properly, a "Vary" header 
should be added to the response indicating what header was used to vary 
on.  One such header would be "If-None-Match".

- Sam Ruby

[1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec12.html