Subject: PaceDateModified

From: Tim Bray <[email-masked]>
Date: Thu, 28 Oct 2004 12:06:28 -0700

-1  Absolutely, I'm lying down in the road here.

This is *compulsory*.  As a publisher, I do not accept that, as a 
condition of publishing Atom, I am obligated to tell the world every 
time I make a spelling correction or trivial update.  This is a totally 
unreasonable proposition.  In point of real-life fact, the publisher 
decides when a change is a change, and PaceDateUpdated respects that 
reality.

Make it optional and I'll reconsider. -Tim

---

From: Danny Ayers <[email-masked]>
Date: Thu, 28 Oct 2004 21:26:44 +0200

> decides when a change is a change, and PaceDateUpdated respects that

The publisher corrects a spelling mistake and it goes into the feed -
isn't that the real-life fact? So why shouldn't the publishing
software give the client software a helping hand, and give it explicit
notification? What of the Winerwatch offend-and-delete scenario?
Shouldn't the publisher have at least the obligation to say something
has changed?

> Make it optional and I'll reconsider. -Tim

Devil's advocacy above. I wouldn't oppose it being optional (and
certainly wouldn't want to ding my tailpipe).

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Robert Sayre <[email-masked]>
Date: Thu, 28 Oct 2004 15:26:05 -0400

> -1  Absolutely, I'm lying down in the road here.
...
> Make it optional and I'll reconsider. -Tim

-1 even if it's optional. This is protocol-level data.

Robert Sayre

---

From: Graham <[email-masked]>
Date: Thu, 28 Oct 2004 20:43:19 +0100

>-1 even if it's optional. This is protocol-level data.

+1 to that. The only suggested use cases have been about caching etc.

Graham

---

From: Danny Ayers <[email-masked]>
Date: Thu, 28 Oct 2004 22:39:50 +0200

> +1 to that. The only suggested use cases have been about caching etc.

Consider a situation where I'm getting entries indirectly from a
variety of other sources:

entry atom:updated =  10:30pm contains a spelling mistake
entry atom:updated =  10:30pm is fixed

Which do I choose?

Yes, this is protocol and cache-related. But the question remains,
which do I choose?

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 28 Oct 2004 22:43:26 +0200

On Thu, 28 Oct 2004 12:06:28 -0700, Tim Bray <[email-masked]> wrote:

> As a publisher, I do not accept that, as a condition of publishing
> Atom, I am obligated to tell the world every time I make a spelling
> correction or trivial update.

You most often do this already in the HTTP protocol's 'Last-Modified'  
header. Why refuse to duplicate this value into the entry whenevery you  
change it?

The reason 'Last-Modified' isn't enough, is as I see it:

   1. Not all entries are retreivable outside a feed. They only exist
      in the feed they're published in, and may not even have an
      alternate representation (that is directly retreivable over
      HTTP or other protocols that has a notion of «last modified»).

   2. The feeds (and entries) might be served dynamically. Dynamically
      served content seldom provides the correct 'Last-Modified' date
      of the resource.

-- 
Asbjørn Ulsberg         -=|=-        [email-masked]
«He's a loathsome offensive brute, yet I can't look away»

---

From: Sam Ruby <[email-masked]>
Date: Thu, 28 Oct 2004 16:55:56 -0400

> -1 even if it's optional. This is protocol-level data.

But... a feed can contain multiple atom:entries.

- Sam Ruby

---

From: Robert Sayre <[email-masked]>
Date: Thu, 28 Oct 2004 17:02:55 -0400

Asbjørn Ulsberg wrote:
 >   1. Not all entries are retreivable outside a feed.

Aggregators just restore entries when they appear in feeds, AFAIK. Our 
two most active aggregator authoring participants don't see a need for 
it.[0]

Duplicating protocol data as an optimization would be worth considering 
if there was a big payoff. Aggregator authors don't see one, so who are 
we helping?

Robert Sayre

[0] http://www.imc.org/atom-syntax/mail-archive/msg07645.html

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 28 Oct 2004 14:19:50 -0700 (PDT)

> +1 to that. The only suggested use cases have been

+1 to that as well (i.e. -1 to PaceDateModified)

Having atom:updated makes atom:modified unnecessary
except for edge cases which to me shouldn't be catered
to in the core. 

=====
THINGS TO DO IF I BECOME AN EVIL OVERLORD #23
I will keep a special cache of low-tech weapons and train my troops in their use. That way -- even if the heroes manage to neutralize my power generator and/or render the standard-issue energy weapons useless -- my troops will not be overrun by a handful of savages armed with spears and rocks.

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 28 Oct 2004 23:31:05 +0200

On Thu, 28 Oct 2004 17:02:55 -0400, Robert Sayre <[email-masked]>  
wrote:

> Duplicating protocol data [...]

If you read my whole post, and Sam Ruby's comment to this, you'd note that  
this isn't always available in the protocol either. Entries may appear  
only in feeds, which means that there are absolutely no protocol metadata  
available for them, whatsoever.

Where should you look for 'Last-Modified' information when you can't get  
it from HTTP?

---

From: Danny Ayers <[email-masked]>
Date: Thu, 28 Oct 2004 23:39:27 +0200

> Sam Ruby wrote:

 Our
> two most active aggregator authoring participants don't see a need for

Sorry Sam, but the post you link to says:

"...a model like NetNewsWire that shows the differences using an HTML diff."

-- 

http://dannyayers.com

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 28 Oct 2004 14:54:28 -0700 (PDT)

> information when you can't get  

The real question is why you'd be looking for "last
modified" information from an entry in the first place
when atom:updated exists. 

=====

__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 28 Oct 2004 15:04:02 -0700 (PDT)

> differences using an HTML diff."

Exactly. If I was going to have some way of showing
users each minute change that occured to an entry then
I'd simply run a diff between old and new each time I
fetched the entry not test against some optional or
mandatory last-modified element then "alert" the user.

So as the the post said, I don't plan to use this
element and even if I was going to implement a feature
that targets the scenario this element is meant for I
still wouldn't use it. 

=====

_______________________________
Do you Yahoo!?
Express yourself with Y! Messenger! Free. Download now. 
http://messenger.yahoo.com

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 00:04:12 +0200

One question: if an aggegrator is designed to get the most-recently
updated entries (which doesn't sound unreasonable), how will the user
ever get to see spelling corrections and so on that have appeared
after the atom:updated date?

Cheers,
Danny.

btw, Movable Type also considers it useful information:

http://www.movabletype.org/docs/mtmanual_tags.html#item_MTEntryModifiedDate

---

From: Robert Sayre <[email-masked]>
Date: Thu, 28 Oct 2004 18:24:14 -0400

Asbjørn Ulsberg wrote:

> 
> If you read my whole post, and Sam Ruby's comment to this, you'd note 
> that  this isn't always available in the protocol either. Entries may 
> appear  only in feeds, which means that there are absolutely no protocol 
> metadata  available for them, whatsoever.
> 

It's not that I didn't read and understand your post. I just think 
you're wrong.

The feed delivery format isn't perfect, but sticking crufty date 
elements in the entries won't fix it.

If per-entry protocol data is important to you, the best course of 
action is to figure out a protocol that delivers entries individually. A 
pipelined HTTP request might be one solution.

Robert Sayre

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 28 Oct 2004 15:28:09 -0700 (PDT)

> after the atom:updated date?

This has NOTHING to do with the atom:modified date.
RSS Bandit, NetNewsWire and a bunch of other
aggregators always show the user the most recently
fetched content for the site. The only question is
whether the aggregator decides whether to announce
changes to the user based on some internal heuristic
or some notice from the server that indicates the last
time *anything* in the entry changed. 

=====

---

From: Sam Ruby <[email-masked]>
Date: Thu, 28 Oct 2004 19:25:07 -0400

>>[0] http://www.imc.org/atom-syntax/mail-archive/msg07645.html

Note: that link was posted by Robert Sayre.  My contribution to this 
thread was simply to point out that entry level information is not 
something I would expect to find at the protocol level.

FYI: Mod speedyfeed[1] makes use of the atom:modified element.

- Sam Ruby

[1] http://asdf.blogs.com/asdf/2004/09/mod_speedyfeed__1.html

---

From: Garrett Rooney <[email-masked]>
Date: Thu, 28 Oct 2004 19:52:01 -0400

> [1] http://asdf.blogs.com/asdf/2004/09/mod_speedyfeed__1.html

True, but as it's author I'd also be just as happy with atom:updated as 
described in PaceDateUpdated.  The point of delta encoding the feed is 
to only send down what the user needs to see.  The publisher can just as 
well denote that with atom:updated as they can with atom:modified.

As long as there's some way to say for each entry "has this changed 
since $DATE" I'll be happy, and I'm not too picky about the difference 
between "changed" and "changed in a way that matters to the reader".

-garrett

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 10:00:19 +1000

On 29/10/04 5:06 AM, "Tim Bray" <[email-masked]> wrote:

> As a publisher, I do not accept that, as a
> time I make a spelling correction or trivial update.

You already do via http Date-Modified, and if you break that you break all
sorts of http things. How is this different?

If PubSub or similar finds two copies of an entry you made, identified by
<id> to be the same thing, but which are different in minor ways, how will
it know which is most recent, and thus which one to propogate?

e.

---

From: Tim Bray <[email-masked]>
Date: Thu, 28 Oct 2004 17:31:06 -0700

> FYI: Mod speedyfeed[1] makes use of the atom:modified element.

But could, and arguably should, use atom:updated  -Tim

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 28 Oct 2004 17:35:58 -0700 (PDT)

> it know which is most recent, and thus which one to

How did PubSub find both entries? Via the same feed or
on in different feeds? 

=====

---

From: Tim Bray <[email-masked]>
Date: Thu, 28 Oct 2004 17:39:37 -0700

> sorts of http things. How is this different?

Well, I may change atom:updated on an entry even if the part of the 
entry that's in the feed hasn't changed.  How about atom:modified?  How 
about if I do some whitespace normalization?  How about if I change the 
stylesheet?  In a material way?  In a way that's unnoticeable except 
for it fixes a bug in the latest Safari release?  How about if I change 
only the serif version of the stylesheet, leaving sans-serif untouched? 
  How about if I change the Javascript?  All the stylesheet varations, 
on the javascript?  How about if I decide to change Unicode 
canonicalization forms without changing the content?

The notion that there is such a thing as an unambiguous change date is 
just silly.  The virtue of atom:updated is that it's unambiguous: the 
publisher asserted that something is different.  -Tim

---

From: Joe Gregorio <[email-masked]>
Date: Thu, 28 Oct 2004 20:46:45 -0400

> -1 even if it's optional. This is protocol-level data.

No it is not. This is an attribute of an entry, not the whole feed.

    -joe

-- 
Joe Gregorio        http://bitworking.org

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 10:59:39 +1000

> How did PubSub find both entries? Via the same feed or

how does that matter?

assume different sources or mechanisms for now (ie. not in the same feed
document, or the same feed source polled at different times). assume random
latency in the system such that you can't be sure which is most recent
simply per arrival time.

e.

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 11:21:30 +1000

On 29/10/04 10:39 AM, "Tim Bray" <[email-masked]> wrote:

> entry that's in the feed hasn't changed.

That's ok. atom:updated is more an editorial thing than a mechanics thing.
(I hadn't realised until now that's a possible use case though. Interesting)

> How about atom:modified?  How
> about if I do some whitespace normalization?

hmm .. the pace at some point was meant to state that changes due simply to
XML re-serialisation shouldn't be signalled by atom:modified ... currently
it only implies that.

> How about if I change the stylesheet?  In a material way?  In a way that's
> unnoticeable except for it fixes a bug in the latest Safari release?  How
> about if I change only the serif version of the stylesheet, leaving sans-serif
> untouched? 

Is the stylesheet content actually in the atom entry, or is it referenced
from there? If it's external, then it has it's own http:last-modified value.

> on the javascript?

Is the javascript content actually in the atom entry, or is it referenced
from there? If it's external, then it has it's own http:last-modified value.

> How about if I decide to change Unicode

Is this an XML serialisation change? atom:modified shouldn't be incremented
for that, same as it shouldn't be incremented if you simply change the order
of the entry elements (since order is not significant). Similarly
atom:modified shouldn't be incremented if <foo /> is re-serialised as
<foo></foo>. I hope I'm making this point clear. I hope this point is
something which can be clear, or are there all sorts of messy edge cases
where something could be re-serialisation or it could be content change?
(please don't mention QNames)

The pace currently says:

     Publishers SHOULD NOT change the value of atom:modified when no
     changes in the entry have occurred (eg. when re-serialising or
     re-exporting the entry).

that was written with the thought that "entry" is the conceptual thing, not
the particular xml serialisation. That is, if the data inputs are the same,
regardless of the xml-serialisation outputs. The wording can be improved, I
can see that. Suggestions?

e.

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 11:21:45 +1000

> when atom:updated exists.

Why? When you two versions of the same entry, and they both have the same
atom:updated value, yet they are different in content. How do you determine
which is the most recently modified?

e.

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 28 Oct 2004 18:26:02 -0700 (PDT)

> how does that matter?

The PubSub folks have said they don't plan to consider
entries with the same ID in different feeds as the
same due to spoofing. So that would throw out one of
the cases in your question. If they were in the same
feed then the content from the most recent time the
feed was fetched can be used. 

=====
THINGS TO DO IF I BECOME AN EVIL OVERLORD #222
I reserve the right to execute any henchmen who appear to be a little too intelligent, powerful, or devious. However if I do so, I will not at some subsequent point shout "Why am I surrounded by these incompetent fools?!"

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 11:29:40 +1000

>> differences using an HTML diff."

I use NNW, and the HTML diff is good ... when it works!

When it doesn't work, it's just a bunch of noise. It doesn't always fail
gracefully, imho.

> mandatory last-modified element then "alert" the user.

oh great. one of my complaints about NNW's html diff is that if a publisher
tweaks their publishing system which cause certain changes in the XML
serialisation, then every darn entry in their feed is marked as "modified".
Very tedious to go through that mess of noise.

By the way, Brent Simmons (of NNW fame) is in favour of a date-modified
element, last I heard.

e.

---

From: Robert Sayre <[email-masked]>
Date: Thu, 28 Oct 2004 21:33:54 -0400

> which is the most recently modified?

The definition of atom:updated holds the answer to that question--it 
doesn't matter.

Robert Sayre

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 11:39:32 +1000

On 29/10/04 11:26 AM, "Dare Obasanjo" <[email-masked]> wrote:

> the cases in your question.

Only if you restrict the case to pubsub with that policy in place. The case
remains for other sources, and may even remain for pubsub if the entries
were signed.

> If they were in the same feed then the content from the most recent time the

(assuming they don't have multiple bots doing the low level polling work and
feeding that into their system - do they now note the polling time, or the
input into the system time?)

how about two feeds from the same publisher (eg. a main feed and a category
feed)?

and it's not just Pubsub to consider. Even desktop aggregators might need to
do this kind of thing... it might be receiving some entries via XMPP, and
others by polling, some direct from the source, some via some intermediary
filtering service, some might get routed via a proxy, some might not.

e.

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 28 Oct 2004 18:38:22 -0700 (PDT)

> which is the most recently modified?

The one you fetched most recently from the feed. Is
this a trick question? 

=====

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 11:45:25 +1000

On 29/10/04 11:38 AM, "Dare Obasanjo" <[email-masked]> wrote:

>> How do you determine
> The one you fetched most recently from the feed.

that's not a good assumption to make as to which is most recently modified.
lots of things can go wrong to break that assumption.

e.

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 11:52:30 +1000

> The definition of atom:updated holds the answer to that question--it

then why would anyone even bother fixing spelling errors in their entries?

it does matter, it just doesn't need shouting from the rooftops, or if it
does you use atom:updated which is expressly for that purpose:

    [...] which the publisher wishes to bring
          to the attention of subscribers [...]

Consider other variations of modified entries, not just spelling fixes ...
say I have the capability to include various bits of meta-data in my
entries, and I do so because I consider that to be of value. I don't always
have the meta-data to immediately to hand (eg. some has to wait until I get
back to the office). I make those changes. It's not screamingly important
that readers are alerted to this augmentation, but it's still useful to
those that look for it.

e.

---

From: Robert Sayre <[email-masked]>
Date: Thu, 28 Oct 2004 22:05:15 -0400

>           to the attention of subscribers [...]

If the definition of atom:updated depends on atom:modified for its 
semantics, the two dates should have been proposed together. I would say 
there is no such dependency, and the idea of storing a modification date 
in the document itself is silly. In particular, editing an atom:entry in 
a text editor but forgetting to update atom:modified results in a 
non-compliant Atom Document.

Robert Sayre

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 28 Oct 2004 19:07:00 -0700 (PDT)

> lots of things can go wrong to break that

Like what?

=====

---

From: "Bob Wyman" <[email-masked]>
Date: Thu, 28 Oct 2004 22:31:18 -0400

>FYI: Mod speedyfeed[1] makes use of the atom:modified element.
	I think that in general, most implementations of RFC3229+feed[2]
would also want to atom:modified available to them. A developer of the
delta-generating code would be seeking to determine, as accurately and
efficiently as possible, what changes have occurred in a feed since some
particular point in time. Developers would probably prefer to be able to
rely on atom:modified, in the manner that mod_speedyfeed does, in order to
avoid having to do the more-expensive byte-oriented diffs which would be
needed to determine which entries have been modified. Without something like
atom:modified, RFC3229+feed implementors might end up relying on
atom:updated, which would result in some modifications never being published
to at least some readers of the feed. One other alternative might be to
maintain a database of MD5 hashes that it could use to recognize changes.
Yet another alternative would be for feed generating software to insert
"temporary" atom:modified that would be filtered when generating the "real"
or delta feed.
	Most of the alternatives to having atom:modified are pretty ugly.
Given that RFC3229+feed can result in very large bandwidth savings[3] and
appears to be gaining reasonable deployment, it might make sense to support
atom:modified to facilitate the generation of delta feeds. It should also be
noted that atom:modified would potentially make duplicate checking easier
for intermediary filtering services like PubSub, or for multi-feed database
applications like the search engines delivered by Feedster, Technorati,
BlogPulse, etc.

		bob wyman

[1] http://asdf.blogs.com/asdf/2004/09/mod_speedyfeed__1.html
[2] http://bobwyman.pubsub.com/main/2004/09/implementations.html
[3] At PubSub.com, our average byte/request for all requests that return
feeds (i.e. not 302 responses or errors) has been reduced to 28% of what it
was before we implemented RFC3229+feed in our servers. As more clients
support delta-feeds, the savings will increase even more.

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 12:53:56 +1000

>> lots of things can go wrong to break that

I'm writing this from my laptop, connected via wifi. Sometimes I connect via
an ethernet cable instead (much faster!). Those two connections use
different routes, one of them might have a proxy in place, one might not. I
also visit clients where I can connect to the internet via their corporate
connection. At least one is a huge multinational networking mob, whose
Sydney, Australia office is routed via some place in the USA, not via the
domestic ISP network. Thus I sometimes appear to be someplace geographically
quite different, based on the IP address I happen to use. Some publishers
have geographically dispersed web servers, and have different publishing
schedules, or at least delays as they push the bits around the world. Add
proxies to the mix. Add round-robin web farms to the mix. Pray nothing
breaks.

It is thus quite possible that I retrieve the most recently updated feed in
one moment, and then a short time later retrieve an older version from the
same URI.

e.

---

From: "Bob Wyman" <[email-masked]>
Date: Thu, 28 Oct 2004 22:53:49 -0400

Eric Scheid wrote:
> If PubSub or similar finds two copies of an entry you made, 
> identified by <id> to be the same thing, but which are different
> in minor ways, how will it know which is most recent, and thus
> which one to propogate?
	Other intermediaries might make different choices, but I'm afraid
that what we at PubSub would probably send both copies of the entry. We'd
have to do this since in many cases we would have no idea which was the more
recent entry.
	On the other hand, if atom:modified was supported and required, we
might be able to stop doing many of the heuristics we currently do and rely
on the publisher to tell us - via atom:modified - which is the most recent
version of an entry. In general, I prefer any solution that empowers the
publisher, rather than an in-channel intermediary (even one I'm building),
to make authoritative statements about messages and message sequences.

		bob wyman

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 13:01:30 +1000

On 29/10/04 12:05 PM, "Robert Sayre" <[email-masked]> wrote:

> If the definition of atom:updated depends on atom:modified for its semantics,
> the two dates should have been proposed together.
> 
a moot point since it was mandated that all date elements be proposed
singly, one at a time.

> I would say  there is no such dependency, and the idea of storing a
> modification date in the document itself is silly.

I agree there is no dependency. On the second part, by extension the idea of
storing a modification date in the CMS database is also silly. Or is it? Is
the meta-data of date-modified important or useful to the publisher?

> In particular, editing an atom:entry in a text editor but forgetting to update
> atom:modified results in a non-compliant Atom Document.

by the same argument we shouldn't be asking for xml well-formedness either,
since that can be screwed up with manual editing. Not convinced.

e.

---

From: "Bob Wyman" <[email-masked]>
Date: Thu, 28 Oct 2004 23:08:50 -0400

> the cases in your question.
	I don't think this is a fair characterization of my position on the
subject.
	Certainly, I have pointed out a number of problems with trusting
atom:id across feeds. However, there *are* situations in which we would be
able to use such trust. For instance, we might know of specific sets of
feeds that were known to be under stable and responsible management (perhaps
a cluster of "main feed" plus category feeds) or at some time in the future
we might get a mechanism to generate digital signatures. It should also be
recognized that in some cases, we'll be consuming "aggregate" feeds without
knowing it. In any of these cases (and perhaps others) we might, in fact,
chose or be forced to support cross-feed atom:ids.
	So, while Dare is correct that I have many concerns about cross-feed
atom:ids, these concerns are *not* universal and won't exist in all cases.
It would be good, I think, to design features such as atom:modified as
though cross-feed atom:ids *were* trust-worthy in order to ensure that we
don't block paths to implementing truley trust-worthy id systems in the
future.

		bob wyman

---

From: Robert Sayre <[email-masked]>
Date: Thu, 28 Oct 2004 23:22:55 -0700

>since that can be screwed up with manual editing. Not convinced.

No, it's not the same argument. A well-formedness error would result in 
an invalid document, whereas the situation I described would not.
There are lots of reasons last-modified timestamps are stored outside a 
given piece of data, in stuff like file systems, http, etc. I suppose we 
can work through each and every reason for this state of affairs, and 
why we've worked out something better.

I don't buy any of the mod_speedyfeed arguments, btw. If you can install 
such a module, you'd do better to point it at a directory of entries and 
work from the files' timestamps, as opposed to firing up an XML parser 
and dissecting a feed.

Robert Sayre

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Fri, 29 Oct 2004 07:45:08 +0200

On Thu, 28 Oct 2004 23:22:55 -0700, Robert Sayre <[email-masked]>  
wrote:

> given piece of data, in stuff like file systems, http, etc.

Yet again: Not for (all) entries. Most entries doesn't live outside the  
feed, and thus is never transported singularly over HTTP and thus hasn't  
got any 'Last-Modified' HTTP header attached to them at any time in their  
lifecycle.

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 12:00:38 +0200

Ok, Dare mentioned that RSS Bandit and some other aggregators use
internal heuristics to determine what has changed in a feed and what
should be presented to the user. This is fine assuming entries come
from a single source. But how does, and how *should* these tools
behave in the following *real* scenario:

I am subscribed to the folllowing feeds:

http://planetrdf.com/index.rdf
http://planet.xmlhack.com/index.rdf
http://rss.pubsub.com/37/b5/3778cec38f25c9f8282681622a.xml

The first is Planet XMLhack, the second Planet RDF. The third is a
PubSub subscription.

Whenever Norm Walsh posts to his blog, the items are picked up on the
first two feeds. If he happens to mention "environment-friendly lawn
mowers" it will be picked up on the third as well. So my aggregator
will receive three copies of the same item.

Now assume all these systems are using Atom. Norm corrects a spelling
mistake. My aggregator receives two copies of the entry prior to the
correction, one after. They all have the same atom:updated value, as
it was only a minor change.

How does my aggregator know which version of the entry to display?

Note that I'm not even subscribed to Norm's original feed, so Dare's
feed-refresh based heuristics are dead in the water.

I'm not going to lie in the road for anything, I just think Atom will
be hobbled as the syndication environment gets more distributed
without atom:modified (or equivalent).

If  doesn't go into Atom 1.0, I can assure you it will be in the next
version, after the issues have stacked up.

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Graham Parks <[email-masked]>
Date: Fri, 29 Oct 2004 10:32:04 +0100

>one moment, and then a short time later retrieve an older version from the
You know, you could have saved everybody's time by abbreviating that to 
"I've lost the argument".

Graham

---

From: Graham Parks <[email-masked]>
Date: Fri, 29 Oct 2004 11:13:47 +0100

>How does my aggregator know which version of the entry to display?
That depends entirely on the UI of your aggregator.

Graham

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 20:36:02 +1000

> "I've lost the argument".

Your logic is broken.

Dare asked how it is possible, I gave multiple examples of just how.

If you'd rather just hand wave at the assumptions, just say so.

e.

---

From: Graham Parks <[email-masked]>
Date: Fri, 29 Oct 2004 12:48:14 +0100

Eric, you're trying to argue that the most recent version of a resource 
is not the most recent version of a resource. Even if the conditions you 
describe prevail, there is absolutely nothing wrong with an aggregator 
that displays the most recent version of a resource as the most recent 
version of that resource.

Graham

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 14:09:05 +0200

> That depends entirely on the UI of your aggregator.

Ok. Put it another way:

My aggregator database contains 3 entries with have the same values
for atom:id and atom:updated. They came from three different source
feeds, as described earlier. Two of these have spelling mistakes in
the content. The third was produced later, and the mistake has been
corrected.

How does my aggregator select the latest (corrected) version from my database? 

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 22:26:45 +1000

On 29/10/04 9:48 PM, "Graham Parks" <[email-masked]> wrote:

> is not the most recent version of a resource.

I'm doing nothing of the kind. Please go back and re-read the email ...

I'm arguing that the most recently *retrieved* version is not necessarily
the most recently *modified* version, particularly so when we're talking
about specific published instances of those resources.

retrieved vs modified. two different things.

e.

---

From: Dare Obasanjo <[email-masked]>
Date: Fri, 29 Oct 2004 05:27:28 -0700 (PDT)

> then why would anyone even bother fixing spelling

Why do you keep pretending that atom:updated does not
exist? If I fix a typo in my feed, I can update the
atom:updated value if I feel it is that important that
people pick up the change. Even better my CMS/blogging
tool can do it automagically.  
 
> that readers are alerted to this augmentation, but

I can't support having both atom:updated &
atom:modified in the Atom syndication format. The
amount of confusion caused by having both elements
will not be worth the few edge cases that you and
Danny Ayers are trumpeting as being helped by having
both elements. 

-1 PaceDateModified 

It is extremely unlikely that anything anyone else
will say in this thread will convince me otherwise.  

=====

---

From: Graham Parks <[email-masked]>
Date: Fri, 29 Oct 2004 13:18:13 +0100

>How does my aggregator select the latest (corrected) version from my database? 
Depends what you clicked on to get to it. If you clicked on a specific 
feed to get to the entry, then obviousdly it would show the entry from 
that feed. So I guess we're talking about an aggregated list of entries 
(something I personally never use). The aggregator would then need to 
have functionality that removed duplicates, based on id. Because of the 
dangers of spoofing I'd imagine this would be off by default. At that 
point, yes, a  modified date would be useful. But it would be part of 
special functionality that removed duplicates, and not part of basic 
display functionality, as you're question sets it up to be.

Graham

---

From: Dare Obasanjo <[email-masked]>
Date: Fri, 29 Oct 2004 05:44:47 -0700 (PDT)

> How does my aggregator know which version of the

First of all this assumes the aggregator tracks items
across different feeds with the same ID. So far I
haven't seen an aggregator that does this, I could do
this in RSS Bandit but don't due to the high cost of
tracking this memory/perf wise. 

More importantly, your scenario assumes that all 3
feeds are republishing the atom:modified date from
Norm's feed which in many cases may not be true. What
if PlanetXMLHack upgrades their server software which
fixes some transcription errors that have occured in
how they emit feeds (I believe Sam pointed some issues
out in his blog) won't they change atom:modified? What
if this atom:modified date for the now correctly
transcribed entry is later than the typo fixed entry
in Norm's feed? If the aggregator blindly accepted
atom:modified across feeds then it would replace the
typo fixed entry with the entry still containing the
typo because one of the Planet* feeds changed some of
their feed generating code.  

Using atom:modified in this scenario is a hack that
will most likely break down more often than it would
work. 

=====

---

From: Eric Scheid <[email-masked]>
Date: Fri, 29 Oct 2004 22:44:28 +1000

On 29/10/04 10:27 PM, "Dare Obasanjo" <[email-masked]> wrote:

> exist?

I don't, so please don't mis-characterise my position as such.

If the change, even a spelling fix, is significant, then yes, increment
atom:updated. That's what its there for.

However, if the change isn't significant *enough* to bring to the attention
of subscribers who might have already seen the entry, but still important
enough to fix the public record, then what ... ?

Without atom:modified, such changes might well never propagate through
systems, even if more recently retrieved. I've got at least one system
elsewhere where propagation is optimised to ignore those things which have
apparently not been modified. Saves bandwidth, saves CPU, saves time, saves
resources. I won't be able to do the same optimisations with Atom if I don't
have atom:modified.

e.

---

From: Dare Obasanjo <[email-masked]>
Date: Fri, 29 Oct 2004 06:10:26 -0700 (PDT)

> enough to fix the public record, then what ... ?

It sounds like you are contradicting yourself. It
isn't important enough to fix yet it is. 

I've said all I have to say about this issue. -1 on
this Pace. So far there haven't been any convincing
arguments outside of highly contrived, hypothetical
scenarios where it could be useful. 

On the other hand, it is highly likely that the
existence of this atom:updated and atom:modified will
confuse consumers and producers of Atom feeds. 

=====

__________________________________
Do you Yahoo!?
Yahoo! Mail - Helps protect you from nasty viruses.
http://promotions.yahoo.com/new_mail

---

From: Graham Parks <[email-masked]>
Date: Fri, 29 Oct 2004 14:08:43 +0100

> resources. I won't be able to do the same optimisations with Atom if I don't

Why do you keep pretending that hash functions do not exist?

Graham

---

From: Bill de hÓra <[email-masked]>
Date: Fri, 29 Oct 2004 14:40:44 +0100

> Why do you keep pretending that hash functions do not exist?

They don't exist. Not in this spec.

After countless posts on dates, I still don't understand the 
expected behavioural difference between modified and updated. What 
is it you all arguing about?

cheers
Bill

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 15:45:07 +0200

> tracking this memory/perf wise.

That's down to the implementation of RSS Bandit. Tracking duplicates
is cheap and trivial if you're using a SQL database, you can even use
the <id> as a key. Same applies with RDF-based aggregators, where the
item URI is the key.

> in Norm's feed?

No. The atom:modified value would be set at source (in Norm's feed).
No intermediary should interfere with this.

 If the aggregator blindly accepted
> their feed generating code.

Incorrect assumption.

> will most likely break down more often than it would

With atom:updated alone, detecting the most recent version of an entry
is impossible.

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 15:50:35 +0200

On Fri, 29 Oct 2004 06:10:26 -0700 (PDT), Dare Obasanjo <[email-masked]> wrote:

So far there haven't been any convincing
> scenarios where it could be useful.

I gave a practical, real, current scenario with the feeds containing
Norm's posts. It would be useful in the sense that:

1. I don't want 3 copies of the same entry (with or without minor changes) 
2. I would prefer to see the most recent version of a post (after any
minor changes)

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Robert Sayre <[email-masked]>
Date: Fri, 29 Oct 2004 09:57:32 -0400

Bill de hÓra wrote:

> 
> After countless posts on dates, I still don't understand the expected 
> behavioural difference between modified and updated. What is it you all 
> arguing about?
> 

This is what the modified people want:

Title: "I love HTLM!"

oops, typo

Title: "I love HTML!"
[atom:modifed changed]

Title: "I love Atom!"
[atom:updated changed, loud sirens go off in the aggregator, etc]

Why they think this is a useful distinction and/or something that will 
work is beyond me. It's just a timestamp version of the X-Priority email 
header.

They don't want to use atom:updated for typos and such because that's 
been reserved for "radical changes in content" in their minds.

Robert Sayre

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 16:08:30 +0200

> been reserved for "radical changes in content" in their minds.

Yep, as a direct result of what it says in the spec:

[[
The "atom:updated" element is a Date Construct indicating the most
recent date and time when a change was made to the entry which the
publisher wishes to bring to the attention of subscribers. Such
changes will typically not include minor adjustments like spelling and
grammatical corrections.
]]

-- 

http://dannyayers.com

---

From: Robert Sayre <[email-masked]>
Date: Fri, 29 Oct 2004 10:13:38 -0400

> changes will typically not include minor adjustments like spelling and

You might want to read the spec, since that's not quite what it says

[[
The "atom:updated" element is a Date construct indicating the most 
recent instant in time when a change to the entry was made that the 
publisher wishes to bring to the attention of subscribers. For example, 
such changes might not include minor adjustments like spelling and 
grammatical corrections.
]]

Robert Sayre

---

From: Eric Scheid <[email-masked]>
Date: Sat, 30 Oct 2004 00:24:38 +1000

> isn't important enough to fix yet it is.

simple concept: there are degrees of importance.

> confuse consumers and producers of Atom feeds.

This is something we can work on, as it mostly involves writing better spec
text. Please elaborate.

e.

---

From: Eric Scheid <[email-masked]>
Date: Sat, 30 Oct 2004 00:27:46 +1000

> Why do you keep pretending that hash functions do not exist?

sheesh. grow up.

hash functions are very blunt instruments, and would require a lot of
complexifying to get them to ignore differences due to XML serialisation,
unicode encoding, etc.

e.

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 16:38:55 +0200

> such changes might not include minor adjustments like spelling and

I apologise, my mental aggregator only had the atom:updated
information to work from...

-- 

http://dannyayers.com

---

From: Graham Parks <[email-masked]>
Date: Fri, 29 Oct 2004 15:02:01 +0100

> They don't exist. Not in this spec.

Why do they need to be in the spec? I don't know what system's Eric was 
talking about specifically, but I can't imagine a scenario where you 
need modification dates to detect changes. Calculate a hash of the old 
and new data, and if the hashes are the same, it hasn't changed. It 
can't be more expensive than parsing XML to extract the date inside it.

> behavioural difference between modified and updated. What is it you all 

Updated: Changed only when the publisher wants to flag their data as 
having changed, which the client can then flag to the suer
Modified: Updated whenever any data is changed, for God knows what reason.

Graham

---

From: David Powell <[email-masked]>
Date: Fri, 29 Oct 2004 15:51:53 +0100

> is it you all arguing about?

This is how I believe it works, but the anti-atom:modifed people out there
presumably have a different model (using hashes I guess?):

Assuming you have a copy of an entry in your aggregator, and you then refetch
the feed:

If the atom:updated for the original entry is newer than atom:updated in the
recently fetched entry, then the aggregator should mark the entry as unread -
or something similar that is appopriate for the user-agent.

atom:modified on the other hand, is the equivalent of HTTP's Last-Modified
header.  It acts as a cache validator.  If atom:modified is updated, then the
entry should still be fully processed by clients, but it shouldn't be marked as
unread.

If you modify an entry without updating atom:modifed, then you can expect that
some readers may miss the change, because of intermediaries, or aggregators
which only reprocess entries that have changed.

-- 
Dave

---

From: Bill de hÓra <[email-masked]>
Date: Fri, 29 Oct 2004 15:50:45 +0100

> been reserved for "radical changes in content" in their minds.

Thanks Robert.

I don't see how having modified+updated can be easily specified for 
a syndication format. Making "changed" a grayscale of states rather 
than a bitflip* seems rather complicated (even if it's only two or 
three states).

On the offchance that this is a terminology spat, how about dropping 
both and using atom:changed? We can leave the inevitable "I'm 
changed? Changed how? Changed like a clown?" for non-core extensions.

cheers
Bill

* which reminds me a bit of those issue trackers or log tools that 
have plenty of fine-grained states but which no-one uses in practice.

---

From: Eric Scheid <[email-masked]>
Date: Sat, 30 Oct 2004 00:54:07 +1000

> such changes might not include minor adjustments like spelling and

Danny quoted the Pace, rather than the most recent iteration of the spec. No
harm, no foul. Can you explain what the *real* difference is between the two
wordings, because I can't see any difference.

e.

ps: and there shouldn't be any real difference in meanings, right? otherwise
the spec editors have overstepped their mark, right?

---

From: Bill de hÓra <[email-masked]>
Date: Fri, 29 Oct 2004 15:59:42 +0100

Graham Parks wrote:

> Why do they need to be in the spec? 

All I'm saying is that we can't make assumptions about the presence 
of a hash function unless we spec a hash function.

I don't know what system's Eric was
> can't be more expensive than parsing XML to extract the date inside it.

I use something very this to manage RDF literals. But it's a private 
detail of a system, not a public detail of a spec.

> Modified: Updated whenever any data is changed, for God knows what reason.

Thanks.

cheers
Bill

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 17:01:01 +0200

> can't be more expensive than parsing XML to extract the date inside it.

It's not the changing that's a problem, it's the chronological order.
You are assuming you know which is the new and which is the old data.
If the entries have come down different routes, as in the scenario I
described, you can tell they're different, but not which is newer.

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Eric Scheid <[email-masked]>
Date: Sat, 30 Oct 2004 01:04:49 +1000

On 30/10/04 12:02 AM, "Graham Parks" <[email-masked]> wrote:

> I don't know what system's Eric was talking about specifically, but I can't
> imagine a scenario where you need modification dates to detect changes.
> 

Here's something I did just tonite: I accessed a large online data store,
used FTP to get directory listings, and then downloaded just the files which
have a different date from what I have locally. (this is not a perfect
analog for atom entries being passed around, but bear with me)

The alternative is to download all the files just so a hash could be
calculated, and then ignore those whose hash was no different.

But lets say we avoid that by having the remote system calculate the hashes
... this would result in another, different problem: the remote file has
hash ABC, while the local file has hash XYZ. They are thus "different", but
which is the more recent file? Should the remote file be downloaded to
replace the local, or should the local be uploaded to replace the remote.
Hashes won't tell you that.

Changes in atom:modified tell us (1) something has changed, without having
to try to spot the change amongst the XML serialisation, and (2) which of
the two versions is the more recent.

e.

---

From: Robert Sayre <[email-masked]>
Date: Fri, 29 Oct 2004 11:08:03 -0400

> the spec editors have overstepped their mark, right?

You are free to squawk if you feel the spec editors have written down 
their own opinion instead of WG consensus. If the co-chairs agree, the 
editors will cheerfully revise the specification as instructed.

In this case, I'd say 3 or 4 people have bigger plans for atom:updated 
than other supporters do.

Robert Sayre

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 17:20:16 +0200

> than other supporters do.

Care to elaborate? 

I'd like to find out why there's such resistance to something that
fixes something that seems, as it stands, a pretty obvious
shortcoming.

Are they planning on using atom:updated to signal typo changes?

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: "Bob Wyman" <[email-masked]>
Date: Fri, 29 Oct 2004 11:42:43 -0400

I'm concerned that the current discussion seems to indicate that
change detection may be irrelevant and inappropriate for both desktop
aggregators and in-channel intermediaries. If this is correct, then any
discussion of hashing is irrelevant to the discussion. I believe what is
being suggested is something like the following:

	If atom:updated is used to flag only those changes that the author
of an entry wishes to bring to the attention of readers, then it seems to me
that intermediaries should respect the publisher's statements and ignore any
updates that are not so flagged. If this is correct, then any change
detection on the part of intermediaries should be considered a bad thing.
Intermediaries should not be generating entry hashes or otherwise trying to
detect changes.
	The model seems to be that an aggregator needs to build a database
of entries that has columns for atom:id and atom:updated. I should then only
process those entries that have atom:updated values that are greater then
what was previously stored for any given atom:id. No matter how great the
difference may be between two versions of an entry, I should not pay any
attention to the difference unless explicitly "told to" by a new
atom:updated value.
	This implies that if I find multiple versions of an entry, I should
ignore all but the *first* instance of an entry found with any particular
atom:updated value. I should not be concerned with identifying the "latest"
version since any version that shares the same atom:updated value is
equivalent in the opinion of the publisher. The opinion of the publisher is
paramount in these issues.

	Does what I've said above match what is believed by those who
support atom:updated to the exclusion of atom:modified?

		bob wyman

---

From: Robert Sayre <[email-masked]>
Date: Fri, 29 Oct 2004 11:49:33 -0400

> Are they planning on using atom:updated to signal typo changes?

That's what I'll be doing.

Robert Sayre

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 17:49:42 +0200

> than a bitflip* seems rather complicated (even if it's only two or

It isn't really grayscale, the values are qualitatively different, maybe:

subjective change : atom:updated 
objective change : atom:modified

(subjective from the publisher's point of view, objective from a diff
point of view)

Here's a use case:

Publisher:
A CMS is used for software project documentation. The documentation is
under continuous review, minor changes being made to the wording,
typos etc. With every change, the entry is republished with a
corresponding change in the atom:modified date. Periodically there is
a significant update - a new version release. The entry is republished
with a new value in the atom:updated element.

Consumer:
Receives all entries, changing content in the local store whenever the
atom:modified value has changed. Whenever the user looks in their
archives, they see the latest content. *But* the UI only re-displays
the entry when there has been a significant change.

This isn't contrived, here are two examples that might benefit from
this approach, both of which have been pushed into a feed:

http://norman.walsh.name/2004/projects/rdftwig
http://www.wasab.dk/morten/blog/archives/2004/07/05/wordpress-plugin-foaf-output

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Dare Obasanjo <[email-masked]>
Date: Fri, 29 Oct 2004 09:08:03 -0700 (PDT)

> being suggested is something like the following:

Nope. 

There are 2 scenarios. 

(1) Entry is updated in a single feed: Regardless of
what dates are attached to the entry, the aggregator
can tell if the content differs or not as well as
which is newer. 

(2) Same Entry in Multiple feeds: Most aggregators
don't track this today. Even if they did there's still
the problem of distinguishing changes to the entry
from a republishing feed to changes to the entry in
the original feed. 

Basically atom:modified buys you little if anything. 

=====

__________________________________
Do you Yahoo!?
Take Yahoo! Mail with you! Get it on your mobile phone.
http://mobile.yahoo.com/maildemo

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 18:21:39 +0200

> Basically atom:modified buys you little if anything.

It buys you 1 and 2.

1. The aggregator being able to tell which items are newer is based on
the assumption that the order of items in a feed will be significant.
I can't see anything about this in the spec, and if I remember
correctly the consensus was around the entry order in the feed not
being significant, i.e. "It's the Entries, Stupid!".

2. Can't be done on atom:updated alone (unless you accept that all
minor-changed version are equivalent to the first). Can be done with
atom:modified. Republishers shouldn't be changing the dates.

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Dare Obasanjo <[email-masked]>
Date: Fri, 29 Oct 2004 09:34:45 -0700 (PDT)

> being significant, i.e. "It's the Entries, Stupid!".

It seems you are arguing while having little idea of
the implementation of what you are talking about.
Order of items in the feed doesn't have anything to do
with this discussion.  

> atom:modified. Republishers shouldn't be changing

Aggregators don't need atom:updated to know they have
the most recent version of the feed since they just
fetched it. 

atom:updated is merely a *hint* that change should be
indicated to the user. 

=====

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 19:01:37 +0200

> Order of items in the feed doesn't have anything to do

Ok, I was only thinking as far as multiple versions of an entry in a
single feed document. Now that you mention it, there's nothing to say
that more recently published feeds will contain the more recent
entries. Another assumption that isn't found in the spec.

> the most recent version of the feed since they just

The problem is at entry level.

> atom:updated is merely a *hint* that change should be

It states that the publisher wishes to bring the change to the user's
attention. Sure, the aggregator can ignore the publisher's wishes.

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Danny Ayers <[email-masked]>
Date: Fri, 29 Oct 2004 19:10:36 +0200

Ok, I'm pulling out of this discussion. I anticipate atom:modified
will be ditched, ugly kludges will happen and atom:modified or
something similar will reappear in a later version of the spec.

But I would hope that the two use cases I presented (Norm's entries
from multiple sources, a CMS used for software doc publication) are at
least considered, and Bob's earlier question is answered
satisfactorily, especially the bits around:

[[
 I should not be concerned with identifying the "latest"
version since any version that shares the same atom:updated value is
equivalent in the opinion of the publisher. The opinion of the publisher is
paramount in these issues.
]]

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: "Bob Wyman" <[email-masked]>
Date: Fri, 29 Oct 2004 13:17:07 -0400

Using Robert Sayre's example (slightly modified):

T1: Title: "I love HTLM!" [Atom:updated set]
T2: Title: "I love HTML!" 
T3: Title: "I love Atom!" [Atom:updated modified]

Should PubSub forward 2 or 3 instances of this entry? Should we publish the
entry published at T2 that is "modified" but does not change the
atom:updated value?

At some time T4, after T3, should a retrospective search engine like
Feedster or Google return 1, 2 or 3 results given this stream of entries?

If you argue that a retrospective search engine should only deliver 1 or 2
results at T4, then which result should be returned at some time T2.5 which
is after T2 yet before T3? Should it return the first version of the entry
or the second?

I realize that all behaviors might be user definable or modified on request.
My question concerns the *default* expected behavior. What is the "ideal"
behavior?

	bob wyman

---

From: Robert Sayre <[email-masked]>
Date: Fri, 29 Oct 2004 13:46:38 -0400

> My question concerns the *default* expected behavior. What is the "ideal"

3 instances. It's data on the web. It's not a protocol.

Robert Sayre

---

From: Dare Obasanjo <[email-masked]>
Date: Fri, 29 Oct 2004 10:56:53 -0700 (PDT)

> Feedster or Google return 1, 2 or 3 results given
 
A search engine should cache the most recent version
of the page. Period. That's what Feedster and every
aggregator I've seen does today. 

This concept of 'change to content that is not
significant enough to be cached' is a bogus notion
that should not be encouraged. 

I fail to see how you drew the conclusion from the
current discussion that you should do otherwise. 

Your question is like asking what Google should do if
they fetch a page at time T3 and it has a
Last-Modified date of T2 then when they fetch it at
time T4 it has a Last-Modified date of T1. Should they
throw away the version they fetched previously, keep
it or keep both? 

Google and evey other search engine that implements
caching has already solved this problem one way or the
other today. I don't see why Atom has to reinvent the
wheel. 

=====

---

From: Graham Parks <[email-masked]>
Date: Fri, 29 Oct 2004 18:51:10 +0100

Those scenarios are all valid, except they have nothing to do with Atom 
or syndication. Let's go back to your example of a client retrieving an 
old version of a feed from a cache, after retrieving the current one. 
The optimization here is "I've been accidentally sent an old version of 
this entry, I won't bother processing it further". I don't think that's 
going to buy anyone much, especially as the XML and date parsing is 
already done at that point.

When else is it useful for optimizing?

Graham

---

From: "Bob Wyman" <[email-masked]>
Date: Fri, 29 Oct 2004 15:47:31 -0400

Dare Obasanjo wrote:
> A search engine should cache the most recent version of the page.
> Period. That's what Feedster and every aggregator I've seen does today. 
	Right. But, there are some issues in determining what is the "most
recent version of the page" in some environments *if* dissimilar versions
published or retrieved at different times can share common atom:updated
values.

> This concept of 'change to content that is not significant enough to be
> cached' is a bogus notion that should not be encouraged. I fail to see
> how you drew the conclusion from the current discussion that you should
> do otherwise. 
	The current draft empowers the publisher with a means for indicating
that "a change to the feed was made that the publisher wishes to bring to
the attention of subscribers." There is an implication here that if the
publisher does *not* change the atom:updated value, then the publisher "does
not wish to bring this change to the attention of subscribers." If the
publisher does not wish attention to be drawn to the change, then why would
aggregators, in-channel intermediaries, search-engines, etc. process the
change? Why are we providing mechanisms to make it easier for publishers to
publish data that they don't care if anyone reads?

	Personally, I think that the atom:updated is simply too confusing
and should be replaced by the much simpler and more easily understood
atom:modified. I am not convinced that there are a usefully large number of
cases in which the publisher will change and republish an entry yet not wish
to "bring this change to the attention of subscribers." It seems to me that
we've made a mistake by first adopting atom:updated, an edge case solution,
when we should have first adopted atom:modified and then had folk try to
justify why atom:updated should be in the core rather than in an extension.
The debate so far indicates that if atom:updated remains in the core, we'll
be suffering through debates about the meaning and implications of
"significant change" for the next few decades... A new perma-thread has been
born... Not all births should be celebrated...

		bob wyman

---

From: Tim Bray <[email-masked]>
Date: Fri, 29 Oct 2004 11:37:43 -0700

> Intermediaries should not be generating entry hashes or otherwise 

It depends whether you trust the publisher.  If not, it's entirely 
appropriate to go ahead and ignore dates and do data-level comparison.  
But if not, it's inappropriate to trust atom:modified when you don't 
trust atom:updated.

> attention to the difference unless explicitly "told to" by a new

That's how I'd do it.

> paramount in these issues.

Unless, of course, you distrust the publisher, in which case you should 
ignore each and every date field and draw your own conclusions. -Tim

---

From: Tim Bray <[email-masked]>
Date: Fri, 29 Oct 2004 11:31:54 -0700

On Oct 29, 2004, at 5:44 AM, Eric Scheid wrote:

> systems, even if more recently retrieved.

Which, doesn't matter, unless you've decided you don't trust the 
publisher's choice in setting atom:updated.  In which case, you 
probably shouldn't trust their truthfulness setting atom:modified 
either.  -Tim

---

From: Tim Bray <[email-masked]>
Date: Fri, 29 Oct 2004 11:28:30 -0700

> from there? If it's external, then it has it's own http:last-modified 

My point is that as the publisher, I can look at what I'm doing to the 
stylesheet or normalization and make up my mind whether it's important 
enough to consider an update.  There is no automatic decision procedure 
("objective", feh) that can meaningfully define what constitutes an 
update.  Per your argument, if I drastically changed the *external* 
stylesheet in a way that I thought mandated re-reading the content, I 
should change atom:updated, but not change atom:modified, because the 
content itself hadn't changed.  ????

  -Tim

---

From: Tim Bray <[email-masked]>
Date: Fri, 29 Oct 2004 11:26:00 -0700

> which is the most recently modified?

You could assume that if it mattered, the publisher would have changed 
atom:updated, and not stress out; just pick whichever you picked up 
most recently. -Tim

---

From: Tim Bray <[email-masked]>
Date: Fri, 29 Oct 2004 11:33:51 -0700

> the two versions is the more recent.

Only if you trust atom:modified.  But you've already decided not to 
trust atom:updated, so why should you? -Tim

---

From: Tim Bray <[email-masked]>
Date: Fri, 29 Oct 2004 11:30:26 -0700

On Oct 28, 2004, at 7:31 PM, Bob Wyman wrote:

> would also want to atom:modified available to them.

Only on the assumption that you had made a deliberate policy decision 
to ignore the expressed opinion of the publisher as to when the content 
had meaningfully changed.  In which case, you have to acknowledge that 
a publisher might choose to lie about atom:modified to.  In which case, 
to be sure, you have to compare the content.  In which case, there is 
no case for atom:modified. -Tim

---

From: Eric Scheid <[email-masked]>
Date: Sat, 30 Oct 2004 10:05:18 +1000

> That's what I'll be doing.

perhaps then the spec should be updated such that atom:updated is specced
the way you want it - all changes get reflected in atom:updated.

what we'll need then is some way of letting a publisher selectively signal
that the change is qualitatively different from all the noisy changes which
are minor trivialities of spelling fixes and the like. Something like
atom:updated-significantly.

seriously.

e.

---

From: Eric Scheid <[email-masked]>
Date: Sat, 30 Oct 2004 10:38:25 +1000

On 30/10/04 4:28 AM, "Tim Bray" <[email-masked]> wrote:

> Per your argument, if I drastically changed the *external*
> content itself hadn't changed.

atom:updated is part of the entry. it has changed.

ergo, atom:modified should also be changed in this use case.

e.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sat, 30 Oct 2004 14:37:43 +0200

On Fri, 29 Oct 2004 13:18:13 +0100, Graham Parks <[email-masked]> wrote:

> (something I personally never use).

Have you ever used USENET with a decent news reader? If so, you've  
probably noticed that all cross-posted messages is marked as «read» once  
for all newsgroups, based on their Message-ID. I'd say that the  
above-mentioned scenario is just the same; a cross-post of a single Atom  
article.

Regardless of which feed, group, context or mood I read the single entry  
in, I would _not_ want to have it show up again whenever I change either  
of these variables. An entry is an entry is an entry, because it has the  
same atom:id. When I've read it, I've read it, and don't want to read it  
again. I'd say that's one of the main values in atom:id.

But. I would like to read updates in the entry, and I would like to read  
the latest version of the entry if I haven't read it before.

> The aggregator would then need to have functionality that removed
> duplicates, based on id.

Yes.

> Because of the dangers of spoofing I'd imagine this would be off by
> default.

This is not off by default in news readers that supports supersede  
mechanisms. It's also not off to mark an article as read regardless of  
which group it's posted to.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sat, 30 Oct 2004 14:47:46 +0200

On Fri, 29 Oct 2004 05:44:47 -0700 (PDT), Dare Obasanjo <[email-masked]>  
wrote:

> First of all this assumes the aggregator tracks items across different
> feeds with the same ID. So far I haven't seen an aggregator that does
> this

Have a look at a decent newsreader. Agent, Xnews, Gnus and slrn for  
example. They all do this, and have done it for quite some years. They  
store the read Message-ID's across newsgroups so users shouldn't be  
bothered with reading already read articles over and over again.

> More importantly, your scenario assumes that all 3 feeds are
> republishing the atom:modified date from Norm's feed which in many
> cases may not be true.

Why not? And if they aren't republishing the date, that won't be  
catastrophic. The user agent would just have less metadata to base the  
decision of which entry is the most recent on.

> What if PlanetXMLHack upgrades their server software which fixes some
> transcription errors that have occured in how they emit feeds (I
> believe Sam pointed some issues out in his blog) won't they change
> atom:modified?

No. Not at all. No intermediaries should change the value of  
atom:modified. Nor atom:content or atom:title, for that matter.

> Using atom:modified in this scenario is a hack that will most
> likely break down more often than it would work.

I disagree. In your idea of atom:modified, there are no rules for who can  
modify its content and how it is transmitted. If you feel that the problem  
with the proposed text for atom:modified lies in the lack of these rules,  
then please say so.

I think it's pretty obvious that an aggregator writer can't change the  
content of atom:content. I'd assume the same for atom:modified.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sat, 30 Oct 2004 15:39:01 +0200

On Thu, 28 Oct 2004 18:26:02 -0700 (PDT), Dare Obasanjo <[email-masked]>  
wrote:

> The PubSub folks have said they don't plan to consider entries with the
> same ID in different feeds as the same due to spoofing.

So because there _might_ be spoofers, we shold drop all features from the  
Atom protocol and format that _might_ be used for spoofing? Instead of  
killing the symptoms one by one in fear of what _might_ happen, can't we  
instead look for a solid solution for everything?

Signatures, for example: <url: http://www.w3.org/TR/xmldsig-core/>.

---

From: Bill de hÓra <[email-masked]>
Date: Sat, 30 Oct 2004 14:42:13 +0100

> content of atom:content. I'd assume the same for atom:modified.

Are you saying we're missing a few SHOULD NOTs?

cheers
Bill

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sat, 30 Oct 2004 16:05:47 +0200

On Fri, 29 Oct 2004 11:33:51 -0700, Tim Bray <[email-masked]> wrote:

> trust atom:updated, so why should you?

This goes back four months to when we discussed atom:issued. Some thought  
it was an authorative (subjective) date, while others thought it was a  
machine-managed (objective) date.

While most people agree on that atom:updated is authorative (or do we? I  
think the specification text for that date is incredibly ambiguous), I  
think there is not a common understanding of what atom:modified is. HTTP  
servers manages 'Last-Modified' automatically. I expect Atom tools to do  
the same.

atom:updated is the authorative date in an entry, while atom:modified is  
the machine-generated, objective one. The first one tells you when the  
author wants you to re-read the entry, the last one tells the tool you're  
using when to update the entry in its database. The first one probably  
marks the entry as unread in your reader, the last one doesn't do anything  
visually in your reader at all.

So, this is the subjective/objective discussion all over again, imho.  
We've had it before, and need to have it again, because it's a profound  
distinction we need to make in a publishing format and protocol.

...Which again brings me to the subject of separating protocol-metadata  
 from format-metadata. In my opinion, atom:modified belongs to the protocol  
while atom:updated belongs along with the entry in the format.  
atom:updated is closer to the content, while atom:modified is closer to  
the protocol. No one else seems to think this distinction is useful, but I  
mention it again, nonetheless.

---

From: Tim Bray <[email-masked]>
Date: Sat, 30 Oct 2004 10:16:51 -0700

On Oct 30, 2004, at 10:11 AM, Dare Obasanjo wrote:

> RSS Bandit will always update the entry in regardless
> of whatever the values of date fields are on the
> entry. Entries change today in RSS & Atom feeds
> without date fields changing and I don't expect this
> behavior to change when Atom 1.0 is published.

But Dare, with Atom, you really don't have to any more.  It would be 
perfectly acceptable, when you read an Atom feed, to look at the 
atom:id and the atom:updated and just move along if you've seen the 
entry before with the same or a later date.  There's nothing wrong with 
doing what you do, but I bet a lot of new implementors take the 
two-column-table approach that Bob suggested. -Tim

---

From: Dare Obasanjo <[email-masked]>
Date: Sat, 30 Oct 2004 10:11:43 -0700 (PDT)

=====

---

From: Dare Obasanjo <[email-masked]>
Date: Sat, 30 Oct 2004 10:21:06 -0700 (PDT)

--- Tim Bray <[email-masked]> wrote:
> two-column-table approach that Bob suggested. 

I doubt I'll take your advice. It costs nothing to
replace the existing content with the content from the
feed since I've already downloaded and parsed it.
Especially since there is the off chance that the user
fixed a typo or some inserted some new text but didn't
update the date field. 

=====

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sat, 30 Oct 2004 20:00:44 +0200

> the way you want it - all changes get reflected in atom:updated.

Indeed. It would be interesting to hear e.g. Sam Ruby's opinion (since  
he's the creator of atom:updated) on what an aggregator should do whenever  
atom:updated changes for a given entry. Should it:

   1. Update the entry's content.
   2. Mark the entry as unread.
   3. Score it upwards in the message pool.
   4. (And then) do nothing.

The points above are not mutually exclusive, one might think that an  
aggregator can do 1 and 4, for example. I'd think it would be most useful  
if a change in atom:updated made the aggregator do 1 and 2, and if it had  
some kind of scoring system, then 3.

If atom:updated is going to be used for both trivial and non-trivial  
changes, most users would probably like their aggregator to do 1 and 4 and  
not ever be bothered again, because else there would be too much noise.

---

From: "Bob Wyman" <[email-masked]>
Date: Sat, 30 Oct 2004 15:48:48 -0400

Dare Obasanjo wrote:
>> I bet a lot of new implementors take the two-column-table approach
>> that Bob suggested. 
	I wasn't really "suggesting" the two-column approach. Rather, I was
just documenting what I thought were the logical implications of the
statements that were being made by various "anti-modified" folk...
	I agree with Dare on this one. As long as you have a more recent
copy of the entry, it makes sense that you would make it available to the
user. At that point, the interesting question is whether or not you "bring
it to the user's attention" by doing something like flagging it as unread.
People's expectation for atom:updated seems to be that you would only re-set
an entry to "unread" if atom:updated had been modified. 
	It appears that there are at least four types of entries that may be
handled by a news aggregator:
	1. New entry. First entry instance which bears a specific atom:id
	2. Updated entry. First enty instance which, while not a "New
Entry", has a specific combination of atom:id + atom:updated value.
	3. Modified entry. An entry which differs from an New or Updated
Entry only in elements other than atom:id and atom:updated.
	4. Duplicate entry. An entry which is identical, in all elements, to
an entry previously received or processed.

	Is this list exhaustive? Am I missing something here?

		bob wyman

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sun, 31 Oct 2004 00:07:16 +0200

> Are you saying we're missing a few SHOULD NOTs?

If we need to make it that explicit, then yes, maybe we are.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sun, 31 Oct 2004 00:15:39 +0200

On Sat, 30 Oct 2004 10:11:43 -0700 (PDT), Dare Obasanjo <[email-masked]>  
wrote:

> RSS Bandit will always update the entry in regardless of whatever the
> values of date fields are on the entry.

Then RSS Bandit is vulnerable to spoofing already. How does atom:modified  
change that situation, specifically for RSS Bandit? I.e., how can the  
situation get any worse, when RSS Bandit already just eats everything it  
receives and overwrites everything it has?

---

From: Dare Obasanjo <[email-masked]>
Date: Sat, 30 Oct 2004 18:02:43 -0700 (PDT)

> receives and overwrites everything it has?

I love it when people tell me how my own code works. 

RSS Bandit is vulnerable to spoofing if somehow
someone manages to use DNS redirection or hack the web
server to change the content of the feed. If that
happens, then Yes RSS Bandit would have been
"spoofed". However I'd be very impressed to see what
techniques you propose aggregator implementations use
to prevent such "spoofing". 

Basically I have no idea what your statement above has
to do with the discussion at hand besides stating the
blindingly obvious. Can you clarify? 

=====

---

From: Walter Underwood <[email-masked]>
Date: Sat, 30 Oct 2004 18:29:23 -0700

> an entry to "unread" if atom:updated had been modified. 

Maybe not.

My current preference would be for separate "read" and "changed"
flags. I don't mind being notified that something has changed,
but the "new on every change" behavior in NetNewWire is bugging me.

wunder
--
Walter Underwood
Principal Architect, Verity