Subject: Re: PaceDateline

From: Sam Ruby <[email-masked]>
Date: Sat, 24 Jul 2004 13:46:14 -0400

Arve Bersvendsen wrote:

>>http://www.intertwingly.net/wiki/pie/PaceDateline
> 
> I'm -1 on this as the pace stands at the time of writing. This looks like
> pubDate (almost) all over again, with CMSes having to invent their own uses
> of DC dates.

I understand and appreciate this argument.  If it helps, let me retrace 
my train of thought.  From http://www.intertwingly.net/wiki/pie/DateSurvey :

     [GrahamParks]: I don't think optional core elements cause problems
     if there are clear rules about what to do when one is missing

     [RogerBenningfield]: That means throwing out the subtle bits,
     leaving them for extensions.

     [SamRuby] I prefer names that either don't already have strong
     predefined associations or have associations that match the
     definition we intend.

So, I went back to dcterms.  What I found there was a set of dates, 
*each* of which I could imagine *somebody* making a clear and compelling 
argument for.

What to do?

First, let's look at the 80% use case.  In the case of weblogs, this 
unquestionably is the case where created, valid, available, issued, 
modified, accepted, copyrighted, and submitted are all simultaneous.

Then I picked a name which does not have a strong predefined association 
with any of those "refined" elements.  I tentatively picked dateline. 
If you look at how that term is used by The Media (e.g., newspapers, 
magazines, television), and apply it to a media (the internet) where 
publishing is often accomplished without recourse to an editor, then it 
does seem to fit.

So, how is this different than pubDate?  Well, for starters, there are 
clear rules.  But, is this enough?  I thought about it a bit, and the 
worst thing I could imagine happening is for everybody to pick a 
different refined date to sort on.

So, I thought about the aggregators that I have used.  When presenting 
views of "fresh news", what they mostly care about can be summed up as 
"if I were to poll your site once a second, when would I have first seen 
this entry?"  The refined date that most closely matches this 
description is <available>, which coincidentally matches the spec 
definition (as opposed to the popular usage) of the RSS pubDate.

I figure that by drawing attention to this one date, this problem can be 
solved to the extent that it can be solved.  There always will be the 
case of people who are unwilling and unable to follow instructions. 
Those that don't will likely not place this additional element in, and 
this tend to have the effect of penalizing them (their updates won't be 
seen) without placing any blame on the aggregators.  And, even in the 
worst case (users being penalized for missing news), this is no worse 
than the current state.

I then thought about the subtle distinction between updated and 
modified.  If aggregators are sorting on available, then I'm not sure 
what they would do with another field.  Thinking about it a bit more, 
one could argue that if somebody made a change and chose NOT to update 
the value of available, then one could imply a bit of intent.

At this point, the suggestion seemed worth enough to write up, even if 
it doesn't attract consensus.  I was troubled by the definition of 
dateline also implying a place of origin, so I did a minimal amount of 
research (google rocks!) before committing the Pace.

If the event that this Pace is accepted, some form of the above text 
belongs either as an informative note, or in a separate usage guide.

- Sam Ruby

---

From: Graham <[email-masked]>
Date: Sat, 24 Jul 2004 17:15:57 -0400

> modified, accepted, copyrighted, and submitted are all simultaneous.

The problem here is requiring aggregators to infer something that may 
not be true. My example was <issued> could safely be used for a 
dateline in <dateline>'s absence. However it is not safe to say 
something has not been modified since creation in <modified>'s absence. 
You need to include wording in your proposal that says "you must 
include <dc:xxx> when <dc:xxx> is different from <dateline>" - 
currently it is only implied.

> "if I were to poll your site once a second, when would I have first 
> seen this entry?"  The refined date that most closely matches this 

Again, to ensure uniformity, the spec needs to encourage and/or mandate 
<available> (or equivalent) when it differs from dateline.

> the value of available, then one could imply a bit of intent.

I don't like this line of thought. I'd probably use <updated> to flag 
an update in the listing (eg mark as unread). In fact I'd rather do 
that than sort by it (or at least have the choice of not sorting by 
it).

Graham

---

From: Dare Obasanjo <[email-masked]>
Date: Sun, 25 Jul 2004 12:15:32 -0700 (PDT)

> that than sort by it (or at least have the choice of

+1 

As an aggregator author I don't plan to sort on any
other date field besides available/issued. All the
other dates like modified/updated give me is the
option to do other things with the content like show
an HTML diff or highlight as unread. 

=====
THINGS TO DO IF I BECOME AN EVIL OVERLORD #25
No matter how well it would perform, I will never construct any sort of machinery which is completely indestructible except for one small and virtually inaccessible vulnerable spot.

_______________________________
Do you Yahoo!?
Express yourself with Y! Messenger! Free. Download now. 
http://messenger.yahoo.com

---

From: Sam Ruby <[email-masked]>
Date: Sun, 25 Jul 2004 16:42:36 -0400

> an HTML diff or highlight as unread. 

Here's a concrete example that I will use to set up my question below:

http://www.tbray.org/ongoing/When/200x/2004/07/20/AuthoringPain

This was originally published on 2004/07/20.  It was significantly 
updated four days later.  If you look at http://www.tbray.org/ongoing/, 
you will see that this entry was sorted back to the top.

It clearly was Tim's intent that readers who saw the original be 
presented with the updated version.  Presumably, there are other changes 
(typo fixes) for which there isn't this intent.

It is desirable for there to be a way for Tim to unambiguously to convey 
his intent.  Tim attempted to signal his intent by updating the value of 
pubDate.

Testing with RSSBandit, I found that to my surprise, it appears that if 
the value of pubDate changes from the first time the feed is fetched, 
RSSBandit will continue to present, and optionally sort by, the original 
value.

At the present time, with the version of RSSBandit that I have installed 
(1.2.0.112), it appears that the only way to achieve this is to change 
the value of the link tag.

I take it that you are with Asbjørn in suggesting that this is the right 
way to signal significant changes?

- Sam Ruby

---

From: Dare Obasanjo <[email-masked]>
Date: Sun, 25 Jul 2004 14:04:02 -0700 (PDT)

> (typo fixes) for which there isn't this intent.

Probably. 

> his intent.  Tim attempted to signal his intent by

OK. 

> RSSBandit will continue to present, and optionally

Yes. I'm debating with myself on whether I should
consider this a bug or not. 

> the value of the link tag.

Yes. 

> way to signal significant changes?

No. As has been pointed out before Asbjørn's solution
makes aggregators that support his supersede mechanism
appear broken to users since they end up with
"duplicate" entries. 

RSS Bandit does not visually identify modifications
and the like because it is not functionality I have
found desirable nor is it something that the several
thousand people who've used RSS Bandit or the hundreds
of people who've submitted feature requests and bug
reports have ever asked for. So it shouldn't be used
as an example of an aggregator that cares about
modifications. I'd suggest trying NetNewsWire or
SharpReader to see what they do with Tim's post. 

=====

__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
http://promotions.yahoo.com/new_mail

---

From: Dare Obasanjo <[email-masked]>
Date: Sun, 25 Jul 2004 14:26:13 -0700 (PDT)

Missing word in my response. Added in ALL CAPS. 

--- Dare Obasanjo <[email-masked]> wrote:
> makes aggregators that DON'T support his supersede
> mechanism

=====

__________________________________
Do you Yahoo!?
Yahoo! Mail - 50x more storage than other providers!
http://promotions.yahoo.com/new_mail

---

From: Tim Bray <[email-masked]>
Date: Sun, 25 Jul 2004 16:28:56 -0700

>> (typo fixes) for which there isn't this intent.

It was.

> consider this a bug or not.

 From my point of view, in this case, it's clearly a bug.  I don't knonw 
whether I'm an edge case, but what I did certainly feels like a natural 
idiomatic use of the Web.

> I'd suggest trying NetNewsWire or

NNW lets you choose whether to sort on date or title, I've never sorted 
by anything but date, which brings the revised version to the top. -Tim

---

From: Dare Obasanjo <[email-masked]>
Date: Sun, 25 Jul 2004 16:45:51 -0700 (PDT)

--- Tim Bray <[email-masked]> wrote:

> >>
> >> This was originally published on 2004/07/20.  It
> was
> >> significantly
> >> you will see that this entry was sorted back to
> the
> >> top.

> >> RSSBandit will continue to present, and
> optionally
> >> sort by, the original
> >> value.
> >
>  From my point of view, in this case, it's clearly a
> bug. 

What spec backs up this position? Reading the RSS 2.0
description of pubDate it seems to map quite well to
dc:issued. I find it quote illogical to expect that
the date of issuance for a particular entry should
change, so RSS Bandit does not change the original
pubDate of an entry. 

It seems you are trying to add additional semantics to
those of pubDate above and beyond the intent and
letter of the RSS 2.0 specification. 

> I don't knonw 

If it is idiomatic can you point at 5 other entities
amongst the millions that use the WWW that also engage
in this behavior? 

> by anything but date, which brings the revised

The question isn't whether they sort on date but
whether they replace old pubDate values with the new
one or do other things to imply that the post has been
updated. 

=====

---

From: Sam Ruby <[email-masked]>
Date: Sun, 25 Jul 2004 19:57:10 -0400

> by anything but date, which brings the revised version to the top. -Tim

Now we are getting to the heart of the issue that is bothering me.

In the absense of guidance in the existing specifications, Tim did what 
made sense to him, and what worked with his preferred tool.

Other people have different ideas of what makes sense to them, and what 
works in their preferred tools.  Trying to transpose what Tim did to the 
terminology used by Atom and/or Dublin Core results in a thumbs down by 
Graham and Dare.

That's not the way this should work.

- Sam Ruby

---

From: Tim Bray <[email-masked]>
Date: Sun, 25 Jul 2004 17:01:14 -0700

> change, so RSS Bandit does not change the original

http://blogs.law.harvard.edu/tech/rss#hrelementsOfLtitemgt says "Its 
value is a date, indicating when the item was published".  Well, I 
clearly did something to that entry yesterday which resulted in 
substantially new content showing up on the web.  I had naively thought 
that "published" was a reasonable word to describe what I did.  But if 
the correct reading is "same as dc:issued" then clearly I'm mis-using 
it.  Uh, let's agree that in Atom, we're going to be a little more 
explicit so that if someone like me screws up then someone like Dare 
has a better basis for the pointing of fingers.

> amongst the millions that use the WWW that also engage

It may be the case that I've been mis-using <pubDate>.  However, the 
notion of "Last-Modified" is important enough to have its own HTTP 
header.  In what I would consider normal syndication workflows 
last-modified is generally seen as important information to have.

Surely you would agree that one of the defining differences between the 
Web and prior publishing media is that you can update in place.

> one or do other things to imply that the post has been

Both.  That is to say, if you change the content of the feed NNW will 
notice and highlight it even if you don't change the date.  But if you 
change the date, NNW definitely respects that. -Tim

---

From: "Bill Kearney" <[email-masked]>
Date: Sun, 25 Jul 2004 21:19:12 -0400

> made sense to him, and what worked with his preferred tool..
>

Bearing in mind that the tools could only work with what was known.  Shitty data
often made for shitty tools.  Some tools, however, have risen about the data
available at the time and built in better functionality.  FeedDemon is another
such tool that uses item dates when present as well as sorting.  The question
here isn't whether crappy data should persist, nor whether we should cater to
weak tools.  The question is what core data should we specify as required so as
to encourage the growth of BETTER tools.

The users have, consistently, requested that their reader programs be able to
remember what they've already considered 'seen'.  Given the badly written specs,
bad tools and, frankly, bad editorial practices on the part of some feed
producers it has been a real pain in the ass to handle this consistently.
Perhaps an argument from the other end, looking at what users expect and how the
extant tools can or can't be made to meet those expectations, is in order.

-Bill Kearney

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Tue, 27 Jul 2004 20:05:45 +0200

On Sun, 25 Jul 2004 14:04:02 -0700 (PDT), Dare Obasanjo <[email-masked]>  
wrote:

> RSS Bandit does not visually identify modifications and the like because
> it is not functionality I have found desirable nor is it something that
> the several thousand people who've used RSS Bandit or the hundreds of
> people who've submitted feature requests and bug reports have ever asked
> for.

I think that a feature like this is primarily interesting from a  
publisher's point of view, not a consumer. That's why you probably haven't  
seen this as a feature request in RSS Bandit.

As Atom is going to fulfill (at least I hope so) both producer's and  
consumer's wishes, Atom may need to support revisioning some day. How and  
when is still to be sorted out, but it _may_ be that ID's will change for  
each new version of an entry. This is to accommodate producer's needs. As  
users subscribe to more and more feeds that shows duplicate entries,  
because their RSS aggregator doesn't support Atom's revisioning mechanism,  
they will report this to the aggregator developers, e.g. you.

Would you build support for a revisioning model into RSS Bandit before, or  
only after, users make requests for it?

-- 
Asbjørn Ulsberg         -=|=-        [email-masked]
«He's a loathsome offensive brute, yet I can't look away»

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 29 Jul 2004 12:51:21 -0700 (PDT)

> only after, users make requests for it?

RSS Bandit is an Open Source project. It is all about
scratching an itch. If no one has an itch for
revisioning support in RSS Bandit then it won't get
implemented. 

=====