Subject: PaceAllowDuplicateIdsWithModified

From: David Powell <[email-masked]>
Date: Sun, 15 May 2005 23:12:12 +0100

PaceAllowDuplicateIDs received some opposition because of its use of
atom:updated to differentiate multiple revisions of an entry [1][2][3].

I've posted a couple of Paces - basically a single a proposal, split
into two bite-size pieces:

<http://intertwingly.net/wiki/pie/PaceAllowDuplicateIdsWithModified>
<http://intertwingly.net/wiki/pie/PaceDateModified2>

Apologies in advance. I know what happened last time we discussed
dates, and this is hardly the best timing, but it seemed like an
obvious fix for the problem.

[1] http://www.imc.org/atom-syntax/mail-archive/msg14738.html
[2] http://www.imc.org/atom-syntax/mail-archive/msg14750.html
[3] http://www.imc.org/atom-syntax/mail-archive/msg15144.html

-- 
Dave

---

From: Graham <[email-masked]>
Date: Mon, 16 May 2005 00:04:53 +0100

> <http://intertwingly.net/wiki/pie/PaceDateModified2>

+1

But mostly symbolically, because I don't think the atom:modified will  
fly, and I don't like it much. But it's still better and more  
complete than the original duplicate ids pace.

Graham

---

From: Eric Scheid <[email-masked]>
Date: Mon, 16 May 2005 11:19:30 +1000

+1

There was opposition to atom:modified because we couldn't see a need for it.
Now we have a need for it.

e.

---

From: Henry Story <[email-masked]>
Date: Mon, 16 May 2005 11:54:50 +0200

Should one not then to make atom:update a MAY?
We don't really need 2 MUST dates.

Henry

---

From: Graham <[email-masked]>
Date: Mon, 16 May 2005 11:31:04 +0100

I'd like to propose an alternative to atom:modified? I've mentioned  
this a couple of times before over the last two years, but anyway:

The only properties of atom:modified we're interested in are:
1) That different versions have different dates
2) Sorting chronologically puts the versions in the correct order

This is to say, the absolute value of the date is irrelevant. The  
only thing that matters is whether it is before, after or the same as  
other dates that have appeared on a particular entry.

Proposal:

atom:version

atom:version is a Date Construct that determines the chronology of  
instances of atom:entry. Each time an atom:entry is modified,  
atom:version MUST be given a chronologically later value than other  
values than have appeared in any atom:entry with the same value for  
atom:id.

The value does not need to correspond to any particular event in an  
entries life-cycle. Suitable values include date of last  
modification, or the date the atom:entry is generated by an Atom  
producer.

Is this plausible?

Graham

---

From: Sam Ruby <[email-masked]>
Date: Mon, 16 May 2005 07:07:23 -0400

Graham wrote:
> This is to say, the absolute value of the date is irrelevant. The  only 
> thing that matters is whether it is before, after or the same as  other 
> dates that have appeared on a particular entry.
> 
> Proposal:
> 
> atom:version
> 
> entries life-cycle. Suitable values include date of last  modification, 
> or the date the atom:entry is generated by an Atom  producer.
> 
> Is this plausible?

Potential concerns / observations:

  1) cvs versions are numbered with things like 1.99 and 1.103.  This
     makes such versions unsuitable for this purpose.  How do we make
     this clear?

  2) in the general case, sorting of unicode is culturally sensitive.
     This isn't a concern for RFC 3339 based dates given the constrained
     set of characters that that grammar permits.

  3) Perhaps version/modified need not be mandatory except in those
     instances where entries with duplicate ids are present in a feed?

  4) No semantics can be inferred by two different entries with two
     different ids sharing the same version.

- Sam Ruby

---

From: Graham <[email-masked]>
Date: Mon, 16 May 2005 12:56:08 +0100

>     makes such versions unsuitable for this purpose.  How do we make

"atom:version is a Date Construct"?

It might be worth giving it a clearer name, though.

>     set of characters that that grammar permits.

Timezones prevent sorting of RFC 3339 dates alphabetically anyway, so  
why is this a concern? Dates have to be parsed before the point in  
time and therefore the chronology can be established.

>     instances where entries with duplicate ids are present in a feed?

Yes.

>     different ids sharing the same version.

Can they be for atom:modified? Usefully?

Graham

---

From: Eric Scheid <[email-masked]>
Date: Mon, 16 May 2005 23:02:21 +1000

> other dates that have appeared on a particular entry.

what about comparing that entry against other entries? how do you sort a set
of entries into chrono order?

> modification, or the date the atom:entry is generated by an Atom

does this mean that a dynamic system could cause the same entry to have a
new modification date on every retrieval?

-1

e.

---

From: Graham <[email-masked]>
Date: Mon, 16 May 2005 14:28:09 +0100

> of entries into chrono order?

We have atom:updated and atom:published for that.

> new modification date on every retrieval?

i) It's not a modification date
ii) Yes. The point of atom:modified and atom:version is to  
differentiate 2 different versions. Dynamic dates do that perfectly  
well.

Graham

---

From: Eric Scheid <[email-masked]>
Date: Tue, 17 May 2005 02:16:37 +1000

On 16/5/05 11:28 PM, "Graham" <[email-masked]> wrote:

>> what about comparing that entry against other entries? how do you
>> sort a set of entries into chrono order?
> 

if you want to sort by updated or published, but not for most recently
changed (even if not 'significantly')

>> does this mean that a dynamic system could cause the same entry to
>> have a new modification date on every retrieval?
> 

What if the Archivist is not the Publisher? They will then see many many
instances, all with different 'versions' ... are they to assume they are
different versions/editions, and put each and every dynamic instance into
the archive?

e.

---

From: Graham <[email-masked]>
Date: Mon, 16 May 2005 17:39:17 +0100

> changed (even if not 'significantly')

Well yes. So? I consider atom:modified to be unfeasible anyway for  
all sorts of reasons, so we're not losing any capability here.

> different versions/editions, and put each and every dynamic  

You'd be keying off a semantic atom:version doesn't have, which is a  
daft thing to do. If you want to determine which versions to store,  
wait for the content itself to change.

Graham

---

From: Eric Scheid <[email-masked]>
Date: Tue, 17 May 2005 03:06:01 +1000

On 17/5/05 2:39 AM, "Graham" <[email-masked]> wrote:

> You'd be keying off a semantic atom:version doesn't have, which is a  daft
> thing to do. 

I'm saying that atom:version is an inferior proposition as compared to
atom:modified. Your original premise of there being only two properties
we're interested in is faulty.

> If you want to determine which versions to store,  wait for the
> content itself to change.

blech. We all know just how successful that approach is.

e.

---

From: Graham <[email-masked]>
Date: Mon, 16 May 2005 18:22:32 +0100

> we're interested in is faulty.

With respect to allowing duplicate ids, it isn't.

Graham

---

From: David Powell <[email-masked]>
Date: Wed, 18 May 2005 01:03:38 +0100

> all sorts of reasons, so we're not losing any capability here.

Can you explain some? I don't see atom:version would be more feasible.
atom:version doesn't support some cases that atom:modified does, and
it doesn't really seem easier to explain in the spec or implement.

Are you thinking of some scenarios where a publisher wouldn't be able
to produce a modified date, but would be able to produce a version? -
or do you mean it wouldn't be feasible for other non-technical
reasons?

-- 
Dave

---

From: David Powell <[email-masked]>
Date: Wed, 18 May 2005 01:12:26 +0100

>      instances where entries with duplicate ids are present in a feed?

If we want to support the cases of aggregating multiple feeds and of
full archiving (other than by the publisher), then it needs to be
mandatory doesn't it?  The subscriber/archiver may be a different
entity than the publisher; if the publisher doesn't provide
modified/version, then what is the subscriber supposed to do?

-- 
Dave

---

From: Graham <[email-masked]>
Date: Wed, 18 May 2005 01:32:21 +0100

> it doesn't really seem easier to explain in the spec or implement.

The first problem is that not all systems track a modified date. If  
you're obtaining entries using an API on a closed system, and the  
system doesn't supply a modified date for whatever data you're  
syndicating, you're screwed.

The second problem is that "modification" is an incredibly hard thing  
to pin down, eg:
- I modify my atom-generating script to change which optional  
elements are included. Does the modification date change?
- I modify my atom-generating script to change the order elements  
appear in. Does the modification date change?
- I change the location of my entries, and therefore the atom:link  
element values. Does the modification date change?
- I change the location of my entries, and therefore the xml:base  
attribute on a parent element. Does the modification date change?
- I change the email address of an entry's author, but not the entry  
itself. Does the modification date change?
etc etc

Don't bother answering yes or no to any of these here. The point is  
that even if you do pin down exactly which count as modifications,  
you have to demonstrate it can easily be implemented and tracked  
exactly that way on the average CMS (Note adding new columns to the  
database may not be possible).

Graham

---

From: Eric Scheid <[email-masked]>
Date: Wed, 18 May 2005 15:01:48 +1000

> syndicating, you're screwed.

Not so very long ago you suggested that aggregators look at the content to
determine if it's changed. If it's good enough for aggregators, it's good
enough for publishers (actually better than good enough since the publisher
would be able to do the test before other transformations occur, thus
eliminating some false positives).

> itself. Does the modification date change?

Some of these are format level changes, which would also include
- the publisher changes the text encoding
- the publisher starts using xml:base and relative references, but without
changing the fully resolved location of hrefs

Some are content changes, or metadata changes.

One suggests a problem with the object model.

> Don't bother answering yes or no to any of these here.

ok, I won't.

> The point is  that even if you do pin down exactly which count as
> modifications,  you have to demonstrate it can easily be implemented and
> tracked  exactly that way on the average CMS (Note adding new columns to the

which is more likely to be supported by the average CMS?

    an automated date stamp of last modification
    a user selectable date stamp of last "significant" update

According to http://www.intertwingly.net/wiki/pie/BlogToolDateSurvey it
appears that the former is already well supported.

e.

---

From: Graham <[email-masked]>
Date: Wed, 18 May 2005 10:41:42 +0100

> eliminating some false positives).

But if the publisher does that, atom:modified is going to end up  
being the date the atom-generating program is run rather than the  
date the modification happened. You may argue that functionally this  
doesn't matter, and you'd be right. You'd also be admitting that the  
absolute date is not important, as long as the dates are in the right  
order - aka atom:version.

> Some are content changes, or metadata changes.

I see no discussion of this distinction in the atom:modified proposal.

>     a user selectable date stamp of last "significant" update

atom:updated does not have to be user selectable. It's perfectly  
valid to leave it as the first publishing date, or to use a last  
modification date.

(Not that either of these are useful - I think atom:updated should be  
optional, but Tim doesn't listen)

Graham

---

From: "Roger B." <[email-masked]>
Date: Wed, 18 May 2005 07:16:15 -0500

> Now we have a need for it.

Eric: That most definitely wasn't the core reason for opposition, to
my recollection. It was opposed because it can't be reliably produced
by existing systems, among other things. (See the wiki for a survey of
tools and the dates they support.)

A big -1 for any pace that depends upon a mandatory atom:modified.

--
Roger Benningfield

---

From: Eric Scheid <[email-masked]>
Date: Thu, 19 May 2005 01:11:21 +1000

> I see no discussion of this distinction in the atom:modified proposal.

I meant "content or metadata changes". I don't care so much about the
distinction between those two senses. I was however making a distinction
between those and representation/format changes.

The atom:modified proposal refers to changes in "the entry", not changes in
"atom:entry". Subtle, I know.

> (Not that either of these are useful

and one is actively harmful, in the sense of being polluting.

>I think atom:updated should be optional, but Tim doesn't listen)

the problem is that if atom:updated is optional we then have to spec the
fall-back precedence for when it is missing, and also spec what it means if
an entry had atom:updated, but is then published without atom:updated. can
of worms.

e.

---

From: Eric Scheid <[email-masked]>
Date: Thu, 19 May 2005 01:12:22 +1000

On 18/5/05 10:16 PM, "Roger B." <[email-masked]> wrote:

> by existing systems, among other things.

The exact same argument can be made against atom:published and atom:updated,
and pretty much any date for that matter.

Sutely you are not suggesting that atom:published and atom:updated should be
stricken from the spec?

>(See the wiki for a survey of tools and the dates they support.)

hmmm ... Blogger, Moveable Type, JournURL, bloxsom, ExpressionEngine,
ongoing, Roller, Macsanomat, WordPress, and BigBlogTool all provide dates
which represent the last date/time the entry was modified, and there is no
info for LiveJournal. Sam has mentioned they have the same three dates as
blogger, but if not it is open-source code and can be patch.

Real world practice thus suggests that atom:modified is not such a great
burden.

Could there be systems incapable of storing/producing atom:modified, or
incapable of being patched to do so? Yes. But the same can be said for
atom:updated, atom:published, atom:id, atom:link, atom:content, etc etc etc.

e.

---

From: Walter Underwood <[email-masked]>
Date: Wed, 18 May 2005 09:51:39 -0700

--On Thursday, May 19, 2005 01:12:22 AM +1000 Eric Scheid <[email-masked]> wrote:
> info for LiveJournal.

We abaondoned full LiveJournal compatability a long time ago by requiring
time zones. Older LJ posts do not have time zones. Don't know about the
current ones.

wunder
--
Walter Underwood
Principal Architect
Verity Ultraseek

---

From: David Powell <[email-masked]>
Date: Wed, 18 May 2005 21:32:13 +0100

> my recollection.

I just checked the PaceDateModified thread, and Eric's recollection
seems correct. I couldn't see a single objection on the basis of
software support.

> It was opposed because it can't be reliably produced

Am I looking at the same page? BlogToolDateSurvey shows atom:modified
as having widespread support. Support for atom:updated isn't shown,
but I would bet on it being a lot scarcer than atom:modified.

-- 
Dave

---

From: "Roger B." <[email-masked]>
Date: Wed, 18 May 2005 17:00:38 -0500

Eric: My bad... I was looking at Ye Olde PaceDateModified, not
PaceDateModified2. Template-based systems can't reliably produce a
PaceDateModified, but PDM2 is a different story.

Consider the -1 moved to +0.

--
Roger Benningfield

---

From: Eric Scheid <[email-masked]>
Date: Thu, 19 May 2005 18:49:57 +1000

On 18/5/05 7:41 PM, "Graham" <[email-masked]> wrote:

> But if the publisher does that, atom:modified is going to end up  being the
> date the atom-generating program is run rather than the  date the modification
> happened. You may argue that functionally this  doesn't matter, and you'd be
> right. You'd also be admitting that the  absolute date is not important, as
> long as the dates are in the right  order - aka atom:version.

not quite. I agree "the absolute date of modification" is open to
philosophical argument (eg. an author retrieves an entry into their editor,
changes some bit, goes off for 5 minutes to answer the phone, comes back and
presses the submit button ... what is the true date of modification now?)

The absolute or true date of modification isn't the quality I'm interested
in though. What I am interested in is having one such timestamp for any
given modified content. Not multiple timestamps for the exact same content,
which is what atom:version sloppily allows.

e.