Subject: Handling updated or deleted entries

From: Chris <[email-masked]>
Date: Wed, 08 Aug 2007 17:58:18 -0500

Is there a standard way to indicate that an entry has been updated or 
deleted?

Note that I'm not asking how to update or delete an entry; that's what 
APP does. I'm just looking for a flag in the read-only feed that 
indicates that the author made a change to an entry that was previously 
published.

(Apologies if this message shows up twice. Couldn't quite get the 
email/gmane connection to work correctly).

---

From: James M Snell <[email-masked]>
Date: Wed, 08 Aug 2007 19:13:56 -0700

There's no standard.  I have toyed with this in the past and had come up
with a x:deleted-entry stand in for a deleted entry.

e.g.,

  <feed>
    <x:deleted-entry>
      <atom:id>{id of deleted entry}</atom:id>
    </x:deleted-entry>
    <entry>...</entry>
    <entry>...</entry>
    <entry>...</entry>
  </feed>

The challenge is knowing how long to keep the x:deleted-entry things
around in the feed.

- James

---

From: Tim Bray <[email-masked]>
Date: Wed, 08 Aug 2007 19:45:35 -0700

> with a x:deleted-entry stand in for a deleted entry.

I was chatting with Steven Lees of Microsoft about this during the  
interop.  One of the thing that makes me nervous about SSE is the  
sse:deleted element which says "This element isn't here any more"...  
this is going to be mega-confusing to a non-SEE-savvy client.

  -Tim

---

From: John Panzer <[email-masked]>
Date: Wed, 08 Aug 2007 22:52:17 -0700

> The challenge is knowing how long to keep the x:deleted-entry things

Perhaps it's sufficient in many cases for the feed to indicate to 
consumers how long it keeps deleted-entry items around.  Then a consumer 
who checks back, finds that they may have missed a tombstone, has the 
option of doing a complete sync by paging back through the feed history.

---

From: James M Snell <[email-masked]>
Date: Wed, 08 Aug 2007 21:59:46 -0700

John Panzer wrote:
>>[snip]

One idea I had was that a feed would only contain deleted-entry elements
for entries that fall within the same atom:updated date range as the
atom:entry's in the feed.  That is, if the deleted entry *would* have
appeared in a particular feed page based on it's app:edited or
atom:updated value, then there would be a deleted-entry element.  The
downside of this would be that deleted-entry elements would be spread
out across multiple feed pages.  I'm not sure if that's a really bad
thing or not.

- James

---

From: Steven Lees <[email-masked]>
Date: Thu, 9 Aug 2007 00:04:59 -0700

>out across multiple feed pages.  I'm not sure if that's a really bad

That actually seems like a good thing, because it means that the deleted items will be in the "right" place in the feed. Anyone who wants the complete update will need to read all of those pages eventually. For SSE (spec is on http://msdn.microsoft.com/xml/rss/sse), we've been thinking that a sync client would use ETag headers on a GET to the feed as a way to ask the server for all of the changes since the last sync, but I'm not sure how well that would mix with paging.

Regarding John's comment about about missing the tombstones--we have a mechanism for this in SSE. You can have an optional feed-level sx:sharing element with an "expires" attribute that gives the client guidance about how often to sync.

Steven

---

From: Steven Lees <[email-masked]>
Date: Thu, 9 Aug 2007 00:15:41 -0700

>this is going to be mega-confusing to a non-SEE-savvy client.

That concern makes sense to me, given the way that we're used to using feeds and feed readers today. It would be strange if a blog post was gone when viewed from the web page of the blog, but the post was still present in the feed.

I think the deleted entry issue will come up more often as people start to use Atom and AtomPub for exchanging and syncing data. There may never be a web page at all in that case, and the feed becomes a way (or the only way?) for the server to hand state changes out to clients. In that case, you must have a way to represent deleted items, otherwise the clients will never know that they're gone. This could be useful even in the typical feed reader case today. It doesn't happen very often, but if someone actually deleted a post, I'd be happy to have my reader keep the cached copy around, but still give me some indication that the post was deleted by the publisher.

As we discussed on IRC during the interop, there are some ways to make the two schemes coexist. You could have different URIs depending on whether the client understood deleted items; or you could have the client provide some HTTP header indicating whether or not it wanted deleted items to appear in the feed.

Steven

---

From: James M Snell <[email-masked]>
Date: Thu, 09 Aug 2007 00:25:34 -0700

Another approach that we use in the Lotus Connections Activities
component is to have a Trash collection.  When an item is deleted from
an Activity (Collection) it is moved to the Trash.  It's url changes, it
is removed completely from the original feed, and it shows up in the
Trash feed.  The item can be undeleted and moved back to the original or
it can be purged from the trash.  There are no tombstones at all. A user
can determine whether or not a previously existing entry has been
deleted either by doing a HEAD on it (and getting a redirect to the
trash url) or by checking for it in the trash collection.

- James

---

From: Antone Roundy <[email-masked]>
Date: Thu, 09 Aug 2007 08:01:00 -0500

James M Snell wrote:
> an Activity (Collection) it is moved to the Trash.

I was imagining something like this:

<feed ...>
    <link rel="deleted-entries" href="..." 
ext:updated="2007-08-09T12:45:00+0000" />
    ...

That would point to a document listing deleted entries.  It could of 
course be a set of linked documents with tombstones for the most 
recently deleted entries appearing first.  ext:updated is the timestamp 
marking when the most recent deletion occurred.

To avoid creating a new document type, perhaps the document could be an 
Atom feed document containing only tombstones and not live entries.

Antone Roundy

---

From: "James Holderness" <[email-masked]>
Date: Thu, 9 Aug 2007 14:40:11 +0100

> with a x:deleted-entry stand in for a deleted entry.

If you could get some server implementers interested in supporting this I'd 
love to add it to Snarfer (in fact I've been wanting to do this for a long 
time). It would be really useful for comment feeds and public forums that 
get a lot of spam. I tend to set my refresh interval very low on forums, so 
I'm almost guaranteed to receive any spam that is posted, no matter how 
diligent the moderators are about deleting it. Tombstones could fix that.

Regards
James

---

From: Keith Fahlgren <[email-masked]>
Date: Thu, 09 Aug 2007 07:51:12 -0700

On 8/9/07 12:25 AM, James M Snell wrote:
> Trash feed. 

The Trash collection/feed is exactly the solution we've used (though we don't
have an un-delete mechanism).

HTH,
Keith

---

From: Chris <[email-masked]>
Date: Thu, 09 Aug 2007 13:54:49 -0500

> out across multiple feed pages.  I'm not sure if that's a really bad

This problem isn't confined to just deletes; we also need a way to 
signal updates. This problem is also related to Brian Smith's post "How 
to implement the "Give me all the new & updated entries" use case" on 
7/13. It's a more general issue.

The general problem is how to keeping two databases in sync via a feed. 
The normal way to do it is with something akin to a transaction log. A 
log has a timestamp, and then it has record of the change that happened, 
which is always an insert, an update, or a delete.

An Atom feed can look like a transaction log if we just add a flag to 
indicate an update or a delete. If the flag is not present, it's an 
insert. The timestamp is the <updated> value. The flag could look like this:

<atom:action>delete</atom:action>

or possibly

<atom:delete/> or <atom:updated>

I like the "action" tag because it's more obvious.

If we combine this with a mechanism to get every entry since a certain 
timestamp (per Brian Smith's request), then we have a reliable way to 
support simple transactions, and Atom can be a general mechanism for 
data distribution. I suggest adding a "since" parameter to the feed's URL:

http://mydomain/myfeed?since=2007-08-09T14:00:00

---

From: Chris <[email-masked]>
Date: Thu, 09 Aug 2007 14:01:20 -0500

or possibly