Subject: PaceDuplicateIDWithSource

From: Graham <[email-masked]>
Date: Mon, 2 May 2005 19:41:47 +0100

http://www.intertwingly.net/wiki/pie/PaceDuplicateIDWithSource

Graham

===
Abstract

Allow multiple version of an entry within a feed where the source id  
is different.

Status

Proposed

Rationale

The current spec does not allow multiple versions of the same entry  
within a feed. In normal usage, this is OK, but when merging two  
feeds into a single feed, it requires duplicate detection and the  
discarding of data.

Note it does not make legal having an entry without a source and a  
entry with a source, both of which have the same entry id, since the  
feed without a source may come from the same feed as the one with.

Proposal

In relation to atompub-format-08:

In section 4.1.1, replace:

atom:feed elements MUST NOT contain atom:entry elements with  
identical atom:id values
With:

atom:feed elements MUST NOT contain an atom:entry element that has  
the same atom:id value as any other atom:entry element, unless both  
atom:entry elements each contain an atom:source element, and the two  
atom:id values in each atom:source element are different.
Impacts

Allows duplicate ids within feeds, so atom:source must also be  
examined when looking for duplicates.

---

From: Antone Roundy <[email-masked]>
Date: Mon, 2 May 2005 13:40:06 -0600

On Monday, May 2, 2005, at 12:41  PM, Graham wrote:
> within a feed. In normal usage, this is OK, but when merging two feeds 
> into a single feed, it requires duplicate detection and the discarding 
> of data.

The abstract and rationale made me think this was proposing something 
entirely different from what it is--that multiple representations of 
the same entry resource could appear in a feed document as long as they 
were aggregated into the feed by different routes.  What the actual 
proposed text allows is not multiple versions of the same entry, but 
multiple entries with the same id as long as they're from different 
feeds.  When merging two feeds, duplicate detection will still have to 
be done, because both feeds could contain the same entry, with the same 
atom:source/atom:id.

> feed without a source may come from the same feed as the one with.

I don't see any reason to disallow this--the one without an atom:source 
would look to the current feed for its id.  I don't see how this makes 
it any more likely that they'd come from the same feed.  We could 
require that the feed have an atom:id in such a case if we want to, so 
that the IDs can be compared (ie. existing atom:id != missing atom:id 
is not good enough).

> atom:feed elements MUST NOT contain atom:entry elements with identical 
> atom:id values
> With:
>
> atom:feed elements MUST NOT contain an atom:entry element that has the 
> same atom:id value as any other atom:entry element, unless both

---

From: "Bob Wyman" <[email-masked]>
Date: Mon, 2 May 2005 17:37:01 -0400

Antone Roundy wrote:
> multiple representations of the same entry resource could appear 
> in a feed document as long as they were aggregated into the feed 
> by different routes.
Two rules:
	1. You should only write a source element into an entry if it does
not already have one. 
	2. If you copy an entry that doesn't have a source element, then you
SHOULD insert one that indicates source from which you copied the entry.
(Note: I wish that SHOULD was a MUST...)
	
	The effect of these two rules is that the path from the original
source to the final destination is irrelevant. An entry may have been copied
100's of times between many feeds; however, it should always carry
identification of its *original* source.
	Remember, we explicitly decided not to deal with the issue of paths
when these discussions and those on the more general issue of "provenance"
were going on.

		bob wyman

---

From: "Bob Wyman" <[email-masked]>
Date: Mon, 2 May 2005 17:47:42 -0400

> atom:id values in each atom:source element are different.

	-.5

Graham's text would make the *real* entry id the concatenation of the
atom:id of the feed and the atom:id of the entry. This means that we need to
make sure that atom:id is a required element of feed. The problem here is
that it is very easy to "fake" the atom:id of a feed. It is harder to fake
the URI from which a feed is actually read.

I would much prefer if the *real* entry id was built from the concatenation
of the URI of the source feed, as identified in a link element (with
rel="self") and the atom:id of the entry. 

		bob wyman

---

From: Antone Roundy <[email-masked]>
Date: Mon, 2 May 2005 15:58:55 -0600

On Monday, May 2, 2005, at 03:37  PM, Bob Wyman wrote:
> source to the final destination is irrelevant.

Yes, I agree...including, absent some convincing argument to the 
contrary, that it were a MUST.  What I was saying is that the abstract 
and rationale of the Pace led me to think that it was trying to enable 
what I wrote above.

---

From: Tim Bray <[email-masked]>
Date: Mon, 02 May 2005 16:12:42 -0700

>> in a feed document as long as they were aggregated into the feed

Bob, we need spec language.  Right now, we have Paces from Graham and 
Roundy.  Can you please make sure which of these meets your application 
needs, and if neither do, either fix one of them or construct a new 
one?  You're our canary in the coal mine on this issue. -Tim

---

From: Graham <[email-masked]>
Date: Tue, 3 May 2005 00:33:54 +0100

These two statements conflict:

> It is harder to fake the URI from which a feed is actually read.

> as identified in a link element (with rel="self") and the atom:id  
> of the entry.

rel=self doesn't contain where the feed came from, it contains where  
the feed claims it came from, and since it has no constraints on  
uniqueness or persistence, but is just as easy to spoof as id, it's  
an even worse choice.

A better thing to do would be to add a missing feature to the source  
element: Where the feed actually came from. I'd suggest it being a  
uri/href/src attribute on the source element itself, since it  
describes the sourcing of the feed. Adding an element within <source>  
would imply the new element was part of the original feed.

Then we need a new Pace that uses the original URI.

(I'm also starting to think that feed ids are a bit of a waste of  
space, but anyway...)

Graham

---

From: Antone Roundy <[email-masked]>
Date: Mon, 2 May 2005 17:49:29 -0600

On Monday, May 2, 2005, at 05:33  PM, Graham wrote:
>> as identified in a link element (with rel="self") and the atom:id of 
>> the entry.
>
> uniqueness or persistence, but is just as easy to spoof as id, it's an 
> even worse choice.

Huh?  I thought it contained a URI from which the feed could be 
accessed, and that if one were to get the feed from a different URI, 
the preferred action would be to fetch it from the "self" URI in the 
future.  If an aggregator is subscribed to the "self" URI, then it 
knows what entries have been there, and can't be deceived by a feed 
claiming to have an entry from there.  You're correct that there's no 
persistence, but I'd think most feeds are going to have fairly stable 
addresses.  As for uniqueness, how are you going to get multiple feeds 
from the same URI?  Different feeds may CLAIM the same "self" URI, but 
only one can actually come from there, so spoofs can be easily detected.

> (I'm also starting to think that feed ids are a bit of a waste of 

I agree completely, except for cases recently named where a feed 
doesn't have a URI (delivered by email, etc).

---

From: Graham <[email-masked]>
Date: Tue, 3 May 2005 01:51:40 +0100

On 3 May 2005, at 12:49 am, Antone Roundy wrote:

> accessed, and that if one were to get the feed from a different  
> URI, the preferred action would be to fetch it from the "self" URI  
> in the future.

No. The spec says simply this:
"The value "self" signifies that the IRI in the value of the href  
attribute identifies a resource equivalent to the containing element"

If an aggregator is reading the feed from the "wrong" URI, the web  
server can easily be set up to provide HTTP redirects on that address.

> If an aggregator is subscribed to the "self" URI, then it knows  
> what entries have been there, and can't be deceived by a feed  
> claiming to have an entry from there.  You're correct that there's  
> no persistence, but I'd think most feeds are going to have fairly  
> stable addresses.  As for uniqueness, how are you going to get  
> multiple feeds from the same URI?  Different feeds may CLAIM the  
> same "self" URI, but only one can actually come from there, so  
> spoofs can be easily detected.

It's the claiming that's the problem. If the real URI must always  
match the rel="self", then every Atom feed must be discarded when the  
two don't match, and the request attempted again. That seems like  
hazardous behaviour and mixing of layers to me. It also means a feed  
document is only valid when accessed from its original location.

Graham

---

From: "Bob Wyman" <[email-masked]>
Date: Mon, 2 May 2005 17:37:01 -0400

Two rules:

		bob wyman