Subject: atom:id again... Please accept my apologies for doing this now...

From: "Bob Wyman" <[email-masked]>
Date: Sat, 2 Oct 2004 20:40:43 -0400

My apologies for bringing up atom:id again... I've got a problem that I'm
trying to solve. Basically, what I want to do is publish an instance of a
feed that has potentially many entries all with the same atom:id. However, I
think it was decided that one shouldn't do this. (I've been digging through
the list but can't find what I'm looking for.) 

Let me describe what I'm trying to do and please let me know what is the
"right" thing.

At PubSub.com, we publish "airport status alerts" via Atom files and via
"Atom over XMPP PubSub". For instance, at the moment some sort of "Aircraft
Emergency" is slowing down traffic at JFK by between 16 and 30 minutes and
increasing... If you were planning to pick someone up at JFK tonight, it
would be handy if you knew that there is a general delay going on.

My assumption is that for most people, the only interesting information is
going to be the current status. However, there are a number of applications
that would require the ability to track changes in status over time. For
those who only care about current status, we only need to have one entry for
any airport in our feed -- the most recent one. For those who care about
changes over time, we should provide access to all entries for some given
time period (even entries that are now "old" and obsolete) so that they can
be sure to pick up everything they need without gaps.

What I would like to do is create a single tag: URI for each airport. I
would then use that URI as the atom:id for any and every status record for
that airport. Thus, "tag:pubsub.com,2004:JFK" might be the atom:id we would
use for *all* JFK reports from now until the universe ends... Doing this
would make it really easy for applications to handle "replacing" old
entries. They would just make sure that they only had the most recent
instance of any particular atom:id. On the other hand, for the folk that
want history, I'd like to publish the last 10 alerts or even the last 24
hours of alerts to make it easier for them. (I assume that "good"
aggregators would actually use RFC3229+feed and thus wouldn't actually get
the history unless they needed it.) But, if I publish multiple entries in a
single atom file and if each entry has the same atom:id, I think I'm
breaking the rules. Is this the case? (Note: Please, let's not get into the
"versions and variants" debate...) Or, is publishing multiple entries in a
single instance of a feed simply the logical equivalent of publishing
multiple entries in different instances of the feed over time?

What is the correct way to handle this? Below, you'll see what we currently
publish (either in a file or over XMPP) as an airport delay entry> I've
inserted an atom:id for illustrative purposes. Imagine reading an atom file
that contain 10 entries like the one below. Each entry would have the same
atom:id but each entry would also have a different modified time (note: this
is Atom 0.3 -- thus, updated isn't used.)

		bob wyman

<entry>
 <modified>2004-10-02:T19:18:43-0400</modified>
 <issued>2004-10-02:T19:18:43-0400</issued>
 <id>tag:pubsub.com,2004:JFK</id>
 <content type="text/xml">
  <Message>
   <topic>Airport Status</topic>
   <publisher>PubSub Airport Status Publisher</publisher>
   <messageID>102</messageID>
   <revisionID>0</revisionID>
   <publication-date>2004-10-02:T19:18:43-0400</publication-date>
   <contentType>AirportStatusUpdate</contentType>
    <content>
     <AirportStatusUpdate>
      <airport>
       <airportCode>JFK</airportCode>
       <airportCountry>USA</airportCountry>
       <airportName>John F. Kennedy International Airport</airportName>
       <airportLocation>New York, NY</airportLocation>
      </airport>
      <averageDelay>23</averageDelay>
      <reason>A/C EMERGENCY</reason>
      <maxDelay>30</maxDelay>
      <statusTypeCode>3</statusTypeCode>
      <delayTrend>Increasing</delayTrend>
      <statusType>Delay</statusType>
      <minDelay>16</minDelay>
     </AirportStatusUpdate>
   </content>
  </Message>
 </content>
</entry>

---

From: Graham Parks <[email-masked]>
Date: Sun, 03 Oct 2004 10:15:10 -0700

On Sunday, October 03, 2004, at 10:04AM, Tim Bray <[email-masked]> wrote:

>OK.  Who's writing a Pace to clean up the language?

I did, weeks ago:
http://www.intertwingly.net/wiki/pie/PaceIdConstruct2

Graham

---

From: Tim Bray <[email-masked]>
Date: Sun, 03 Oct 2004 09:56:08 -0700

On Oct 3, 2004, at 9:26 AM, Paul Hoffman / IMC wrote:

>> Currently I can point to "universally unique" and claim it bans 
>> serving a feed twice with the same ids.
>
> +0.5
>
>>  Equally I can claim it says a feed must use the same 15 ids for its 
>> entries regardless of which 15 entries it currently contains. It is 
>> one of the most hand-wavey bits of the spec, so claiming it means 
>> anything specific is a bit off.
>
> +1
>
>> (Also " that is universally unique; i.e., it MUST NOT change over 
>> time" is a complete non sequitur.)
>
> +1. Good point. "universally" does not necessarily mean "over time".

OK.  Who's writing a Pace to clean up the language? -Tim

---

From: Robert Sayre <[email-masked]>
Date: Sun, 03 Oct 2004 12:38:10 -0400

Graham Parks wrote:
>  On Sunday, October 03, 2004, at 08:45AM, Tim Bray <[email-masked]> wrote:
> 
> 
>>"Universally unique" means that any two atom:entries that have the same 
>>atom:id are the same atom:entry.
> 
> 
> Does it? Where does it say that?
> 
> 
>>I don't think there's anything in any 
>>of our current draft language that I could point to and say "it's 
>>illegal because of *this*".
> 
> 
> Currently I can point to "universally unique" and claim it bans serving a feed twice with the same ids. Equally I can claim it says a feed must use the same 15 ids for its entries regardless of which 15 entries it currently contains. It is one of the most hand-wavey bits of the spec, so claiming it means anything specific is a bit off.
> 
> (Also " that is universally unique; i.e., it MUST NOT change over time" is a complete non sequitur.)

The spec is trying to say that atom:id is a reference. You can change 
what the reference is bound to, but don't use a different reference for 
the same resource. If I write an entry about losing my cat,

http://example.org/i_lost_my_cat

that URI should never change, but the result of dereferencing it may. I 
understand Bob's dilemma, but I'm not sure he's hit on the right 
formula. The fact that he wants to include more than one instance 
indicates that each update should have its own URI.

     "My assumption is that for most people, the only interesting
     information is going to be the current status. However, there are a
     number of applications that would require the ability to track
     changes in status over time. For those who only care about current
     status, we only need to have one entry for any airport in our feed--
     the most recent one. For those who care about changes over time, we
     should provide access to all entries for some given time period
     (even entries that are now "old" and obsolete) so that they can
     be sure to pick up everything they need without gaps."

The way to provide access to changes in status over time is to give each 
one of them a URI, and relate them. Perhaps <dc:replaces> or a category 
could be used to indicate the relationship between the Airport status 
entries. Or, perhaps the feed should only include one entry per airport 
at any given time. The app can start tracking changes from the current 
point in time, rather than ten entries ago.

In any case, this is not an approach we should endorse in any way. We 
should specifically call it out as undefined or ban it.

Robert Sayre

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Sun, 3 Oct 2004 09:26:17 -0700

+0.5

+1

>  ><atom:updated> is now compulsory.  Hmm... do we want to require that no
>>feed contain two entries with the same <atom:id> and <atom:updated>? 
>>I'm inclined not to worry, this would be stupid but tend not to break
>>software, if I were writing the software I'd be inclined to do
>>something like
>
>Unless there are specific processing rules for finding the most 
>recent entry, it should be banned.

Why? What is the interoperability issue here?

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: Graham Parks <[email-masked]>
Date: Sun, 03 Oct 2004 09:06:20 -0700

>software, if I were writing the software I'd be inclined to do 

Unless there are specific processing rules for finding the most recent entry, it should be banned.

Graham

---

From: Tim Bray <[email-masked]>
Date: Sun, 03 Oct 2004 08:45:36 -0700

On Oct 3, 2004, at 8:20 AM, Graham Parks wrote:

>> It's not obvious which rule you're breaking.
>
> What interpretation of "universally unique" are you using that's 
> narrow enough to not include other entries in the same XML doc?

"Universally unique" means that any two atom:entries that have the same 
atom:id are the same atom:entry.  I don't think there's anything in any 
of our current draft language that I could point to and say "it's 
illegal because of *this*".  If people want to make it illegal it would 
require some extra work, and I'm not sure it's worth doing.

>> Given that there will be
>> multiple different versions of an entry, it seems that receiving
>> software can do one of two things: forget all but the most recent, or
>> store successive versions.
>
> And which is the most recent Tim?

WHEN I find an entry
   IF I have its atom:id on file
       IF its atom:date is later than the atom:date on file
           replace the current one
           flag update for user
       ELSE
           ignore it

-Tim

---

From: Graham Parks <[email-masked]>
Date: Sun, 03 Oct 2004 08:20:14 -0700

>It's not obvious which rule you're breaking.

What interpretation of "universally unique" are you using that's narrow enough to not include other entries in the same XML doc?

I think the only answer here is that each update should have its own id and each aiprort its own feed. That maps it to how blogs work, and we are creating a blogging protocol.

Graham

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Sun, 3 Oct 2004 07:30:01 -0700

I don't see any obvious protocol problem, but I hope you don't expect 
any particular action on the part of the receiver. This kind of 
over-loading is common in protocols; the problem is a few years down 
the line when a new implementer starts up and sees streams like this 
in the wild, they want to know what they are "supposed" to do with 
them. As long as the answer is "it's undefined", no one is the worse 
for it.

You might consider also creating an extension that will give the 
history-keeping readers some context, such as the order from the 
publisher's perspective.

--Paul Hoffman, Director

---

From: "Roger B." <[email-masked]>
Date: Sun, 3 Oct 2004 01:33:22 -0500

Bob: I have a feeling that, unless an aggregator displays diffs within
its interface, history-seeking users aren't going to get what they
want from the feed. Many will simply get one of the following:

(a) The most recent (by date) entry overwriting its predecessor.
(b) The last (by element order) entry in the feed overwriting its predecessors.
(c) All additional entries after the original being ignored as duplicates.

Of course, if the total audience for a "history feed" is small enough,
and is willing to use a specific tool to get specific results, it
might be a pretty good idea.

That aside, does it "break the rules"? Not in my opinion, at least not
at first glance. Variant results aren't necessarily a sign of
breakage.

--
Roger Benningfield
http://admin.support.journurl.com/

---

From: Robert Sayre <[email-masked]>
Date: Sun, 03 Oct 2004 19:21:21 -0400

> http://www.intertwingly.net/wiki/pie/PaceIdConstruct2

How about this:

    3.5 Identification Constructs

    An Identity construct is an element whose content conveys a
    permanent, universally unique identifier for the construct's
    parent. Its content MUST be an absolute URI [RFC2396].

    When an Atom document is relocated, migrated, syndicated,
    republished, exported or imported, the content its Identity
    construct MUST NOT change. Put another way, an Identity
    construct pertains to all instantiations of a particular Atom
    document; revisions to the Atom document retain the same
    content in their Identity constructs.

Compare and contrast to RFC822:

    4.6.1.  MESSAGE-ID / RESENT-MESSAGE-ID

         This field contains a unique identifier  (the  local-part
    address  unit)  which  refers to THIS version of THIS message.
    The uniqueness of the message identifier is guaranteed by  the
    host  which  generates  it.  This identifier is intended to be
    machine readable and not necessarily meaningful to humans.   A
    message  identifier pertains to exactly one instantiation of a
    particular message; subsequent revisions to the message should
    each receive new message identifiers.

Robert Sayre

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Sun, 3 Oct 2004 16:28:09 -0700

At 7:21 PM -0400 10/3/04, Robert Sayre wrote:
>    construct MUST NOT change.

Wording glitch there....

--Paul Hoffman, Director

---

From: Robert Sayre <[email-masked]>
Date: Sun, 03 Oct 2004 19:36:27 -0400

How about this:

    When an Atom document is relocated, migrated, syndicated,
    republished, exported or imported, the content of its
    Identity construct MUST NOT change. Put another way, an
    Identity construct pertains to all instantiations of a
    particular Atom document; revisions to the Atom document
    retain the same content in their Identity constructs.

Robert Sayre

---

From: "Bob Wyman" <[email-masked]>
Date: Sun, 3 Oct 2004 02:47:05 -0400

> (b) The last (by element order) entry in the feed overwriting its
	We would probably ensure that these were the same by ensuring the
more recent entries were "first" in the feed. However, as a publisher, I
think it is reasonable to expect that Atom would define default replacement
rules and those rules should be based on the combination of atom:id and date
updated.
	My expectation is that the "history feed" applications would
explicitly not implement the default replacement rules and would instead
probably just store all instances of entries with the same atom:id. Thus, my
expectation is that any statements concerning default replacement rules
would be optional. The use of "history" will probably be most important in
specialized feeds that carry things like stock-market data, weather,
earthquake info, airport status, etc. However, I think we'll find that there
will be a number of uses with normal blogging stuff. For instance, we'll
have people who will want to implement versioning systems (i.e. CVS or
subversion type stuff) and they will want to be able to reconstruct each
incremental "state" of an entry.

		bob wyman

---

From: Tim Bray <[email-masked]>
Date: Sun, 03 Oct 2004 00:13:21 -0700

On Oct 2, 2004, at 5:40 PM, Bob Wyman wrote:

> breaking the rules. Is this the case?

It's not obvious which rule you're breaking.  Given that there will be 
multiple different versions of an entry, it seems that receiving 
software can do one of two things: forget all but the most recent, or 
store successive versions.  It doesn't seem to me that whether or not 
they're in one feed is really that material.  -Tim

---

From: Bill de hÓra <[email-masked]>
Date: Tue, 05 Oct 2004 17:49:08 +0100

> use for *all* JFK reports from now until the universe ends... 

http://www.daml.org/2001/10/html/airport-ont#
http://www.w3.org/2002/08/flight/crunchflights.rb
http://jibbering.com/2002/8/sample-air.rdf

cheers
Bill