Subject: Feed History -03

From: Mark Nottingham <[email-masked]>
Date: Mon, 15 Aug 2005 13:31:52 -0700

Draft -03 of feed history is now available, at:
   http://www.ietf.org/internet-drafts/draft-nottingham-atompub-feed- 
history-03.txt

Significant changes in this revision include:
   - add fh:archive element, to indicate that an entry is an archive
   - allow subscription feed to omit fh:stateful if fh:prev is present
   - clarified that fh doesn't add ordering semantics, just allows  
you to reconstruct state
   - cleaned up text, fixed examples, general standards hygiene

There's going to be at least one more draft, as I neglected to  
acknowledge people who have made suggestions and otherwise helped so  
far. Sorry!

--
Mark Nottingham     http://www.mnot.net/

---

From: Robert Sayre <[email-masked]>
Date: Mon, 15 Aug 2005 19:28:56 -0400

>    http://www.ietf.org/internet-drafts/draft-nottingham-atompub-feed-

From <http://www.mnot.net/blog/2005/08/15/feed_history>::
"I registered this PURL namespace for the benefit of format-neutral
syndication extensions; i.e., those that will work equally well in all
versions of RSS as well as Atom."

Why is it desirable to promote mulitple syndication formats?
Practically any RSS2 extension would be ok in Atom, and any Atom
element would be legal in RSS2. If feed-history used atom:link, it
would get xml:base processing and link descriptions "for free."

I'm guessing extensions "that will work equally well in all versions
of RSS" must be valid RDF. This feels like crypto-RDF-advocacy to me,
which seems a little pointless. Five years of syndication software
seem to show that RDF isn't very useful in this space. Most
aggregators don't parse RSS1 with an RDF processor, and the most
popular one that does (Thunderbird) would get a lot fewer bug reports
if it ditched the RDF processor (which it probably will when Mozilla
switches to SQLLite for storage).

Mark, James, and anyone else are obviously free to spec any extension
they want. That said, it feels fishy to call RDF modules Atompub
drafts given that the WG rejected RDF with prejudice (I wouldn't count
myself in that camp, though I do despise RDF-promises and RDF-lessons
offered instead of RDF-benefits and RDF-running-code).

Robert Sayre

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 15 Aug 2005 16:49:14 -0700

> would get xml:base processing and link descriptions "for free."

Because it's not free; publishers have to declare the Atom namespace,  
and that's confusing; atom:link is defined in the context of Atom,  
not in the context of RSS. I'm interested in getting things working,  
not playing the syndication format advocacy game.

There's a difference between promoting multiple syndication formats  
and accommodating the reality that there are many.

> which seems a little pointless.

Er, no.

> offered instead of RDF-benefits and RDF-running-code).

This isn't an atompub draft, it's a draft to the attention of the  
atompub WG (hence draft-nottingham-atompub).

---

From: Robert Sayre <[email-masked]>
Date: Mon, 15 Aug 2005 20:09:47 -0400

On 8/15/05, Mark Nottingham <[email-masked]> wrote:
> and that's confusing; 

Less confusing than the purl namespace? Most people just think those
are magic strings they have to include at the top. You could be right,
though.

> atom:link is defined in the context of Atom,
> not in the context of RSS. 

What problems or ambiguities arise from putting Atom links in RSS? I
can't think of any.

> I'm interested in getting things working,

Not sure how feed-history will work without a unique id.

> and accommodating the reality that there are many.

Well, since any extension element (that I know of) is valid RSS2 or
Atom, I'm confused as to why "extensions that can be used in any
format" is necessary.

> > which seems a little pointless.

So, they must be valid RDF, or no? Pardon my crypto comment, that was
a little over the top.
 
> atompub WG (hence draft-nottingham-atompub).

Well, you could still bring it to the attention of the WG by emailing
us, but the drafts don't purport to solve any of the problems in the
charter. I feel it's (unintentionally) intellectually dishonest, since
most folks will assume it's an Atompub draft. But, that's just my
humble opinion.

Robert Sayre

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 15 Aug 2005 17:19:19 -0700

> Not sure how feed-history will work without a unique id.

? Sorry, you lost me there.

> Atom, I'm confused as to why "extensions that can be used in any

I wanted a neutral place/namespace for them, where the extension can  
be separated from the back-and-forth of what format to use. Put it  
this way; how would it go over here if I came along and suggested  
using an existing RSS element for some functionality in Atom? It's  
all just syntactic sugar, of course, but a clear separation avoids  
the politics and makes the status of the extension clearer to users.

> So, they must be valid RDF, or no? Pardon my crypto comment, that was

I'm not going to be incredibly strict about it; some RSS 1.0 people  
might not like non-RDF syntax getting into RDF 1.0, but I think it's  
a fact of life.

I like the RDF model; hate the XML syntax. So, you can accuse me of  
being anything *but* an RDF/XML syntax advocate :)

> most folks will assume it's an Atompub draft. But, that's just my

Sorry, it's an IETF convention; I assumed that it was well-known (but  
then again, many/most IETF conventions aren't, so I should have known  
better!)

---

From: Stefan Eissing <[email-masked]>
Date: Tue, 16 Aug 2005 10:27:20 +0200

> http://www.ietf.org/internet-drafts/draft-nottingham-atompub-feed- 

Ch 4. Typo: remove period of first sentence

Ch 4: Different use of MAY and SHOULD when compared to Ch. 3.:   
"fh:stateful ...MAY occur" vs. "fh:archive...SHOULD occur..." On third  
read i think i know how you mean it. But it is a circular definition  
nevertheless. Proposal:

"The fh:archive element MAY occur in a document's head section  and  
declares that the document is an archive document."

Ch. 5 similar:  "MUST occur unless". If the document is an archive  
there are only 2 possiblities: either fh:prev is there or not. If not  
it will always "terminate" the archive list, won't it? You seem to have  
a (server-side) model in mind which drives the document structure. From  
a client perspective, there are only the documents and it derives its  
own model from that.

Maybe its a matter of taste.

//Stefan

---

From: Henry Story <[email-masked]>
Date: Tue, 16 Aug 2005 11:04:50 +0200

> ? Sorry, you lost me there.

Yes. If you were to take a subscription feed "feed.atom" and map it  
to a graph
you might get something like this:

_feed
   |----id------> <tag:myblog,2005/feed>
   |---entry----> _
   |              |----id-----> <tag:myblog,2005/e1>
   |              |---title---> "first entry"
   |--fh:next---> <http://my.web/feed/history1.atom>

The graph of the history1.atom archive may be:

_feed2
   |----id------> <tag:myblog,2005/feed>
   |---entry----> _
                  |----id-----> <tag:myblog,2005/e2>
                  |---title---> "another entry"

The role of id is to help us merge the two graphs. IE the point of  
the history
tag is to help us think of the history.atom document as a  
continuation of
the same document, or rather that they are both descriptions of the  
feed with
id <tag:myfeed,2005/feed>. Merging the above two as simple graphs at  
most gives
this

<tag:myblog,2005/feed>
   |--1/id---> _feed
   |           |---entry----> _
   |           |              |----id-----> <tag:myblog,2005/e1>
   |           |              |---title---> "first entry"
   |           |--fh:next---> <http://my.web/feed/history1.atom>
   |--1/id--->_feed2
               |---entry----> _
               |              |----id-----> <tag:myblog,2005/e2>
               |              |---title---> "another entry"
               |--fh:next---> <http://my.web/feed/history1.atom>

But what we really want is something like this:

<tag:myblog,2005/feed>
   |---entry----> _
   |              |----id-----> <tag:myblog,2005/e1>
   |              |---title---> "first entry"
   |---entry----> _
                  |----id-----> <tag:myblog,2005/e2>
                  |---title---> "another entry"

I think we require some form of inferencing beyond what is available  
in OWL
to achieve this result. This could probably be achieved through some  
simple
N3 rules.

Henry Story

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 16 Aug 2005 08:42:50 -0700

On 16/08/2005, at 1:27 AM, Stefan Eissing wrote:

> "fh:stateful ...MAY occur" vs. "fh:archive...SHOULD occur..." On  
> third read i think i know how you mean it. But it is a circular  
> definition nevertheless.

I agree that there's a misalignment here, but would actually resolve  
it the other way. I wanted to encourage fh:archive's presence,  
because it will allow processors to avoid mistakes like subscribing  
to an archive feed. Hence, the SHOULD.

However, fh:stateful would probably be improved by saying it SHOULD  
(instead of MAY) occur in a subscription document's head section, and  
processors MUST (instead of MAY) behave as if it is present and true  
if fh:prev is there.

> there are only 2 possiblities: either fh:prev is there or not. If  
> not it will always "terminate" the archive list, won't it? You seem  
> to have a (server-side) model in mind which drives the document  
> structure. From a client perspective, there are only the documents  
> and it derives its own model from that.

Not sure what you mean here; are you saying that fh:archive is  
superfluous?

---

From: Stefan Eissing <[email-masked]>
Date: Tue, 16 Aug 2005 18:17:49 +0200

> On 16/08/2005, at 1:27 AM, Stefan Eissing wrote:
  [...]
>> it will always "terminate" the archive list, won't it? You seem to 
>> have a (server-side) model in mind which drives the document 

Currently:
The document first defines "archive documents" and *afterwards* 
requires that fh:archive MUST be present in archive documents.

My proposal:
Introduce fh:archive with the semantics that the server garantuees that 
the set of entries in this document will not change over time if 
fh:archive is present. A document with fh:archive in it (and its 
implied semantics) is then called an "archive document".

To tackle it from another view: The spec should say "servers MUST NOT 
break the promises of fh:archive" instead of saying "archive documents 
MUST announce that they do not change". There is possible harm in 
breaking the first, but only suboptimal performance in neglecting the 
latter case.

//Stefan

---

From: Henry Story <[email-masked]>
Date: Tue, 16 Aug 2005 20:35:19 +0200

I think that in section 5. you should specify that the URI reference  
MUST NOT be relative or
MUST BE absolute (if that is the proper W3C Architecture term). I  
agree with the point made by
David Powell in the thread entitled "More about extensions" [1].

Given that we have this problem I was wondering whether it would not  
be better
to use the link element as I think it permits relative references.  
Relative references
really are *extreemly useful*. I tried to work without them in my  
BlogEd editor
because the Sesame database folk mistakenly thought it was not part  
of RDF, and it caused
me no end of trouble: all those problems vanished as soon as they  
allowed relative references.

So if relative references are allowed in links perhaps the following  
would be better:

<link type="http://purl.org/syndication/history/1.0/next" href="./ 
archives/archive1.atom">

Henry Story

[1] http://www.imc.org/atom-syntax/mail-archive/msg16643.html

On 15 Aug 2005, at 22:31, Mark Nottingham wrote:
> acknowledge people who have made suggestions and otherwise helped  
> so far. Sorry!
>
> --

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 16 Aug 2005 12:00:55 -0700

I very much disagree; relative references should be allowable in  
simple extensions, and in fact the rationale that Tim gives is the  
reasoning I assumed regarding Atom extensions; if I had known that  
the division between simple and complex extensions would be used to  
justify a constraint on the use of context in simple extensions, I  
would have objected to it.

If you're using something like RDF to model feeds, you already have a  
number of context-related issues to work through, this isn't an extra  
burden.

I should explicitly allow relative URIs in fh:prev, though.

Cheers,

On 16/08/2005, at 11:35 AM, Henry Story wrote:

> I think that in section 5. you should specify that the URI  
> reference MUST NOT be relative or
> Given that we have this problem I was wondering whether it would  
> not be better
> So if relative references are allowed in links perhaps the  
> following would be better:
>
>>   http://www.ietf.org/internet-drafts/draft-nottingham-atompub- 
>> feed-history-03.txt
>>

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 16 Aug 2005 12:11:30 -0700

On 16/08/2005, at 9:17 AM, Stefan Eissing wrote:
>>> Ch. 5 similar:  "MUST occur unless". If the document is an  
>>> archive there are only 2 possiblities: either fh:prev is there or  
>>> not. If not it will always "terminate" the archive list, won't  
>>> it? You seem to have a (server-side) model in mind which drives  
>>> the document structure. From a client perspective, there are only  
>>> the documents and it derives its own model from that.
>>
> Introduce fh:archive with the semantics that the server garantuees  
> that the set of entries in this document will not change over time  
> if fh:archive is present. A document with fh:archive in it (and its  
> To tackle it from another view: The spec should say "servers MUST  
> NOT break the promises of fh:archive" instead of saying "archive  
> documents MUST announce that they do not change". There is possible  
> harm in breaking the first, but only suboptimal performance in  
> neglecting the latter case.

I made it loose purposefully; I think there are several types of  
archives out there, and it's likely that further specs are going to  
come along that talk about the guarantees surrounding persistence,  
entry deletion, etc. Again, I want to avoid, as much as possible,  
defining what a feed is in this document, as there are many potential  
models for feeds.

For example, an archive in my blog feed can change for spelling  
mistakes and updates, but an archive of telephone records used for  
SOX compliance can't. Mandating a particular definition of what an  
archive is would necessitate ruling some types of archives out, and  
that wasn't my main use case for this; rather, it was to make sure  
that archive feeds (as defined for the purposes of this spec)  
wouldn't be accidentally subscribed to.

Cheers,

---

From: Henry Story <[email-masked]>
Date: Tue, 16 Aug 2005 21:29:31 +0200

> would have objected to it.

The problem is that readers of your xml that wish to store your data  
in some form other than xml (relational database, triple store,  
object oriented database, prolog database,...), and that may
not at the time of consumption know about your extension will  save  
the content in text, which is all the spec says they should do. Since  
they don't at the time know the meaning of fh:prev and
in particular that it should contain a URI  they can't save the  
absolute URI it represents.
All they will be able to save is relative uri which will be  
meaningless if the context of the
original document is lost.

> If you're using something like RDF to model feeds, you already have  
> a number of context-related issues to work through, this isn't an  
> extra burden.

The point of making extensions is that they should be interpretable  
or at least in part useable
by parties that don't understand the extension. RDF has nothing to do  
with this. (Other than
it having some very solid theoretical backing and so being very  
helpful in thinking about these
issues, and also of it having solved quite a few years ago the  
problems that you are just discovering)

> I should explicitly allow relative URIs in fh:prev, though.

-1

This is a question of the Atom spec saying that the content of a  
simple extension element is character data. Yet you are now wishing  
to put in relative references that have complex semantics
not completely understandable without having the original context of  
the document they appear in.
I know it may seem to the writer of an extension as if the meaning of  
his extension were clear
as day, but this will not be the case to all consumers of your  
extension, and even if your
extension becomes world famous, it certainly won't be true for all  
the other extensions that people will come up with. So I think we  
should be setting a good example in these first extensions we write.

Henry Story

>> because the Sesame database folk mistakenly thought it was not  
>> part of RDF, and it caused

---

From: Robert Sayre <[email-masked]>
Date: Tue, 16 Aug 2005 15:50:27 -0400

> original document is lost.

The app should store the relative URI, and it shouldn't lose the context.

> by parties that don't understand the extension.

It still is.

> the document they appear in.

Lots of extensions will be like this. What's one itunes extenstion
without the others? :)

In summary, I agree with Mark.

Robert Sayre

---

From: David Powell <[email-masked]>
Date: Tue, 16 Aug 2005 22:05:33 +0100

> would have objected to it.

The constraint on the context is the main reason for the distinction
between Simple and Structured extensions. Why else would we define two
classes of extension if it didn't affect the processing model, and why
else would we make Simple extensions language-insensitive (a very
similar constraint on context, which is spelt out quite clearly, and
therefore presumably not disputed)?

> number of context-related issues to work through, this isn't an extra

Yeah a few, Structured extensions are obviously a big one (where the
lang and base context do need to be preserved). The reason for there
being two classes of extensions was to reduce this burden, so that
implementations based on RDBMS, RDF, or whatever can process a common
class of unknown extensions generically. The burden of requiring the
lang and base context to be preserved in a legacy CMS database along
with each extension, on the off-chance that they might be significant
seemed to great.

If you think that the context shouldn't be constrained, then maybe
you're right - maybe the constraint was unnecessary; but I think that
the current spec does impose that constraint, albeit too subtly, in
Section 6.4.1 paragraph 2, and that constraint is what I originally
intended.

If a significant number of people have not picked up on this (the lack
of rationale probably didn't help), and would have disputed it, then I
guess that we have a problem.

-- 
Dave

---

From: Robert Sayre <[email-masked]>
Date: Tue, 16 Aug 2005 17:19:33 -0400

On 8/16/05, David Powell <[email-masked]> wrote:
> The reason for there

Um, this makes no sense to me. It seems smart to preserve the entry
from which an extension came. If the extension has no base URI itself,
the one from the entry applies, so you can still find it if you think
you need it later.

Robert Sayre

---

From: Henry Story <[email-masked]>
Date: Tue, 16 Aug 2005 23:56:19 +0200

> The app should store the relative URI, and it shouldn't lose the  

relative URI? The application we are speaking about knows nothing of  
a relative
uri. All you have inside your Simple Extension Element is text! See  
the spec 6.4.1.

An application that does not know your extension cannot know that the  
text inside
your extension is to be interpreted as a relative uri. So what you  
are saying is
that any application that wants to process atom with extension  
elements has to keep
the full context of the document in which they found the extension  
element, even if
the document is a 100s of Megabytes long, and even if there is only  
one extension
element inside of it!  Or take the case of code that wants to place  
the feed inside
some other xml, which has a base element. What is it to do when it  
finds an extension
element it does not understand?

>> by parties that don't understand the extension.

No they are not! See the 2 examples above.

> Lots of extensions will be like this. What's one itunes extenstion

?

> In summary, I agree with Mark.

You agree to accept the problems that go with his position, yes.
But why bother when there clearly is an easy solution to the problem  
that works with
the current atom format and which I outlined earlier, namely using  
the link element.
I suggested writing the next tag like this:

This clearly allows relative uris (if I am not mistaken) without any  
of the problems
we have with the simple extension elements. And it seems to be  
exactly this type of
thing that the link element was designed to do.

Henry Story
http://blogs.sun.com/roller/page/bblfish/

---

From: Robert Sayre <[email-masked]>
Date: Tue, 16 Aug 2005 18:05:15 -0400

On 8/16/05, Henry Story <[email-masked]> wrote:
> element, 

No, Henry, that's not what I'm saying. I'm saying you do have to keep
the Base URI of an atom:entry or atom:feed around, for a variety of
reasons. The app should keep the extension text as it was entered.

> <link type="http://purl.org/syndication/history/1.0/next" href="./

That's what I would do, too. Not my spec, though. Mainly so I could
put a title in that said "Entries from August" or whatever.

Robert Sayre

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 16 Aug 2005 15:14:42 -0700

> put a title in that said "Entries from August" or whatever.

For that matter, if Henry's interpretation were correct, the element  
could be

   <fh:history nonsense="1">./archives/archive1.atom</fh:history>

And Atom processors would magically know that XML Base applies to the  
URI therein. It's the magic that I object to; inferring the  
applicability of context based on the presence or absence of other  
markup isn't good practice, and will lead to practical problems.  
E.g., what if I want to have an optional attribute on an empty  
element? Is it "simple" or "complex"?

This interpretation of extensions seems very fragile to me.

---

From: David Powell <[email-masked]>
Date: Tue, 16 Aug 2005 23:39:56 +0100

> element? Is it "simple" or "complex"?

FYI: The first draft of the proposal used an atom:notation="structured"
attribute on the extension to indicate the extension class which
avoided that problem, but I think it was considered to be too crufty.

-- 
Dave

---

From: "Thomas Broyer" <[email-masked]>
Date: Wed, 17 Aug 2005 10:14:56 +0200 (CEST)

Mark Nottingham wrote:
> markup isn't good practice, and will lead to practical problems.   E.g.,
> what if I want to have an optional attribute on an empty

FYI, I already rose this up in late May [1]. That was before format-09...

[1] http://www.imc.org/atom-syntax/mail-archive/msg15863.html

-- 
Thomas Broyer

---

From: Henry Story <[email-masked]>
Date: Wed, 17 Aug 2005 13:10:07 +0200

On 17 Aug 2005, at 00:14, Mark Nottingham wrote:
> For that matter, if Henry's interpretation were correct, the  
> element could be
>
> And Atom processors would magically know that XML Base applies to  
> the URI therein. It's the magic that I object to; inferring the  

Yes. I agree the problem also exists for complex extensions. My  
question is the following:

How can a parser that parses atom and unknown extensions, know when  
to apply the xml base to
an extension element automatically?

Clearly if the extensions themselves were to use only xlink:link  
elements that would be easy.
(though atom itself does not give a good example by not following  
that principle). Is there
a place that parsers can get information where they can automatically  
tell that an attribute is a
xlink:link copycat? Or how do they tell that the content of an  
element is a URI? Can  DTDs or RelaxNG files help? If they could  
help, would they be retrievable mechanically by a processor of
xml to help it work out how to deal with relative references?

Anyway to summarise: if you don't want to use the atom:link element  
then perhaps it would
be best to use the xlink:link attributes. I have only read that spec  
quickly [1] but this would
mean that the following

   <fh:history xlink:href="./archives.archive1.atom">

would widely be understood to work with the xml:base.

Henry Story

[1] http://www.w3.org/TR/2001/REC-xlink-20010627/

---

From: Tim Bray <[email-masked]>
Date: Wed, 17 Aug 2005 07:40:28 -0700

> an extension element automatically?

It can't.

> be best to use the xlink:link attributes. I have only read that  
> spec quickly [1] but this would
> mean that the following
>

That would work.  I haven't been following this discussion closely  
enough to understand why there's resistance to <atom:link rel="some- 
history-thing" ...>

  -Tim