Subject: Re: Atom and RDF

From: Danny Ayers <[email-masked]>
Date: Tue, 12 Oct 2004 00:21:03 +0200

Thanks guys, I'll collate the posts in a day or two and pass feedback
back to the Atom list.

On Mon, 11 Oct 2004 12:26:08 -0700, Joshua Allen <[email-masked]> wrote:
> > HTTP), so there's clearly some overlap with RDF efforts. RSS
> > 1.0 extensions must follow the constraints of RDF. So RDF is
> 
> Are there actually any RSS 1.0 extensions in popular use?  

In popular terms, the FOAF stuff really seems to be gaining ground,
and I've seen quite a few niche (not /quite/ the right word) things
like Nature's use of the PRISM vocab.

I note that
> RSS 2.0 approach made it very easy for podcasting to catch hold, and
> this may be a consideration when evaluating an extensibility philosophy.

<enclosure> is a core element, so I'm not entirely sure how that
relates to extensibility. Definitely something to learn from though.
As is usually the case I doubt the lessons will be entirely clear cut.
Check out these comments: http://gonze.com/weblog/story/5-17-4

<enclosure> does appear to have significant flaws, and was frozen into
the spec that way. At the time I'm not even sure Userland were using
it, there certainly wasn't what you'd call significant deployment. Yet
now, two or so years later it's suddenly the hottest thing. Ok, maybe
there is a point there in relation to extensibility - had this
appeared in a 3rd-party extension, would it still have been picked up
in the same way? Another question, could it have been better designed
and picked up in the same way?

Speaking philosophically, I'm starting to wonder if the good Mr. Winer
mightn't simply have entered into a pact with some horned deity or
other...

> > reading) support both formats. Most syndication tools only
> > support enough of the RDF model to be able to extract the
> > syndication-domain data (there are notable exceptions, and of
> 
> This is perhaps wishful.  Most syndication tools do not support RDF
> model at all, and simply read the RSS 1.0 file as XML.  Most tools would
> barf given any of a number of isomorphic RDF models which stray even
> slightly from the expected XML syntax.

Ok, it's syntax based, but the format and model is defined through
RDF, not as a direct mapping between characters and syndication-domain
information. Our dog only understands about three words* (only one of
which he generally responds to**), but those words are most definitely
English, not Dog.

> > course virtually all RDF tools can consume & process RSS 1.0
> > out of the box). Developers of syndication tools tend to have
> 
> Again, I would temper this -- they can consume the RDF graph, but not as
> RSS (unless you meant to say "virtually all RDF-based syndication
> tools").

Well no, I meant what I said - they can consume the RDF vocabulary
defined in RSS. They can merge graphs, make inferences based on the
statements in schema and instance data and so on.

> > Why does it matter? I think RDF potentially stands to gain
> > quite a lot from Atom, in the form of a relatively
> > lightweight but versatile and thoroughly spec'ed transport
> > (and editing protocol) which is likely to gain widespread
> 
> How is this use case any different than using RSS (or Atom) as a
> protocol for moving around audio streams, bittorrent pointers to TV
> programs, and other payloads?  

Hmm, I was going to ask then why you distributed your RSS feed as text
rather than as a zipped blob. Then I realised that it probably does
get delivered as a gzipped blob.

The fact that RSS can be useful with a
> wide variety of payloads is evidence IMO that it defines a good
> separation between the protocol and payload, and my suggestion would be
> to treat RDF as an opaque payload exactly like any other.

Perhaps, but I have a strong feeling that such an approach might miss
out on significant potential of the Web. Opaque audio streams are one
thing, they have one primary form of interpretation, but descriptive
data is something else entirely. It's bad enough with documents - the
web is built of HTML, not PDFs. It's clear that descriptive metadata
distributed through syndication channels is useful, thats what RSS and
Atom are. I suspect long term that top-level flexibility will prove
more interesting than efficient tunnelling, though both probably have
their place.

Cheers,
Danny.

* "Basil", "biscuit", "sit"
** "biscuit"

-- 

http://dannyayers.com

---

From: Dare Obasanjo <[email-masked]>
Date: Mon, 11 Oct 2004 18:40:05 -0700 (PDT)

--- Danny Ayers <[email-masked]> wrote:
> the spec that way.

Can you name these significant flaws and point out
where in the RSS 2.0 spec these flaws are enshrined. 

> At the time I'm not even sure
> Userland were using
> there is a point there in relation to extensibility

I knew you'd figure it out. :)
 
> mightn't simply have entered into a pact with some

*sigh* 

Seriously, is such a comment really warranted? 

=====
THINGS TO DO IF I BECOME AN EVIL OVERLORD #23
I will keep a special cache of low-tech weapons and train my troops in their use. That way -- even if the heroes manage to neutralize my power generator and/or render the standard-issue energy weapons useless -- my troops will not be overrun by a handful of savages armed with spears and rocks.

_______________________________
Do you Yahoo!?
Declare Yourself - Register online to vote today!
http://vote.yahoo.com

---

From: Dare Obasanjo <[email-masked]>
Date: Mon, 11 Oct 2004 19:05:12 -0700 (PDT)

Removing cross post. My bad, didn't realize it was
CCed to a bunch of RSS/RDF lists. 

 > now, two or so years later it's suddenly the
hottest
 > thing. Ok, maybe

 > Speaking philosophically, I'm starting to wonder
 if
 > the good Mr. Winer
 
 *sigh* 

=====

---

From: Danny Ayers <[email-masked]>
Date: Tue, 12 Oct 2004 10:08:52 +0200

> where in the RSS 2.0 spec these flaws are enshrined.

Lucas Gonze listed the following problems with the enclosure element:

1 It causes users to download big files that they will never listen to
or watch, creating pointless overload on web hosts.
2 It doesn't allow us to credit the MP3 host, so we can't satisfy the
netiquette of always linking back.
3 For broadband users, MP3s are not big enough to need advance caching
in the first place.
4 The required content-type attribute is a bad idea in the first place. 
5 The required content-length attribute should not be there. 

http://gonze.com/weblog/story/5-17-4

1 and 3 are bigger picture issues, so arguably could be dropped from
the list. 2 could *potentially* be satisfied by a namespace-qualified
extension, but as far as I'm aware no-one is doing so. Had there been
a slot for this in the spec then they may have done. But ok, that
could potentially be fixed. Which still leaves 4 and 5 as genuine
flaws enshrined in the spec.

> Seriously, is such a comment really warranted?

Sorry, I find some things hard to take seriously.

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Dare Obasanjo <[email-masked]>
Date: Tue, 12 Oct 2004 06:58:58 -0700 (PDT)

> http://gonze.com/weblog/story/5-17-4

I debunked most of these claims at
http://www.25hoursaday.com/weblog/PermaLink.aspx?guid=d9c0205d-3cc1-4efb-a62f-7b0f05fb13af

1. The RSS 2.0 spec never describes this automatic
downloading behavior. This was a suggested usage that
Dave Winer began to evangelize outside the spec which
as many have pointed out has its flaws. 

2. This is true, although it did seem like a weird
special case to me. 

3. See (1) above 

4. No it isn't. Client software should be able, to
decide how to react to the enclosure [e.g. if it is
audio/mpeg display a play button] without having to
make additional HTTP requests to the web server even
if it is just HEAD requests. 

5. Agreed. 

=====

---

From: Danny Ayers <[email-masked]>
Date: Tue, 12 Oct 2004 18:45:36 +0200

> as many have pointed out has its flaws.

The spec-linked "use-case narrative" is rather suggestive. If no
automatic behaviour is associated with this element then it isn't
really clear why a separate element is needed rather than a HTML-style
hyperlink in the content. But I guess you're right about this not
really being a format issue, more of a user interface thing - would I
rather have the (possibly unwanted) mp3 today, or (definitely wanted)
mp3 tomorrow.

> 2. This is true, although it did seem like a weird

I don't know, with aggregate-republish systems this could get quite
common. But whatever, although it's a bit untidy structurally this
could be worked around by using the RSS 2.0 <source> element on the
item, sort of reverting to the pre-Userland semantics of <description>
being the description of an item rather than the item/content itself.
 
> if it is just HEAD requests.

Hmm, I'm not sure about this at all. I can see how a button might be
useful, but HTML links don't demand prior notification of the media
type.

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: "Bill Kearney" <[email-masked]>
Date: Tue, 12 Oct 2004 13:29:43 -0400

Goodness but this thread has been cc'd shotgun-style out to waaay too many
lists.  How about choosing one and saving the rest from the thread?

> > 1. The RSS 2.0 spec never describes this

That'd be a good starter for most of what that spec discusses...

> hyperlink in the content.

Sure there is, an enclosure with a TYPE is a lot more informative than
guesswork associated with an HTML href link.

> But I guess you're right about this not

Well, the only tool using it doesn't offer anything in the way of
per-channel or other control over the attachments.  You get 'em all with no
selectiveness.  When there's only attachments coming from one channel and
you know you want it (or narcissus has already assumed you do) then it's
fine.  With many channels or potentially re-aggregated material (lucas'
webjay for example) it's potentially a really horrible waste of bandwidth
all around.

> useful, but HTML links don't demand prior notification of the media

There's a balancing act to be considered here.  One being the idea that
off-peak bandwidth could be used as a way to avoid clogging up the download
time.  This could just as well be handled via other p2 transports
(bitTorrent, ed2k, etc).  However those weren't available (or popular) at
the time the enclosure scheme was foisted onto the spec.  The other being
avoiding needless waste of bandwidth from downloading media that won't ever
be consumed.  This being something the client-side interface could (should)
handle.  Having content type, size and other info greatly aids the client in
being able to make 'smarter' decisions about the content in question.  This
isn't something a regular HREF would usually offer.

-Bill Kearney
Syndic8.com

---

From: Lucas Gonze <[email-masked]>
Date: Tue, 12 Oct 2004 13:44:39 -0400 (EDT)

> lists.  How about choosing one and saving the rest from the thread?

Note to people not sub'd to atom-syntax: I'm going to trim the CC: list 
and reply there only.

- Lucas

---

From: Lance Lavandowska <[email-masked]>
Date: Tue, 12 Oct 2004 12:48:15 -0500

> isn't something a regular HREF would usually offer.

Wouldn't it be nice if it did?  I propose these be added as optional
attributes for the Link discussions.

Lance

---

From: Robert Sayre <[email-masked]>
Date: Tue, 12 Oct 2004 13:57:03 -0400

> attributes for the Link discussions.

+1

Robert Sayre

---

From: Lucas Gonze <[email-masked]>
Date: Tue, 12 Oct 2004 14:41:05 -0400 (EDT)

> webjay for example) it's potentially a really horrible waste of bandwidth

Yup, and that's why I turned off enclosures on Webjay, but I'm working on 
fixing the problems so that I can turn enclosures on again.

My plan is simple: I'm going to change my feeds to Atom, then assume that 
the WG will do a good job with the bugs.  It's a faith based approach, 
really.

I have posted a strawman feed with enclosure support here: 
http://gonze.com/weblog/story/atomenclosures

The relevant bits are:
<entry>
<link rel="enclosure" type="audio/mpeg" href="http://www.earthear.com/realaudio/emergent45.mp3"/>
<link rel="via" type="text/html" href="http://www.earthear.com/catalog/angels.html"/> 
</entry>

I have managed to convince the developer of a feed reader to implement 
that scheme.  This worked out because he already supported Atom, he 
already supported RSS 2.0 enclosures, and the link[@rel="enclosure"] 
element is so similar to RSS 2.0 enclosures that it was trivial to 
code.

In the short term I'd make the case for a straight swipe of enclosure 
semantics based on ease of implementation and likelyhood of adoption, 
which is pretty much 100%.  In the long term I'd advocate that the group 
should take the same approach to enclosures as it has to any work -- adopt 
a set of features based on real world software, then patiently work out 
the bugs and try to raise the general standard of engineering.

>>> if it is just HEAD requests.

I agree that would be a nice feature, but in practice the types will be 
wrong so often that all robust clients still have to HEAD.  Webjay is a 
user-facing tool that reacts to media types in about the way described 
here, and it has to check them very carefully.

I don't mean to imply that design work here should coddle bad system 
administration elsewhere.  My point is that any metadata that's part of an 
href or link has to be wrapped in caveats -- the metadata is only a hint, 
and clients that need it to be correct have to assume that it's incorrect.

- Lucas Gonze

(N.B.: I prefer Atom to RSS 1.0 in this context because it's more easily 
convertible to/from XSPF, which is XML).

---

From: Antone Roundy <[email-masked]>
Date: Tue, 12 Oct 2004 13:04:05 -0600

On Tuesday, October 12, 2004, at 12:41  PM, Lucas Gonze wrote:
> I agree that would be a nice feature, but in practice the types will 
> be wrong so often that all robust clients still have to HEAD.  Webjay 
> is a user-facing tool that reacts to media types in about the way 
> described here, and it has to check them very carefully.
>
> administration elsewhere.  My point is that any metadata that's part 
> of an href or link has to be wrapped in caveats -- the metadata is 
> only a hint, and clients that need it to be correct have to assume 
> that it's incorrect.
>
I'd prefer to have an OPTIONAL type attribute, and encourage people to 
omit it if they don't know how to get it right.  Developers of crappy 
tools that put inaccurate data into their feeds should be shamed into 
fixing them, or pushed from the market by better tools.  Making the 
attribute option provides a sufficient opening for developers that 
don't know how to get the type right.  If the remote resource changes 
types on occasion, that can be dealt with at the time the user tries to 
open the enclosure.

---

From: Lucas Gonze <[email-masked]>
Date: Tue, 12 Oct 2004 15:32:19 -0400 (EDT)

On Tue, 12 Oct 2004, Antone Roundy wrote:
> I'd prefer to have an OPTIONAL type attribute, and encourage people to omit 
> it if they don't know how to get it right.  Developers of crappy tools that 
> put inaccurate data into their feeds should be shamed into fixing them, or 
> pushed from the market by better tools.  Making the attribute option provides 
> a sufficient opening for developers that don't know how to get the type 
> right.  If the remote resource changes types on occasion, that can be dealt 
> with at the time the user tries to open the enclosure.

Well, I think the essential point is that these attributes are assertions, 
not facts, and the definition of related data structures has to 
acknowledge that.

- Lucas

---

From: Sam Ruby <[email-masked]>
Date: Tue, 12 Oct 2004 15:53:55 -0400

Lucas Gonze wrote:
>> types on occasion, that can be dealt with at the time the user tries 
>> to open the enclosure.
> 
> Well, I think the essential point is that these attributes are 
> assertions, not facts, and the definition of related data structures has 
> to acknowledge that.

If approved, PaceLinkAttrDefaults would make the type attribute optional.

Any contemplated spec language should take in account the following:

   http://www.w3.org/2001/tag/doc/mime-respect.html#intro

In short, the answers returned by a simple HEAD request MUST override 
any values of any attributes that may be found in the link element itself.

In addition, I would like to point out that any size attribute would not 
only be non-authoritative, but is also fairly meaningless in a common 
scenario employed with today's enclosures: bit torrents.  What you get 
is the size of the .torrent file, not the size of the file(s) in the 
torrent itself.

- Sam Ruby

---

From: Dare Obasanjo <[email-masked]>
Date: Tue, 12 Oct 2004 13:22:47 -0700 (PDT)

> here, and it has to check them very carefully.

In practice, robust clients will ignore the MIME type
returned by the web server and try to sniff the file
to figure it out anyway. As has been shown on this
list in the past, almost half the XML feeds out there
use the wrong MIME types anyway. I haven't done a
survey of rich media files on the Web but wouldn't be
surprised to see similar discrepancies.  

=====

__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail

---

From: "Bill Kearney" <[email-masked]>
Date: Tue, 12 Oct 2004 16:33:28 -0400

> any values of any attributes that may be found in the link element itself.

So let's say something like an .mp3 file is being served up from a
misconfigured server.  In a HEAD request it's telling you that the file is a
"text/plain" document.  This is supposed to be taken as gospel and the
attributes used otherwise are to be ignored?

Uh, that's sort of the entire reason for using the attributes, isn't it?

The answser suggested by scenario 2.2 is perhaps overly optimistic (and
that's being kind) given REAL WORLD conditions.  I can certainly sympathize
with the point they might be trying to make about the importance of
server-side configurations but, please, that ain't happenin' anytime soon.

Section 4.1 of the spec is almost close to being useful:
http://www.w3.org/2001/tag/doc/mime-respect.html#answer2

    "Query the server, and when there is an inconsistency,
    choose the authoritative metadata, or

    Query the server, and when there is an inconsistency,
    prompt the user for instructions on how to proceed."

The trouble being with their idea of 'authoritative'.  If it's the server
config then we've gotten nowhere.  Or perhaps the documentation for an
enclosure-like element, when used in a feed, would benefit from stating that
it's use of the attributes should be considered authoritative regardless of
server config.

Either situation is bad, the question to ask is which is worse?

Stating the resource is an .mp3 file and it being served improperly as
text/plain isn't a problem when the client understands how to handle mp3
files.  Conversely, should someone be trying to force something 'executable'
on the user while being served up as 'text/plain' or some other
'server-allowed' datatype might introduce other woes.  I'd be less concerned
with these as I don't forsee too many server configurations being properly
setup to block publishing of something 'harmful' such that an attribure
override would 'trick' the client into mishandling it.  Granted, a local
client's 'handler' of such resources could be compromised such that this
trick might work (ms jpeg comes to mind).  But I'm inclined to think the
server-side isn't going to have things figured out either.

> is the size of the .torrent file, not the size of the file(s) in the

Well, of course!  It's not the size of the .torrent file that we care about
but the actual payload it points toward.  While, in some cases, it's
possible to pull a pointer document like those used by BitTorrent and
extract from it relevant metadata that requires additional network requests.
But why bother?  In the context of something like an enclosure it seems
quite a bit more reasonable to indicate the metadata as attributes and save
the extra trip.

I sense we're crossing the wires again between what an atom entry might
benefit from versus it being wrong for a feed-like situation.

-Bill Kearney

---

From: Walter Underwood <[email-masked]>
Date: Tue, 12 Oct 2004 15:07:06 -0700

> attributes used otherwise are to be ignored?

Yes, it is gospel. MSIE can guess text/plain to HTML, then run the
scripts in it. Makes it rather dangerous to look at log files.

Having MIME type override the server's type opens some interesting
opportunities for cross-site scripting attacks.

If the web server is misconfigured, it should be changed. This is
exactly the position we took on well-formed Atom feeds, right?

wunder
--
Walter Underwood
Principal Architect
Verity Ultraseek

---

From: Lucas Gonze <[email-masked]>
Date: Tue, 12 Oct 2004 18:46:51 -0400 (EDT)

> exactly the position we took on well-formed Atom feeds, right?

That's a reasonable position when there's a pressing need, but the need 
has to be pressing because it's very hard to pull off at internet scale.

Let's say the one and only usecase for an element is prefetching media for 
offline rendering.  Let's also say that the usecase for the type attribute 
on that element is to determine whether the URL contained in the element 
is the type of media that offline rendering would be appropriate for. 
But since the element only every holds media intended for offline 
rendering, the type isn't adding information.

Whether those are acceptable or wise restrictions on the element is a 
different conversation.  My first instinct is to say that any other 
usecase for the element is YAGNI.

- Lucas

---

From: Eric Scheid <[email-masked]>
Date: Wed, 13 Oct 2004 10:02:39 +1000

> rendering, the type isn't adding information.

what if @type in a feed item was "application/svg+xml" ... would you want
webjay attempting to prefetch that? (assuming it knows squat about handling
svg+xml, being an mp3 player at heart)

e.

---

From: Eric Scheid <[email-masked]>
Date: Wed, 13 Oct 2004 10:09:34 +1000

> Uh, that's sort of the entire reason for using the attributes, isn't it?

not necessarily.

if a feed contained these elements:

    <link type="image/jpeg" rel="enclosure" .... />
    <link type="application/mpeg" rel="enclosure" .... />

then something like webjay wouldn't need to HEAD the first element at all.
It can ignore it totally. Similarly an aggregator user-agent could do
something intelligent with this...

    <link type="application/mpeg" rel="enclosure" .... />
    <link type="music/x-karaoke" rel="enclosure" .... />
    <link type="audio/midi" rel="enclosure" .... />
    <link type="audio/wav"  rel="enclosure" .... />

... choosing the format the user prefers, and avoiding the formats she hates
(midi! karaoke!!).

Only when attempting to download the "application/mpeg" enclosure might the
aggregator be faced with handling an inconsistency. It doesn't care if the
"audio/midi" file is served as "text/plain" because it will never look at
it.

e.

---

From: Lucas Gonze <[email-masked]>
Date: Tue, 12 Oct 2004 21:14:55 -0400 (EDT)

> svg+xml, being an mp3 player at heart)

I wouldn't want that.  It is better to avoid having to test resources that 
I know are going to come up negative.  But webjay is a web app.  A 
prefetch handler running unattended on a client machine is in a different 
position, since it has plenty of bandwidth and time.

...

Here's a different line of argument.

One of the main usecases for RSS 2.0 enclosures is BitTorrent files.  The 
mime type of a BitTorrent file describes the seed document, not the bytes 
you get via the seed.  Therefore it is already impossible to filter by 
type.  And if we did find a compelling enough story to want to know the 
type of the URL referenced in the enclosure, we would want both the type 
of the seed file and the result file, and that opens up yet another can of 
worms.

- Lucas

---

From: James Aylett <[email-masked]>
Date: Wed, 13 Oct 2004 10:43:53 +0100

> of the seed file and the result file, and that opens up yet another can of 

But if the feed author uses an optional type attribute to indicate
that the enclosure/link is a BitTorrent seed file, then the client can
do something different to what it might find if type is given as SVG
(or absent).

BitTorrent files aren't that big, are they? So it could grab it
automatically, and if it turns out not to be the correct MIME type
from the server, close the connection before it gets too much rubbish
thrown at it. (Admittedly this relies on having more control over the
underlying connection than you get with most HTTP libraries; the
conservative approach in that case would be to HEAD it first.)

James

-- 
/--------------------------------------------------------------------------\
  James Aylett                                                  xapian.org
  [email-masked]                               uncertaintydivision.org

---

From: James Aylett <[email-masked]>
Date: Wed, 13 Oct 2004 10:52:56 +0100

> surprised to see similar discrepancies.  

While I appreciate the interop reasons for doing this, doesn't Firefox
require that the MIME type is appropriate for doing things like loading
stylesheets, XML pages, handing off to third party apps for video and
so forth? So there is a real case of something doing what the specs
say should be done, and I haven't heard as much criticism of that as
the idea has been given on this list.

I'm inclined to take the Google-view [1]: the web is big, and bits of
it will be inconvenient or impossible to get to, but that doesn't
matter because the web is SO big that there'll be something else out
there that you can get to.

After all, this already happens to some extent - RSS Bandit may work
round incorrect MIME types, but it refuses to parse feeds which use
undeclared namespace prefixes. When a feed exists but doesn't work, I
just ignore it and find something else to do. Life is too short. Most
people /don't/ file bug reports, they just move on.

[1] Okay, maybe Google doesn't think like this explicitly :-)

James

---

From: Danny Ayers <[email-masked]>
Date: Wed, 13 Oct 2004 12:49:45 +0200

> exactly the position we took on well-formed Atom feeds, right?

In the case of media files, agreed. I don't think there's anything
like the same mess about providing the right header with multimedia
and other blob types compared to the markup continuum: plain
text/html/xhtml/xml/lisp...
 
Quick glance at an Apache mime.types would seem consistent with this
(though I see no entry for BitTorrent).

Cheers,
Danny. 

-- 

http://dannyayers.com

---

From: Danny Ayers <[email-masked]>
Date: Wed, 13 Oct 2004 12:53:53 +0200

> of the seed file and the result file, and that opens up yet another can of

Sorry if you've already mentioned this before (this thread is a bit
unwieldy), but how do you suggest dealing with this difference
(type->media, type->descriptor->media)?

Cheers,
Danny. 

-- 

http://dannyayers.com

---

From: Lucas Gonze <[email-masked]>
Date: Wed, 13 Oct 2004 10:31:40 -0400 (EDT)

> (type->media, type->descriptor->media)?

By avoiding it if at all possible.  I brought up the example to show that 
the type attribute is not doing any work in existing enclosure software, 
because if it was then BitTorrent types would be breaking things.

This is in support of my general point that the type attribute is YAGNI. 
There are no application requirements which depend on it.

- Lucas

---

From: Lucas Gonze <[email-masked]>
Date: Wed, 13 Oct 2004 10:37:10 -0400 (EDT)

> (though I see no entry for BitTorrent).

The situation is much worse for media files, because:

1) proliferation of media formats means that there are more mime types for
    front line coders to learn.

2) the major vendors (Apple, MS, Real) are in a standards war and
    see open standards like MIME as being in their competitors' interests.

3) a large portion of media files are served by proprietary servers that
    ship misconfigured.  I'd guess that 75% of Windows Media files are
    served as text/plain.

- Lucas

---

From: Lucas Gonze <[email-masked]>
Date: Wed, 13 Oct 2004 10:42:06 -0400 (EDT)

On Wed, 13 Oct 2004, James Aylett wrote:
> BitTorrent files aren't that big, are they?

No, but HEAD is even smaller.

What's missing in this conversation is an application requirement.  What 
would the client do differently for SVG than BitTorrent, and why does it 
belong at this layer?  That's why YAGNI has been my mantra.  The type 
attribute introduces bugs but not features.

- Lucas

---

From: Robert Sayre <[email-masked]>
Date: Wed, 13 Oct 2004 11:01:58 -0400

> attribute introduces bugs but not features.

I would agree with you if HTTP was perfect, but HEAD requests do have a 
cost, and not all resources respond to HEAD requests accurately.

Let's say I'm subscribed to 1000 feeds in Project D.U.[0] or something 
similar, and I have it configured to show me .mp3 enclosures in a 
special pane (I don't want .wma .ra files). Is the aggregator supposed 
HEAD every single enclosure link to find out which ones are mp3s?

I think the type attribute is useful metadata in the same sense that the 
title attribute is. If we omit it, we'll just see stuff like this:

<link ... title="Van Halen - Panama. [5MB, mp3]" />

That's something we often see in HTML. There's your use case.

Robert Sayre

[0] http://projectdu.com/

---

From: Antone Roundy <[email-masked]>
Date: Wed, 13 Oct 2004 09:04:19 -0600

On Wednesday, October 13, 2004, at 08:42  AM, Lucas Gonze wrote:
> That's why YAGNI has been my mantra.  The type attribute introduces 
> bugs but not features.
Crappy feeds introduce bugs.  A few people on this list have chanted in 
opposition to your mantra, and my opinion harmonizes with theirs.  My 
application, CaRP, which only uses image/gif and image/jpeg enclosures, 
uses the type attribute to decide whether to try displaying an 
enclosure.  I have yet to have a customer complain.  I suspect they'd 
be more likely to complain if I did an almost-always-unnecessary HEAD 
on every enclosure to see whether the (possibly misconfigured) server 
agreed with the feed, since that would slow performance.  For me and 
mine, @type is a useful feature.  If it is less useful for some other 
application, let that application HEAD the enclosures.

---

From: Lucas Gonze <[email-masked]>
Date: Wed, 13 Oct 2004 11:09:56 -0400 (EDT)

> ... choosing the format the user prefers, and avoiding the formats she hates

I'm separating out this comment of yours because Alex brought up the same 
issue.

I'd say again: there's no usecase for this; if there were it could be 
satisfied within existing web architecture by sending preferred types in 
the Accept header; if there is really a pressing reason to do this then 
Atom is slipping into playlist functionality and should start by looking 
for a playlist format that already solves the problem.

- Lucas

---

From: James Aylett <[email-masked]>
Date: Wed, 13 Oct 2004 16:21:58 +0100

> No, but HEAD is even smaller.

Erm ... so? If you have reason to believe you'll get a BT file, and
the user wants things downloaded automatically, what reason do you
have not to grab it? A similar mechanism regulates downloading of CSS
stylesheets in browsers ...

> attribute introduces bugs but not features.

If a client doesn't know what to do with SVG, why would it
automatically download it? Even making the HEAD requests has costs
(ignoring the possibility that the HEAD might lie [1]), and if I
/know/ that the type marked is something I can do nothing with, why
should I pay a cost on the off-chance that it's marked wrongly?

The only reason BT files would be treated differently to SVG is if the
client knows how to deal with one and not the other. That on
downloading, BT files should cause another fetch is a wrinkle, but not
a unique one - there are formats where auxiliary resources might want
to be automatically downloaded to enable immediate display, after all.

With type optional, anything that can't guarantee to get it right can
just not produce the type attribute, and then you do have to HEAD or
GET to figure out what it is, but that's at least making life
easier. I wouldn't want my connected MP3 player downloading anything
not explicitly marked as MP3s, but for them it can go and get as many
as possible automatically and I'll be happy - which is, in my
understanding, one of the main uses of enclosures in RSS 2.0 in the
first place.

[1] RFC 2616 is cagey about this; HEAD responses SHOULD contain the
same metadata as the equivalent GET responses. However clients can
rely on that metadata for updating cached entities and figuring out
staleness of cached entities (which, interestingly, may not include
Content-Type - which suggests that the resource is considered the same
even if the MIME type changes; the RFC is silent on this AFAICT, and I
can't think through all the implications of this). This strikes me as
strange, at the very least.

James

---

From: Eric Scheid <[email-masked]>
Date: Thu, 14 Oct 2004 01:31:58 +1000

> There are no application requirements which depend on it.

What about content negotiation?

if an aggregator sees an enclosure link to an "application/mpeg" resource,
could it not use content negotiation to ask for the resource with a high
preference for a BitTorrent stub? If the server has a BT stub, it can send
that instead of the actual mpeg, declaring that it is a BT stub in the
response headers. Of course if the aggregator doesn't have a BT capability
built in it wouldn't request with content negotiation.

e.

---

From: Dare Obasanjo <[email-masked]>
Date: Wed, 13 Oct 2004 08:37:00 -0700 (PDT)

--- Antone Roundy <[email-masked]> wrote:

> 
> I have yet to have a customer complain. 
> I suspect they'd 
> agreed with the feed, since that would slow
> performance. 

Actually according to other posts by Lucas in this
thread the server will MOST LIKELY be misconfigured
(didn't he throw out a number like 75%) so if you are
going to go as far as making network connections you
might as well do a GET and seek into the file to sniff
what type it is since the MIME type returned by the
HEAD request is likely to be wrong anyway. 

> For me and 

+1 

=====

__________________________________
Do you Yahoo!?
Y! Messenger - Communicate in real time. Download now. 
http://messenger.yahoo.com

---

From: Dare Obasanjo <[email-masked]>
Date: Wed, 13 Oct 2004 08:42:27 -0700 (PDT)

> for a playlist format that already solves the

Atom is a content syndication format. Music files are
content. Throwing up a playlist on my site and having
folks subscribe to it either directly or indirectly
(i.e. an MP3 blog) is syndication. 

Sounds like a use case for Atom. 

=====

__________________________________
Do you Yahoo!?
Read only the mail you want - Yahoo! Mail SpamGuard.
http://promotions.yahoo.com/new_mail

---

From: Walter Underwood <[email-masked]>
Date: Wed, 13 Oct 2004 09:00:28 -0700

>     ship misconfigured.  I'd guess that 75% of Windows Media files are

No, the situation is bad because Microsoft Internet Explorere ignores
the content-type and dispatches on file extension. That means that it
works in IE even when misconfigured, so there is little pressure to
fix busted websites.

This isn't a new problem. Most HTML files don't declare their encoding
because Netscape Navigator had an encoding guesser.

Past bad behavior is not a good reason for Atom to do it wrong.
MIME content-type hints are useful, but they should only be hints.

wunder
--
Walter Underwood
Principal Architect
Verity Ultraseek

---

From: Lucas Gonze <[email-masked]>
Date: Wed, 13 Oct 2004 11:35:42 -0400 (EDT)

>> That's why YAGNI has been my mantra.  The type attribute introduces bugs 
>> but not features.

On Wed, 13 Oct 2004, Antone Roundy wrote:
> opposition to your mantra, and my opinion harmonizes with theirs.

The chanting preceded the mantra, so it can't have been in opposition.

Also, please be careful about the flamish tone.

> My 
> application, CaRP, which only uses image/gif and image/jpeg enclosures, uses 
> the type attribute to decide whether to try displaying an enclosure.  I have 
> yet to have a customer complain.  I suspect they'd be more likely to complain 
> if I did an almost-always-unnecessary HEAD on every enclosure to see whether 
> the (possibly misconfigured) server agreed with the feed, since that would 
> slow performance.  For me and mine, @type is a useful feature.  If it is less 
> useful for some other application, let that application HEAD the enclosures.

Can you talk about your application more?  Why put images in enclosures 
rather than in content?  This is unconventional, so the first thing to 
figure out is whether your usage matches up with what we're trying to 
enable.

- Lucas

---

From: Antone Roundy <[email-masked]>
Date: Wed, 13 Oct 2004 10:25:42 -0600

On Wednesday, October 13, 2004, at 09:35  AM, Lucas Gonze wrote:
> Can you talk about your application more?  Why put images in 
> enclosures rather than in content?  This is unconventional, so the 
> first thing to figure out is whether your usage matches up with what 
> we're trying to enable.
>
My application is a feed consumer (it displays feeds on websites), not 
a publisher.  It's consuming images that other publishers, for whatever 
reason, put into enclosures.  I'd guess that the decision whether to 
put it in content or an enclosure is based on a few factors:

1) Whether the publisher wants to dictate how the image is displayed in 
relation to the item or whether they want the consumer to be able to 
configure that.  I find it useful to have them separated so that, for 
example, the feed can be displayed in a table with images in the first 
column and title/link and descriptions in the second.

2) Some consuming applications may strip out inline images from 
content.  Putting it into an enclosure MIGHT increase the probability 
of the application displaying it, since they more control over 
presentation (see the second sentence above for an example).

3) If the content is plain text, the image can't be included in it.

---

From: "Roger B." <[email-masked]>
Date: Wed, 13 Oct 2004 12:20:22 -0500

> rather than in content?  This is unconventional...

Lucas: No, it isn't. Flickr does the same thing, for example. As for
why... so the user can download the image, not a chunk of HTML that
references it.

--
Roger Benningfield
http://admin.support.journurl.com/

---

From: Danny Ayers <[email-masked]>
Date: Wed, 13 Oct 2004 22:11:02 +0200

On Wed, 13 Oct 2004 08:42:27 -0700 (PDT), Dare Obasanjo <[email-masked]> wrote:

> content. 

If that's the definition we're following, then to fulfil the charter
we'll also have to figure out how to do remote audio and video editing
using the protocol too. I would guess to the level of splicing would
be appropriate. Should be fun, though this project may take a while
longer than initially anticipated.

Alternately we could assume that "content" in the charter refers to
text-based formats.

Throwing up a playlist on my site and having
> Sounds like a use case for Atom.

It does, but looking at the list of playlists formats there's a great
risk of inventing Yet Another Playlist format. I would suggest working
with the people [1] already developing openly in this domain to see if
Atom core can feasibly support binary media playlist syndication (i.e.
<content src=""> as bugfixed <enclosure>). Everything else should go
in an extension, left to the domain experts.

I suspect the typing issue in link->content vs. link->meta->content
will need to be tackled on some front sometime whatever we do.

Cheers,
Danny.

[1] http://playlist.musicbrainz.org/playlist/moin.cgi/FrontPage

-- 

http://dannyayers.com

---

From: Danny Ayers <[email-masked]>
Date: Wed, 13 Oct 2004 22:41:49 +0200

> > attribute introduces bugs but not features.

I don't think YAGNI is an appropriate pattern for spec development,
but still I'm not sure putting this kind of data in an attribute is
advisable.

> special pane (I don't want .wma .ra files).

Declaring the type in advance is only useful if it makes a significant
improvement in performance and/or reduction in complexity in a large
proportion of cases. Even then, that only works if the data passed
along this way is going to be significantly more accurate than the
data obtained by other means. Where is the publishing tool going to
get its information from?

 Is the aggregator supposed
> HEAD every single enclosure link to find out which ones are mp3s?

Possibly, yes, if that's the way the trade-off goes. At least one of
the costs of HEADs (bandwidth) is proportionately reduced when dealing
with big media files.

> That's something we often see in HTML. There's your use case.

Well, not really because you can still do a HEAD and it doens't make
the in-content statement any more reliable.

If the type attribute inline will be non-redundant, accurate and
useful information then let's have it . But adding another unreliable
piece of information will just offer another avenue for error. So far
I don't think all the angles have yet been convincingly covered.

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Danny Ayers <[email-masked]>
Date: Wed, 13 Oct 2004 22:56:02 +0200

> why... so the user can download the image, not a chunk of HTML that

Do you happen to know offhand how Flickr handles media typing?

-- 

http://dannyayers.com

---

From: Robert Sayre <[email-masked]>
Date: Wed, 13 Oct 2004 16:55:43 -0400

> the in-content statement any more reliable.

Then why does HTML have @type?

> useful information then let's have it

Ha! Which part of the Atom format makes that guarantee? @type is there 
to present alternatives, just like @title.

Robert Sayre

---

From: "Bill Kearney" <[email-masked]>
Date: Wed, 13 Oct 2004 17:15:58 -0400

>     <link type="application/mpeg" rel="enclosure" .... />

My point about attributes was within the context of an enclosure element.
It's use of type and size are among the reasons to use attributes.  Type
alone is indeed helpful, other data helps too.  The size being that of the
end-result data not the interim pointer URI for a p2p network.  While it
migth, perhaps, be interesting to know the size of the p2p pointer file it's
probably not going to be very useful in making any decisions related to the
handling of the target data.  Knowing the size of the torrent link would be
nearly as useless as knowing the size of an HTML index page that contained
links to other things.

> "audio/midi" file is served as "text/plain" because it will never look at

Well, yeah, that's sort of obvious.  It's in the question of whether or not,
or how to handle retrieval (if at all) that added metadata helps the
decision-making.

I also think there's a difference between the semantics of an enclosure
versus that of streamed data or something that provides an alternative
representation of the data.  An audio recording of the same data presents an
interesting question.  Is it an enclosure or is it an alternative?

The idea of an enclosure is to allow an item to indicate that it's got some
sort of extra 'payload' that can be downloaded locally, perhaps at a
different interval or during off-peak times.  Off-peak being it's own big of
ambiguity as well.  Is it off-peak for the client (as it's assumed now) or
is it off-peak for the serving resource.  I could well see situations where
some might consider the latter might to be useful.

The original use case for enclosures in RSS was to push music and videos out
to the users but not bog down their immediate daytime downloading
experience.

-Bill Kearney

---

From: Danny Ayers <[email-masked]>
Date: Thu, 14 Oct 2004 14:15:09 +0200

> Then why does HTML have @type?

Because the TAG group arrived late?

Lines like this don't exactly inspire confidence:
[[
Authors who use this attribute take responsibility to manage the risk
that it may become inconsistent with the content available at the link
target address.
]]

http://www.w3.org/TR/REC-html40/struct/links.html#adef-type-A

btw, HTML 2.0 had URN (c.f. atom:id) and METHODS (c.f. atom:service) -

http://www.w3.org/MarkUp/html-spec/html-spec_5.html#SEC5.7.3

> Ha! Which part of the Atom format makes that guarantee?

This runs along with why I don't think developer patterns are
appropriate for specs - the MUSTs in the spec do make guarantees. But
their fidelity evaporates when it gets to implementation. Specs are
defined for an idealized world, laws put in as compromises are part of
the idealization. In this case redundancy and utility can be
determined pretty clearly, accuracy is a bit floppy.

@type is there
> to present alternatives, just like @title.

Hmm, it's a grey area but there is something of a qualitative
difference in that one is designed primarily for human interpretation
and the other machine interpretation. Having the wrong title has
little consequence to the application behaviour, the wrong type might
well prevent something from working.

I really don't know. Perhaps there is justification for @type, but I
would think it would have to be optional and qualified with a
Postelian MUST for publisher accuracy and MAY for consumer advisory.

btw, what are the alternatives presented to @title?

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Robert Sayre <[email-masked]>
Date: Thu, 14 Oct 2004 08:35:32 -0400

> determined pretty clearly, accuracy is a bit floppy.

No. You'll find plenty of handwaving in lots of good specs. "Advisory 
media type" is good enough for HTML and it's good enough for me. We're 
not going to pin this one down.

> btw, what are the alternatives presented to @title?

"There you go again" :)

This is an issue you can solve with conneg and alternate links. If 
someone subscribes to a feed with 10 titles for 10 entries that's 100 
titles, of which 10 will be viewed. HTTP users call that "slow". Hanging 
10 entries off of one URI is bad enough, but the hack breaks once you 
start hanging huge subtrees of alternates off of those entries.

Anyway, this is a bikeshed discussion. I suggest you write a Pace that 
outlines how clients are to act in the absence of @type.

Robert Sayre