Subject: xml:base abuse

From: Sjoerd Visscher <[email-masked]>
Date: Mon, 15 Aug 2005 15:20:28 +0200

Hi,

A while ago we had a discussion about how xml:base should be used.
We didn't reach a conclusion, but I think we need to act.

The way Tim Bray's feed and the examples from James Snell on 
developerWorks use xml:base is what Roy T. Fielding calls an abuse.

Now I think that no matter what we decide, we should not do something 
that the writer of the URI spec thinks is an abuse.

More here:
http://w3future.com/weblog/2005/08/#howToUseBaseUris

-- 
Sjoerd Visscher
http://w3future.com/weblog/

---

From: Sam Ruby <[email-masked]>
Date: Mon, 15 Aug 2005 09:52:46 -0400

> http://w3future.com/weblog/2005/08/#howToUseBaseUris

I would prefer to deal with what the spec actually says.  As you point
out, it is "really odd" that nothing was added to the new RFC 3986 to
support your position.

The authors of DOM3 looked at both the xml:base and InfoSet
specifications.  I've taken a look at how two respected XML parsers have
implemented DOM3:

http://www.intertwingly.net/blog/2005/08/12/xml-base-support

- Sam Ruby

---

From: Sjoerd Visscher <[email-masked]>
Date: Mon, 15 Aug 2005 16:12:55 +0200

> http://www.intertwingly.net/blog/2005/08/12/xml-base-support

Yes, that code is perfectly fine, and not the actual problem.

Let me show you some pseudo-code that implements how to dereference an 
atom link according to rfc3986:

dereference(linkElement) =
   baseURI = linkElement.baseURI
   linkURI = makeAbsolute(linkElement.href, baseURI)
   if stripFragmentID(linkURI) == stripFragmentID(baseURI)
     then return linkElement.ownerDocument
     else return loadDocument(linkURI)

You'll see the problem when you pass the link element of Tim Brays feed.

Also, Roy T. Fielding says "[...] a person is deliberately abusing the 
base URI by assigning it an unrelated URI for the purpose of creating an 
artificial shorthand notation for external references."

There is nothing in rfc3986 that contradicts this, it only indirectly 
hints to it. The real purpose of a base URI isn't explained in rfc3986.

---

From: Tim Bray <[email-masked]>
Date: Mon, 15 Aug 2005 07:28:42 -0700

> developerWorks use xml:base is what Roy T. Fielding calls an abuse.

I do plan to revise my feed, will try to do so in the next day or  
two. -Tim

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Mon, 15 Aug 2005 16:29:04 +0200

>We didn't reach a conclusion, but I think we need to act.

How to use xml:base is a matter of the xml:base specification
and (less so) the resource identifier specifications. If you
think xml:base is unclear in this regard, please let the W3C
XML Core Working Group know.

>Now I think that no matter what we decide, we should not do
>something that the writer of the URI spec thinks is an abuse.

"We" as in there is specific text in one of the atompub drafts
that make misleading suggestions that are inconsistent with
the xml:base Recommendation or RFC 3986/7 other than some con-
fusion about how to handle IRIs in non-Unicode encoded Atom
documents that are not in NFC?
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Sjoerd Visscher <[email-masked]>
Date: Mon, 15 Aug 2005 16:41:59 +0200

> documents that are not in NFC?

I have no problem with the spec, only with some high profile Atom feed 
examples.

---

From: Sam Ruby <[email-masked]>
Date: Mon, 15 Aug 2005 11:25:36 -0400

> You'll see the problem when you pass the link element of Tim Brays feed.

Focusing on the <link href=""/> in Tim's existing feed, it seems to me
that Tim intended to express <http://www.tbray.org/ongoing/>.  How you
interpret the spec is that this will actually resolve to
<http://www.tbray.org/ongoing/ongoing.atom>.

Am I correct so far?

If so, what would happen if he replaced this with <link href="/"/>?
linkURI would still resolve to the same value as baseURI.  Do you
interpret the spec is that this will also resolve to
<http://www.tbray.org/ongoing/ongoing.atom>?

How about <link href="./"/>?  I guess that would depend on what the
meaning of "==" is.

 - - -

I am quite willing to add feed validator warnings for edge cases where
reasonable professionals might have differing opinions of how to read
the specification, particularly if there is no compelling use case for
use of the feature.

Can we agree that empty //atom:link/@href values, or //atom:link/@href
values that consist entirely of a fragment are problematic?

- Sam Ruby

---

From: Sjoerd Visscher <[email-masked]>
Date: Mon, 15 Aug 2005 17:46:18 +0200

> <http://www.tbray.org/ongoing/ongoing.atom>.

Not quite, it is a nitpick, but I think the source of our disagreement.
Resolution is the procedure of making absolute paths from relative 
paths. So the link resolves to <http://www.tbray.org/ongoing/>.
But this is equal (string-equal) to the base URI, so this is a 
same-document reference.

> <http://www.tbray.org/ongoing/ongoing.atom>

/ resolves to <http://www.tbray.org/>, not the same value as the base 
URI, and not a same-document reference.

> How about <link href="./"/>?  I guess that would depend on what the
"==" is string equality in my pseudo-code.
I think <./> resolves to <http://www.tbray.org/ongoing/>, so that would 
be a same-document reference as well.

> values that consist entirely of a fragment are problematic?

The problematic values are the ones that when resolved are the same as 
the base URI (ignoring the fragment). In this case even <link 
href="http://www.tbray.org/ongoing/"/> would be problematic.

---

From: Sam Ruby <[email-masked]>
Date: Mon, 15 Aug 2005 12:33:02 -0400

> But this is equal (string-equal) to the base URI, so this is a

In the sentence above, you refer to "the base URI".  I would interpret
that as "the base URI of the document", not "the base URI of the element".

Note that the base URI of the document is not necessarily the same as
the base URI of the first element of the document (often referred to as
the documentElement).

> href="http://www.tbray.org/ongoing/"/> would be problematic.

My reading of this statement is that your feed also contains a
problematic link.  Issuing warnings on what is unambiguously a fully
qualified URI and on your usage are things I would rather avoid.

- Sam Ruby

---

From: Sjoerd Visscher <[email-masked]>
Date: Mon, 15 Aug 2005 18:40:00 +0200

> that as "the base URI of the document", not "the base URI of the element".

It has to the base URI of the element, otherwise links can change 
meaning when you XInclude them in another document.

> qualified URI and on your usage are things I would rather avoid.

Why?

---

From: Robert Sayre <[email-masked]>
Date: Mon, 15 Aug 2005 12:53:11 -0400

> meaning when you XInclude them in another document.

The implementors of Internet Explorer and Mozilla agree with Sam.

http://www.franklinmint.fm/2005/08/15/base.html

Robert Sayre

---

From: "A. Pagaltzis" <[email-masked]>
Date: Mon, 15 Aug 2005 19:39:57 +0200

> http://www.franklinmint.fm/2005/08/15/base.html

That uses html:base, which sets the base URI for the entire
document, not @xml:base, which sets the base URI for the element
and its children. Your example is irrelevant.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>

---

From: "Roger B." <[email-masked]>
Date: Mon, 15 Aug 2005 12:53:07 -0500

Sjoerd: While reading your linked post, I noticed this:

"Finally I'd like to share a trick to set the base URI for escaped
HTML in RSS and Atom: add a BASE element to the beginning of the HTML
content."

Maybe I shouldn't be, but I'd be surprised if that worked in too many
aggregators. After all the warnings (and platypus attacks) over the
years, I know that I strip pretty much any non-essential
markup/attributes from incoming RSS/Atom items, including html:base.

--
Roger Benningfield

---

From: Robert Sayre <[email-masked]>
Date: Mon, 15 Aug 2005 14:08:24 -0400

> and its children. Your example is irrelevant.

Oh no, not irrelevance! :)

Both seem to do the job described in section 5.1.1 of RFC 3986,  "Base
URI embedded in content". <http://rfc.net/rfc3986.html#s5.1.>

Just for fun, I made a version that uses xml:base instead. You can try
it with Firefox, which happens to support xml:base:
<http://www.franklinmint.fm/2005/08/15/base.xhtml>

Robert Sayre

---

From: Tim Bray <[email-masked]>
Date: Mon, 15 Aug 2005 12:25:11 -0700

>> developerWorks use xml:base is what Roy T. Fielding calls an abuse.

I disagree with Roy, but agree that the way my links were set up was  
a little surprising to the eye, so I tweaked.  Check out

http://feedvalidator.org/check.cgi?url=http%3A%2F%2Fwww.tbray.org% 
2Fongoing%2Fongoing.atom

  -Tim

---

From: Sjoerd Visscher <[email-masked]>
Date: Mon, 15 Aug 2005 23:01:40 +0200

> <http://www.franklinmint.fm/2005/08/15/base.xhtml>

Yes, it's a known bug.
https://bugzilla.mozilla.org/show_bug.cgi?id=275689

As a result of that bug, same-document references in documents with 
xml:base or html:base (like documents from the Google cache) don't work. 
Also content-location support is removed in Firefox because it broke 
same-document references (which hadn't be the case if same-document 
references would have been correctly implemented in the first place.)

---

From: Sjoerd Visscher <[email-masked]>
Date: Tue, 16 Aug 2005 00:09:35 +0200

Tim Bray wrote:
> I disagree with Roy, 

Shouldn't he be the one who knows? He actually he made it into an abuse 
by changing the definition of same-document references in the new rfc, 
so some documents that used the base URI in a way it was (apparently) 
never intended break. Read his whole argument if you want:
http://lists.w3.org/Archives/Public/uri/2004Jan/0009

It would be nice to know how bad he actually thinks the abuse is.

 > but agree that the way my links were set up was  a
> little surprising to the eye, so I tweaked.  Check out
> 

It would be really cool if you would move the xml:base of the entry to 
the div, but as you have 2 divs per entry I can imagine you don't want 
to do that. Or you could change the base URI to f.e. 
When/200x/2005/08/14/Java-Net-Terms.atom (even if that doesn't 
dereference to anything)

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 16 Aug 2005 00:27:19 +0200

>When/200x/2005/08/14/Java-Net-Terms.atom (even if that doesn't 

xml:base is not allowed on xhtml1:div elements.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Sjoerd Visscher <[email-masked]>
Date: Tue, 16 Aug 2005 00:45:04 +0200

> xml:base is not allowed on xhtml1:div elements.

Doh. The imperfections of the real world make it sometimes 
overwhelmingly disappointing.

---

From: Robert Sayre <[email-masked]>
Date: Tue, 16 Aug 2005 00:15:08 -0400

> https://bugzilla.mozilla.org/show_bug.cgi?id=275689

Well, it's not clear from that bug that any Mozilla committers feel
it's wise to "fix." Even so, they seem to be leaning towards patching
html:a as a special case, in the event that they do anything.

I'll agree that this should be a warning, and perhaps write something
that willcome back to haunt me--Atom makes a point of leaning many
established specifications. It's unwise to lean on those specs in the
corner cases where they diverge from existing implementations, and a
warning in the feedvalidator is justified in such cases.

> xml:base or html:base...

As long as we're picking on Tim's URI space... what happens in this
hypothetical situation:

<div xml:base="http://www.tbray.org/ongoing/hubble60.jpg">
<a href="stars.html">
<img longdesc="hubble60.html" 
        alt="stars" 
        src="http://www.tbray.org/ongoing/hubble60.jpg" />
</a>
</div>

Robert Sayre

---

From: "A. Pagaltzis" <[email-masked]>
Date: Tue, 16 Aug 2005 10:33:18 +0200

* Robert Sayre <[email-masked]> [2005-08-16 09:40]:
> As long as we're picking on Tim's URI space... what happens in
> this hypothetical situation:
> 

That’s the same as

    <div>
    <a href="stars.html">
            src="" />
    </a>
    </div>

ie the document author made a mistake.

Regards,

---

From: "A. Pagaltzis" <[email-masked]>
Date: Tue, 16 Aug 2005 10:35:26 +0200

* Robert Sayre <[email-masked]> [2005-08-15 20:15]:
> > document, not @xml:base, which sets the base URI for the
> > element and its children. Your example is irrelevant.
> 
> Both seem to do the job described in section 5.1.1 of RFC 3986,
> "Base URI embedded in content".
> <http://rfc.net/rfc3986.html#s5.1.>

Sorry, I misunderstood what you were trying to demonstrate
because I mixed it up with the discussion about @xml:base
providing a base URI for a subtree of a document only.

In other matters, what Sjoerd said.

Regards,

---

From: Sam Ruby <[email-masked]>
Date: Tue, 16 Aug 2005 10:15:03 -0400

> warning in the feedvalidator is justified in such cases.

What I see occurring here is implementations dealing with the attendent
confusions that their users face as the result of an unclear
specification of a corner case in a spec.

After all this discussion, I am *still* unclear as to where this corner
case is.

Let me try again.

Apparently, consuming tools are welcome to aggressively substitute
references to the enclosing parent document of any element for any
references that, when resolved according to xml:base, differ from that
xml:base only in ways that deal with normalization and fragment
identifiers.  This can only cause confusion if the xml:base in effect
differs from original xml:base of the document (i.e., the URI used to
retrieve the document in the first place) in ways other than the
fragment identifier.

Sjoerd: is this your understanding?  Note that I'm sidestepping all
questions about who is right or wrong.  Different implementations may
chose different strategies on how aggressive they wish to be.

If my understanding is correct, then I will agree that it is unwise for
feeds to express URIs that some consumers will interpret in one way, and
others will interpret in an other way; therefore a warning is warranted.

If my understanding is correct, I can produce test cases (and would
welcome any contributions in this area), and fix the feed validator.

The recommendations produced by the feed validator will focus on the
areas where the user is most likely to stumble into this problem.  It
seems to me that the largest problem area is at the feed level, and the
recommendation will be to either make xml:base at the feed level match
the URI from which the feed was loaded, or (paradoxically enough) to
reference a resource that you are unlikely to directly reference later
in the document.  Referencing a parent directory of any given document
is OK, what's important is that it isn't the document itself.

And to include an example that matches what Sjoerd's feed (and now Tim's
feed) now do.

Fair enough?

- Sam Ruby

---

From: Sjoerd Visscher <[email-masked]>
Date: Tue, 16 Aug 2005 19:11:47 +0200

> retrieve the document in the first place) in ways other than the

You've nailed it.

> Note that I'm sidestepping all questions about who is right or wrong. 

I totally agree, there is no right or wrong here. The established usage 
of a base URI is so different from what Roy is saying that he shouldn't 
have changed the RFC in such a way. (The RFC is even an Internet 
Standard, defined as "a specification that is stable and well-understood.")

> is OK, what's important is that it isn't the document itself.

Yes, although I wonder how you would test for "unlikely to directly 
reference later".

> Fair enough?

Totally, thanks!

---

From: Sam Ruby <[email-masked]>
Date: Tue, 16 Aug 2005 14:53:53 -0400

>> retrieve the document in the first place) in ways other than the

Cool.  Took me long enough.

> Standard, defined as "a specification that is stable and well-understood.")

I'm sure that we can find established usages where the consuming tool
has various degrees of "agressiveness".  Pure speculation on my part,
but now that I have been able to see how limited in scope this issue is,
I can see where spec authors might have had trouble outlawing one
behavior or another, and decided that publishing a spec within a
reasonable timeframe was of greater value than resolving this issue.

> Yes, although I wonder how you would test for "unlikely to directly

I will only test for actual references that meet the criteria described
at the top of this note.  When I encounter such a condition, I will emit
a warning containing a one line message accompanied by a link.  That
link will take the user to a web page that repeats the one line message,
provides an explation of that message (probably close to what I said at
the top of the page), provide a recommendation (probably close to what I
said in the second paragraph above this one), and a link to the
feedvalidator mailing list.

- Sam Ruby

---

From: Sam Ruby <[email-masked]>
Date: Sun, 21 Aug 2005 00:12:57 -0400

>> retrieve the document in the first place) in ways other than the

Sjoerd, I'd be interested in your comments on this:

http://tinyurl.com/9o6y2

- Sam Ruby

---

From: Sjoerd Visscher <[email-masked]>
Date: Sun, 21 Aug 2005 12:30:24 +0200

> Sjoerd, I'd be interested in your comments on this:

The explanation in the documentation[1] is perfect. And it says "As the 
current xml:base in effect does not match the URI of the document", but 
this is not the case in my feed, so I'm not sure why you report a warning?

Regarding the solution, my first suggestion would be to change the 
xml:base to reference the atom document, e.g.:

   <link href="." xml:base="http://example.com/blog/feed.atom" />

This is also more consistent with the explanation.

greetings,
Sjoerd

[1] http://www.feedvalidator.org/docs/warning/SameDocumentReference.html

---

From: Sam Ruby <[email-masked]>
Date: Sun, 21 Aug 2005 07:16:37 -0400

> this is not the case in my feed, so I'm not sure why you report a warning?

URI(doc) = http://www.w3future.com/weblog/rss.xml?notransform
xml:base = http://w3future.com/weblog/rss.xml?notransform

>   <link href="." xml:base="http://example.com/blog/feed.atom" />

... which would resolve to http://example.com/blog/.  Is that what we
want here?

- Sam Ruby

---

From: Sjoerd Visscher <[email-masked]>
Date: Sun, 21 Aug 2005 13:25:56 +0200

> xml:base = http://w3future.com/weblog/rss.xml?notransform

Ah, ok, I missed that. (Just to be sure, you added www yourself, or is 
there a link to the feed somewhere with www in it?)

> ... which would resolve to http://example.com/blog/.  Is that what we

The original in your example is

   <link href="." xml:base="http://example.com/blog/" />.

---

From: "A. Pagaltzis" <[email-masked]>
Date: Sun, 21 Aug 2005 14:13:22 +0200

* Sjoerd Visscher <[email-masked]> [2005-08-21 13:40]:
> Regarding the solution, my first suggestion would be to change
> the xml:base to reference the atom document, e.g.:
> 

Except it’s a @rel='self' link, so you really do want it to
resolve to <http://example.com/blog/feed.atom>.

@rel='self' is the one case where I still wonder how it should be
handled. To be helpful and correct, you will want to provide an
atom:feed/@xml:base for your feed, and likewise the conscientous
will want to provide an atom:link[@rel='self'] for the feed – now
regardless of *how* you write one or the other, if you use both
as intended by their specs, then the atom:link is inevitably
going to end up a same-document reference.

I wonder if that’s really harmful, though. After all, @rel='self'
is meant as the URL that should be used for *subscription* when
the aggregator does not know where the document came from. This
does not in itself imply dereferencing. (Semantics, semantics…)
Further, I expect that in practice, clients will fall back to
atom:feed/@xml:base when it exists but atom:link[@rel='self']
does not.

In fact, I’ve been wondering whether atom:feed/@xml:base doesn’t
obsolete the purpose of atom:link[@rel='self'], so that the
former should have been the SHOULD that the latter is, and the
latter not invented at all. It would seem that this notion would
also be less controversial when backported to RSS2, as opposed to
the item of including an atom:link in an RSS2 feed.

Regards,

---

From: Sjoerd Visscher <[email-masked]>
Date: Sun, 21 Aug 2005 14:42:28 +0200

> resolve to <http://example.com/blog/feed.atom>.

This was about the link in the solution in
http://www.feedvalidator.org/docs/warning/SameDocumentReference.html
which is not a self link.

> the item of including an atom:link in an RSS2 feed.

Not only does is port well to RSS2, it ports well to every single xml 
document. I think it is good practice, as it makes the infoset of the 
document context independent.

---

From: "A. Pagaltzis" <[email-masked]>
Date: Sun, 21 Aug 2005 15:04:06 +0200

> which is not a self link.

Sorry for not being clearer. I used the documentation example
because that was handier to reference, but I *was* talking about
the actual warning emitted.

What do you think about what I said? Is @rel='self' being a
same-document reference a problem?

One thing to note is that when retrieving the document from the
location @rel='self' refers to, as the external base URI for
atom:feed is the one which @rel='self' also references. So when
retrieving the feed from its preferred location, @rel='self' will
always implicitly be a same-document reference.

Regards,

---

From: Sjoerd Visscher <[email-masked]>
Date: Sun, 21 Aug 2005 15:38:12 +0200

> same-document reference a problem?

No. As long as xml:base is the same as the document URI.

> always implicitly be a same-document reference.

Yes, good point!

---

From: Sam Ruby <[email-masked]>
Date: Sun, 21 Aug 2005 13:26:58 -0400

> there a link to the feed somewhere with www in it?)

Your feed is available from both of the URI's mentioned above.  The
tinyurl quoted above is based on passing the first one to the feed
validator.

>   <link href="." xml:base="http://example.com/blog/" />.

Not being quite ready to drop the existing suggestion, I simply added
another.

- Sam Ruby

---

From: Sjoerd Visscher <[email-masked]>
Date: Sun, 21 Aug 2005 21:04:08 +0200

> tinyurl quoted above is based on passing the first one to the feed

Oh, this is actually interesting. I send a 301 when you use 
www.w3future.com. It looks like this is handled transparently by your 
http library, giving you the file at w3future.com. In that case it 
should be possible to request the actual uri that is used to get the file.

And then there's also the Content-Location header, which sets the base 
URI. This is used with content negotiation. F.e. (I haven't actually 
implemented this) if you would send a request to
   http://w3future.com/weblog/
with the header
   Accept: application/atom+xml
I could send you the atom file with this header:
   Content-Location: http://w3future.com/weblog/rss.xml?notransform

Afaik this is how the HTTP spec suggest to implement content 
negotiation. Then the value of Content-Location should be considered to 
be the uri of the document.

> http://www.feedvalidator.org/docs/warning/SameDocumentReference.html

That's what I meant by saying "first suggestion".

---

From: Sam Ruby <[email-masked]>
Date: Sun, 21 Aug 2005 19:21:10 -0400

> should be possible to request the actual uri that is used to get the file.

Fixed.

> be the uri of the document.

I now check for content-location too.

- Sam Ruby