Subject: PaceXmlBaseEverywhere

From: Mark Nottingham <[email-masked]>
Date: Fri, 2 Jul 2004 09:49:05 -0700

Doesn't this proposal effectively require xml:base to be considered in 
the processing of ALL atom elements, attributes and extensions? Whether 
or not the extension author wants it to be? Whether or not the 
processor is aware of the semantics of the extension?

Does is require schema information for both Atom as well as extensions?

Shouldn't module authors crisply specify when xml:base is and is not 
used, so that it's crystal-clear when it's to be used?

--
Mark Nottingham   Principal Technologist
Office of the CTO   BEA Systems

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Fri, 02 Jul 2004 19:15:41 +0200

> used, so that it's crystal-clear when it's to be used?

Just as with 'xml:lang'[1], I think it's obvious that 'xml:base' should be  
defined as a part of the Content Construct. Then, we don't need to go  
through the hoops and loops to say what elements it should and should not  
exist on, because that is implied with all the elements defined as Content  
Constructs.

____
[1] <url: http://www.intertwingly.net/wiki/pie/PaceLang>

-- 
Asbjørn Ulsberg         -=|=-        [email-masked]
«He's a loathsome offensive brute, yet I can't look away»

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 2 Jul 2004 12:57:17 -0700

On Jul 2, 2004, at 10:15 AM, Asbjørn Ulsberg wrote:

> Just as with 'xml:lang'[1], I think it's obvious that 'xml:base' 
> should be defined as a part of the Content Construct. Then, we don't 
> need to go through the hoops and loops to say what elements it should 
> and should not exist on, because that is implied with all the elements 
> defined as Content Constructs.

That sounds much more reasonable -- I'm only worried about saying that 
"everything" has xml:base processing applied, because it's so 
imprecise.

--
Mark Nottingham     http://www.mnot.net/

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Fri, 02 Jul 2004 22:28:24 +0200

On Fri, 2 Jul 2004 12:57:17 -0700, Mark Nottingham <[email-masked]> wrote:

>> Just as with 'xml:lang'[1], I think it's obvious that 'xml:base' should  
>> be defined as a part of the Content Construct.
>
> That sounds much more reasonable

Yup. We probably need an alternative Pace, then, won't we?

> I'm only worried about saying that "everything" has xml:base processing
> applied, because it's so imprecise.

I totally agree. I can't see why one would ever want 'xml:base' on  
atom:issued either.

---

From: Antone Roundy <[email-masked]>
Date: Fri, 2 Jul 2004 14:46:50 -0600

> defined as Content Constructs.
...and link.  And service (if PaceServiceElement is adopted).  I think 
we need to consider each element to be sure we don't accidentally 
forget any, and then list specifically which elements it applies to.  
If someone wants to make a wiki page to keep track of elements where 
we've decided that xml:base processing DOESN'T apply, that would save 
us periodically running through the whole list after new elements are 
added (but I don't think that list necessarily needs to be in the spec).

I think it most cases, it will be obvious where xml:base doesn't apply 
because most elements don't have URIs anywhere in them.  We only need 
clarification are places like:

* XHTML inside <content>
* HTML inside <content>
* if we use URIs to extend attributes like link/@rel, then there

---

From: Tim Bray <[email-masked]>
Date: Fri, 02 Jul 2004 14:45:55 -0700

On Jul 2, 2004, at 9:49 AM, Mark Nottingham wrote:

> the processing of ALL atom elements, attributes and extensions? 
> Whether or not the extension author wants it to be? Whether or not the 
> processor is aware of the semantics of the extension?uh

Yes, and I think that's the intent.  You set xml:base saying "this is 
the base URI of the feed" (or of the entry or whatever, depending) and 
that means people who put relative URIs into Atom feeds, unless they 
provide their own value for xml:base, are counting on whatever the 
relative URI happens to be.  And people who write and use extensions 
should be aware of this.

> Does is require schema information for both Atom as well as extensions?

Huh?  Don't understand.

> used, so that it's crystal-clear when it's to be used?

Any time you embed a relative URI in a resource representation, you 
either make it absolute or you rely on how that resource representation 
goes about making a base URI.  That's why the Pace is worded in terms 
relative to the RFCs that govern those issues.

Summary: I think the Pace is pretty OK as it stands. -Tim

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sat, 03 Jul 2004 00:09:35 +0200

On Fri, 2 Jul 2004 14:46:50 -0600, Antone Roundy <[email-masked]>  
wrote:

> ...and link.  And service (if PaceServiceElement is adopted).

Yes, true.

> I think we need to consider each element to be sure we don't accidentally

Correct. I think we agree that it shouldn't be defined as «everywhere», at  
least. Stick it into the correct constructs, and springle it around the  
other elements that needs it. Be explicit. Do we need a pace for each and  
every one of them, you think, or should we create PaceCommonAttributes or  
something?

> we've decided that xml:base processing DOESN'T apply, that would save us  
> periodically running through the whole list after new elements are added  
> (but I don't think that list necessarily needs to be in the spec).

It's just to remember that when you add an element, you loop through the  
list of common attributes and see if they fit on the element. If they do,  
specify it. This could be added as a reminder to section 7 in the format  
specification.

If we either go with a «don't use here» or «use here» list, it will need  
to be updated, and I think it's more intuitive to tell someone that they  
can use an attribute than that they can't. Not mentioning something should  
at least imply that it isn't available.

---

From: Dan Brickley <[email-masked]>
Date: Fri, 2 Jul 2004 18:24:46 -0400

* Tim Bray <[email-masked]> [2004-07-02 14:45-0700]
> >Does is require schema information for both Atom as well as extensions?

Is the idea that extension elements/attributes from other namespaces
might get deployed inside Atom, and that (without some presumably schema-carried
info) we won't know which element/attribute content carries (possibly
local) URI references? Nor which of those are to be interpreted in the
light of the current xml:base. 

Dan

---

From: Eric Scheid <[email-masked]>
Date: Sat, 03 Jul 2004 09:48:05 +1000

> I totally agree. I can't see why one would ever want 'xml:base' on

<issued somens:href="foo.dat">datestring</issued>

??

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 2 Jul 2004 17:06:19 -0700

> light of the current xml:base.

Exactly.

If you want your infrastructure to apply XML Base for you, it has to 
know where to apply it. In a container format like Atom, that doesn't 
work very well, unless you have good Schema information.

So, you end up needing to apply XML Base on a case-by-case basis. Since 
you're already applying it at the level of the module, rather than in 
the XML infrastructure, it seems odd to require all Atom modules for 
all time to use XML Base (which is effectively what this does).

I'm not strongly against it, just seems like an unnecessary constraint 
upon modules, especially since they're going to need to remind 
developers of the need to consider XML Base (when they use it) anyway.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sat, 03 Jul 2004 22:03:31 +0200

> work very well, unless you have good Schema information.

But coming to think of it; how often is really 'base' used? And how often  
is it realisticly to assume it will be used in Atom? Is the base going to  
change for each and every element in the feed? Is there any aggregator in  
existance that could cope with something like this:

   <?xml version="1.0" encoding="utf-8"?>
   <feed version="0.3" xmlns="http://purl.org/atom/ns#"
     xmlns:my="http://example.com/my#namespace"
     xmlns:date="http://date.example.com/date#namespace"
     xml:base="http://example.com/">

     <title>Asbjørn Ulsberg's weblog</title>

     <link rel="alternate" type="text/html"
      xml:base="http://example.org/" href="asbjornu" />

     <modified xml:base="http://date.example.com/
       date:href="modified">2003-12-13T18:30:02Z</modified>

     <author xml:base="http://example.net/" my:href="asbjornu">
       <name>Asbjørn Ulsberg</name>
     </author>

     <entry>
       <title>XML base example</title>
       <id>tag:example.org,2004:6.2397</id>
       <issued>2003-12-13T08:29:29-04:00</issued>
       <modified>2003-12-13T18:30:02Z</modified>

       <content type="application/xhtml+xml" mode="xml"
         xml:base="http://example.org/articles/">
         <div xmlns="http://www.w3.org/1999/xhtml"
           <h1><a href="62397.html">XML base example</a></h1>
         </div>
       </content>
     </entry>
   </feed>

This is probably a bad example, but hopefully it shows how meaningless  
«xml:base everywhere» is. I can understand the need to change the URI base  
for each entry, and maybe for each content element, but for each and every  
element in the feed?

---

From: Ryan Tomayko <[email-masked]>
Date: Sun, 4 Jul 2004 00:23:52 -0400

> existance that could cope with something like this:

I think so. I've always thought about xml:base much like I think about
xml namespaces. Would you consider specifying where in an atom
document namespaces can be defined?

In fact, I was under the impression that "supporting xml:base" meant
supporting it everywhere in a document. I didn't know it was legal to
support it only on certain elements. Running over the recommendation
[1] quickly doesn't seem to confirm or reject this point of view. Does
anyone know for sure whether it is legal to use xml:base only on
specific elements?

Also, I must be misunderstanding how "infrastructure to apply XML Base
for you" is perceived to work. Processors that support xml:base
natively (libxml2 for example) make obtaining the base URI for a given
element trivial and, to my knowledge, assume xml:base everywhere. But
all these processors do is keep track of the base URI, they do not
actually change any data in the document. It makes the base URI for
any element available so that it may be retrieved by the application.
I get the feeling we're expecting some infrastructure to automatically
"absolutize" URIs for us. I'm not sure I've ever seen a tool capable
of that because it would require a schema or some other augmentation
to specify which items in a document are URIs. Processors make the
base URI available given an element, it's the role of the code
resolving the URI to use this information when retrieving URIs.

I would also like to note that I've implemented xml:base generically
on top of SAX, Pull, and DOM based parsers without much trouble. The
only time I've had real problems with xml:base is within XSLT and
there only because there is no standard way of resolving URIs relative
to a base URI.

Ryan

[1] http://www.w3.org/TR/xmlbase/

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sun, 04 Jul 2004 14:26:21 +0200

> document namespaces can be defined?

No, of course not.

> support it only on certain elements.

Afaik (I'm no XSD champion) you need to explicitly specify on what  
elements you are to allow xml:lang, xml:base and so on in XSD. That's some  
part of the problem. So not being explicit in the specification about this  
means that we can't be explicit in XSD schemas either, which again means  
xml:base or xml:lang can't be used «everywhere».

If xml:lang and xml:base is to be used «everywhere», then this needs to go  
into the specification. I think «everywhere» is a bit too much, and think  
Content Constructs and maybe a couple of other elements are more than  
enough. But no matter what elements these attributes are allowed on, it  
needs to be stated explicitly. And the XSD needs to implement this just as  
explicitly (I presume the same goes for RNG).

> Running over the recommendation[1] quickly doesn't seem to confirm
> or reject this point of view. Does anyone know for sure whether it is
> legal to use xml:base only on specific elements?

Not including validation and XSD into the equation; yes, xml:base is  
allowed everywhere. The XML specification and its friends are independent  
of the schema languages, after all. But when you begin to talk about  
validation, you need to be more explicit.
> element trivial and, to my knowledge, assume xml:base everywhere.

Ok, cool. I have really no idea what libraries support xml:base natively.  
Could Dare tell us if System.Xml and MSXML supports xml:base natively, as  
libxml2?

> But all these processors do is keep track of the base URI, they do not
> actually change any data in the document. It makes the base URI for any
> element available so that it may be retrieved by the application.

How is the URI base retrieved? Does each node of a DOM document, or the  
context of a SAX based parser have a .base property or similar that at any  
given time contains the base of the element in scope?

> "absolutize" URIs for us.

Well, that's the hard part of this, imho. Just have a look at how  
amazingly difficult this is in HTML:

<url: http://diveintomark.org/archives/2004/01/02/relative-uris>

I'm not saying it's equally hard in Atom, but it's sure hard enough. I  
think we will see a lot of xml:base-bugs in Atom implementations.

> I'm not sure I've ever seen a tool capable of that because it would
> require a schema or some other augmentation to specify which items
> in a document are URIs.

Atom will hopefully have a normative schema, but I've raised the question  
about having 'href' in no namespace at all, while having 'xml:base' in the  
XML namespace. Maybe 'xlink:href' would be more appropriate.

> Processors make the base URI available given an element, it's the role
> of the code resolving the URI to use this information when retrieving
> URIs.

Yes, I agree, but it's resolving the URI that's difficult.

> on top of SAX, Pull, and DOM based parsers without much trouble.

Maybe you could write something up on the Wiki or somewhere else on how it  
should be done? A step by step explanation of what to consider each time  
you meet a new xml:base attribute? That would have been really nice. The  
Atom specification can never have too many attached tutorials or examples.

---

From: Mark Pilgrim <[email-masked]>
Date: Sun, 4 Jul 2004 09:33:46 -0400

> think we will see a lot of xml:base-bugs in Atom implementations.

A bug is a test case we haven't written yet.

http://intertwingly.net/wiki/pie/ConformanceTests
http://diveintomark.org/tests/client/base/

Meanwhile, my own Universal Feed Parser passes all of these test cases:

http://feedparser.org/tests/wellformed/base/

Note that some of UFP's cases test conditions that are unsupported by
spec text, and indeed test formats which do not mention XML:Base at
all.  I do not wish to discuss what to do with such non-Atom specs,
but merely to point out that comprehensive test suites are possible.

> Atom specification can never have too many attached tutorials or examples.

The XML:Base specification is short and surprisingly easy to read, if
you like that sort of thing.

http://www.w3.org/TR/xmlbase/

-- 
Cheers,
-Mark

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sun, 04 Jul 2004 15:51:59 +0200

> A bug is a test case we haven't written yet.

True.

> http://feedparser.org/tests/wellformed/base/

Goodie. I guess this is no big concern, then. Great! :-)

> you like that sort of thing.

Yes, it is not a hard read, fortunately.

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 4 Jul 2004 08:29:35 -0700

Well, that's true of any general mechanism; it can be abused if people 
put their minds to it.

In terms of code, I think the most obtrustive impact of "XML Base 
everywhere" is that, because it can inherit from above but not be 
processed in a generic way, implementations will be required to pass 
XML base context along to extension handlers, and those handlers will 
have to know how to deal with it and compute their own internal base 
URI.

This could be as simple as passing it the parse handler, which will 
contain the context itself, but that depends on the feed parser's 
architecture.

On Jul 3, 2004, at 1:03 PM, Asbjørn Ulsberg wrote:

> But coming to think of it; how often is really 'base' used? And how 
> often is it realisticly to assume it will be used in Atom? Is the base 
> going to change for each and every element in the feed? Is there any 
> aggregator in existance that could cope with something like this:
>
> «xml:base everywhere» is. I can understand the need to change the URI 
> base for each entry, and maybe for each content element, but for each 
> and every element in the feed?

---

From: Ryan Tomayko <[email-masked]>
Date: Sun, 4 Jul 2004 16:16:55 -0400

> given time contains the base of the element in scope?

Yep, that's pretty much it. The libxml2 C interface has an
xmlNodeGetBase function [1] that takes a pointer to a Document and
Node and gives you back the base URI. The Python for libxml2 bindings
have a wrapper for this as well [2].

Probably the best place to look at how DOM and DOMlike APIs do, or
will, implement this is the the recently completed DOM Level 3
Recommendation [3], which adds support for XML Base to standard DOM
implementations:

  This version enhances DOM Level 2 Core by completing the 
  mapping between DOM and the XML Information Set [XML 
  Information Set], including the support for XML Base [XML Base], 
  ...

Especially Section 1.3.4 "Base URIs" [4], which confirms your guess of
a ".base property":

  The DOM Level 3 adds support for the [base URI] property defined 
  in [XML Information Set] by providing a new attribute on the Node 
  interface that exposes this information. However, unlike the 
  Node.namespaceURI attribute, the Node.baseURI attribute is not a 
  static piece of information that every node carries. Instead, it is a value 
  that is dynamically computed according to [XML Base]. 

To my earlier point, I don't think there's any way to tell DOM Level 3
parsers to only process xml:base for certain elements. Perhaps Mike
Champion can tell us a little about how xml:base was approached the
way it was.

> you meet a new xml:base attribute?

The XML Base Recommendation does a great job of explaining the rules.
As for implementation details, here's a quick overview of how I
attacked base processing for different processing models:

Tree based APIs (like DOM) are probably the easiest to implement this
for. Obtaining the base uri for any node consists of walking the
ancestor-or-self sequence looking for xml:base attributes. If the API
supports XPath, this is usually a simple matter of evaluating the
following expression: ancestor-or-self::*/@xml:base and running
through the nodeset in reverse.

If you're using SAX, I suggest creating an XMLFilter that maintains a
stack of base URIs. The filter looks for xml:base attributes during
startElement and pushes their value onto the stack. When the
corresponding endElement method is called, the URI is popped off the
stack. If you then start from the bottom of the stack and move up, you
can obtain an absolute base uri for the current event at any time
during the parse.

Implementing on top of Pull Based APIs is similar to implementing on
top of SAX.

Ryan

[1] http://xmlsoft.org/html/libxml-tree.html#xmlNodeGetBase
[2] http://pyxmlsec.labs.libre-entreprise.org/docs/html/libxml2.xmlNode-class.html#getBase
[3] http://www.w3.org/TR/2004/REC-DOM-Level-3-Core-20040407/
[4] http://www.w3.org/TR/2004/REC-DOM-Level-3-Core-20040407/core.html#baseURIs-Considerations

---

From: Dare Obasanjo <[email-masked]>
Date: Sun, 4 Jul 2004 14:10:01 -0700 (PDT)

> xml:base or xml:lang can't be used «everywhere».

There's nothing stopping one from specifying that
every element can have xml:base and xml:lang in the
schema. 

> Could Dare tell us if System.Xml and MSXML supports

Nope. The xml:base spec was produced after the last
major coding work was done on either System.Xml or
MSXML in their most recent major releases. Since There
hasn't really been much [if any] demand for xml:base
support in our APIs which is unsurprising since the
primary users of xml:base in the wild (XHTML &
XInclude) aren't getting that much usage from the
average XML developer.  

The next version of System.Xml will not have xml:base
support but will fix the bug where the XSD validator
complains about xml:base attributes being undefined.
This means that currently if you try to validate an
XML document that uses xml:base using the
System.Xml.XmlValidatingReader class you will get
validation errors using current versions of the .NET
Framework. 

I don't think MSXML has any plans to add xml:base
support in a future version but will double check with
the PM responsible for it when she gets back from
vacation. 

> XML namespace. Maybe 'xlink:href' would be more

I don't see why this is a problem. Most XML formats do
not have attributes in a namespace unless they are
global attributes which can appear elements from
different namespaces.  

=====
THINGS TO DO IF I BECOME AN EVIL OVERLORD #95
My dungeon will have its own qualified medical staff complete with bodyguards. That way if a prisoner becomes sick and his cellmate tells the guard it's an emergency, the guard will fetch a trauma team instead of opening up the cell for a look.

		
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - 100MB free storage!
http://promotions.yahoo.com/new_mail

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sun, 04 Jul 2004 23:39:39 +0200

On Sun, 4 Jul 2004 14:10:01 -0700 (PDT), Dare Obasanjo <[email-masked]>  
wrote:

> There's nothing stopping one from specifying that every element
> can have xml:base and xml:lang in the schema.

No, but you have to explicitly allow it, no? If you don't assert that an  
element can contain a 'xml:base' attribute in the schema, can it contain  
the attribute nonetheless?

>> Could Dare tell us if System.Xml and MSXML supports xml:base
>> natively, as libxml2?
>
> Nope. The xml:base spec was produced after the last major
> coding work was done on either System.Xml or MSXML in their
> most recent major releases.

Ok.

> Since There hasn't really been much [if any] demand for xml:base
> support in our APIs which is unsurprising since the primary
> users of xml:base in the wild (XHTML & XInclude) aren't getting
> that much usage from the average XML developer.

True. That might change now that Atom and other vocabularies utilize  
'xml:base', though.

> The next version of System.Xml will not have xml:base support but
> will fix the bug where the XSD validator complains about xml:base
> attributes being undefined.

What is this bug, really? Is it so that an XSD should not really need to  
explicitly state where 'xml:base' (or 'xml:*' really) is allowed, but now  
you have to in order to make the XSD work with System.Xml?

> This means that currently if you try to validate an XML document
> that uses xml:base using the System.Xml.XmlValidatingReader class
> you will get validation errors using current versions of the .NET
> Framework.

Right.

> I don't think MSXML has any plans to add xml:base support in a
> future version but will double check with the PM responsible for it
> when she gets back from vacation.

Cool. Support for xml:base (and xml:lang, for that matter) in the XML  
libraries would help Atom development a great deal.

>> I've raised the question about having 'href' in no namespace at all,
>> while having 'xml:base' in the XML namespace. Maybe 'xlink:href'
>> would be more appropriate.
>
> I don't see why this is a problem. Most XML formats do not have
> attributes in a namespace unless they are global attributes which
> can appear elements from different namespaces.

Yes, well, I guess there isn't much of a problem. I just think it's  
inconsistent that @href belongs to Atom, while the attribute controlling  
@href's behaviour in respect to relative URI's is in the XML namespace. No  
biggie.

---

From: Dare Obasanjo <[email-masked]>
Date: Sun, 4 Jul 2004 17:04:26 -0700 (PDT)

--- Asbjørn_Ulsberg <[email-masked]> wrote:
> every element

This should lead to an error during validation because
no xml:base attribute is defined for the element in
the schema. 

> @href's behaviour in respect to relative URI's is in

Then you are raising a fundamental problem with XML as
a whole since this is how xml:base, xml:space and
xml:lang all work. 

=====

__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
http://promotions.yahoo.com/new_mail