Subject: Some thoughts on XCAP's resource architecture

From: Lisa Dusseault <[email-masked]>
Date: Sun, 21 Nov 2004 18:58:47 -0800

During the DC IETF, I expressed some reservations about XCAP to Ted and 
Jonathan. Jonathan asked me to send a message to the SIMPLE list with 
my comments, so here it is...

Based on the mailing list on the traffic, it appears that  XCAP is 
supposed to be an extension or profile of HTTP, rather than just a 
protocol that mimics the HTTP interaction style, and that as such it is 
intended to be compatible with other extensions of HTTP.  I'm concerned 
that the current architecture of XCAP makes this difficult.  In 
particular the XCAP resource ontology and the URL addressing style that 
goes with it shifts the HTTP design along two major axes:

1) Resource granularity
2) Dependency between resource

The first shift is in size and number of resources.  Because the path 
part of the URL allows for XML node selection, there are many more 
resources for a given volume of content material.  This affects us in a 
number of ways.

1a) HTTP intermediaries and scaling: caches aren't designed to cache 
huge numbers of tiny resources.  It would probably be wise to disable 
caching on XCAP responses.

1b) HTTP servers aren't designed for that many small resources.  
There's a certain amount of overhead to maintaining the metadata (at a 
minimum, the ETag) for so many small resources.  An HTTP server might 
have to be rearchitected to do a scalable job of supporting XCAP, which 
increases the XCAP implementation costs in the long run.

1c) Performance: HTTP is designed to batch requests in a certain way 
based on the granularity assumptions.  Recall that latency is a much 
bigger problem than bandwidth above a certain (low) bandwidth, and in 
modern Internet applications it's usually the latency that kills you.  
A more granular approach to resources doesn't in itself kill 
performance but it does if you stay with HTTP's request granularity.  
What XCAP is saving in bandwidth it will lose, in many use cases, in 
latency costs.

1d) Extensions to HTTP have also been designed with HTTP's current 
granularity in mind.  RFC2518, RFC3253, RFC3229, RFC3744 all extend 
HTTP in useful ways, and they're all written with the assumption that 
the granularity of resources is pretty much what it is today.  Access 
control, in particular, has a lot of overhead per resource

2)  Dependencies:  HTTP servers are designed such that static resources 
are handled independently of each other. Their ETag management is 
stand-alone, the request and response handling and concurrency are 
designed for that independence.  By contrast, XCAP contemplates a large 
number of resources which really map to parts of the same underlying 
file.  As far as I can tell, that introduces dependencies between 
resources (for example that a PUT to one URL would require the ETag of 
another URL to change).

2a) HTTP implementation barriers.  The last HTTP server I developed 
would have to be rearchitected in several places to handle XCAP's 
interdependencies, work beyond what you'd expect from adding XCAP 
support.  Throughout the server, the code uses exact matching of URLs 
to figure out what to do -- not URL pattern matching. So for example:
  - The way ETags were generated and stored and changed would have to be 
thrown out because ETags were generated independently for every 
resource.
  - Since resources were independent, write requests for different 
resources could be handled concurrently with ease, but that would have 
to change.

2b) How interdependencies work within existing HTTP extensions: For 
one, somebody would have to write a specification for how the existing 
access control standard (RFC 3744) might work with XCAP.  Since XCAP 
can have two different URLs that point to the same underlying piece of 
data, what does it mean to apply certain access control settings to 
either or both of those URLs?

I haven't examined every feature of HTTP and its extensions to see how 
well it deals with interdependencies, but that's a sampling.

So, what to do? It doesn't seem to me that XCAP is going to go back to 
the drawing board (or needs to), but it would be sufficient for most of 
the above concerns to simply make the definition of "resource" stay 
with the root XML documents that XCAP deals with.  The existing 
extensions to HTTP work a lot better on that size resource.  Part of 
this change involves putting the XPATH-like part of the XCAP URL out of 
the path part of the URL.  It could go in a header or even in the URL 
after the query delimiter (the question mark).  There is a theoretical 
problem with using query parameters on HTTP URLs in PUT requests if 
write-through caches don't know how to handle those, but there aren't a 
lot of write-through caches and overall it's a smaller problem and less 
work to deal with.

Full disclosure: I'm partially responsible for the current design 
because I pointed out the write-through cache problem with a previous 
design that used query params in PUT URLs. Unfortunately, I think that 
on balance the problems with the current architecture are worse.

Lisa

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 22 Nov 2004 04:25:34 +0000

I am not sure I follow all of the concerns you are raising, and I could 
easily be missing some important point.

However, we have implemented an XCAP server using an off-the-shelf HTTP 
server as a starting point.  We did not have to do significant violence to 
the HTTP code to support XCAP, the XCAP granularity, or the XCAP 
dependency.  (The places we had dependency issues had nothing to do with 
XCAP, but rather were simply that sometimes setting configuration 
information has side effects.)

We probably could also handle it if the path were moved into the 
parameters.  Hawever, the earlier discussion indicated that would not be a 
good idea.  Moving the path into a different HTTP parameter would be a very 
bad idea from my perspective.  It would be tantamount to treating HTTP as a 
transport for a non-HTTP protocol.  The fact that I can currently use a 
standard browser to get XCAP information, and a browser with PUT support 
can be used to update information is a feature, and one we should try to 
preserve.

Yours,
Joel M. Halpern

At 09:58 PM 11/21/2004, Lisa Dusseault wrote:
>Jonathan. Jonathan asked me to send a message to the SIMPLE list with my 
>comments, so here it is...
>
>that the current architecture of XCAP makes this difficult.  In particular 
>the XCAP resource ontology and the URL addressing style that goes with it 
>shifts the HTTP design along two major axes:
>
>1) Resource granularity

---

From: Joe Gregorio <[email-masked]>
Date: Mon, 22 Nov 2004 18:25:14 +0000

> resources for a given volume of content material.  This affects us in a

<Lots of granularity arguments elided>

This is just silly, HTTP is used to serve up everything from
100MB mpegs to a myriad of spacer GIFs. There is no such
thing as a 'regular level of resource granularity' on the web.

> resources could be handled concurrently with ease, but that would have

You are talking about implementation details, a choice on the part of 
the server writer. If you choose wrong, you may have 
difficulties, nothing earth-shattering there.

    -joe

-- 
Joe Gregorio        http://bitworking.org

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 24 Nov 2004 14:58:40 +0000

I am not sure I follow the question here.  Apologies if I am missing the 
point in attempting to answer.

Whether the underlying data is stored in an XML document, an XML document 
plus indexing and related structures, a database, or something else 
entirely is a local implementation choice.  I expect it to be influenced by 
the amount of related data to be handled, and by other product 
issues.  (Similarly, with a web server, the question of whether resources 
are stored as HTML, stored as HTML fragments which are dynmically combined, 
fully dynamically generated, or some other mechanism is a local application 
dependent choice.)

As for extension, it depends upon the problem.  For what we are doing, 
extensions will occur only when the underlying system has new 
capabilities.  And we will then extend the XML Schema.  For some uses, 
being able to store uninterpretted but validated parts of the XML is 
important.  In those cases, the schema provides for extension, and the XCAP 
server ought to be built so it can handle the data.
Adding new features is going to mean adding capabilities to the parts that 
work with the server.  Whether that requires updating the "database" 
probably depends upon the exact underlying implementation.  The degree of 
change required has almost nothing to do with whether XCAP is used to 
communicate the configuration of the new features.

Yours,
Joel

At 09:07 PM 11/23/2004, Cullen Jennings wrote:
>In thinking about implementing something, it seems that XCAP would more or
>less require me to use a database not a file to manage this stuff. Perhaps
>someone like Joel or the Nokia folks who has actually implemented stuff can
>straighten me out. As the XCAP schemas get extended do I need to extend my
>DB schemas at the same time? Do I also need to extend server code to compute
>new things.

---

From: "Alex Rousskov" <[email-masked]>
Date: Wed, 24 Nov 2004 09:18:48 -0700

On Sun, 2004/11/21 (MST), <[email-masked]> wrote:

> the XCAP resource ontology and the URL addressing style that goes with  
> it shifts the HTTP design along two major axes:
>
> 1) Resource granularity

I disagree that HTTP defines some specific size and number of server  
resources (what you define as resource granularity). From HTTP point of  
view, URL paths are almost opaque. I agree that some server  
implementations are less suitable than others to support XCAP, but I do  
not see that as a showstopper. Will handling 1000 1-byte objects be as  
efficient as handling 1 1000-byte object with HTTP? No, of course not.  
However, handling 1000 1-byte objects may be efficient enough for a given  
application/environment. And if some proxy breaks while handling large  
number of small objects, that proxy is not HTTP compliant and should be  
fixed (to prevent DoS attacks and such).

I agree that HTTP assumes that resources are mostly independent. There are  
no HTTP mechanisms to, say, invalidate a large group of resources with a  
single response. However, individual applications and environments can  
deal with it. For example, Apache provides per-directory access controls.  
ICAP has ISTag header to invalidate all cached responses from a given ICAP  
server at once. These examples do not use HTTP features, but work fine on  
top of HTTP. Again, some existing server implementations would be less  
appropriate for supporting XCAP, but that should not be a showstopper for  
XCAP.

$0.02,

Alex.

---

From: Lisa Dusseault <[email-masked]>
Date: Wed, 24 Nov 2004 11:33:26 -0800

We're actually on the same page about granularity.  HTTP does not 
*define* a specific granularity, as you said (and as others have 
pointed out, many HTTP implementations are capable of handling a very 
small granularity).  However since HTTP is one of the most widely 
deployed protocol systems we have, where browsers and intermediaries 
interact with a wide variety of servers and not just one host server, 
the practice matters as well as the definition.  And the extensions 
that have been written to HTTP most definitely assume a larger 
granularity.

I thought of another way to describe the resource granularity problem.  
When we say that something is "An HTTP Resource", here's what we imply 
(particularly for static, authorable resources):
A resource can be queried for the current Entity Tag if the server 
supports ETags.
A resource has its own last-modified timestamp, and supports the 
If-Modified-Since and If-Unmodified conditional headers.
A resource has a Content-Type and a Content-Length, and may have an 
entity Digest (Content-MD5)
A resource may be cacheable.
You can ask an HTTP server what methods may be applied to a resource.
A resource may be downloadable in byte-ranges.
Hit-metering is typically measured per resource (see RFC2227 in 
particular)
A resource has a set of queryable properties, including 'getetag', 
'getlastmodified', 'creationdate' and 'getcontentype', if the server 
supports WebDAV.
A resource can be locked (with its own independent lock token, lock 
owner) if the server supports WebDAV level 2.  Each resource has its 
own lock info property.
A resource can be moved with MOVE or copied with COPY if the server 
supports WebDAV.
A resource supports the creation of user-defined properties if the 
server supports WebDAV.
A 'diff' from a previously downloaded copy of a resource can be 
obtained if the server supports RFC3229.
A resource may have a version history if the server supports RFC3253.
A resource may have a "working copy" in another location if the server 
supports RFC3253.
A resource may have a 'comment' property if the server supports RFC3253.
A resource may be checked out and checked in if the server supports 
RFC3253.
A resource may be given an ordering within a collection if the server 
supports RFC3648.
A resource has its own access control list if the server supports 
RFC3744 (with any number of principals named in the list)

Dynamic resources tend not to have all the same characteristics, but 
then they're not authorable.

So when I look at what a resource is across all these HTTP extensions 
and in HTTP itself, and what XCAP wants to do, it seems to me that more 
often than not the XML document is the resource, and the XML node 
should simply be a part of that resource.  We might think it would be 
nice to lock and add access control independently for every XML node 
but I don't think that will be manageable.  Certainly the per-node 
version history seems prohibitive while the ability to view past 
versions of the whole document (and revert to a past version) seems 
potentially useful.  That's what I mean by the granularity problem with 
extensions -- the choice of granularity for "what is a resource" has a 
lot of implications.

Lisa

On Nov 24, 2004, at 8:18 AM, Alex Rousskov wrote:

>> the XCAP resource ontology and the URL addressing style that goes 
>> with it shifts the HTTP design along two major axes:
>>
>> 1) Resource granularity
> resources (what you define as resource granularity). From HTTP point 
> of view, URL paths are almost opaque. I agree that some server 
> implementations are less suitable than others to support XCAP, but I 
> do not see that as a showstopper. Will handling 1000 1-byte objects be 
> as efficient as handling 1 1000-byte object with HTTP? No, of course 
> not. However, handling 1000 1-byte objects may be efficient enough for 
> a given application/environment. And if some proxy breaks while 
> handling large number of small objects, that proxy is not HTTP 
> compliant and should be fixed (to prevent DoS attacks and such).
>
> I agree that HTTP assumes that resources are mostly independent. There 
> are no HTTP mechanisms to, say, invalidate a large group of resources 
> with a single response. However, individual applications and 
> environments can deal with it. For example, Apache provides 
> per-directory access controls. ICAP has ISTag header to invalidate all 
> cached responses from a given ICAP server at once. These examples do 
> not use HTTP features, but work fine on top of HTTP. Again, some 
> existing server implementations would be less appropriate for 
> supporting XCAP, but that should not be a showstopper for XCAP.
>
> $0.02,
>
> Alex.
>

---

From: Lisa Dusseault <[email-masked]>
Date: Wed, 24 Nov 2004 11:49:11 -0800

> I'm not understanding here. How we can best design stuff to batch in a 
> way
> to have optimal performance.

If you have multiple changes to make to a 1 MB or smaller document, 
batch them up together if possible, even if it requires uploading the 
whole document afresh.  The current design of XCAP encourages changes 
to be made independently, and each change will require a full 
round-trip (no pipelining possible because you need to wait for the 
server to respond with an ETag each time).

Similarly, even though a whole document may be large, up to a MB or so 
it's probably better to download (synch) the whole document rather than 
do several roundtrips to get individual nodes.  Even though download 
requests can be pipelined not all client libraries support that yet.  
If you can pipeline, then you can do more node requests before the 
tradeoff of preferring to get the whole document kicks in.

The optimal granularity for performance is probably somewhere between 1 
kB and 10 MB for XCAP's use cases.  A single XML node is probably too 
small; an undivisible 10 MB file is probably too large.   Compare to 
byte ranges -- 10 MB files are often broken into byte ranges for 
reliable download, but the byte ranges aren't chosen to be too small or 
the performance cost would be greater than the reliability win.

My size estimates here are from intuition based on experience; but I'm 
confident in saying that HTTP is not ideal for ferrying around tiny 
things at a high performance  (just try using the MSDAIPP library to do 
client applications pretending that the HTTP server is a local 
database).  The overhead per request/response is not insignificant.

Since XCAP does allow addressable nodes to be grouped into larger 
addressable nodes it will be possible for a smart XCAP client to do 
some of this batching anyway, but many clients will do the simple and 
obvious thing regardless.  So I don't expect this to be the killer 
argument for changing XCAP -- if this were the only consideration, the 
answer would likely be to "let the implementor beware".  It's not the 
only consideration however -- I consider the implementability and 
interoperability of HTTP extensions to be the bigger consideration.

Lisa

---

From: Jeffrey Mogul <[email-masked]>
Date: Wed, 24 Nov 2004 11:54:06 -0800

So when I look at what a resource is across all these HTTP
    extensions and in HTTP itself, and what XCAP wants to do, it seems
    to me that more often than not the XML document is the resource,
    and the XML node should simply be a part of that resource.  We
    might think it would be nice to lock and add access control
    independently for every XML node but I don't think that will be
    manageable.  Certainly the per-node version history seems
    prohibitive while the ability to view past versions of the whole
    document (and revert to a past version) seems potentially useful.
    That's what I mean by the granularity problem with extensions --
    the choice of granularity for "what is a resource" has a lot of
    implications.

I admit that I haven't been following this discussion in great
depth.  But when I read this paragraph, I thought that maybe
the way to get at "node resources" is to treat them as a kind
of "Range".  Remember that the Range and Accept-Ranges headers
in RFC2616 are not necessarily limited to byte ranges, although
the spec is biased in that direction:

3.12 Range Units

   HTTP/1.1 allows a client to request that only part (a range of) the
   response entity be included within the response. HTTP/1.1 uses range
   units in the Range (section 14.35) and Content-Range (section 14.16)
   header fields. An entity can be broken down into subranges according
   to various structural units.

      range-unit       = bytes-unit | other-range-unit
      bytes-unit       = "bytes"
      other-range-unit = token

   The only range unit defined by HTTP/1.1 is "bytes". HTTP/1.1
   implementations MAY ignore ranges specified using other units.
   HTTP/1.1 has been designed to allow implementations of applications
   that do not depend on knowledge of ranges.

One would hope that any extant implementation of Accept-Range
is smart enough to ignore something *like*

	Accept-Ranges: xcap-nodes

which would allow you to define a new specification for "ranges"
of XCAP XML nodes that would be backwards compatible with HTTP/1.1.

Then (if you pardon me for my complete ignorance of XCAP details)
you could replace Lisa's example:

	http://xcap.example.com/services/resource-lists/users/bill/fr.xml/~~/ 
		resource-lists/list%5b@name=%22friends%22%5d/entry

with something like

	GET /services/resource-lists/users/bill/fr.xml/ HTTP/1.1
	Host: xcap.example.com
	Range: xcap-nodes="resource-lists/list%5b@name=%22friends%22%5d/entry"

with the obvious syntax if you wanted to get multiple nodes at
once.

Like I said, this could be totally naive.

-Jeff

---

From: "Alex Rousskov" <[email-masked]>
Date: Wed, 24 Nov 2004 13:29:54 -0700

On Wed, 2004/11/24 (MST), <[email-masked]> wrote:

> And the extensions that
> have been written to HTTP most definitely assume a larger granularity.

I am not sure that is true, let's go through your (edited) list :

> A resource can be queried for the current Entity Tag if ...
> A resource has its own last-modified timestampA resource has a  
> Content-Type and a Content-Length
> A resource may have an entity Digest (Content-MD5)

All of the above can still "work" when resource is a result of a query,  
IMO.
Some servers will support some of the above features for XCAP, some will  
not.

> Hit-metering is typically measured per resource

Hit metering (as specified in IETF) is dead, but would still "work" for  
XCAP with post-processing scripts merging individual hit counters, for  
example; and nothing prevents folks specifying a "more XCAP-friendly" hit  
metering.

> A resource .... if the server supports WebDAV.

I agree that editing some XCAP resources via WebDAV would be difficult,  
but I think it would be still technically possible. Just do not think of  
an XML document as a single file. Can you name a single WebDAV feature  
that would be impossible to implement for most XCAP resources?

> A 'diff' from a previously downloaded copy of a resource can be obtained  
> if ...
> A resource may have a version history if ...
> A resource may have a "working copy" in another location if ...
> A resource may have a 'comment' property if ...
> A resource may be checked out and checked in if ...
> A resource has its own access control list if ...

Again, all seem to be technically possible with XCAP resources, and not  
that difficult if XML storage model is more XML-friendly than a regular  
"file on disk".

> A resource may be given an ordering within a collection if the server  

Not sure about this one, but I think that some DTDs allow for any ordering  
of XML nodes inside an XML element, so perhaps the above is also  
applicable, in some environments.

> often than not the XML document is the resource, and the XML node should  
> simply be a part of that resource.

That view is one of the many valid views, IMO.

> We might think it would be nice to lock and add access control  
> manageable.

Why not? If XML document is a collection of individually-managed nodes, I  
do not see a problem. As a simple example, imagine a CGI script that  
assembles an XML document from 100 nodes, each stored in its own file. As  
a more flexible example, imagine an XML-friendly database that allows  
users to have user-defined attributes for every XML query result (which is  
nothing but a "view" in a database terminology).

> Certainly the per-node version history seems prohibitive

Why? If my XML document is a collection of client information nodes, I can  
write an interface that will show per-client history of changes.

> while the ability to view past versions of the whole document (and  
> revert to a past version) seems potentially useful.

One does not preclude the other, I think.

> That's what I mean by the granularity problem with extensions -- the  
> choice of granularity for "what is a resource" has a lot of implications.

It seems to me that you have an "XML document is a file" assumption that  
is not true in general.

Also, I am not sure we should attack one HTTP application based on what  
other HTTP applications or extensions can or cannot do. Each extension has  
its niche and they do not have to perfectly overlap all the time  
(otherwise, they should be included into the core protocol!).

Thanks,

Alex.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 24 Nov 2004 23:00:38 +0100

Lisa Dusseault wrote:
> *define* a specific granularity, as you said (and as others have pointed 
> out, many HTTP implementations are capable of handling a very small 
> granularity).  However since HTTP is one of the most widely deployed 
> protocol systems we have, where browsers and intermediaries interact 
> with a wide variety of servers and not just one host server, the 
> practice matters as well as the definition.  And the extensions that 

Nit: it's not the resource that has etags and such, it's the resource's 
variants. The assumption that each resource has only a single variant in 
general is incorrect.

 > ...

Best regards, Julian

-- 
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: Julian Reschke <[email-masked]>
Date: Wed, 24 Nov 2004 23:18:41 +0100

> caching on XCAP responses.

I'd call that an implementation issue, not a problem with the design as 
such. You can avoid it by making responses non-cacheable, ot you can 
kick the vendor of the intermediary to fix the product (for instance, by 
modifying it not to cache very small entitities).

> 1b) HTTP servers aren't designed for that many small resources.  There's 
> a certain amount of overhead to maintaining the metadata (at a minimum, 
> the ETag) for so many small resources.  An HTTP server might have to be 
> rearchitected to do a scalable job of supporting XCAP, which increases 
> the XCAP implementation costs in the long run.

<http://www.ietf.org/internet-drafts/draft-ietf-simple-xcap-05.txt>, 
Section 8.5 seems to indicate that all subordinate resources of the 
document share the same etag, so this wouldn't be a problem. For 
instance, if you would view a subversion server as an XCAP server, the 
global revision number of the repository could be used as an etag.

> modern Internet applications it's usually the latency that kills you.  A 
> more granular approach to resources doesn't in itself kill performance 
> but it does if you stay with HTTP's request granularity.  What XCAP is 
> saving in bandwidth it will lose, in many use cases, in latency costs.

But it enables the client to decide on it's own. If latenty proves to be 
a problem, it can still update larger chunks in single requests.

> granularity in mind.  RFC2518, RFC3253, RFC3229, RFC3744 all extend HTTP 
> in useful ways, and they're all written with the assumption that the 
> granularity of resources is pretty much what it is today.  Access 

I disagree that these extensions have been designed with some specific 
particular granularity in mind (having been active member of the WQebDAV 
WG for the last 3,5 years and being one of the authors of RFC3744).

> resources (for example that a PUT to one URL would require the ETag of 

Yes. But why is that a problem? It simply means that it's hard to adapt 
*specific* implementation for XCAP.

> support.  Throughout the server, the code uses exact matching of URLs to 
> figure out what to do -- not URL pattern matching. So for example:
> thrown out because ETags were generated independently for every resource.

That would IMHO be a problem anyway, because ETags *can't* work reliably 
if they are independant of each other if the server allows namespace 
operations.

> resources could be handled concurrently with ease, but that would have 

I think you're arguing from a very specific implementer's point of view. 
There are already WebDAV servers in place that handle XML in a similar 
way to XCAP (Slide/Tamino and (I think) Oracle10 come to mind).

> 2b) How interdependencies work within existing HTTP extensions: For one, 
> somebody would have to write a specification for how the existing access 
> control standard (RFC 3744) might work with XCAP.  Since XCAP can have 
> two different URLs that point to the same underlying piece of data, what 
> does it mean to apply certain access control settings to either or both 
> of those URLs?

If two URLs map to the same underlying piece of data, they identify the 
same resource. Thus they would have the same ACLs. What am I missing?

> the above concerns to simply make the definition of "resource" stay with 
> the root XML documents that XCAP deals with.  The existing extensions to 
> HTTP work a lot better on that size resource.  Part of this change 
> involves putting the XPATH-like part of the XCAP URL out of the path 
> part of the URL.  It could go in a header or even in the URL after the 
> query delimiter (the question mark).  There is a theoretical problem 
> with using query parameters on HTTP URLs in PUT requests if 

Putting the selector into the query part doesn't change anything it all. 
That would be merely a change in syntax with almost no impact in 
implementations (possibly except working around implementation problems 
in intermediaries).

Putting it into separate headers would make the document fragments 
non-resources (not even second-class resources), thus I fail to see how 
that would be an improvement. You would simple have to reinvent a lot of 
syntax to do something that HTTP was already doing for you.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 24 Nov 2004 23:28:28 +0100

Best regards, Julian

---

From: Lisa Dusseault <[email-masked]>
Date: Wed, 24 Nov 2004 16:17:53 -0800

> I agree that editing some XCAP resources via WebDAV would be 
> difficult, but I think it would be still technically possible. Just do 
> not think of an XML document as a single file. Can you name a single 
> WebDAV feature that would be impossible to implement for most XCAP 
> resources?

Nope, no impossibilities -- just difficulties and undesirabilities.

> Why not? If XML document is a collection of individually-managed 
> nodes, I do not see a problem. As a simple example, imagine a CGI 
> script that assembles an XML document from 100 nodes, each stored in 
> its own file. As a more flexible example, imagine an XML-friendly 
> database that allows users to have user-defined attributes for every 
> XML query result (which is nothing but a "view" in a database 
> terminology).

It's manageable from a coding point of view, I'm sure we're all 
excellent coders.  It's not manageable from an ease-of-use, application 
design and GUI point of view, nor does it scale as far.  I really can't 
think how a GUI client/user can possibly manage all the ACLs on all the 
node resources -- the complexity is enormous if it's not scoped down 
somehow, and if it's not scoped down now, then the natural diversity of 
implementations will make it much harder to scope down later.

Lisa

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 25 Nov 2004 00:20:50 +0000

> implementations (possibly except working around implementation problems 

No, actually there is an important reason why XCAP should be changed
to be after the query part.

Clearly, generic origin XCAP servers should not have to analyse and
transform the text of XML document - they should simply know how to
select portions of it based on the XCAP selector.

With the "/~~/" syntax, think about what happens when an XCAP resource
is fetched by non-XCAP aware software that is given an XCAP URL.  For
example, a web browser told to view a subtree of some XML document.

That is one of the most useful reasons why XCAP is operating using
HTTP URLs, after all.

Relative URLs appearing in the text of the subtree will be resolved by
the browser including components of the selector.

This is fine when the selector path represents other accessible
resources.  For example, when a selector is used to access resources
in a filesystem-like hierarchy, but in this case, why use XCAP at all?

However, imagine a browser is told to view some limited portion of an
XHTML document (or any other document format), which contains links to
other resources on the same server.  In this case, the URL is used to
restrict the view of the document to a portion of it.

When XCAP is used in this way, almost certainly you want URLs in the
text of the restricted view to resolve relative to the path _before_
the selector - the selector itself should not count in relative URL
resolution.

To fix this, thereby making XCAP more widely useful, why not use "??":

     - "??" seems likely to be unambiguous in practice

     - it's valid, and RFC1808 path resolvers will do the right thing

     - it can be appended to any existing path including those which
       already have a "?" query part, so select sub-resources of
       all existing resources

"??" would allow a generic XCAP server module to be written which
would serve a restricted view of _any_ resource that the server
already serves, automatically maintaining correct relative URLs in the
resource whatever its syntax.

For example, such a module could be written for Apache, and it would
automatically add a facility for fetching any resource that Apache is
able to serve now, but restricting the response to a portion of it -
while keeping those relative URLs working.

With the "/~~/" syntax, XCAP has the flaw (a big flaw IMHO) that a
generic XCAP module for restricting views cannot be written: such a
module would have to know how to transform relative URLs appearing in
the body, and that implies document-type-specific parsers.

To whom do I write to be sure this issue is properly considered?

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 25 Nov 2004 00:26:11 +0000

> implementations will make it much harder to scope down later.

Imagine an XML document which is a web page.

It seems quite reasonable to allow writing to selected portions of the
page, but not to other portions or the page as a whole.

That's a simple example of per-selector ACLs.

There is no way a client-side GUI can handle that.  ACLs of different
selectors are related.  In this case, a server might have logic which
says a selected portion of a document is writable _if and only if_ it
contains no read-only nodes.  Changing permission of portions of the
document would effectively modify the ACLs of other selectable
portions - in ways which a generic client cannot know about.

So the best a generic client GUI could do would be to try operations
on a selected portion to discover if they are permitted, or query for
the permitted operations on a portion before telling the user it's ok
to start editing this portion.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 25 Nov 2004 00:28:58 +0000

> server to respond with an ETag each time).

That seems like a flaw which should be fixed.

Maybe XCAP could take advantage of PATCH? :)

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Thu, 25 Nov 2004 08:45:15 +0100

> implementations will make it much harder to scope down later.

The difference is that in XCAP, you leave the decision about what 
granularity the ACLs will have to the server. It's perfectly ok if the 
server chooses to have just one for the whole document. But if it wishes 
to support a higher granularity, it can do that as well.

Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 25 Nov 2004 09:26:36 +0100

> That is one of the most useful reasons why XCAP is operating using

I tend to disagree.

a) That you can use a browser is a useful feature for debugging, but 
unlikely to be the primary way XCAP would be used,

b) As long as each addressable fragment *does* get it's own URI, every 
solution will work equally well in a browser.

> the browser including components of the selector.

...by any XML compliant recipient. Anyway, validaty of relative (URI) 
references is a problem, but a bigger one. Consider

<x xml:base="foo"><y/></x>

If you just address y, you'll loose the base URI information. So to make 
this work, an XCAP server would need to know about base URIs and 
xml:base anyway.

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 26 Nov 2004 07:59:25 +0000

> solution will work equally well in a browser.

I wasn't really thinking about normal web browsers, but rather clients
which can fetch resources from XML URIs and do useful things with
them, including following the URIs within the resources.

I'm thinking that the selectors work rather like the Range header in
some ways, but because they're in the URL it's reasonable to pass them
around among different programs.  For example, program A computes the
selector for a certain kind of XCAP query, and passes it to program B
which can retrieve the query result, which might contain references to
other resources (or even other query results).

> this work, an XCAP server would need to know about base URIs and 

xml:base at the top-most tag is essentially document metadata.  (I
don't know about xml:base - is it allowed to appear anywhere?)

Other things such as the charset in the <?xml?> declaration also need
to be passed.  Any XCAP server will have a mechanism for extracting
and passing document metadata along with the response, and xml:base or
HTML's <base/> are just another part of that.

My main point is that when an XML document contains relative URIs,
those are relative to the _document as a whole_.

It's a conceptual thing.

The concept with XCAP is to address portions of an XML document.  The
XML rules and general practice mean that relative URIs appearing in
those portions are still _supposed_ to be relative to the whole document.

Therefore _conceptually_, the XCAP selectors should not change the
base path for relative URI resolution.

As it is now, an XCAP server would have to parse and modify the entire
selected portion to transform every relative URI, not just the single
base URI.

(Alternatively it could add its own xml:base to the served response,
but that might not be so reliably understood by recipients.)

This is because of the mismatch between the two concepts.

If XML URIs normally refered to subtrees in the XML, there would be no
mismatch, but they don't so there is.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Fri, 26 Nov 2004 10:11:39 +0100

> don't know about xml:base - is it allowed to appear anywhere?)

Yes.

> to be passed.  Any XCAP server will have a mechanism for extracting

Not necessarily.  A server could serve all fragments encoded in UTF-8 
and would never have to send an XML declaration.  Strictly speaking, it 
wouldn't need to send it as well if it's specified in the Content-Type 
response header.

> those are relative to the _document as a whole_.

That's not true. The relative references need to get resolved to 
whatever the *current* base URI is. The base URI may change due to 
entity inclusion and xml:base processing.

> those portions are still _supposed_ to be relative to the whole document.

No, that's incorrect.

> base path for relative URI resolution.

Yes, but as demonstrated, that's fixable.

> selected portion to transform every relative URI, not just the single

No, that's incorrect.

> This is because of the mismatch between the two concepts.

That's what I already proposed (I think). Just clarify it in the spec, 
and there should be a problem with it.

Best regards, Julian

---

From: KAWAGUTI Ginga <[email-masked]>
Date: Sun, 28 Nov 2004 01:27:36 +0000

In Sun, Nov 21, 2004 at 06:58:47PM -0800,
> goes with it shifts the HTTP design along two major axes:

..snip..

> lot of write-through caches and overall it's a smaller problem and less 

I agree with this in general.
Also, there might be some other anxiety, which is encoding.

XCAP's target document is usually written in UTF-8(it's XML),
so XCAP requires Xpath stuff(utf-8) into URL part, which is usually
considered as US-ASCII.
There are ways to encode utf-8 string into URL part, but
I don't think there's any unique, robust and common way for doing it.

If that argument was in body part(header might also be recepted),
this consideration is much less problem, because there are ways to
declare encodings, and usual gateways/clients are aware of them.

(People who lives in us-ascii world, or even iso-8859-* world,
 might not realize what these problems are...)
-- 
Office:  NTT Communications Innovative IP Architecture Center 1PT 1P
Address: KAWAGUTI Ginga <[email-masked]>
Phone:   voice=+81-3-6800-3032; fax=+81-3-5388-0550

---

From: Julian Reschke <[email-masked]>
Date: Sun, 28 Nov 2004 12:33:46 +0100

> XCAP's target document is usually written in UTF-8(it's XML),

Unless XCAP says something normative, it could be any encoding. As far 
as I can tell, the actual decoding doesn't matter at all as long as it 
is one of the XML required ones (thus UTF-8 or UTF-16).

> I don't think there's any unique, robust and common way for doing it.

No matter what encoding the document uses, XCAP needs to define that 
mapping (and yes, that mapping needs to be based on UTF-8).

> declare encodings, and usual gateways/clients are aware of them.

Don't understand. How are you sending the document selector inside the 
request body for PUT?

>  might not realize what these problems are...)

Oh yes, I do :-)

Best regards, Julian