Subject: Re: PaceIntrospection: Is another file format really needed?

From: Beecher Greenman <[email-masked]>
Date: Thu, 01 Jul 2004 18:07:08 -0400

>> On Thu, Jul 01, 2004 at 03:35:34PM -0000, 
kwark.1511609@xxxxxxxxxxxxx wrote:
 >>
 >>> > But I do have an answer
 >>> > to your question: "What does the PaceIntrospection stop you from 
doing, exactly?"
 >>> >
 >>> > The proposed introspection format does not allow me to subscribe 
to the
 >>> > introspection file containing my blog sites, with any of the 
current aggregators.
 >>> >  ;-)
 >>
 >> I've only just read PaceIntrospection, because I didn't think it would
 >> be that interesting or contentious, and I don't really have the right
 >> background to contribute without thinking much (:-). But now I have
 >> and I think that this entire argument is bogus, and due to
 >> a misconception of what the introspection (at least as it was when
 >> originally debated pre-IETF) was for.

I have seen these original drafts. Although the name "introspection" 
might be poorly given, the concept behind PaceIntrospection -the ability 
to find out the sites for which a given user is responsible- remains 
something for which there is a real-world need.

It is true that if either PaceIntrospection or my feed-based proposal is 
used along with the current spec as written, a certain amount of data is 
duplicated: namely, the links to discover each service's API points. 
PaceIntrospection and my feed-based introspection proposal both call for 
these to be included in the introspection file via <link> tags, and the 
current specification calls for these links to be included in the 
service's main HTML, again via <link> tags.

At this point in time -just judging from the requirements of 
PaceIntrospection, which I understand are not agreed upon yet- any 
service being introspected will have between one and four <link> tags in 
the introspection file (again, whichever format is being used):
   - Almost all services should have a <link rel="alternate" />, which 
points to the service itself. Without a link to the service, there is 
little point in having it in an introspection file in the first place.
   - Most services -those which can be syndicated, including sites, 
articles within sites, and such- should have a <link rel="service.feed" 
/> tag. The only example I can think of at the moment which might not 
have this tag, actually, would be a comment within an article on a site.
   - Those services to which "entries" can be added should have a <link 
rel="service.post" /> tag. This might include a site where entries 
represent articles, or an article within a site, where entries represent 
comments. Theoretically this could be extended to a user where entries 
represent sites, allowing Atom to be used to create or delete new sites 
-Weblogs, for example- but that's outside the scope of these proposals.
   - Those services which can be edited directly should have a <link 
rel="service.edit" /> tag. This would include articles within a site or 
comments within an article. Theoretically this could be extended to 
templates within a site as well. I don't immediately see any use for 
directly editing a site itself, but I see no reason to exclude it 
either; no doubt someone will come up with a use I haven't thought of.

Of these, up to three tags might be duplicated in the HTML (or other 
applicable markup language) of the service itself. The first one on the 
list, which points to the service itself, doesn't make sense for 
duplication in the service, since the user-agent is by definition 
already pointing there. This seems to be the kind of introspection that 
you're talking about, such that a user-agent could find a site's 
service.feed, service.edit, and service.post links by looking at the 
service itself.

 >> You introspect a feed to find its service API points.

You don't introspect a feed at all, because a feed is not a "thing" to 
be introspected. You could introspect sites, entries, users, or whatever 
else, but a feed is a representation of something, not the thing itself. 
What exactly a feed can represent is at the core of this debate, but I 
don't think there's any argument that a feed is an actual object instead 
of a representation.

 >> PaceIntrospection seems to want to provide a single file that allows
 >> you to find out information about /several/ feeds in one go. In my
 >> view that (a) shouldn't be called introspection; (b) shouldn't
 >> actually happen. (It doesn't help that it reminds me a little of P3P,
 >> which is always guaranteed to make me shudder.)

You're not finding out information about several feeds in one go; you're 
finding out information about several services in one go. A feed 
represents a service but it is not the service itself.

Anyway, an honest question, first: why should this not be called 
introspection? Although the user is performing it against an author 
rather than a service, the principle seems to still apply.

Second: why should this not happen? There is certainly a real need to 
list the services for which a user is responsible; this necessity is at 
the core of both PaceIntrospection and my feed-based introspection 
proposal. A common use case comes to mind: a publishing tool where a 
user logs in and is presented with a list of his services, from which he 
may then choose one or several to update. Should this process also 
reveal information on each service's capabilities? This is, admittedly, 
debatable.

If we decide that capabilities should not be revealed in such a list, 
then the <link rel="service.xxxxx" /> links could be cut out of the 
files (the <link rel="alternate" /> links would need to remain). In that 
case, then things seem much more appropriate to my feed-based 
"introspection" proposal, though you're right that it shouldn't really 
be called introspection at that point. If, however, we decide that 
capabilities should be revealed in such a list, then the question of 
which proposal is more appropriate becomes less obvious. I believe that 
my proposal is still the more appropriate choice, as it takes advantage 
of the power of an existing Atom technology rather than reinventing the 
wheel, but I can see where my opposition might not believe this.

For the record, I believe that revealing capability introspection in a 
service list would be highly useful. Although there are use cases for a 
user-agent which simply lists the services and presents a user with Web 
links to each one, most user-agents are likely to want to actually do 
something with one or more of these services. Keep in mind that the 
content of the service request is likely to be larger than the 
introspection file, and most of the content of that request cannot be 
used by Atom anyway. Requiring the client to request the service to get 
its capabilities will at least double network overhead in most cases, 
and most of that extra overhead will be wasted. In this age of broadband 
that extra bandwidth may not mean much on the client end, where 
everything is measured in seconds per request, but it makes a world of 
difference on the server end, where things are more often measured in 
requests per second.

Does this mean that introspection links should be taken out of the 
service itself, leaving only a service.introspection or service.feed 
link? I don't know. The current method does embed a little data into 
each service request whether or not Atom is involved, and that does mean 
that we create overhead even outside of Atom. A little overhead of this 
type -for the link to an introspection file, if nothing else- is 
unavoidable, but there is a school of thought saying that we should 
minimize this. At the same time, let us consider the overhead for Atom 
itself. For small deployments, particularly those where there is only 
one service per user, it is unclear whether any bandwidth can really be 
saved by keeping this information in the service. For larger 
deployments, however, when one user might be responsible for many sites, 
the bandwidth savings for in-service introspection are clear, since a 
request for an introspection file may involve many services, only one of 
which the user may want.

It seems to me, then, as though if we want to have minimal impact on the 
network, it is probably safest to include the capability information in 
both places: an introspection-capable file and the service itself. Each 
place has unique advantages that the other cannot duplicate, and in both 
cases the bandwidth advantage actually goes to showing this data in both 
areas. The separate file is useful when we only know which user is 
accessing the system, but not what service he might want to update, and 
this is a more common scenario than might initially be believed. 
Meanwhile, the in-service introspection links are more useful when we 
know which service we want to update, because we do not waste bandwidth 
on other services that we know we don't want.

 >> What's the problem with having each feed point to an introspection
 >> file which lists its service API points (URIs)? We have a way of any
 >> web page that is associated with the feed to point to the Atom feed;
 >> then the feed points to the introspection file; that tells you how to
 >> edit the feed (and perhaps the underlying resource group, depending on
 >> what you're doing). It's disadvantage is mostly in the number of
 >> requests required, but without bloating either the HTML page or the
 >> Atom feed, that looks unavoidable.

Indeed, it's unavoidable, but is it necessarily bloat? It seems that 
putting the information in both places actually ends up doing less 
damage overall than putting the data in only one place. It's not good 
database theory, but we're not a database; we're an API which has to 
communicate over a medium where latency is still relatively high. It's 
not uncommon nowadays to hear that the biggest problem with Web-based 
applications is the inherently high-latency interfaces with which we 
must interact; we cannot completely solve this problem, but this can 
still go a fair way towards alleviating the worst of it.

 >> If we want to roll up several feeds in one so people can easily keep
 >> track of (for instance) the feeds a site publishes, that is a
 >> different issue and should be addressed separately.

Again, we're not talking about feeds; we're talking about services which 
may or may not publish feeds of their own.

- Beecher Greenman

---

From: James Aylett <[email-masked]>
Date: Fri, 2 Jul 2004 13:29:50 +0100

On Thu, Jul 01, 2004 at 06:07:08PM -0400, Beecher Greenman wrote:

> be introspected.

My thinking is as follows:

 * a feed is a collection of entries
 * we provide a serialisation of this feed's state in Atom XML
 * this (the Atom document) is a resource, in that it has a URL

So 'feed' is part of the conceptual model, just like 'entry', so it's
equally something you can introspect against.

> You could introspect sites, entries, users, or whatever 

I'd argue that the feed is a group of entries, and that the Atom XML
document is the resource representation of that feed. I think this is
mostly a difference in how we're using the words though, rather than a
real difference in opinion.

> don't think there's any argument that a feed is an actual object instead 

I think there is. Further, I think that the argument on these lines
descends into TAG territory, which probably means we aren't going to
resolve it in the lifetime of atompub :-)
 
> represents a service but it is not the service itself.

That's a little too metaphysical for me :-)
 
> rather than a service, the principle seems to still apply.

You're not introspecting the author (which is a person, possibly), but
looking at a resource representing the collection of some of the feeds
that author can access.

You're not even introspecting the res rep of the /author/, because the
service points that you get from the introspection are not intrinsic
to the author, they are intrinsic to the feeds that the author can
access.
 
> Second: why should this not happen?

I accept that it is. (I accept this now, because I now understand what
the use case for it is.)

> may or may not publish feeds of their own.

I have an issue with this view, namely that we don't have a model for
such services.

If we introspect on a feed, and get a list of service points for that
feed (how to post, edit, delete, comment, whatever) then it's simple
and clear what's going on. We can possibly expand that to have the
introspection file contain labelled information about /several/ feeds;
and if we link to this introspection file from an HTML page, we (I
think) get the effects most people seem to want.

I'd be opposed to explicitly talking about a 'service' behind one or
more feeds, and heavily opposed to doing so unless we have a very
clear idea of what we mean by that. (The best I can think of is that
you could consider a MovableType installation as a service - that's an
example however, not a definition.)

James

-- 
/--------------------------------------------------------------------------\
  James Aylett                                                  xapian.org
  [email-masked]                               uncertaintydivision.org