Subject: AtomPub accept

From: Erik Wilde <[email-masked]>
Date: Tue, 24 Aug 2010 15:13:07 -0700

hello.

we are working on an atompub implementation where we don't just want to 
limit the accepted entries to media types, but to certain subsets of 
documents (specifically, not just KML, but only KML with, for example, 
polygons, or only points). is there an established way of how to express 
these additional constraints in the service document? adding schema 
information? adding "profiles" (which probably would be opaque and 
specific to the service)? we can of course invent our own mechanism, but 
i am trying to find out whether there is something we can reuse.

thanks and kind regards,

erik wilde   tel:+1-510-6432253 - fax:+1-510-6425814
        [email-masked]  -  http://dret.net/netdret
        UC Berkeley - School of Information (ISchool)

---

From: Erik Wilde <[email-masked]>
Date: Wed, 25 Aug 2010 18:23:42 -0700

hello jan.

>> still requires some rather static "registration" of those profile 
>> names, right? 
> Yes, that is the idea :-) ... it is all about avoiding coupling clients 
> to specific services. Better to couple to a 'standard'.

my 2min research did not yield a conclusive answer to this, but it is 
actually possiblt to define a parameter to a media type when the media 
type is already existing and does not have that parameter? it kind of 
feels weird, but my media type definition/registration knowledge is a 
bit rusty in this particular department...

>> or could these be any kind of resource, including something like 
>> schemas or templates that would be fully self-describing?
> IMHO anything will do as long as the specification is a 'standard' (in 
> the sense of: not specific to the single service) and as long as it 
> enables developers to use the specification when implementing the client 
> and server. IOW: developers must understand it

a schema would be nice because it could be machine-readable, so that 
there would be no need to have a predefined set of profile values. you 
find a profile URI, read the schema, and then know what additional 
constraints are defined for the KML mathing this profile. probably ok, 
but i am a bit concerned about the retrofitting of parameters.

>> client behavior, generally speaking ;-) specifically, our UI should 
>> only provide tools to create features that then are accepted, so that 
>> users don't get a UI that allows them to create polygons, only to have 
>> them rejected by the server. our work revolves around managing spatial 
>> objects using atom and atompub.
> Curious due to some other discussions I have been having: Why did you 
> not use HTML to drive your user agent?

because we are building a feed-oriented API where the main abstraction 
are KML-described spatial features, and those are exposed through a 
specialization of the general atom/atompub API to collections. for our 
scenario, i wouldn't even know ho9w we could utilize HTML.

cheers,

dret.

---

From: Peter Keane <[email-masked]>
Date: Wed, 25 Aug 2010 21:23:56 -0500

I have used the app:category element to something sort of like this.
Per RFC5023 section 8.3.6
(http://bitworking.org/projects/atom/rfc5023.html#categories-elem) a
collection can advertise that members should have a particular
category to be accepted.  Of course you'd need a category schema that
mapped to whatever combinations of KML elements were acceptable, then
add a "app:category" element to the KML doc itself say it was, say, of
the "point+polygon" category.  But at least you'd have a standard
AtomPub mechanism that a client that knew about it could use.

My use of this mechanism has been a means to "type" atom entries,
since we use atom:entry to represent different "types" in the
application. Certain collections only accept certain types of
atom:entries and that is advertised in the app:categories section of
the service doc for the collection.

--peter

On Wed, Aug 25, 2010 at 8:23 PM, Erik Wilde <[email-masked]> wrote:
>
> hello jan.
>
>>> still requires some rather static "registration" of those profile names,
>>> right?
>>
>> Yes, that is the idea :-) ... it is all about avoiding coupling clients to
>> specific services. Better to couple to a 'standard'.
>
> actually possiblt to define a parameter to a media type when the media type
> is already existing and does not have that parameter? it kind of feels
> weird, but my media type definition/registration knowledge is a bit rusty in
> this particular department...
>
>>> or could these be any kind of resource, including something like schemas
>>> or templates that would be fully self-describing?
>>
>> IMHO anything will do as long as the specification is a 'standard' (in the
>> sense of: not specific to the single service) and as long as it enables
>> developers to use the specification when implementing the client and server.
>> IOW: developers must understand it
>
> a schema would be nice because it could be machine-readable, so that there
> would be no need to have a predefined set of profile values. you find a
> profile URI, read the schema, and then know what additional constraints are
> defined for the KML mathing this profile. probably ok, but i am a bit
> concerned about the retrofitting of parameters.
>
>>> client behavior, generally speaking ;-) specifically, our UI should only
>>> provide tools to create features that then are accepted, so that users don't
>>> get a UI that allows them to create polygons, only to have them rejected by
>>> the server. our work revolves around managing spatial objects using atom and
>>> atompub.
>>
>> Curious due to some other discussions I have been having: Why did you not
>> use HTML to drive your user agent?
>
> because we are building a feed-oriented API where the main abstraction are
> KML-described spatial features, and those are exposed through a

---

From: Jan Algermissen <[email-masked]>
Date: Thu, 26 Aug 2010 20:56:39 +0200

On Aug 26, 2010, at 3:23 AM, Erik Wilde wrote:

> hello jan.
> 
>>> still requires some rather static "registration" of those profile names, right? 
>> Yes, that is the idea :-) ... it is all about avoiding coupling clients to specific services. Better to couple to a 'standard'.
> 
> my 2min research did not yield a conclusive answer to this, but it is actually possiblt to define a parameter to a media type when the media type is already existing and does not have that parameter?

The rule should be (though I have no pointer) to ignore unknown parameters. So, it would be fine.

> it kind of feels weird, but my media type definition/registration knowledge is a bit rusty in this particular department...
> 
>>> or could these be any kind of resource, including something like schemas or templates that would be fully self-describing?
>> IMHO anything will do as long as the specification is a 'standard' (in the sense of: not specific to the single service) and as long as it enables developers to use the specification when implementing the client and server. IOW: developers must understand it
> 
> a schema would be nice because it could be machine-readable, so that there would be no need to have a predefined set of profile values. you find a profile URI, read the schema, and then know what additional constraints are defined for the KML mathing this profile. probably ok, but i am a bit concerned about the retrofitting of parameters.
> 
>>> client behavior, generally speaking ;-) specifically, our UI should only provide tools to create features that then are accepted, so that users don't get a UI that allows them to create polygons, only to have them rejected by the server. our work revolves around managing spatial objects using atom and atompub.
>> Curious due to some other discussions I have been having: Why did you not use HTML to drive your user agent?
> 
> because we are building a feed-oriented API where the main abstraction are KML-described spatial features, and those are exposed through a specialization of the general atom/atompub API to collections. for our scenario, i wouldn't even know ho9w we could utilize HTML.

I was just asking because I had the impression you have a completely human targeted user agent.

Is there anything in public to read that explains what you are building?

Jan

---

From: Erik Wilde <[email-masked]>
Date: Sat, 28 Aug 2010 11:28:35 -0700

hello jan.

> The rule should be (though I have no pointer) to ignore unknown parameters. So, it would be fine.

yes, unknown parameters MUST be ignored, but unless there is some 
registry for parameter values that can populated with parameter 
definitions independently of the media type registration, it is hard for 
developers to understand what some parameter is supposed to mean.

> I was just asking because I had the impression you have a completely human targeted user agent.

no, we're building an API (targeted on spatial information services) and 
for our demos we need something that is a UI, but the API should be 
equally useful for machine-to-machine scenarios as it is for UIs.

> Is there anything in public to read that explains what you are building?

http://dret.net/netdret/publications#liu10a describes the overall 
design, and we're currently working on a paper that highlights the 
decentralized nature of our approach (as opposed to many other LBS 
scenarios that are more vertically integrated).

cheers,

dret.

---

From: Jan Algermissen <[email-masked]>
Date: Sat, 28 Aug 2010 21:08:02 +0200

On Aug 28, 2010, at 8:28 PM, Erik Wilde wrote:

> no, we're building an API (targeted on spatial information services) and for our demos we need something that is a UI, but the API should be equally useful for machine-to-machine scenarios as it is for UIs.

Can you provide an example of a m2m use? (would greatly help to wrap my brain around other stuff I am currently doing).

Jan

---

From: Jan Algermissen <[email-masked]>
Date: Sat, 28 Aug 2010 21:18:02 +0200

On Aug 28, 2010, at 8:55 PM, mike amundsen wrote:

> Erik:
> 
> It seems to me that you need a new negotiation level here.

Re the other reply. This defines Accept-Features: http://www.faqs.org/rfcs/rfc2295.html

(Not quite what you need, but maybe it is helpful).

Jan

> that
> clients could advertise support for certain kml element and servers
> could respond accordingly.
> for example:
> - kml-accept for a request header
>  contains the list of KML elements supported by the client
> - kml-type for response header
>  contains the list of KML elements returned within the response representation
> 
> These could be included in the Vary header listing to help caches sort
> out the details.
> Advantage is that the details are clearly worked out by both parties.
> Downside is that clients and servers must agree to these new headers.
> 
> Other options could be:
> - clients send KML support information as an argument in the URL
> ?kml={URL-encoded-list}
> - servers generate specific representation of the same map for each
> client and return a Content-Location header that points to the exact
> representation
> - Use Agent-drive conneg: servers respond w/ 300 See Other that
> contains a representation listing all options for this map
> representation based on KML types and the client picks the best fit.
> 
> mca
> http://amundsen.com/blog/
> http://mamund.com/foaf.rdf#me
> 
> 
> 
> 
> On Sat, Aug 28, 2010 at 14:28, Erik Wilde <[email-masked]> wrote:
>> 
>> hello jan.
>> 
>> Jan Algermissen wrote:
>>>> 
>>>> is already existing and does not have that parameter?
>>> 
>> yes, unknown parameters MUST be ignored, but unless there is some registry
>> for parameter values that can populated with parameter definitions
>> independently of the media type registration, it is hard for developers to
>> understand what some parameter is supposed to mean.
>> 
>> no, we're building an API (targeted on spatial information services) and for
>> our demos we need something that is a UI, but the API should be equally
>> useful for machine-to-machine scenarios as it is for UIs.
>> 
>> http://dret.net/netdret/publications#liu10a describes the overall design,
>> and we're currently working on a paper that highlights the decentralized
>> nature of our approach (as opposed to many other LBS scenarios that are more
>> vertically integrated).
>> 
>> cheers,
>> 
>> dret.
>> 
>>

---

From: Erik Wilde <[email-masked]>
Date: Sat, 28 Aug 2010 12:38:50 -0700

hello jan.

> Can you provide an example of a m2m use? (would greatly help to wrap my brain around other stuff I am currently doing).

since we're building on atom and atompub, you can simply look at atom's 
canonical "blogging" use case. if you build a mobile client for creating 
location-based information (let's say, snapping geotagged pictures with 
some app that accesses our API), it would use our API to POST new 
pictures (in that case the accepted media type might be JPEG instead of 
some KML geometry), and to GET other pictures (such as your friends 
snapping pictures). we are not in any way restricting the way in which 
this app's UI would be built (or whether it even had a UI or would be 
some fully automated app with no UI at all), we are only concerned with 
the API design for LBS-oriented collections.

cheers,

dret.

---

From: Erik Wilde <[email-masked]>
Date: Sat, 28 Aug 2010 13:16:54 -0700

hello mike.

mike amundsen wrote:
> If the list of allowed elements is decided solely by the server (e.g.
> "we only allow polygons at this server" or "today we accept only
> circles and arcs", etc.):
> - use a media-type extension specific for your needs:
>   appcation/kml;allowed="{,,,,,}"
>   where the value for allowed is a  comma-sep list of allowed KML elements

that might be a bit too rigid, but i think we haven't even fully made up 
our minds what kind of constraints we want to be able to encode.

> - use the profile extension to point to a document with the list of
> allowed XML elements
>   application/kml;profile="{uri}"
>   where the document @ {uri} is in a form the client knows (based on
> documentation) how to parse for a list of allowed elements

that sounds more flexible and powerful and was what jan suggested; i am 
still a bit unclear, though, how "profile" would become a well-known 
parameter for the KML media type. i tend to think that you cannot just 
retroactively register additional parameters for an already registered 
media type, but i may be wrong.

> - instruct clients to use the OPTIONS method to return a list of
> allows elements via a new header (kml-allowed) and/or the body of the
> response

but would this still count as atompub?

> if the list of allowed elements can be actively "negotiated" based on
> client capabilities (e.g."oh, i know you, you can only do arcs" or
> "all clients from your location are allowed the full list", etc.):

no, this is not in our requirements. we want a service to be able to 
expose what it accepts, and clients will have to deal with that.

thanks,

dret.

---

From: Erik Wilde <[email-masked]>
Date: Sat, 28 Aug 2010 13:46:23 -0700

hello mike.

> It seems to me that you need a new negotiation level here. that

i guess we have to discuss this in bit more depth in our team, but my 
initial reaction to this is that all we need is a way for a server to 
expose what it is willing/able to accept. clients will have to live with 
that (as they do in plain atompub), and if they don't support what the 
server accepts, then they simply cannot edit or create new entries.

> Downside is that clients and servers must agree to these new headers.

i think i'd like to avoid using new headers. it makes an API much more 
complicated to understand and use for the average web developer, even if 
it may be an elegant and architecturally sound way to design it.

thanks a lot for your input and the other suggestions! most importantly, 
i am now pretty certain that there isn't "the one way" that our problem 
is typically solved in other applications or APIs. we now have to 
discuss our requirements in more detail, and then will try to come up 
with a simple and effective way of solving our problem.

thanks and cheers,

dret.

---

From: Erik Wilde <[email-masked]>
Date: Sat, 28 Aug 2010 17:54:25 -0700

hello peter.

> AtomPub mechanism that a client that knew about it could use.

hm. to me, categories are metadata labels that i can attach to entries, 
and i am able to require that only certain labels can be used. defining 
the semantics of those labels as restricting the entry's representation 
to me looks a little bit like overloading the semantics of categories in 
a way that is not intended by the spec, and i think its better to 
clearly separate the category of something (a classification of an 
entry) from its representation (a constraint on the entry itself). i 
think i would stay away from this pattern, but i'd be curious to hear 
what others think about this approach.

> the service doc for the collection.

i think i understand why you're doing it this way, but i think this 
couples categories and entries (too) tightly. we would like to use 
categories in their normal classification role, too, and mixing this 
usage of categories with the magic "this category constrains the entry 
representation" categories would results in categories being used for 
very different purposes.

thanks for the suggestion! cheers,

dret.

---

From: Peter Keane <[email-masked]>
Date: Sat, 28 Aug 2010 23:45:57 -0500

On Sat, Aug 28, 2010 at 7:54 PM, Erik Wilde <[email-masked]> wrote:
> hm. to me, categories are metadata labels that i can attach to entries, and
> i am able to require that only certain labels can be used. defining the
> semantics of those labels as restricting the entry's representation to me
> looks a little bit like overloading the semantics of categories in a way
> that is not intended by the spec, and i think its better to clearly separate
> the category of something (a classification of an entry) from its
> representation (a constraint on the entry itself). i think i would stay away
> from this pattern, but i'd be curious to hear what others think about this
> approach.
>
> i think i understand why you're doing it this way, but i think this couples
> categories and entries (too) tightly. we would like to use categories in
> their normal classification role, too, and mixing this usage of categories
> with the magic "this category constrains the entry representation"
> categories would results in categories being used for very different
> purposes.

I actually agree w/ you here.  We've done lots of overloading of
atom:category as it seemed to the most natural pure-atom extension
point.  But ultimately its not really in the spirit of the spec. While
I do not regret at all our use of Atom/AtomPub, it served mainly to
simply rationalize a growing/evolving system (a digital media
repository) and keep us relatively RESTful and enjoying the benefits
of some tooling (feed readers, feed validator, Tim Bray's APE, etc.).
To that extent it was a good idea.  But interop w/ other systems that
we didn't control, no so much (other than lots of aspects of our
system can be followed w/ a feed reader).

I'll be following your efforts closely -- it sounds like a much
truer-to-the-spirit application of Atom/AtomPub.  I'd been of the
belief that Atom/AtomPub was going to be the ideal generic read/write
format & protocol that could be almost universally applicable.  I've
come to believe that the genericity we were after is probably best
achieved with intelligent use of JSON (I think CouchDB basically got
that right), and that's the direction we are moving in.  How to do
REST properly w/o JSON being a hypertext format is another can o'
worms (which we address simply by using a convention for hyperlinks).

--peter

---

From: James Snell <[email-masked]>
Date: Sun, 29 Aug 2010 08:27:54 -0700

I've got a draft spec for representing a number of common object types in
JSON that I've been working on... basically just defines the conventions...
e.g.

link object:
{
  "rel":"...",
  "type":"...",
  "href":"...",
  "hreflang":"...",
  "length":"..."
}

category:
{
  "scheme":"..."
  "term":"..."
}

Was likely going to publish the draft tomorrow.

- James

On Sat, Aug 28, 2010 at 9:45 PM, Peter Keane <[email-masked]> wrote:

> and
> couples

-- 
- James Snell
  http://www.snellspace.com
  [email-masked]