Subject: [ogpx] An Inquiry into the Nature and Causes of Web Capabilities

From: Infinity Linden <[email-masked]>
Date: Wed, 3 Jun 2009 15:41:52 -0700

So there were a few questions off-line recently about capabilities and
their use in OGP. Some people understand them, some people don't. Some
people think they're cool, some people want to hurl inanimate objects
around at the mention of their name. But it does seem that there's a
lot of confusion about the motivation for caps in general and web
capabilities in OGP specifically. So i figured i would take a few
moments to try to demystify them.

1. What _Are_ Capabilities Anyway?

In General, capabilities are authorization tokens designed so that
possession of the token implies the authority to perform a specific
action. Consider a file handle in POSIX like systems. When you open a
file, you get back an integer representing the open file. You only get
this file handle back if you have access rights to the file. When you
spawn a new child process, the child inherits your open file handles
and it too can access those files via the file handle, even though the
child process lives in a completely different process space. Later
versions of *nix even allow you to move open file handles between
unrelated processes. So... the takeaway here is... it's an opaque bit
of data (i.e. - a token) and if you have it, it means you have the
authority to use it. And... you can pass it around if need be.

Capabilities on the web extend the concept. In addition to the token
connoting authorization to access some resource, it usually also
provides the address to access the resource. In other words, a web
capability is a URL possessed by a client that the client may use to
create, read, update or delete a resource. A web capabilities in OGP
are in the form of a "well known" portion of a URL (something like
"http://service.example.org/s/") and a large, unguessable, randomly
generated string (like "CE7A3EA0-2948-405D-A699-AA9F6293BDFE".)
Putting them together, you get a valid URL a client can use to access
a resource via HTTP. In this example, that URL would be
"http://service.example.org/s/CE7A3EA0-2948-405D-A699-AA9F6293BDFE".

2. Why the Heck Would I EVER Want to Do THAT !?

Yup. No doubt about it, this is not a "standard pattern" for web
services. Usually if you have a resource, you publish a well known
resource and if it's sensitive you require the client to log in. For
example, you might have a RESTful resource at
"http://service.example.org/s/group/AWGroupies" that represents the
group "AWGroupies". You then define an API that says if you want to
post a message to the group, you use HTTP POST with data (XML, JSON or
whatever) that implies "post this message to this group". For the sake
of discussion, let's say the JSON of the message would look like: {
from: "Zha Ewry", message: "I'm giving 50 L$ to anyone who IMs me in
the next 5 minutes!" }. This is a perfectly acceptable solution,
though readers familiar with AWGroupies will know that i am not, in
fact, the first life avatar of Zha Ewry, so i probably shouldn't be
posting messages to this group promising to disburse game specie in
her name. So authentication is in order, but this is a well known
problem, i simply use HTTP digest auth over HTTPS (or something
similar) and we're done. This is a perfectly acceptable solution.

But there are a couple of issues with this solution.

Most notably... every service that presents an interface to a
sensitive resource MUST understand authentication. So not only does
"http://service.example.org/s/group/AWGroupies" need to understand
authentication, so does
"http://service.example.org/s/user/Linden/Infinity" and
"http://service.example.org/s/parcel/Levenhall/Infinity%20is%20full%20of%stars"
and so on. Not a problem really, until you start adding new
authentication techniques. So one day your boss comes to you and tells
you.. "hey! we're using Secure/IDs for everything now!" Ugh. But it's
still not _that_ painful. You've probably abstracted out
authentication, so you have a map of service URLs to authentication
techniques and common libraries that actually authenticate requests
throughout your system.

And this works until the day that your boss comes in and says... "hey!
we just bought our competitor Cthulhos.Com! we're going to integrate
their FooWidget service into our service offering! isn't it great!"
And then you get that sinking feeling cause you know that this means
you've got to figure a way to get their service to talk to your
identity provider so their service can authenticate your customers.
People who have been through this know that depending on the
technology this can turn out to be a bag full of pain. The standard
way of doing this is something like:

a. service request come into http://service.example.org/s/foo/Infinity
b. http://service.example.org/s/foo/ redirects with a 302 to
http://foo.cthulhos.com/Infinity
c. http://foo.cthulhos.com/Infinity responds with a 401 getting the
client to resubmit the request with a WWW-Authenticate: header.
d. the client resubmits to http://foo.cthulhos.com/Infinity with the
proper WWW-Authenticate: header, but remember, these are example.org's
customers, so
e. http://foo.cthulhos.com/Infinity sends a message to a private
interface on example.org, asking it to authenticate the user
credentials.
f. let's assume the client is using valid credentials, so example.org
responds to cthulhos.com with the digital equivalent of a thumbs up,
and finally...
g. http://foo.cthulhos.com/Infinity responds to the request.

And this works pretty well up until the point that the new CIO comes
in and say.. "infocard! we're moving everything to infocard!" There's
nothing wrong with infocard, of course, but i this situation you've
got to implement it at both example.org and cthulhos.com. And when we
start adding to the mix the fact that the biz dev people keep trying
to buy new companies and you get a new CIO every 18 months who wants a
new user authentication technology, things can get out of hand. And...
i didn't even talk about the fact that each time you change the
authentication scheme, thick client developers have to go through the
code, looking for every place a request is made.

Web Capabilities are not a magic panacea, but they can help out in
this situation. Rather than having each request authenticated, the
user's identity is authenticated once at a central location and that
central location (example.org) coordinates with it's services
(cthulhos.com) to provide a unique, unguessable URL (the capability)
known only to that specific client and trusted systems (example.org
and cthulhos.com). So the flow would be something like...

a. client logs in at http://service.example.org/s/authme and asks for
a capability to use a particular service
b. http://service.example.org/s/authme verifies the user's credentials
and verifies the user can access that service
c. http://service.example.org/s/authme sends a request to a private
interface on cthulhos.com asking for the capability.
d. cthulhos.com generates the unguessable capability
http://foo.cthulhos.com/EE409B12-6E9B-4F5B-90BF-161AE5DE410C and
returns it to http://service.example.org/s/authme
e. http://service.example.org/s/authme returns the capability
http://foo.cthulhos.com/EE409B12-6E9B-4F5B-90BF-161AE5DE410C to the
client
f. the client uses the capability
http://foo.cthulhos.com/EE409B12-6E9B-4F5B-90BF-161AE5DE410C to access
the sensitive resource.

Both approaches require establishing a trusted interface between
example.org and cthulhos.com, but in the case of the capability
example, only service.example.org has to know about the specific
details of user authentication. Thick client developers may also
notice that they access the capability as if it were a public
resource; that is, they don't need to authenticate each request.

Another benefit to capabilities is that they are pre-authorized. If
you have a resource that is accessed frequently (like maybe "get the
next 10 inventory items" or "check to see if there are any messages on
the event queue for me") you don't have to do the username ->
permissions look up each time the server receives a request. For
environments where the server makes a network request for each
permissions check, this can lead to reduced request latency.

So... to recap... capabilities are not magic panaceas... there's still
some work involved in implementing them. and they start making a lot
more sense when you have a cluster of machines offering service to a
client, but deployers want identity and identity to permissions
mapping functions to live elsewhere in the network than the machine
offering a service. (i.e. - "the cloud" or "the grid".)

3. Okay... But How Do I Provision a Capability ?

I'm sure there are several ways to provision capabilities, but the
approach we take in OGP is to use the "seed capability."

Like most other distributed protocols involving sensitive resources,
OGP protocol interactions begin with user authentication. (This is not
strictly true, 'cause i'm ignoring the case where two machines want to
communicate outside the context of a user request, but let me hand
wave that use case away for the moment while we talk about using seed
caps.)

The process begins with user authentication. The OGP : Auth
specification describes this process; it is essentially the client
sending an avatar name and a password to a server in the agent domain.
Assuming the authentication request can be validated, the agent domain
returns a "seed cap." The client then sends a list of capability names
to the seed cap and awaits the response.

What the host behind the seed cap is doing while the client waits for
a reply is verifying the requested capability exists and the user is
permitted to perform the operation implied by the capability. (and it
does this for each capability requested.)

So, for example, let's say you are a client that only wants to update
a user profile and send/receive group messages. The protocol
interaction might look something like this...

a. authentication : client -> server at https://example.org/login

{
  identifier: {
    type: "agent",
    first_name: "Random",
    last_name: "Avatar"
  },
  authenticator: {
    type: "hash",
    algorithm: "md5",
    secret: "i1J8B0rOmekRn8ydeup6Dg=="
  }
}

b. auth response : server -> client

{
  condition: "success",
  agent_seed_capability:
"https://service.example.org/seed/CF577955-3E0D-4299-8D13-F28345D843F3"
}

c. asking the seed for the other caps : client -> server at
https://service.example.org/seed/CF577955-3E0D-4299-8D13-F28345D843F3

{
  capabilities : [
    "profile/update",
    "groups/search"
  ]
}

d. the response with the URIs for the caps : server -> client

{
  capabilities : {
    profile/update :
"http://service.example.org/user/35A59C5D-315C-4D50-B78D-A38D41D2C90A",
    groups/search : "http://cthulhos.com/8579CE1F-9C05-43E8-8677-A645859DCD64"
  }
}

4. Expiring Capabilities

Readers may notice that there is a potential "TOCTOU vulnerability."
TOCTOU stands for "time of check, time of use," and refers to a common
security problem... what happens if the permissions on an object
change between the time the code managing the resource checks the
permission and the time it performs an operation on the resource.

This is a common problem with many systems, including POSIX file
descriptors. (seriously.. if you change the permissions on a file to
disallow writing AFTER the file has been opened, subsequent writes on
the file descriptor will not fail in many POSIX systems.)

OGP capabilities address this problem by expiring capabilities when
they get old. So if you request a capability, then wait a LONG time
before you access it, you may find that you get a 404 response code.
The OGP specifications do not require all caps to expire, but the do
require servers to signal their expiration by removing them (thus the
404 response) and require clients to understand what to do when a
capability has been expired. In most cases, the appropriate response
is to re-request the capability from the seed cap. If the seed cap is
expired, clients should re-authenticate.

Capabilities may also "expire after first use." Also called "single
shot capabilities," they are used to communicate sensitive or highly
volatile information to the client.

The current practice is to include an Expires: header in the response
from the server so the client will know when the resource expires.

5. Introspection on Capabilities

Finally, RESTful resources represented by a capability are described
by LLIDL, the interface description language used by LLSD. A request
has been made that introspection be supported so clients may receive
the LLIDL description of a capability and more accurately reason about
the semantics of its use.

The proposed solution to this problem for OGP messages carried over
HTTP is to use the OPTIONS method when accessing the capability
(instead of GET, POST, PUT or DELETE.) Upon receipt of the OPTIONS
request, the server should respond with the LLIDL describing the
resource.

6. Conclusion

Capabilities are cool, especially in clouds or grids.

7. References

A pair of Google Tech Talks at:
  http://video.google.com/videoplay?docid=-8527120258517176598 , and
  http://video.google.com/videoplay?docid=8799856896828158583
provide a pretty good technical introduction to the concept of capabilities.

OGP's description of capabilities is at:
  http://tools.ietf.org/html/draft-lentczner-ogp-base-00#section-2.3

OGP uses capabilities to access RESTful resources. Roy Fielding's
paper on REST is at:
  http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm

---

From: Carlo Wood <[email-masked]>
Date: Fri, 5 Jun 2009 00:58:40 +0200

I think it might be better to separate capability and the hostname
of the url. That is, if you are passed:

http://db33.example.com/foo/?5923828591293879468287348955

then the capability should be "foo/?5923828591293879468287348955".

In particular, an capability url

http://db33.example.com/foo/?5923828591293879468287348955

and

http://db34.example.com/foo/?5923828591293879468287348955

should be garanteed to be the same capability.

The main reason for this idea is because I'm worried about
unreachable hosts, or the lifetime that capabilities need
to be kept alive.

Now assume that a client knows that it can choose from
a pool of hostnames for a given capability (ie, the two
above), then a problem might occur if access of the
capability is supposed to change a state. For example,
it cannot be allowed to use the same capability twice;
then accessing both hosts at the same time still needs
to result in the access of a single database.

Do you think the hostname has to be fixed? Would the
above idea have too great disadvantages?

-- 
Carlo Wood <[email-masked]>

---

From: Christian Scholz <[email-masked]>
Date: Fri, 05 Jun 2009 10:22:15 +0200

Hi!

First I personally like my components to do authorization themselves, it 
feels more secure to me. Also as in the Linden Lab setup I don't really 
like that I need an internal network I need to protect. I could possibly 
do this by running my backend servers on 127.0.0.1 but that relies for 
me too much on configuration. Moreover it gets harder if you want to 
implement caps in a standalone server as you cannot use the toolset 
common web frameworks give you. You'd need to implement an internal 
mapping yourself.

That's why I like OAuth better, mostly because it already has wider 
adoption and tools are around.

As for your use case of adding new components (which seem to have no 
authorization mechanism yet of themselves) why not simply adding an 
OAuth proxy in front?

And as for InfoCards etc.: Wouldn't this be mostly the login step? After 
that you'd either get a seed cap or an oauth access token.

Here some brainstorming:

- you have all your components listed in an XRD file describing which 
services are located where, that can be possible done on a per user 
basis so that the client once retrieves all the necessary URLs.
- you have some Access Broker authorized via an OAuth Access Token which 
can give out additional OAuth tokens for those services (similar to a 
seed cap).
- you use those access tokens to perform requests on the URLs derived 
from the XRD file
- in case your component has no authorization code of itself you put a 
proxy in front which checks the OAuth token and either forwards the 
request or denies it.
- The Access Broker talks to the subcomponents (and it is allowed to by 
OAuth, too) to retrieve specific service based tokens (similar to 
non-seed caps) and returns them to the client.

I haven't thought too deep about this but if this works it would have 
the benefits of being able to reuse the OAuth signing method and thus 
the OAuth libraries. It also would have the benefits of many more people 
having had a look at it (and as we know one security problem was already 
discovered).

(and btw, I think even the LL SLIM app would have benefitted from using 
OAuth as you wouldn't need to create a new account with vivox, you 
simply login to secondlife.com and this would directly send an access 
token to SLIM which then would use this to get access to agent data. 
Then again probably the same would have been possible with caps I guess. 
I wonder why it hasn't been done that way).

-- christian

Infinity Linden schrieb:
> _______________________________________________
> ogpx mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ogpx

-- 
COM.lounge GmbH
http://comlounge.net
Hanbrucher Strasse 33, 52064 Aachen
Amtsgericht Aachen HRB 15170
Geschäftsführer: Dr. Ben Scheffler, Christian Scholz

email: [email-masked]
fon: +49-241-4007300
fax: +49-241-97900850

personal email: [email-masked]
personal blog: http://mrtopf.de/blog
personal podcasts: http://openweb-podcast.de, http://datawithoutborders.net

---

From: Meadhbh Siobhan <[email-masked]>
Date: Fri, 5 Jun 2009 08:57:35 -0700

dang. i keep hitting "reply" instead of "reply to all"

---------- Forwarded message ----------
From: Meadhbh Siobhan <[email-masked]>
Date: Fri, Jun 5, 2009 at 8:56 AM
Subject: Re: [ogpx] An Inquiry into the Nature and Causes of Web Capabilities
To: Carlo Wood <[email-masked]>

ah! i see where you're going.

there is something i didn't put in the previous message.

when a capability expires and the client attempts to access it, it
should get a 404. if the system is down, the TCP connection will
eventually time out. the processing expectations for the client are
that if it needs to access an expired (or otherwise unavailable) cap,
it goes back to the seed cap and says... "hey! gimme another one of
these caps, please!"

at that point, it is up to the system hosting the seed cap to return a
cap that is accessible to the client. if the seed cap is expired, the
expectation is that the client will re-authorize (with OAuth) or
re-authenticate (with user login or transport layer identity /
authentication.)

i think this points to a philosophical difference that's plagued
protocol developers from the early days... where does the intelligence
live? in the POTS phone system, the intelligence lies with the network
and the leaf nodes are relatively dumb. tcp based networks do devolve
application layer and some network layer smarts to client.

the systems that have been deployed to date make a "network has the
smarts" assumption and current implementations don't have the feature
you're describing. personally.. i'm not a big fan of what you're
describing 'cause it increases the burden on network services (my
systems now have to have a distributed cap router and understand that
endpoints from multiple locations on the net route to a particular
system.)

but.

i'm also an experimentalist at heart. if you wanted to re-write the
caps section of the base document to describe what you're doing, and
then investigate the behavior of the two different systems under
typical failure cases (host down, host unreachable, etc.) i would love
to see if there's a major difference in behavior.

that being said, LL, PyOGP and at least one of the OpenSim
implementations have been implemented using caps as described here. so
it might be a better idea to go ahead with what we've got described
now, so we can get to the real work of describing the application
layer messages, but keep the "multi-caps" as a future expansion
option.

-cheers
-meadhbh

On Fri, Jun 5, 2009 at 8:09 AM, Carlo Wood<[email-masked]> wrote:
> Ok... but what if that host is unreachable/down for a while?
> If the hostname is a fixed part of the capability then it
> is impossible to continue.
>
> We could simply redefine things a bit, and say that a service
> returns:
>
> { range or set of hosts / the rest of the URL }
>
> Or more indetail, it could return:
>
> http://assetX-NNN.example.com/?91839838
>
> where then, as part of the protocol, the 'NNN' has
> to be replaced by by a number between 1 and N, where
> N was given to the client upon login (or whenever,
> some grid-dependent configurable value).
>
> In that case, the capability would be
> "http://assetX-NNN.example.com/?91839838"
> and thus not includes (one) host.
>
> The clients would know how to route it (they
> just fill in some number for NNN), and if that
> host is unreachable they can try another host.
>
> --

---

From: Nexii Malthus <[email-masked]>
Date: Tue, 16 Jun 2009 18:36:14 +0100

@Infinity Linden, that explanation was absolutely fantastic to read for a
newcomer like me to this Capability stuff. Now I understand this a lot
better. The theory behind it makes a lot of sense.

---

From: Charles Krinke <[email-masked]>
Date: Tue, 16 Jun 2009 12:20:37 -0700 (PDT)

Is there going to be an "OGP" booth on the regions for SL6B? If so, do we know where it might be, yet?

OSGrid has gone ahead and put in for a booth and we would like to be near other OGP activity and have a few prim signs describing OGP in our booth.

Charles

---

From: Infinity Linden <[email-masked]>
Date: Tue, 16 Jun 2009 16:40:17 -0700

hmm.. i think we're focusing more on getting revisions of various
drafts cranked out before the BoF, so not much bandwidth to have an
official presentation at SL6B. Hopefully, by SL7B we'll have more
consensus, more working code and more presentations for peeps of all
levels of tech / business savvy. But alas, it's not there today.

-cheers
-meadhbh / infinity

On Tue, Jun 16, 2009 at 12:20 PM, Charles Krinke<[email-masked]> wrote:
> OSGrid has gone ahead and put in for a booth and we would like to be near
> other OGP activity and have a few prim signs describing OGP in our booth.
>
> Charles
>